2023-11-28T12:47:25.827 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5 2023-11-28T12:47:25.827 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469974 --name neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} --owner scheduled_neesingh@teuthology -- /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469974/orig.config.yaml 2023-11-28T12:47:25.883 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T12:47:25.976 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469974 branch: wip-neesingh-testing-231123 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7469974' kernel: client: branch: testing kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-neesingh-testing-231123 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: issue_54557 sha1: 3ae99413970476692dfab1fe38dadb056700a61d owner: scheduled_neesingh@teuthology priority: 10 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6754 sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 sleep_before_teardown: 0 suite: fs suite_branch: issue_54557 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: 3ae99413970476692dfab1fe38dadb056700a61d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi156.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi186.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 5ae09a5cc7064ca2bf994df19f3006b79794789a timestamp: 2023-11-28_12:38:30 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-28T12:47:25.976 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa; will attempt to use it 2023-11-28T12:47:25.977 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks 2023-11-28T12:47:25.977 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-28T12:47:25.979 INFO:teuthology.task.internal:Checking packages... 2023-11-28T12:47:26.002 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '2f5f6028b893ad41357cd06e827373f0fe2a1232' 2023-11-28T12:47:26.003 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-28T12:47:26.003 INFO:teuthology.packaging:ref: None 2023-11-28T12:47:26.003 INFO:teuthology.packaging:tag: None 2023-11-28T12:47:26.003 INFO:teuthology.packaging:branch: wip-neesingh-testing-231123 2023-11-28T12:47:26.003 INFO:teuthology.packaging:sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T12:47:26.003 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-neesingh-testing-231123 2023-11-28T12:47:26.222 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-2697.g2f5f6028 2023-11-28T12:47:26.223 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-28T12:47:26.228 INFO:teuthology.task.internal:no buildpackages task found 2023-11-28T12:47:26.228 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-28T12:47:26.256 INFO:teuthology.task.internal:Saving configuration 2023-11-28T12:47:26.275 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-28T12:47:26.281 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T12:47:26.307 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469974', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 12:42:17.443992', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T12:47:26.328 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469974', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 12:42:17.445454', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T12:47:26.328 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-28T12:47:26.335 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-28T12:47:26.336 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-28T12:47:26.336 INFO:teuthology.run_tasks:Running task console_log... 2023-11-28T12:47:26.442 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3e38dafa60>, signals=[15]) 2023-11-28T12:47:26.443 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-28T12:47:26.448 INFO:teuthology.task.internal:Opening connections... 2023-11-28T12:47:26.448 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T12:47:26.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T12:47:26.558 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T12:47:26.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T12:47:26.626 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-28T12:47:26.633 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2023-11-28T12:47:26.654 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2023-11-28T12:47:26.655 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2023-11-28T12:47:26.715 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T12:47:26.716 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T12:47:26.717 INFO:teuthology.orchestra.run.smithi156.stdout:ID="rhel" 2023-11-28T12:47:26.717 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE="fedora" 2023-11-28T12:47:26.717 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="8.6" 2023-11-28T12:47:26.717 INFO:teuthology.orchestra.run.smithi156.stdout:PLATFORM_ID="platform:el8" 2023-11-28T12:47:26.717 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T12:47:26.718 INFO:teuthology.orchestra.run.smithi156.stdout:ANSI_COLOR="0;31" 2023-11-28T12:47:26.718 INFO:teuthology.orchestra.run.smithi156.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T12:47:26.718 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T12:47:26.718 INFO:teuthology.orchestra.run.smithi156.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T12:47:26.718 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T12:47:26.718 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T12:47:26.718 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T12:47:26.718 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T12:47:26.718 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T12:47:26.718 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T12:47:26.719 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2023-11-28T12:47:26.741 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2023-11-28T12:47:26.762 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2023-11-28T12:47:26.763 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2023-11-28T12:47:26.822 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T12:47:26.822 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T12:47:26.822 INFO:teuthology.orchestra.run.smithi186.stdout:ID="rhel" 2023-11-28T12:47:26.822 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="fedora" 2023-11-28T12:47:26.822 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="8.6" 2023-11-28T12:47:26.822 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el8" 2023-11-28T12:47:26.823 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T12:47:26.823 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2023-11-28T12:47:26.823 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T12:47:26.823 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T12:47:26.823 INFO:teuthology.orchestra.run.smithi186.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T12:47:26.823 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T12:47:26.823 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T12:47:26.823 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T12:47:26.823 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T12:47:26.823 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T12:47:26.823 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T12:47:26.824 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2023-11-28T12:47:26.841 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-28T12:47:26.853 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-28T12:47:26.859 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-28T12:47:26.860 DEBUG:teuthology.orchestra.run.smithi156:> test '!' -e /home/ubuntu/cephtest 2023-11-28T12:47:26.863 DEBUG:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2023-11-28T12:47:26.883 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-28T12:47:26.890 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-28T12:47:26.890 DEBUG:teuthology.orchestra.run.smithi156:> test -z $(ls -A /var/lib/ceph) 2023-11-28T12:47:26.922 DEBUG:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2023-11-28T12:47:26.948 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-28T12:47:27.010 INFO:teuthology.run_tasks:Running task kernel... 2023-11-28T12:47:27.026 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-11-28T12:47:27.026 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-11-28T12:47:27.026 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-11-28T12:47:27.027 DEBUG:teuthology.orchestra.run.smithi156:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T12:47:27.027 DEBUG:teuthology.orchestra.run.smithi186:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T12:47:27.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:47:27.050 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-11-28T12:47:27.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:47:27.059 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-11-28T12:47:27.217 DEBUG:teuthology.task.kernel:sha1 for client.1 is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T12:47:27.218 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-28T12:47:27.218 DEBUG:teuthology.orchestra.run.smithi186:> uname -r 2023-11-28T12:47:27.228 DEBUG:teuthology.task.kernel:sha1 for client.0 is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T12:47:27.228 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-28T12:47:27.230 DEBUG:teuthology.orchestra.run.smithi156:> uname -r 2023-11-28T12:47:27.238 INFO:teuthology.orchestra.run.smithi186.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-28T12:47:27.239 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T12:47:27.239 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-28T12:47:27.240 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum upgrade -y linux-firmware 2023-11-28T12:47:27.249 INFO:teuthology.orchestra.run.smithi156.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-28T12:47:27.249 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T12:47:27.250 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-28T12:47:27.250 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum upgrade -y linux-firmware 2023-11-28T12:47:28.403 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-11-28T12:47:28.403 INFO:teuthology.orchestra.run.smithi186.stdout:Unable to read consumer identity 2023-11-28T12:47:28.457 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-11-28T12:47:28.457 INFO:teuthology.orchestra.run.smithi156.stdout:Unable to read consumer identity 2023-11-28T12:47:29.795 INFO:teuthology.orchestra.run.smithi186.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-11-28T12:47:29.903 INFO:teuthology.orchestra.run.smithi156.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-11-28T12:47:30.529 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 23 MB/s | 16 MB 00:00 2023-11-28T12:47:30.654 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2023-11-28T12:47:34.054 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 470 kB/s | 24 kB 00:00 2023-11-28T12:47:34.312 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 125 kB/s | 24 kB 00:00 2023-11-28T12:47:34.842 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-11-28T12:47:34.843 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2023-11-28T12:47:34.844 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-11-28T12:47:34.944 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.1... 2023-11-28T12:47:34.945 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T12:47:35.054 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-11-28T12:47:35.055 INFO:teuthology.orchestra.run.smithi156.stdout:Nothing to do. 2023-11-28T12:47:35.055 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-11-28T12:47:35.163 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-11-28T12:47:35.163 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T12:47:35.176 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/ 2023-11-28T12:47:35.177 DEBUG:teuthology.orchestra.run.smithi186:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/ --input-file=- 2023-11-28T12:47:35.179 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi186.front.sepia.ceph.com to complete... 2023-11-28T12:47:35.307 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/ 2023-11-28T12:47:35.308 DEBUG:teuthology.orchestra.run.smithi156:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/ --input-file=- 2023-11-28T12:47:35.310 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi156.front.sepia.ceph.com to complete... 2023-11-28T12:47:53.714 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28 12:47:53 URL:https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [383178732/383178732] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-28T12:47:53.714 INFO:teuthology.orchestra.run.smithi186.stderr:FINISHED --2023-11-28 12:47:53-- 2023-11-28T12:47:53.714 INFO:teuthology.orchestra.run.smithi186.stderr:Total wall clock time: 19s 2023-11-28T12:47:53.715 INFO:teuthology.orchestra.run.smithi186.stderr:Downloaded: 1 files, 365M in 18s (20.0 MB/s) 2023-11-28T12:47:53.716 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.1... 2023-11-28T12:47:53.716 DEBUG:teuthology.orchestra.run.smithi186:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-28T12:47:55.313 INFO:teuthology.orchestra.run.smithi186.stdout:Verifying... ######################################## 2023-11-28T12:47:55.367 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing... ######################################## 2023-11-28T12:47:55.453 INFO:teuthology.orchestra.run.smithi186.stdout:Updating / installing... 2023-11-28T12:48:01.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28 12:48:01 URL:https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [383178732/383178732] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-28T12:48:01.129 INFO:teuthology.orchestra.run.smithi156.stderr:FINISHED --2023-11-28 12:48:01-- 2023-11-28T12:48:01.129 INFO:teuthology.orchestra.run.smithi156.stderr:Total wall clock time: 26s 2023-11-28T12:48:01.129 INFO:teuthology.orchestra.run.smithi156.stderr:Downloaded: 1 files, 365M in 25s (14.3 MB/s) 2023-11-28T12:48:01.130 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-11-28T12:48:01.130 DEBUG:teuthology.orchestra.run.smithi156:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-28T12:48:02.734 INFO:teuthology.orchestra.run.smithi156.stdout:Verifying... ######################################## 2023-11-28T12:48:02.789 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing... ######################################## 2023-11-28T12:48:02.885 INFO:teuthology.orchestra.run.smithi156.stdout:Updating / installing... 2023-11-28T12:48:33.967 INFO:teuthology.orchestra.run.smithi186.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-28T12:48:38.014 INFO:teuthology.orchestra.run.smithi186.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-28T12:48:41.536 INFO:teuthology.orchestra.run.smithi156.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-28T12:48:43.078 INFO:teuthology.orchestra.run.smithi156.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-28T12:48:58.408 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi186.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-28T12:48:58.409 DEBUG:teuthology.orchestra.run.smithi186:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-28T12:48:58.473 INFO:teuthology.orchestra.run.smithi186.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-28T12:48:58.473 INFO:teuthology.orchestra.run.smithi186.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-28T12:48:58.473 INFO:teuthology.orchestra.run.smithi186.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-28T12:48:58.473 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-28T12:48:58.473 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-28T12:48:58.473 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-28T12:48:58.473 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-28T12:48:58.473 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-28T12:48:58.473 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-28T12:48:58.474 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-28T12:48:58.474 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-28T12:48:58.474 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-28T12:48:58.474 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-28T12:48:58.474 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-28T12:48:58.474 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-28T12:48:58.474 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-28T12:48:58.474 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-28T12:48:58.474 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-28T12:48:58.474 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-28T12:48:58.474 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-28T12:48:58.474 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-28T12:48:58.475 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-28T12:48:58.475 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-28T12:48:58.475 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-28T12:48:58.475 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-28T12:48:58.475 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-28T12:48:58.475 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-28T12:48:58.475 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-28T12:48:58.475 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-28T12:48:58.475 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-28T12:48:58.475 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-28T12:48:58.475 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-28T12:48:58.475 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-28T12:48:58.476 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-28T12:48:58.476 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-28T12:48:58.476 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-28T12:48:58.476 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-28T12:48:58.476 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-28T12:48:58.476 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-28T12:48:58.476 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-28T12:48:58.476 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-28T12:48:58.476 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-28T12:48:58.476 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-28T12:48:58.476 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-28T12:48:58.476 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-28T12:48:58.476 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-28T12:48:58.476 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-28T12:48:58.477 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-28T12:48:58.477 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-28T12:48:58.477 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-28T12:48:58.477 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-28T12:48:58.477 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-28T12:48:58.477 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-28T12:48:58.477 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-28T12:48:58.477 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-28T12:48:58.477 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-28T12:48:58.477 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-28T12:48:58.477 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-28T12:48:58.477 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-28T12:48:58.478 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-28T12:48:58.478 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-28T12:48:58.478 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-28T12:48:58.478 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-28T12:48:58.478 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-28T12:48:58.478 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-28T12:48:58.478 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-28T12:48:58.478 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-28T12:48:58.478 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-28T12:48:58.478 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-28T12:48:58.478 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-28T12:48:58.478 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-28T12:48:58.479 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-28T12:48:58.479 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-28T12:48:58.479 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-28T12:48:58.479 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-28T12:48:58.479 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-28T12:48:58.479 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-28T12:48:58.479 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-28T12:48:58.479 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-28T12:48:58.479 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-28T12:48:58.479 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-28T12:48:58.479 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-28T12:48:58.479 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-28T12:48:58.479 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-28T12:48:58.480 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-28T12:48:58.480 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-28T12:48:58.480 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-28T12:48:58.480 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-28T12:48:58.480 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-28T12:48:58.480 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-28T12:48:58.480 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-28T12:48:58.480 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-28T12:48:58.480 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-28T12:48:58.480 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-28T12:48:58.480 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-28T12:48:58.480 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-28T12:48:58.481 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-28T12:48:58.481 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-28T12:48:58.481 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-28T12:48:58.481 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-28T12:48:58.481 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-28T12:48:58.481 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-28T12:48:58.481 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-28T12:48:58.481 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-28T12:48:58.481 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-28T12:48:58.481 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-28T12:48:58.481 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-28T12:48:58.481 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-28T12:48:58.481 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-28T12:48:58.481 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-28T12:48:58.482 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-28T12:48:58.482 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-28T12:48:58.482 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-28T12:48:58.482 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-28T12:48:58.482 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-28T12:48:58.482 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-28T12:48:58.482 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-28T12:48:58.482 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-28T12:48:58.482 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-28T12:48:58.482 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-28T12:48:58.482 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-28T12:48:58.482 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-28T12:48:58.483 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-28T12:48:58.483 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-28T12:48:58.483 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-28T12:48:58.483 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-28T12:48:58.483 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-28T12:48:58.483 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-28T12:48:58.483 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-28T12:48:58.483 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-28T12:48:58.483 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-28T12:48:58.483 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-28T12:48:58.483 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-28T12:48:58.483 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-28T12:48:58.484 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-28T12:48:58.484 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-28T12:48:58.484 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-28T12:48:58.484 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-28T12:48:58.484 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-28T12:48:58.484 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-28T12:48:58.484 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-28T12:48:58.484 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-28T12:48:58.484 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-28T12:48:58.484 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-28T12:48:58.484 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-28T12:48:58.484 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-28T12:48:58.484 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-28T12:48:58.485 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-28T12:48:58.485 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-28T12:48:58.485 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-28T12:48:58.485 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-28T12:48:58.485 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-28T12:48:58.485 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-28T12:48:58.485 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-28T12:48:58.485 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-28T12:48:58.485 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-28T12:48:58.485 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-28T12:48:58.485 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-28T12:48:58.485 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-28T12:48:58.485 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-28T12:48:58.486 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-28T12:48:58.486 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-28T12:48:58.486 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-28T12:48:58.486 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-28T12:48:58.486 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-28T12:48:58.486 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-28T12:48:58.486 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-28T12:48:58.486 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-28T12:48:58.486 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-28T12:48:58.486 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-28T12:48:58.486 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-28T12:48:58.486 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-28T12:48:58.487 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-28T12:48:58.487 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-28T12:48:58.487 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-28T12:48:58.487 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-28T12:48:58.487 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-28T12:48:58.487 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-28T12:48:58.487 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-28T12:48:58.487 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-28T12:48:58.487 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-28T12:48:58.487 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-28T12:48:58.487 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-28T12:48:58.487 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-28T12:48:58.487 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-28T12:48:58.488 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-28T12:48:58.488 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-28T12:48:58.488 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-28T12:48:58.488 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-28T12:48:58.488 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-28T12:48:58.488 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-28T12:48:58.488 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-28T12:48:58.488 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-28T12:48:58.488 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-28T12:48:58.488 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-28T12:48:58.488 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-28T12:48:58.488 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-28T12:48:58.488 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-28T12:48:58.489 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-28T12:48:58.489 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-28T12:48:58.489 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-28T12:48:58.489 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-28T12:48:58.489 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-28T12:48:58.489 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-28T12:48:58.489 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-28T12:48:58.489 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-28T12:48:58.489 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-28T12:48:58.489 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-28T12:48:58.489 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-28T12:48:58.489 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-28T12:48:58.489 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-28T12:48:58.490 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-28T12:48:58.490 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-28T12:48:58.490 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-28T12:48:58.490 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-28T12:48:58.490 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-28T12:48:58.490 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-28T12:48:58.490 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-28T12:48:58.490 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-28T12:48:58.490 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-28T12:48:58.490 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-28T12:48:58.490 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-28T12:48:58.490 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-28T12:48:58.491 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-28T12:48:58.491 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-28T12:48:58.491 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-28T12:48:58.491 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-28T12:48:58.491 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-28T12:48:58.491 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-28T12:48:58.491 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-28T12:48:58.491 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-28T12:48:58.491 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-28T12:48:58.491 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-28T12:48:58.491 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-28T12:48:58.491 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-28T12:48:58.491 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-28T12:48:58.492 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-28T12:48:58.492 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-28T12:48:58.492 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-28T12:48:58.492 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-28T12:48:58.492 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-28T12:48:58.492 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-28T12:48:58.492 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-28T12:48:58.492 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-28T12:48:58.492 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-28T12:48:58.492 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-28T12:48:58.492 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-28T12:48:58.492 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-28T12:48:58.493 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-28T12:48:58.493 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-28T12:48:58.493 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-28T12:48:58.493 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-28T12:48:58.493 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-28T12:48:58.493 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-28T12:48:58.493 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-28T12:48:58.493 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-28T12:48:58.493 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-28T12:48:58.493 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-28T12:48:58.493 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-28T12:48:58.493 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-28T12:48:58.493 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-28T12:48:58.494 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-28T12:48:58.494 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-28T12:48:58.494 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-28T12:48:58.494 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-28T12:48:58.494 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-28T12:48:58.494 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-28T12:48:58.494 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-28T12:48:58.494 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-28T12:48:58.494 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-28T12:48:58.494 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-28T12:48:58.494 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-28T12:48:58.494 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-28T12:48:58.495 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-28T12:48:58.495 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-28T12:48:58.495 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-28T12:48:58.495 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-28T12:48:58.495 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-28T12:48:58.495 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-28T12:48:58.495 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-28T12:48:58.495 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-28T12:48:58.495 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-28T12:48:58.495 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-28T12:48:58.495 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-28T12:48:58.495 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-28T12:48:58.496 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-28T12:48:58.496 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-28T12:48:58.496 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-28T12:48:58.496 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-28T12:48:58.496 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-28T12:48:58.496 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-28T12:48:58.496 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-28T12:48:58.496 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-28T12:48:58.496 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-28T12:48:58.496 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-28T12:48:58.496 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-28T12:48:58.496 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-28T12:48:58.496 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-28T12:48:58.496 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-28T12:48:58.497 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-28T12:48:58.497 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-28T12:48:58.497 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-28T12:48:58.497 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-28T12:48:58.497 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-28T12:48:58.497 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-28T12:48:58.497 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-28T12:48:58.497 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-28T12:48:58.497 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-28T12:48:58.497 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-28T12:48:58.497 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-28T12:48:58.497 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-28T12:48:58.498 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-28T12:48:58.498 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-28T12:48:58.498 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-28T12:48:58.498 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-28T12:48:58.498 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-28T12:48:58.498 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-28T12:48:58.498 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-28T12:48:58.498 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-28T12:48:58.498 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-28T12:48:58.498 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-28T12:48:58.498 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-28T12:48:58.498 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-28T12:48:58.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-28T12:48:58.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-28T12:48:58.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-28T12:48:58.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-28T12:48:58.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-28T12:48:58.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-28T12:48:58.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-28T12:48:58.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-28T12:48:58.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-28T12:48:58.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-28T12:48:58.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-28T12:48:58.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-28T12:48:58.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-28T12:48:58.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-28T12:48:58.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-28T12:48:58.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-28T12:48:58.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-28T12:48:58.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-28T12:48:58.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-28T12:48:58.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-28T12:48:58.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-28T12:48:58.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-28T12:48:58.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-28T12:48:58.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-28T12:48:58.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-28T12:48:58.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-28T12:48:58.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-28T12:48:58.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-28T12:48:58.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-28T12:48:58.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-28T12:48:58.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-28T12:48:58.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-28T12:48:58.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-28T12:48:58.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-28T12:48:58.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-28T12:48:58.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-28T12:48:58.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-28T12:48:58.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-28T12:48:58.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-28T12:48:58.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-28T12:48:58.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-28T12:48:58.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-28T12:48:58.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-28T12:48:58.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-28T12:48:58.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-28T12:48:58.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-28T12:48:58.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-28T12:48:58.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-28T12:48:58.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-28T12:48:58.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-28T12:48:58.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-28T12:48:58.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-28T12:48:58.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-28T12:48:58.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-28T12:48:58.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-28T12:48:58.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-28T12:48:58.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-28T12:48:58.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-28T12:48:58.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-28T12:48:58.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-28T12:48:58.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-28T12:48:58.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-28T12:48:58.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-28T12:48:58.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-28T12:48:58.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-28T12:48:58.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-28T12:48:58.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-28T12:48:58.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-28T12:48:58.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-28T12:48:58.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-28T12:48:58.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-28T12:48:58.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-28T12:48:58.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-28T12:48:58.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-28T12:48:58.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-28T12:48:58.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-28T12:48:58.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-28T12:48:58.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-28T12:48:58.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-28T12:48:58.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-28T12:48:58.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-28T12:48:58.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-28T12:48:58.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-28T12:48:58.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-28T12:48:58.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-28T12:48:58.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-28T12:48:58.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-28T12:48:58.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-28T12:48:58.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-28T12:48:58.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-28T12:48:58.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-28T12:48:58.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-28T12:48:58.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-28T12:48:58.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-28T12:48:58.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-28T12:48:58.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-28T12:48:58.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-28T12:48:58.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-28T12:48:58.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-28T12:48:58.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-28T12:48:58.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-28T12:48:58.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-28T12:48:58.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-28T12:48:58.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-28T12:48:58.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-28T12:48:58.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-28T12:48:58.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-28T12:48:58.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-28T12:48:58.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-28T12:48:58.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-28T12:48:58.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-28T12:48:58.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-28T12:48:58.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-28T12:48:58.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-28T12:48:58.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-28T12:48:58.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-28T12:48:58.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-28T12:48:58.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-28T12:48:58.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-28T12:48:58.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-28T12:48:58.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-28T12:48:58.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-28T12:48:58.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-28T12:48:58.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-28T12:48:58.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-28T12:48:58.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-28T12:48:58.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-28T12:48:58.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-28T12:48:58.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-28T12:48:58.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-28T12:48:58.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-28T12:48:58.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-28T12:48:58.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-28T12:48:58.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-28T12:48:58.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-28T12:48:58.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-28T12:48:58.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-28T12:48:58.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-28T12:48:58.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-28T12:48:58.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-28T12:48:58.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-28T12:48:58.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-28T12:48:58.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-28T12:48:58.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-28T12:48:58.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-28T12:48:58.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-28T12:48:58.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-28T12:48:58.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-28T12:48:58.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-28T12:48:58.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-28T12:48:58.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-28T12:48:58.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-28T12:48:58.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-28T12:48:58.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-28T12:48:58.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-28T12:48:58.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-28T12:48:58.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-28T12:48:58.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-28T12:48:58.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-28T12:48:58.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-28T12:48:58.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-28T12:48:58.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-28T12:48:58.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-28T12:48:58.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-28T12:48:58.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-28T12:48:58.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-28T12:48:58.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-28T12:48:58.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-28T12:48:58.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-28T12:48:58.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-28T12:48:58.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-28T12:48:58.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-28T12:48:58.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-28T12:48:58.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-28T12:48:58.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-28T12:48:58.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-28T12:48:58.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-28T12:48:58.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-28T12:48:58.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-28T12:48:58.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-28T12:48:58.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-28T12:48:58.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-28T12:48:58.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-28T12:48:58.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-28T12:48:58.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-28T12:48:58.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-28T12:48:58.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-28T12:48:58.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-28T12:48:58.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-28T12:48:58.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-28T12:48:58.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-28T12:48:58.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-28T12:48:58.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-28T12:48:58.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-28T12:48:58.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-28T12:48:58.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-28T12:48:58.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-28T12:48:58.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-28T12:48:58.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-28T12:48:58.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-28T12:48:58.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-28T12:48:58.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-28T12:48:58.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-28T12:48:58.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-28T12:48:58.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-28T12:48:58.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-28T12:48:58.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-28T12:48:58.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-28T12:48:58.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-28T12:48:58.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-28T12:48:58.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-28T12:48:58.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-28T12:48:58.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-28T12:48:58.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-28T12:48:58.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-28T12:48:58.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-28T12:48:58.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-28T12:48:58.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-28T12:48:58.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-28T12:48:58.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-28T12:48:58.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-28T12:48:58.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-28T12:48:58.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-28T12:48:58.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-28T12:48:58.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-28T12:48:58.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-28T12:48:58.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-28T12:48:58.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-28T12:48:58.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-28T12:48:58.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-28T12:48:58.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-28T12:48:58.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-28T12:48:58.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-28T12:48:58.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-28T12:48:58.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-28T12:48:58.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-28T12:48:58.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-28T12:48:58.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-28T12:48:58.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-28T12:48:58.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-28T12:48:58.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-28T12:48:58.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-28T12:48:58.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-28T12:48:58.518 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-28T12:48:58.518 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-28T12:48:58.518 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-28T12:48:58.518 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-28T12:48:58.518 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-28T12:48:58.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-28T12:48:58.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-28T12:48:58.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-28T12:48:58.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-28T12:48:58.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-28T12:48:58.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-28T12:48:58.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-28T12:48:58.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-28T12:48:58.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-28T12:48:58.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-28T12:48:58.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-28T12:48:58.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-28T12:48:58.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-28T12:48:58.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-28T12:48:58.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-28T12:48:58.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-28T12:48:58.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-28T12:48:58.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-28T12:48:58.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-28T12:48:58.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-28T12:48:58.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-28T12:48:58.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-28T12:48:58.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-28T12:48:58.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-28T12:48:58.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-28T12:48:58.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-28T12:48:58.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-28T12:48:58.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-28T12:48:58.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-28T12:48:58.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-28T12:48:58.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-28T12:48:58.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-28T12:48:58.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-28T12:48:58.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-28T12:48:58.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-28T12:48:58.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-28T12:48:58.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-28T12:48:58.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-28T12:48:58.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-28T12:48:58.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-28T12:48:58.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-28T12:48:58.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-28T12:48:58.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-28T12:48:58.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-28T12:48:58.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-28T12:48:58.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-28T12:48:58.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-28T12:48:58.524 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-28T12:48:58.524 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-28T12:48:58.524 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-28T12:48:58.524 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-28T12:48:58.524 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-28T12:48:58.524 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-28T12:48:58.524 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-28T12:48:58.524 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-28T12:48:58.524 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-28T12:48:58.524 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-28T12:48:58.524 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-28T12:48:58.524 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-28T12:48:58.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-28T12:48:58.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-28T12:48:58.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-28T12:48:58.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-28T12:48:58.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-28T12:48:58.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-28T12:48:58.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-28T12:48:58.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-28T12:48:58.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-28T12:48:58.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-28T12:48:58.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-28T12:48:58.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-28T12:48:58.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-28T12:48:58.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-28T12:48:58.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-28T12:48:58.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-28T12:48:58.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-28T12:48:58.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-28T12:48:58.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-28T12:48:58.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-28T12:48:58.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-28T12:48:58.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-28T12:48:58.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-28T12:48:58.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-28T12:48:58.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-28T12:48:58.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-28T12:48:58.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-28T12:48:58.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-28T12:48:58.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-28T12:48:58.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-28T12:48:58.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-28T12:48:58.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-28T12:48:58.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-28T12:48:58.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-28T12:48:58.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-28T12:48:58.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-28T12:48:58.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-28T12:48:58.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-28T12:48:58.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-28T12:48:58.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-28T12:48:58.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-28T12:48:58.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-28T12:48:58.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-28T12:48:58.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-28T12:48:58.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-28T12:48:58.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-28T12:48:58.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-28T12:48:58.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-28T12:48:58.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-28T12:48:58.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-28T12:48:58.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-28T12:48:58.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-28T12:48:58.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-28T12:48:58.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-28T12:48:58.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-28T12:48:58.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-28T12:48:58.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-28T12:48:58.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-28T12:48:58.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-28T12:48:58.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-28T12:48:58.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-28T12:48:58.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-28T12:48:58.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-28T12:48:58.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-28T12:48:58.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-28T12:48:58.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-28T12:48:58.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-28T12:48:58.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-28T12:48:58.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-28T12:48:58.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-28T12:48:58.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-28T12:48:58.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-28T12:48:58.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-28T12:48:58.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-28T12:48:58.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-28T12:48:58.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-28T12:48:58.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-28T12:48:58.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-28T12:48:58.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-28T12:48:58.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-28T12:48:58.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-28T12:48:58.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-28T12:48:58.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-28T12:48:58.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-28T12:48:58.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-28T12:48:58.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-28T12:48:58.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-28T12:48:58.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-28T12:48:58.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-28T12:48:58.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-28T12:48:58.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-28T12:48:58.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-28T12:48:58.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-28T12:48:58.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-28T12:48:58.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-28T12:48:58.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-28T12:48:58.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-28T12:48:58.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-28T12:48:58.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-28T12:48:58.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-28T12:48:58.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-28T12:48:58.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-28T12:48:58.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-28T12:48:58.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-28T12:48:58.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-28T12:48:58.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-28T12:48:58.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-28T12:48:58.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-28T12:48:58.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-28T12:48:58.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-28T12:48:58.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-28T12:48:58.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-28T12:48:58.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-28T12:48:58.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-28T12:48:58.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-28T12:48:58.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-28T12:48:58.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-28T12:48:58.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-28T12:48:58.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-28T12:48:58.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-28T12:48:58.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-28T12:48:58.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-28T12:48:58.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-28T12:48:58.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-28T12:48:58.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-28T12:48:58.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-28T12:48:58.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-28T12:48:58.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-28T12:48:58.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-28T12:48:58.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-28T12:48:58.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-28T12:48:58.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-28T12:48:58.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-28T12:48:58.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-28T12:48:58.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-28T12:48:58.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-28T12:48:58.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-28T12:48:58.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-28T12:48:58.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-28T12:48:58.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-28T12:48:58.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-28T12:48:58.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-28T12:48:58.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-28T12:48:58.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-28T12:48:58.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-28T12:48:58.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-28T12:48:58.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-28T12:48:58.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-28T12:48:58.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-28T12:48:58.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-28T12:48:58.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-28T12:48:58.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-28T12:48:58.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-28T12:48:58.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-28T12:48:58.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-28T12:48:58.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-28T12:48:58.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-28T12:48:58.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-28T12:48:58.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-28T12:48:58.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-28T12:48:58.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-28T12:48:58.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-28T12:48:58.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-28T12:48:58.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-28T12:48:58.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-28T12:48:58.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-28T12:48:58.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-28T12:48:58.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-28T12:48:58.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-28T12:48:58.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-28T12:48:58.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-28T12:48:58.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-28T12:48:58.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-28T12:48:58.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-28T12:48:58.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-28T12:48:58.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-28T12:48:58.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-28T12:48:58.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-28T12:48:58.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-28T12:48:58.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-28T12:48:58.541 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-28T12:48:58.541 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-28T12:48:58.541 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-28T12:48:58.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-28T12:48:58.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-28T12:48:58.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-28T12:48:58.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-28T12:48:58.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-28T12:48:58.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-28T12:48:58.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-28T12:48:58.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-28T12:48:58.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-28T12:48:58.543 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-28T12:48:58.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-28T12:48:58.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-28T12:48:58.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-28T12:48:58.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-28T12:48:58.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-28T12:48:58.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-28T12:48:58.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-28T12:48:58.545 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-28T12:48:58.545 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-28T12:48:58.545 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-28T12:48:58.545 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-28T12:48:58.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-28T12:48:58.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-28T12:48:58.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-28T12:48:58.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-28T12:48:58.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-28T12:48:58.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-28T12:48:58.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-28T12:48:58.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-28T12:48:58.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-28T12:48:58.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-28T12:48:58.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-28T12:48:58.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-28T12:48:58.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-28T12:48:58.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-28T12:48:58.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-28T12:48:58.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-28T12:48:58.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-28T12:48:58.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-28T12:48:58.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-28T12:48:58.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-28T12:48:58.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-28T12:48:58.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-28T12:48:58.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-28T12:48:58.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-28T12:48:58.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-28T12:48:58.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-28T12:48:58.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-28T12:48:58.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-28T12:48:58.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-28T12:48:58.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-28T12:48:58.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-28T12:48:58.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-28T12:48:58.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-28T12:48:58.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-28T12:48:58.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-28T12:48:58.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-28T12:48:58.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-28T12:48:58.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-28T12:48:58.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-28T12:48:58.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-28T12:48:58.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-28T12:48:58.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-28T12:48:58.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-28T12:48:58.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-28T12:48:58.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-28T12:48:58.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-28T12:48:58.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-28T12:48:58.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-28T12:48:58.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-28T12:48:58.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-28T12:48:58.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-28T12:48:58.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-28T12:48:58.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-28T12:48:58.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-28T12:48:58.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-28T12:48:58.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-28T12:48:58.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-28T12:48:58.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-28T12:48:58.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-28T12:48:58.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-28T12:48:58.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-28T12:48:58.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-28T12:48:58.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-28T12:48:58.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-28T12:48:58.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-28T12:48:58.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-28T12:48:58.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-28T12:48:58.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-28T12:48:58.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-28T12:48:58.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-28T12:48:58.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-28T12:48:58.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-28T12:48:58.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-28T12:48:58.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-28T12:48:58.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-28T12:48:58.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-28T12:48:58.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-28T12:48:58.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-28T12:48:58.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-28T12:48:58.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-28T12:48:58.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-28T12:48:58.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-28T12:48:58.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-28T12:48:58.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-28T12:48:58.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-28T12:48:58.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-28T12:48:58.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-28T12:48:58.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-28T12:48:58.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-28T12:48:58.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-28T12:48:58.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-28T12:48:58.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-28T12:48:58.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-28T12:48:58.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-28T12:48:58.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-28T12:48:58.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-28T12:48:58.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-28T12:48:58.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-28T12:48:58.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-28T12:48:58.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-28T12:48:58.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-28T12:48:58.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-28T12:48:58.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-28T12:48:58.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-28T12:48:58.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-28T12:48:58.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-28T12:48:58.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-28T12:48:58.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-28T12:48:58.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-28T12:48:58.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-28T12:48:58.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-28T12:48:58.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-28T12:48:58.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-28T12:48:58.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-28T12:48:58.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-28T12:48:58.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-28T12:48:58.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-28T12:48:58.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-28T12:48:58.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-28T12:48:58.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-28T12:48:58.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-28T12:48:58.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-28T12:48:58.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-28T12:48:58.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-28T12:48:58.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-28T12:48:58.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-28T12:48:58.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-28T12:48:58.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-28T12:48:58.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-28T12:48:58.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-28T12:48:58.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-28T12:48:58.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-28T12:48:58.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-28T12:48:58.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-28T12:48:58.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-28T12:48:58.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-28T12:48:58.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-28T12:48:58.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-28T12:48:58.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-28T12:48:58.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-28T12:48:58.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-28T12:48:58.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-28T12:48:58.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-28T12:48:58.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-28T12:48:58.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-28T12:48:58.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-28T12:48:58.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-28T12:48:58.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-28T12:48:58.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-28T12:48:58.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-28T12:48:58.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-28T12:48:58.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-28T12:48:58.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-28T12:48:58.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-28T12:48:58.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-28T12:48:58.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-28T12:48:58.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-28T12:48:58.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-28T12:48:58.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-28T12:48:58.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-28T12:48:58.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-28T12:48:58.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-28T12:48:58.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-28T12:48:58.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-28T12:48:58.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-28T12:48:58.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-28T12:48:58.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-28T12:48:58.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-28T12:48:58.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-28T12:48:58.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-28T12:48:58.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-28T12:48:58.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-28T12:48:58.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-28T12:48:58.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-28T12:48:58.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-28T12:48:58.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-28T12:48:58.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-28T12:48:58.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-28T12:48:58.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-28T12:48:58.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-28T12:48:58.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-28T12:48:58.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-28T12:48:58.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-28T12:48:58.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-28T12:48:58.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-28T12:48:58.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-28T12:48:58.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-28T12:48:58.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-28T12:48:58.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-28T12:48:58.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-28T12:48:58.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-28T12:48:58.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-28T12:48:58.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-28T12:48:58.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-28T12:48:58.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-28T12:48:58.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-28T12:48:58.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-28T12:48:58.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-28T12:48:58.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-28T12:48:58.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-28T12:48:58.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-28T12:48:58.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-28T12:48:58.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-28T12:48:58.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-28T12:48:58.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-28T12:48:58.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-28T12:48:58.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-28T12:48:58.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-28T12:48:58.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-28T12:48:58.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-28T12:48:58.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-28T12:48:58.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-28T12:48:58.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-28T12:48:58.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-28T12:48:58.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-28T12:48:58.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-28T12:48:58.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-28T12:48:58.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-28T12:48:58.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-28T12:48:58.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-28T12:48:58.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-28T12:48:58.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-28T12:48:58.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-28T12:48:58.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-28T12:48:58.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-28T12:48:58.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-28T12:48:58.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-28T12:48:58.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-28T12:48:58.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-28T12:48:58.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-28T12:48:58.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-28T12:48:58.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-28T12:48:58.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-28T12:48:58.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-28T12:48:58.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-28T12:48:58.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-28T12:48:58.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-28T12:48:58.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-28T12:48:58.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-28T12:48:58.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-28T12:48:58.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-28T12:48:58.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-28T12:48:58.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-28T12:48:58.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-28T12:48:58.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-28T12:48:58.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-28T12:48:58.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-28T12:48:58.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-28T12:48:58.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-28T12:48:58.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-28T12:48:58.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-28T12:48:58.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-28T12:48:58.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-28T12:48:58.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-28T12:48:58.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-28T12:48:58.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-28T12:48:58.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-28T12:48:58.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-28T12:48:58.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-28T12:48:58.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-28T12:48:58.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-28T12:48:58.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-28T12:48:58.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-28T12:48:58.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-28T12:48:58.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-28T12:48:58.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-28T12:48:58.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-28T12:48:58.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-28T12:48:58.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-28T12:48:58.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-28T12:48:58.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-28T12:48:58.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-28T12:48:58.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-28T12:48:58.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-28T12:48:58.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-28T12:48:58.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-28T12:48:58.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-28T12:48:58.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-28T12:48:58.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-28T12:48:58.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-28T12:48:58.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-28T12:48:58.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-28T12:48:58.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-28T12:48:58.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-28T12:48:58.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-28T12:48:58.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-28T12:48:58.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-28T12:48:58.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-28T12:48:58.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-28T12:48:58.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-28T12:48:58.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-28T12:48:58.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-28T12:48:58.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-28T12:48:58.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-28T12:48:58.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-28T12:48:58.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-28T12:48:58.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-28T12:48:58.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-28T12:48:58.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-28T12:48:58.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-28T12:48:58.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-28T12:48:58.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-28T12:48:58.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-28T12:48:58.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-28T12:48:58.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-28T12:48:58.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-28T12:48:58.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-28T12:48:58.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-28T12:48:58.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-28T12:48:58.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-28T12:48:58.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-28T12:48:58.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-28T12:48:58.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-28T12:48:58.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-28T12:48:58.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-28T12:48:58.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-28T12:48:58.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-28T12:48:58.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-28T12:48:58.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-28T12:48:58.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-28T12:48:58.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-28T12:48:58.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-28T12:48:58.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-28T12:48:58.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-28T12:48:58.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-28T12:48:58.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-28T12:48:58.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-28T12:48:58.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-28T12:48:58.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-28T12:48:58.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-28T12:48:58.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-28T12:48:58.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-28T12:48:58.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-28T12:48:58.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-28T12:48:58.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-28T12:48:58.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-28T12:48:58.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-28T12:48:58.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-28T12:48:58.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-28T12:48:58.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-28T12:48:58.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-28T12:48:58.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-28T12:48:58.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-28T12:48:58.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-28T12:48:58.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-28T12:48:58.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-28T12:48:58.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-28T12:48:58.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-28T12:48:58.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-28T12:48:58.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-28T12:48:58.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-28T12:48:58.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-28T12:48:58.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-28T12:48:58.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-28T12:48:58.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-28T12:48:58.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-28T12:48:58.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-28T12:48:58.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-28T12:48:58.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-28T12:48:58.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-28T12:48:58.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-28T12:48:58.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-28T12:48:58.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-28T12:48:58.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-28T12:48:58.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-28T12:48:58.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-28T12:48:58.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-28T12:48:58.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-28T12:48:58.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-28T12:48:58.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-28T12:48:58.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-28T12:48:58.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-28T12:48:58.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-28T12:48:58.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-28T12:48:58.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-28T12:48:58.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-28T12:48:58.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-28T12:48:58.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-28T12:48:58.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-28T12:48:58.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-28T12:48:58.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-28T12:48:58.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-28T12:48:58.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-28T12:48:58.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-28T12:48:58.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-28T12:48:58.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-28T12:48:58.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-28T12:48:58.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-28T12:48:58.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-28T12:48:58.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-28T12:48:58.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-28T12:48:58.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-28T12:48:58.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-28T12:48:58.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-28T12:48:58.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-28T12:48:58.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-28T12:48:58.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-28T12:48:58.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-28T12:48:58.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-28T12:48:58.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-28T12:48:58.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-28T12:48:58.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-28T12:48:58.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-28T12:48:58.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-28T12:48:58.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-28T12:48:58.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-28T12:48:58.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-28T12:48:58.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-28T12:48:58.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-28T12:48:58.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-28T12:48:58.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-28T12:48:58.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-28T12:48:58.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-28T12:48:58.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-28T12:48:58.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-28T12:48:58.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-28T12:48:58.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-28T12:48:58.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-28T12:48:58.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-28T12:48:58.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-28T12:48:58.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-28T12:48:58.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-28T12:48:58.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-28T12:48:58.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-28T12:48:58.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-28T12:48:58.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-28T12:48:58.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-28T12:48:58.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-28T12:48:58.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-28T12:48:58.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-28T12:48:58.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-28T12:48:58.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-28T12:48:58.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-28T12:48:58.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-28T12:48:58.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-28T12:48:58.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-28T12:48:58.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-28T12:48:58.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-28T12:48:58.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-28T12:48:58.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-28T12:48:58.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-28T12:48:58.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-28T12:48:58.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-28T12:48:58.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-28T12:48:58.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-28T12:48:58.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-28T12:48:58.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-28T12:48:58.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-28T12:48:58.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-28T12:48:58.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-28T12:48:58.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-28T12:48:58.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-28T12:48:58.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-28T12:48:58.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-28T12:48:58.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-28T12:48:58.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-28T12:48:58.586 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-28T12:48:58.586 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-28T12:48:58.586 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-28T12:48:58.586 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-28T12:48:58.586 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-28T12:48:58.587 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-28T12:48:58.587 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-28T12:48:58.587 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-28T12:48:58.587 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-28T12:48:58.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-28T12:48:58.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-28T12:48:58.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-28T12:48:58.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-28T12:48:58.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-28T12:48:58.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-28T12:48:58.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-28T12:48:58.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-28T12:48:58.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-28T12:48:58.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-28T12:48:58.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-28T12:48:58.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-28T12:48:58.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-28T12:48:58.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-28T12:48:58.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-28T12:48:58.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-28T12:48:58.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-28T12:48:58.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-28T12:48:58.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-28T12:48:58.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-28T12:48:58.590 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-28T12:48:58.590 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-28T12:48:58.590 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-28T12:48:58.590 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-28T12:48:58.590 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-28T12:48:58.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-28T12:48:58.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-28T12:48:58.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-28T12:48:58.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-28T12:48:58.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-28T12:48:58.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-28T12:48:58.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-28T12:48:58.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-28T12:48:58.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-28T12:48:58.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-28T12:48:58.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-28T12:48:58.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-28T12:48:58.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-28T12:48:58.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-28T12:48:58.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-28T12:48:58.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-28T12:48:58.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-28T12:48:58.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-28T12:48:58.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-28T12:48:58.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-28T12:48:58.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-28T12:48:58.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-28T12:48:58.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-28T12:48:58.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-28T12:48:58.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-28T12:48:58.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-28T12:48:58.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-28T12:48:58.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-28T12:48:58.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-28T12:48:58.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-28T12:48:58.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-28T12:48:58.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-28T12:48:58.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-28T12:48:58.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-28T12:48:58.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-28T12:48:58.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-28T12:48:58.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-28T12:48:58.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-28T12:48:58.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-28T12:48:58.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-28T12:48:58.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-28T12:48:58.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-28T12:48:58.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-28T12:48:58.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-28T12:48:58.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-28T12:48:58.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-28T12:48:58.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-28T12:48:58.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-28T12:48:58.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-28T12:48:58.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-28T12:48:58.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-28T12:48:58.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-28T12:48:58.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-28T12:48:58.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-28T12:48:58.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-28T12:48:58.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-28T12:48:58.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-28T12:48:58.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-28T12:48:58.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-28T12:48:58.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-28T12:48:58.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-28T12:48:58.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-28T12:48:58.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-28T12:48:58.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-28T12:48:58.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-28T12:48:58.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-28T12:48:58.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-28T12:48:58.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-28T12:48:58.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-28T12:48:58.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-28T12:48:58.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-28T12:48:58.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-28T12:48:58.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-28T12:48:58.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-28T12:48:58.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-28T12:48:58.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-28T12:48:58.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-28T12:48:58.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-28T12:48:58.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-28T12:48:58.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-28T12:48:58.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-28T12:48:58.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-28T12:48:58.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-28T12:48:58.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-28T12:48:58.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-28T12:48:58.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-28T12:48:58.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-28T12:48:58.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-28T12:48:58.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-28T12:48:58.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-28T12:48:58.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-28T12:48:58.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-28T12:48:58.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-28T12:48:58.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-28T12:48:58.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-28T12:48:58.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-28T12:48:58.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-28T12:48:58.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-28T12:48:58.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-28T12:48:58.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-28T12:48:58.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-28T12:48:58.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-28T12:48:58.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-28T12:48:58.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-28T12:48:58.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-28T12:48:58.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-28T12:48:58.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-28T12:48:58.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-28T12:48:58.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-28T12:48:58.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-28T12:48:58.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-28T12:48:58.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-28T12:48:58.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-28T12:48:58.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-28T12:48:58.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-28T12:48:58.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-28T12:48:58.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-28T12:48:58.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-28T12:48:58.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-28T12:48:58.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-28T12:48:58.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-28T12:48:58.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-28T12:48:58.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-28T12:48:58.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-28T12:48:58.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-28T12:48:58.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-28T12:48:58.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-28T12:48:58.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-28T12:48:58.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-28T12:48:58.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-28T12:48:58.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-28T12:48:58.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-28T12:48:58.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-28T12:48:58.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-28T12:48:58.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-28T12:48:58.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-28T12:48:58.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-28T12:48:58.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-28T12:48:58.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-28T12:48:58.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-28T12:48:58.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-28T12:48:58.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-28T12:48:58.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-28T12:48:58.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-28T12:48:58.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-28T12:48:58.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-28T12:48:58.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-28T12:48:58.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-28T12:48:58.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-28T12:48:58.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-28T12:48:58.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-28T12:48:58.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-28T12:48:58.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-28T12:48:58.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-28T12:48:58.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-28T12:48:58.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-28T12:48:58.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-28T12:48:58.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-28T12:48:58.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-28T12:48:58.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-28T12:48:58.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-28T12:48:58.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-28T12:48:58.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-28T12:48:58.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-28T12:48:58.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-28T12:48:58.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-28T12:48:58.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-28T12:48:58.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-28T12:48:58.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-28T12:48:58.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-28T12:48:58.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-28T12:48:58.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-28T12:48:58.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-28T12:48:58.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-28T12:48:58.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-28T12:48:58.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-28T12:48:58.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-28T12:48:58.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-28T12:48:58.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-28T12:48:58.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-28T12:48:58.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-28T12:48:58.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-28T12:48:58.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-28T12:48:58.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-28T12:48:58.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-28T12:48:58.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-28T12:48:58.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-28T12:48:58.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-28T12:48:58.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-28T12:48:58.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-28T12:48:58.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-28T12:48:58.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-28T12:48:58.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-28T12:48:58.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-28T12:48:58.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-28T12:48:58.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-28T12:48:58.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-28T12:48:58.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-28T12:48:58.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-28T12:48:58.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-28T12:48:58.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-28T12:48:58.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-28T12:48:58.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-28T12:48:58.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-28T12:48:58.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-28T12:48:58.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-28T12:48:58.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-28T12:48:58.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-28T12:48:58.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-28T12:48:58.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-28T12:48:58.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-28T12:48:58.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-28T12:48:58.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-28T12:48:58.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-28T12:48:58.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-28T12:48:58.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-28T12:48:58.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-28T12:48:58.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-28T12:48:58.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-28T12:48:58.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-28T12:48:58.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-28T12:48:58.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-28T12:48:58.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-28T12:48:58.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-28T12:48:58.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-28T12:48:58.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-28T12:48:58.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-28T12:48:58.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-28T12:48:58.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-28T12:48:58.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-28T12:48:58.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-28T12:48:58.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-28T12:48:58.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-28T12:48:58.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-28T12:48:58.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-28T12:48:58.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-28T12:48:58.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-28T12:48:58.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-28T12:48:58.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-28T12:48:58.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-28T12:48:58.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-28T12:48:58.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-28T12:48:58.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-28T12:48:58.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-28T12:48:58.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-28T12:48:58.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-28T12:48:58.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-28T12:48:58.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-28T12:48:58.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-28T12:48:58.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-28T12:48:58.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-28T12:48:58.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-28T12:48:58.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-28T12:48:58.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-28T12:48:58.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-28T12:48:58.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-28T12:48:58.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-28T12:48:58.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-28T12:48:58.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-28T12:48:58.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-28T12:48:58.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-28T12:48:58.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-28T12:48:58.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-28T12:48:58.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-28T12:48:58.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-28T12:48:58.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-28T12:48:58.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-28T12:48:58.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-28T12:48:58.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-28T12:48:58.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-28T12:48:58.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-28T12:48:58.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-28T12:48:58.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-28T12:48:58.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-28T12:48:58.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-28T12:48:58.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-28T12:48:58.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-28T12:48:58.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-28T12:48:58.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-28T12:48:58.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-28T12:48:58.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-28T12:48:58.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-28T12:48:58.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-28T12:48:58.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-28T12:48:58.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-28T12:48:58.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-28T12:48:58.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-28T12:48:58.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-28T12:48:58.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-28T12:48:58.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-28T12:48:58.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-28T12:48:58.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-28T12:48:58.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-28T12:48:58.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-28T12:48:58.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-28T12:48:58.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-28T12:48:58.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-28T12:48:58.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-28T12:48:58.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-28T12:48:58.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-28T12:48:58.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-28T12:48:58.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-28T12:48:58.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-28T12:48:58.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-28T12:48:58.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-28T12:48:58.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-28T12:48:58.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-28T12:48:58.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-28T12:48:58.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-28T12:48:58.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-28T12:48:58.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-28T12:48:58.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-28T12:48:58.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-28T12:48:58.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-28T12:48:58.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-28T12:48:58.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-28T12:48:58.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-28T12:48:58.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-28T12:48:58.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-28T12:48:58.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-28T12:48:58.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-28T12:48:58.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-28T12:48:58.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-28T12:48:58.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-28T12:48:58.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-28T12:48:58.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-28T12:48:58.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-28T12:48:58.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-28T12:48:58.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-28T12:48:58.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-28T12:48:58.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-28T12:48:58.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-28T12:48:58.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-28T12:48:58.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-28T12:48:58.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-28T12:48:58.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-28T12:48:58.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-28T12:48:58.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-28T12:48:58.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-28T12:48:58.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-28T12:48:58.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-28T12:48:58.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-28T12:48:58.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-28T12:48:58.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-28T12:48:58.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-28T12:48:58.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-28T12:48:58.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-28T12:48:58.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-28T12:48:58.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-28T12:48:58.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-28T12:48:58.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-28T12:48:58.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-28T12:48:58.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-28T12:48:58.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-28T12:48:58.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-28T12:48:58.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-28T12:48:58.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-28T12:48:58.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-28T12:48:58.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-28T12:48:58.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-28T12:48:58.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-28T12:48:58.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-28T12:48:58.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-28T12:48:58.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-28T12:48:58.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-28T12:48:58.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-28T12:48:58.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-28T12:48:58.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-28T12:48:58.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-28T12:48:58.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-28T12:48:58.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-28T12:48:58.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-28T12:48:58.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-28T12:48:58.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-28T12:48:58.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-28T12:48:58.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-28T12:48:58.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-28T12:48:58.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-28T12:48:58.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-28T12:48:58.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-28T12:48:58.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-28T12:48:58.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-28T12:48:58.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-28T12:48:58.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-28T12:48:58.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-28T12:48:58.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-28T12:48:58.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-28T12:48:58.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-28T12:48:58.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-28T12:48:58.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-28T12:48:58.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-28T12:48:58.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-28T12:48:58.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-28T12:48:58.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-28T12:48:58.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-28T12:48:58.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-28T12:48:58.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-28T12:48:58.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-28T12:48:58.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-28T12:48:58.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-28T12:48:58.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-28T12:48:58.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-28T12:48:58.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-28T12:48:58.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-28T12:48:58.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-28T12:48:58.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-28T12:48:58.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-28T12:48:58.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-28T12:48:58.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-28T12:48:58.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-28T12:48:58.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-28T12:48:58.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-28T12:48:58.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-28T12:48:58.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-28T12:48:58.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-28T12:48:58.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-28T12:48:58.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-28T12:48:58.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-28T12:48:58.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-28T12:48:58.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-28T12:48:58.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-28T12:48:58.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-28T12:48:58.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-28T12:48:58.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-28T12:48:58.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-28T12:48:58.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-28T12:48:58.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-28T12:48:58.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-28T12:48:58.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-28T12:48:58.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-28T12:48:58.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-28T12:48:58.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-28T12:48:58.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-28T12:48:58.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-28T12:48:58.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-28T12:48:58.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-28T12:48:58.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-28T12:48:58.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-28T12:48:58.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-28T12:48:58.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-28T12:48:58.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-28T12:48:58.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-28T12:48:58.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-28T12:48:58.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-28T12:48:58.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-28T12:48:58.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-28T12:48:58.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-28T12:48:58.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-28T12:48:58.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-28T12:48:58.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-28T12:48:58.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-28T12:48:58.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-28T12:48:58.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-28T12:48:58.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-28T12:48:58.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-28T12:48:58.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-28T12:48:58.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-28T12:48:58.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-28T12:48:58.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-28T12:48:58.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-28T12:48:58.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-28T12:48:58.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-28T12:48:58.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-28T12:48:58.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-28T12:48:58.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-28T12:48:58.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-28T12:48:58.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-28T12:48:58.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-28T12:48:58.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-28T12:48:58.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-28T12:48:58.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-28T12:48:58.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-28T12:48:58.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-28T12:48:58.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-28T12:48:58.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-28T12:48:58.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-28T12:48:58.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-28T12:48:58.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-28T12:48:58.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-28T12:48:58.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-28T12:48:58.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-28T12:48:58.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-28T12:48:58.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-28T12:48:58.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-28T12:48:58.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-28T12:48:58.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-28T12:48:58.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-28T12:48:58.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-28T12:48:58.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-28T12:48:58.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-28T12:48:58.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-28T12:48:58.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-28T12:48:58.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-28T12:48:58.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-28T12:48:58.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-28T12:48:58.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-28T12:48:58.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-28T12:48:58.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-28T12:48:58.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-28T12:48:58.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-28T12:48:58.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-28T12:48:58.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-28T12:48:58.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-28T12:48:58.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-28T12:48:58.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-28T12:48:58.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-28T12:48:58.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-28T12:48:58.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-28T12:48:58.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-28T12:48:58.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-28T12:48:58.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-28T12:48:58.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-28T12:48:58.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-28T12:48:58.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-28T12:48:58.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-28T12:48:58.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-28T12:48:58.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-28T12:48:58.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-28T12:48:58.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-28T12:48:58.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-28T12:48:58.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-28T12:48:58.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-28T12:48:58.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-28T12:48:58.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-28T12:48:58.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-28T12:48:58.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-28T12:48:58.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-28T12:48:58.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-28T12:48:58.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-28T12:48:58.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-28T12:48:58.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-28T12:48:58.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-28T12:48:58.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-28T12:48:58.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-28T12:48:58.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-28T12:48:58.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-28T12:48:58.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-28T12:48:58.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-28T12:48:58.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-28T12:48:58.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-28T12:48:58.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-28T12:48:58.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-28T12:48:58.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-28T12:48:58.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-28T12:48:58.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-28T12:48:58.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-28T12:48:58.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-28T12:48:58.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-28T12:48:58.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-28T12:48:58.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-28T12:48:58.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-28T12:48:58.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-28T12:48:58.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-28T12:48:58.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-28T12:48:58.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-28T12:48:58.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-28T12:48:58.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-28T12:48:58.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-28T12:48:58.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-28T12:48:58.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-28T12:48:58.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-28T12:48:58.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-28T12:48:58.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-28T12:48:58.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-28T12:48:58.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-28T12:48:58.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-28T12:48:58.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-28T12:48:58.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-28T12:48:58.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-28T12:48:58.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-28T12:48:58.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-28T12:48:58.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-28T12:48:58.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-28T12:48:58.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-28T12:48:58.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-28T12:48:58.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-28T12:48:58.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-28T12:48:58.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-28T12:48:58.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-28T12:48:58.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-28T12:48:58.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-28T12:48:58.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-28T12:48:58.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-28T12:48:58.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-28T12:48:58.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-28T12:48:58.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-28T12:48:58.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-28T12:48:58.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-28T12:48:58.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-28T12:48:58.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-28T12:48:58.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-28T12:48:58.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-28T12:48:58.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-28T12:48:58.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-28T12:48:58.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-28T12:48:58.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-28T12:48:58.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-28T12:48:58.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-28T12:48:58.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-28T12:48:58.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-28T12:48:58.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-28T12:48:58.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-28T12:48:58.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-28T12:48:58.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-28T12:48:58.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-28T12:48:58.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-28T12:48:58.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-28T12:48:58.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-28T12:48:58.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-28T12:48:58.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-28T12:48:58.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-28T12:48:58.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-28T12:48:58.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-28T12:48:58.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-28T12:48:58.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-28T12:48:58.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-28T12:48:58.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-28T12:48:58.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-28T12:48:58.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-28T12:48:58.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-28T12:48:58.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-28T12:48:58.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-28T12:48:58.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-28T12:48:58.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-28T12:48:58.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-28T12:48:58.649 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-28T12:48:58.649 DEBUG:teuthology.orchestra.run.smithi186:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-28T12:48:58.676 INFO:teuthology.orchestra.run.smithi186.stdout:postinstall scriptlet (using /bin/sh): 2023-11-28T12:48:58.676 INFO:teuthology.orchestra.run.smithi186.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-28T12:48:58.676 INFO:teuthology.orchestra.run.smithi186.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-28T12:48:58.676 INFO:teuthology.orchestra.run.smithi186.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T12:48:58.676 INFO:teuthology.orchestra.run.smithi186.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-28T12:48:58.676 INFO:teuthology.orchestra.run.smithi186.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T12:48:58.677 INFO:teuthology.orchestra.run.smithi186.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T12:48:58.677 INFO:teuthology.orchestra.run.smithi186.stdout:fi 2023-11-28T12:48:58.677 INFO:teuthology.orchestra.run.smithi186.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-28T12:48:58.677 INFO:teuthology.orchestra.run.smithi186.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-28T12:48:58.677 INFO:teuthology.orchestra.run.smithi186.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-28T12:48:58.677 INFO:teuthology.orchestra.run.smithi186.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-28T12:48:58.678 INFO:teuthology.orchestra.run.smithi186.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-28T12:48:58.678 INFO:teuthology.orchestra.run.smithi186.stdout:fi 2023-11-28T12:48:58.678 INFO:teuthology.orchestra.run.smithi186.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-28T12:48:58.678 INFO:teuthology.orchestra.run.smithi186.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-28T12:48:58.678 INFO:teuthology.orchestra.run.smithi186.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-28T12:48:58.678 INFO:teuthology.orchestra.run.smithi186.stdout:fi 2023-11-28T12:48:58.679 DEBUG:teuthology.orchestra.run.smithi186:> sudo rpm -qi grub2-tools 2023-11-28T12:48:58.752 INFO:teuthology.orchestra.run.smithi186.stdout:Name : grub2-tools 2023-11-28T12:48:58.752 INFO:teuthology.orchestra.run.smithi186.stdout:Epoch : 1 2023-11-28T12:48:58.752 INFO:teuthology.orchestra.run.smithi186.stdout:Version : 2.02 2023-11-28T12:48:58.753 INFO:teuthology.orchestra.run.smithi186.stdout:Release : 123.el8 2023-11-28T12:48:58.753 INFO:teuthology.orchestra.run.smithi186.stdout:Architecture: x86_64 2023-11-28T12:48:58.753 INFO:teuthology.orchestra.run.smithi186.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-11-28T12:48:58.753 INFO:teuthology.orchestra.run.smithi186.stdout:Group : System Environment/Base 2023-11-28T12:48:58.753 INFO:teuthology.orchestra.run.smithi186.stdout:Size : 9265450 2023-11-28T12:48:58.753 INFO:teuthology.orchestra.run.smithi186.stdout:License : GPLv3+ 2023-11-28T12:48:58.753 INFO:teuthology.orchestra.run.smithi186.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-11-28T12:48:58.753 INFO:teuthology.orchestra.run.smithi186.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-11-28T12:48:58.753 INFO:teuthology.orchestra.run.smithi186.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-11-28T12:48:58.754 INFO:teuthology.orchestra.run.smithi186.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-11-28T12:48:58.754 INFO:teuthology.orchestra.run.smithi186.stdout:Relocations : (not relocatable) 2023-11-28T12:48:58.754 INFO:teuthology.orchestra.run.smithi186.stdout:Packager : Red Hat, Inc. 2023-11-28T12:48:58.754 INFO:teuthology.orchestra.run.smithi186.stdout:Vendor : Red Hat, Inc. 2023-11-28T12:48:58.754 INFO:teuthology.orchestra.run.smithi186.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-28T12:48:58.754 INFO:teuthology.orchestra.run.smithi186.stdout:Summary : Support tools for GRUB. 2023-11-28T12:48:58.754 INFO:teuthology.orchestra.run.smithi186.stdout:Description : 2023-11-28T12:48:58.754 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T12:48:58.754 INFO:teuthology.orchestra.run.smithi186.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-28T12:48:58.754 INFO:teuthology.orchestra.run.smithi186.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-28T12:48:58.754 INFO:teuthology.orchestra.run.smithi186.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-28T12:48:58.755 INFO:teuthology.orchestra.run.smithi186.stdout:hardware devices. 2023-11-28T12:48:58.755 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T12:48:58.755 INFO:teuthology.orchestra.run.smithi186.stdout:This subpackage provides tools for support of all platforms. 2023-11-28T12:48:58.756 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-28T12:48:58.756 INFO:teuthology.task.kernel:Updating grub on smithi186 to boot 6.6.0-g8247bbb5b470 2023-11-28T12:48:58.756 DEBUG:teuthology.orchestra.run.smithi186:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-28T12:49:02.885 INFO:teuthology.orchestra.run.smithi186.stderr:Generating grub configuration file ... 2023-11-28T12:49:05.345 INFO:teuthology.orchestra.run.smithi186.stderr:done 2023-11-28T12:49:05.347 DEBUG:teuthology.orchestra.run.smithi186:> mktemp 2023-11-28T12:49:05.363 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.TaOOxwCVqs 2023-11-28T12:49:05.363 DEBUG:teuthology.orchestra.run.smithi186:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.TaOOxwCVqs 2023-11-28T12:49:05.437 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.TaOOxwCVqs 2023-11-28T12:49:05.586 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.TaOOxwCVqs is 6KB 2023-11-28T12:49:05.635 DEBUG:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.TaOOxwCVqs 2023-11-28T12:49:05.651 DEBUG:teuthology.orchestra.run.smithi186:> sudo /bin/ls /boot/loader/entries || true 2023-11-28T12:49:05.726 INFO:teuthology.orchestra.run.smithi186.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-11-28T12:49:05.727 INFO:teuthology.orchestra.run.smithi186.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-11-28T12:49:05.727 INFO:teuthology.orchestra.run.smithi186.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.6.0-g8247bbb5b470.conf 2023-11-28T12:49:05.728 DEBUG:teuthology.orchestra.run.smithi186:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.6.0-g8247bbb5b470 2023-11-28T12:49:05.894 DEBUG:teuthology.orchestra.run.smithi186:> sudo shutdown -r now 2023-11-28T12:49:08.654 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi156.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-28T12:49:08.654 DEBUG:teuthology.orchestra.run.smithi156:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-28T12:49:08.712 INFO:teuthology.orchestra.run.smithi156.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-28T12:49:08.712 INFO:teuthology.orchestra.run.smithi156.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-28T12:49:08.712 INFO:teuthology.orchestra.run.smithi156.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-28T12:49:08.712 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-28T12:49:08.712 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-28T12:49:08.712 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-28T12:49:08.712 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-28T12:49:08.712 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-28T12:49:08.712 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-28T12:49:08.712 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-28T12:49:08.713 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-28T12:49:08.713 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-28T12:49:08.713 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-28T12:49:08.713 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-28T12:49:08.713 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-28T12:49:08.713 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-28T12:49:08.713 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-28T12:49:08.713 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-28T12:49:08.713 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-28T12:49:08.713 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-28T12:49:08.713 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-28T12:49:08.713 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-28T12:49:08.713 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-28T12:49:08.714 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-28T12:49:08.714 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-28T12:49:08.714 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-28T12:49:08.714 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-28T12:49:08.714 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-28T12:49:08.714 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-28T12:49:08.714 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-28T12:49:08.714 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-28T12:49:08.714 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-28T12:49:08.714 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-28T12:49:08.714 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-28T12:49:08.714 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-28T12:49:08.715 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-28T12:49:08.715 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-28T12:49:08.715 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-28T12:49:08.715 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-28T12:49:08.715 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-28T12:49:08.715 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-28T12:49:08.715 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-28T12:49:08.715 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-28T12:49:08.715 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-28T12:49:08.715 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-28T12:49:08.715 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-28T12:49:08.715 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-28T12:49:08.715 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-28T12:49:08.716 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-28T12:49:08.716 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-28T12:49:08.716 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-28T12:49:08.716 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-28T12:49:08.716 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-28T12:49:08.716 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-28T12:49:08.716 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-28T12:49:08.716 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-28T12:49:08.716 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-28T12:49:08.716 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-28T12:49:08.716 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-28T12:49:08.716 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-28T12:49:08.717 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-28T12:49:08.717 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-28T12:49:08.717 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-28T12:49:08.717 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-28T12:49:08.717 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-28T12:49:08.717 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-28T12:49:08.717 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-28T12:49:08.717 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-28T12:49:08.717 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-28T12:49:08.717 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-28T12:49:08.717 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-28T12:49:08.717 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-28T12:49:08.717 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-28T12:49:08.718 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-28T12:49:08.718 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-28T12:49:08.718 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-28T12:49:08.718 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-28T12:49:08.718 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-28T12:49:08.718 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-28T12:49:08.718 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-28T12:49:08.718 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-28T12:49:08.718 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-28T12:49:08.718 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-28T12:49:08.718 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-28T12:49:08.719 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-28T12:49:08.719 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-28T12:49:08.719 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-28T12:49:08.719 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-28T12:49:08.719 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-28T12:49:08.719 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-28T12:49:08.719 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-28T12:49:08.719 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-28T12:49:08.719 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-28T12:49:08.719 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-28T12:49:08.719 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-28T12:49:08.719 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-28T12:49:08.719 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-28T12:49:08.720 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-28T12:49:08.720 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-28T12:49:08.720 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-28T12:49:08.720 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-28T12:49:08.720 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-28T12:49:08.720 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-28T12:49:08.720 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-28T12:49:08.720 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-28T12:49:08.720 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-28T12:49:08.720 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-28T12:49:08.720 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-28T12:49:08.720 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-28T12:49:08.720 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-28T12:49:08.721 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-28T12:49:08.721 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-28T12:49:08.721 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-28T12:49:08.721 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-28T12:49:08.721 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-28T12:49:08.721 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-28T12:49:08.721 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-28T12:49:08.721 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-28T12:49:08.721 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-28T12:49:08.721 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-28T12:49:08.721 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-28T12:49:08.721 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-28T12:49:08.722 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-28T12:49:08.722 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-28T12:49:08.722 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-28T12:49:08.722 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-28T12:49:08.722 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-28T12:49:08.722 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-28T12:49:08.722 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-28T12:49:08.722 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-28T12:49:08.722 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-28T12:49:08.722 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-28T12:49:08.722 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-28T12:49:08.723 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-28T12:49:08.723 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-28T12:49:08.723 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-28T12:49:08.723 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-28T12:49:08.723 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-28T12:49:08.723 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-28T12:49:08.723 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-28T12:49:08.723 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-28T12:49:08.723 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-28T12:49:08.723 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-28T12:49:08.723 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-28T12:49:08.723 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-28T12:49:08.723 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-28T12:49:08.724 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-28T12:49:08.724 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-28T12:49:08.724 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-28T12:49:08.724 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-28T12:49:08.724 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-28T12:49:08.724 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-28T12:49:08.724 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-28T12:49:08.724 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-28T12:49:08.724 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-28T12:49:08.724 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-28T12:49:08.724 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-28T12:49:08.724 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-28T12:49:08.725 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-28T12:49:08.725 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-28T12:49:08.725 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-28T12:49:08.725 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-28T12:49:08.725 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-28T12:49:08.725 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-28T12:49:08.725 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-28T12:49:08.725 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-28T12:49:08.725 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-28T12:49:08.725 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-28T12:49:08.725 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-28T12:49:08.725 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-28T12:49:08.726 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-28T12:49:08.726 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-28T12:49:08.726 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-28T12:49:08.726 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-28T12:49:08.726 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-28T12:49:08.726 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-28T12:49:08.726 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-28T12:49:08.726 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-28T12:49:08.726 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-28T12:49:08.726 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-28T12:49:08.726 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-28T12:49:08.726 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-28T12:49:08.727 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-28T12:49:08.727 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-28T12:49:08.727 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-28T12:49:08.727 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-28T12:49:08.727 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-28T12:49:08.727 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-28T12:49:08.727 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-28T12:49:08.727 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-28T12:49:08.727 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-28T12:49:08.727 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-28T12:49:08.727 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-28T12:49:08.727 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-28T12:49:08.727 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-28T12:49:08.728 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-28T12:49:08.728 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-28T12:49:08.728 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-28T12:49:08.728 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-28T12:49:08.728 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-28T12:49:08.728 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-28T12:49:08.728 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-28T12:49:08.728 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-28T12:49:08.728 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-28T12:49:08.728 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-28T12:49:08.728 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-28T12:49:08.728 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-28T12:49:08.729 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-28T12:49:08.729 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-28T12:49:08.729 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-28T12:49:08.729 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-28T12:49:08.729 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-28T12:49:08.729 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-28T12:49:08.729 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-28T12:49:08.729 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-28T12:49:08.729 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-28T12:49:08.729 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-28T12:49:08.729 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-28T12:49:08.729 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-28T12:49:08.730 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-28T12:49:08.730 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-28T12:49:08.730 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-28T12:49:08.730 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-28T12:49:08.730 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-28T12:49:08.730 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-28T12:49:08.730 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-28T12:49:08.730 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-28T12:49:08.730 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-28T12:49:08.730 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-28T12:49:08.730 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-28T12:49:08.730 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-28T12:49:08.731 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-28T12:49:08.731 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-28T12:49:08.731 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-28T12:49:08.731 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-28T12:49:08.731 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-28T12:49:08.731 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-28T12:49:08.731 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-28T12:49:08.731 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-28T12:49:08.731 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-28T12:49:08.731 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-28T12:49:08.731 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-28T12:49:08.731 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-28T12:49:08.731 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-28T12:49:08.732 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-28T12:49:08.732 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-28T12:49:08.732 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-28T12:49:08.732 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-28T12:49:08.732 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-28T12:49:08.732 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-28T12:49:08.732 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-28T12:49:08.732 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-28T12:49:08.732 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-28T12:49:08.732 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-28T12:49:08.732 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-28T12:49:08.733 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-28T12:49:08.733 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-28T12:49:08.733 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-28T12:49:08.733 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-28T12:49:08.733 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-28T12:49:08.733 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-28T12:49:08.733 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-28T12:49:08.733 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-28T12:49:08.733 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-28T12:49:08.733 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-28T12:49:08.733 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-28T12:49:08.733 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-28T12:49:08.733 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-28T12:49:08.734 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-28T12:49:08.734 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-28T12:49:08.734 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-28T12:49:08.734 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-28T12:49:08.734 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-28T12:49:08.734 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-28T12:49:08.734 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-28T12:49:08.734 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-28T12:49:08.734 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-28T12:49:08.734 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-28T12:49:08.734 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-28T12:49:08.734 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-28T12:49:08.735 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-28T12:49:08.735 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-28T12:49:08.735 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-28T12:49:08.735 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-28T12:49:08.735 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-28T12:49:08.735 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-28T12:49:08.735 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-28T12:49:08.735 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-28T12:49:08.735 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-28T12:49:08.735 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-28T12:49:08.735 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-28T12:49:08.735 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-28T12:49:08.736 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-28T12:49:08.736 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-28T12:49:08.736 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-28T12:49:08.736 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-28T12:49:08.736 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-28T12:49:08.736 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-28T12:49:08.736 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-28T12:49:08.736 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-28T12:49:08.736 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-28T12:49:08.736 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-28T12:49:08.736 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-28T12:49:08.736 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-28T12:49:08.736 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-28T12:49:08.737 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-28T12:49:08.737 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-28T12:49:08.737 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-28T12:49:08.737 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-28T12:49:08.737 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-28T12:49:08.737 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-28T12:49:08.737 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-28T12:49:08.737 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-28T12:49:08.737 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-28T12:49:08.737 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-28T12:49:08.737 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-28T12:49:08.737 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-28T12:49:08.738 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-28T12:49:08.738 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-28T12:49:08.738 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-28T12:49:08.738 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-28T12:49:08.738 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-28T12:49:08.738 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-28T12:49:08.738 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-28T12:49:08.738 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-28T12:49:08.738 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-28T12:49:08.738 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-28T12:49:08.738 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-28T12:49:08.738 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-28T12:49:08.739 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-28T12:49:08.739 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-28T12:49:08.739 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-28T12:49:08.739 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-28T12:49:08.739 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-28T12:49:08.739 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-28T12:49:08.739 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-28T12:49:08.739 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-28T12:49:08.739 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-28T12:49:08.739 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-28T12:49:08.739 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-28T12:49:08.739 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-28T12:49:08.739 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-28T12:49:08.740 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-28T12:49:08.740 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-28T12:49:08.740 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-28T12:49:08.740 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-28T12:49:08.740 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-28T12:49:08.740 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-28T12:49:08.740 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-28T12:49:08.740 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-28T12:49:08.740 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-28T12:49:08.740 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-28T12:49:08.740 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-28T12:49:08.740 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-28T12:49:08.741 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-28T12:49:08.741 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-28T12:49:08.741 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-28T12:49:08.741 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-28T12:49:08.741 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-28T12:49:08.741 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-28T12:49:08.741 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-28T12:49:08.741 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-28T12:49:08.741 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-28T12:49:08.741 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-28T12:49:08.741 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-28T12:49:08.741 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-28T12:49:08.741 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-28T12:49:08.742 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-28T12:49:08.742 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-28T12:49:08.742 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-28T12:49:08.742 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-28T12:49:08.742 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-28T12:49:08.742 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-28T12:49:08.742 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-28T12:49:08.742 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-28T12:49:08.742 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-28T12:49:08.742 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-28T12:49:08.742 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-28T12:49:08.743 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-28T12:49:08.743 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-28T12:49:08.743 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-28T12:49:08.743 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-28T12:49:08.743 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-28T12:49:08.743 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-28T12:49:08.743 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-28T12:49:08.743 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-28T12:49:08.743 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-28T12:49:08.743 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-28T12:49:08.743 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-28T12:49:08.743 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-28T12:49:08.743 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-28T12:49:08.744 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-28T12:49:08.744 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-28T12:49:08.744 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-28T12:49:08.744 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-28T12:49:08.744 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-28T12:49:08.744 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-28T12:49:08.744 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-28T12:49:08.744 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-28T12:49:08.744 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-28T12:49:08.744 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-28T12:49:08.744 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-28T12:49:08.744 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-28T12:49:08.745 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-28T12:49:08.745 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-28T12:49:08.745 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-28T12:49:08.745 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-28T12:49:08.745 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-28T12:49:08.745 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-28T12:49:08.745 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-28T12:49:08.745 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-28T12:49:08.745 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-28T12:49:08.745 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-28T12:49:08.745 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-28T12:49:08.745 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-28T12:49:08.745 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-28T12:49:08.746 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-28T12:49:08.746 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-28T12:49:08.746 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-28T12:49:08.746 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-28T12:49:08.746 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-28T12:49:08.746 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-28T12:49:08.746 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-28T12:49:08.746 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-28T12:49:08.746 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-28T12:49:08.746 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-28T12:49:08.746 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-28T12:49:08.746 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-28T12:49:08.747 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-28T12:49:08.747 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-28T12:49:08.747 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-28T12:49:08.747 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-28T12:49:08.747 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-28T12:49:08.747 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-28T12:49:08.747 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-28T12:49:08.747 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-28T12:49:08.747 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-28T12:49:08.747 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-28T12:49:08.747 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-28T12:49:08.747 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-28T12:49:08.748 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-28T12:49:08.748 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-28T12:49:08.748 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-28T12:49:08.748 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-28T12:49:08.748 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-28T12:49:08.748 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-28T12:49:08.748 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-28T12:49:08.748 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-28T12:49:08.748 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-28T12:49:08.748 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-28T12:49:08.748 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-28T12:49:08.748 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-28T12:49:08.749 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-28T12:49:08.749 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-28T12:49:08.749 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-28T12:49:08.749 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-28T12:49:08.749 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-28T12:49:08.749 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-28T12:49:08.749 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-28T12:49:08.749 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-28T12:49:08.749 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-28T12:49:08.749 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-28T12:49:08.749 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-28T12:49:08.749 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-28T12:49:08.749 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-28T12:49:08.750 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-28T12:49:08.750 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-28T12:49:08.750 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-28T12:49:08.750 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-28T12:49:08.750 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-28T12:49:08.750 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-28T12:49:08.750 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-28T12:49:08.750 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-28T12:49:08.750 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-28T12:49:08.750 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-28T12:49:08.750 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-28T12:49:08.750 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-28T12:49:08.751 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-28T12:49:08.751 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-28T12:49:08.751 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-28T12:49:08.751 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-28T12:49:08.751 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-28T12:49:08.751 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-28T12:49:08.751 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-28T12:49:08.751 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-28T12:49:08.751 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-28T12:49:08.751 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-28T12:49:08.751 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-28T12:49:08.751 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-28T12:49:08.752 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-28T12:49:08.752 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-28T12:49:08.752 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-28T12:49:08.752 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-28T12:49:08.752 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-28T12:49:08.752 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-28T12:49:08.752 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-28T12:49:08.752 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-28T12:49:08.752 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-28T12:49:08.752 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-28T12:49:08.752 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-28T12:49:08.752 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-28T12:49:08.753 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-28T12:49:08.753 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-28T12:49:08.753 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-28T12:49:08.753 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-28T12:49:08.753 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-28T12:49:08.753 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-28T12:49:08.753 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-28T12:49:08.753 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-28T12:49:08.753 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-28T12:49:08.753 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-28T12:49:08.753 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-28T12:49:08.753 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-28T12:49:08.754 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-28T12:49:08.754 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-28T12:49:08.754 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-28T12:49:08.754 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-28T12:49:08.754 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-28T12:49:08.754 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-28T12:49:08.754 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-28T12:49:08.754 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-28T12:49:08.754 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-28T12:49:08.754 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-28T12:49:08.754 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-28T12:49:08.754 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-28T12:49:08.755 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-28T12:49:08.755 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-28T12:49:08.755 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-28T12:49:08.755 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-28T12:49:08.755 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-28T12:49:08.755 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-28T12:49:08.755 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-28T12:49:08.755 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-28T12:49:08.755 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-28T12:49:08.755 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-28T12:49:08.755 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-28T12:49:08.755 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-28T12:49:08.756 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-28T12:49:08.756 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-28T12:49:08.756 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-28T12:49:08.756 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-28T12:49:08.756 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-28T12:49:08.756 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-28T12:49:08.756 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-28T12:49:08.756 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-28T12:49:08.756 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-28T12:49:08.756 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-28T12:49:08.756 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-28T12:49:08.756 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-28T12:49:08.757 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-28T12:49:08.757 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-28T12:49:08.757 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-28T12:49:08.757 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-28T12:49:08.757 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-28T12:49:08.757 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-28T12:49:08.757 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-28T12:49:08.757 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-28T12:49:08.757 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-28T12:49:08.757 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-28T12:49:08.757 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-28T12:49:08.757 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-28T12:49:08.758 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-28T12:49:08.758 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-28T12:49:08.758 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-28T12:49:08.758 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-28T12:49:08.758 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-28T12:49:08.758 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-28T12:49:08.758 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-28T12:49:08.758 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-28T12:49:08.758 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-28T12:49:08.758 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-28T12:49:08.758 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-28T12:49:08.758 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-28T12:49:08.758 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-28T12:49:08.759 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-28T12:49:08.759 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-28T12:49:08.759 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-28T12:49:08.759 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-28T12:49:08.759 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-28T12:49:08.759 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-28T12:49:08.759 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-28T12:49:08.759 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-28T12:49:08.759 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-28T12:49:08.759 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-28T12:49:08.759 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-28T12:49:08.759 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-28T12:49:08.760 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-28T12:49:08.760 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-28T12:49:08.760 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-28T12:49:08.760 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-28T12:49:08.760 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-28T12:49:08.760 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-28T12:49:08.760 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-28T12:49:08.760 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-28T12:49:08.760 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-28T12:49:08.760 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-28T12:49:08.760 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-28T12:49:08.760 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-28T12:49:08.761 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-28T12:49:08.761 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-28T12:49:08.761 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-28T12:49:08.761 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-28T12:49:08.761 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-28T12:49:08.761 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-28T12:49:08.761 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-28T12:49:08.761 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-28T12:49:08.761 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-28T12:49:08.761 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-28T12:49:08.761 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-28T12:49:08.761 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-28T12:49:08.761 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-28T12:49:08.761 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-28T12:49:08.762 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-28T12:49:08.762 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-28T12:49:08.762 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-28T12:49:08.762 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-28T12:49:08.762 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-28T12:49:08.762 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-28T12:49:08.762 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-28T12:49:08.762 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-28T12:49:08.762 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-28T12:49:08.762 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-28T12:49:08.762 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-28T12:49:08.763 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-28T12:49:08.763 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-28T12:49:08.763 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-28T12:49:08.763 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-28T12:49:08.763 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-28T12:49:08.763 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-28T12:49:08.763 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-28T12:49:08.763 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-28T12:49:08.763 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-28T12:49:08.763 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-28T12:49:08.763 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-28T12:49:08.763 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-28T12:49:08.764 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-28T12:49:08.764 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-28T12:49:08.764 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-28T12:49:08.764 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-28T12:49:08.764 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-28T12:49:08.764 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-28T12:49:08.764 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-28T12:49:08.764 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-28T12:49:08.764 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-28T12:49:08.764 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-28T12:49:08.764 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-28T12:49:08.764 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-28T12:49:08.765 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-28T12:49:08.765 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-28T12:49:08.765 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-28T12:49:08.765 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-28T12:49:08.765 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-28T12:49:08.765 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-28T12:49:08.765 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-28T12:49:08.765 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-28T12:49:08.765 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-28T12:49:08.765 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-28T12:49:08.765 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-28T12:49:08.765 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-28T12:49:08.766 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-28T12:49:08.766 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-28T12:49:08.766 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-28T12:49:08.766 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-28T12:49:08.766 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-28T12:49:08.766 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-28T12:49:08.766 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-28T12:49:08.766 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-28T12:49:08.766 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-28T12:49:08.766 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-28T12:49:08.766 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-28T12:49:08.766 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-28T12:49:08.767 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-28T12:49:08.767 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-28T12:49:08.767 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-28T12:49:08.767 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-28T12:49:08.767 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-28T12:49:08.767 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-28T12:49:08.767 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-28T12:49:08.767 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-28T12:49:08.767 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-28T12:49:08.767 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-28T12:49:08.767 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-28T12:49:08.767 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-28T12:49:08.768 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-28T12:49:08.768 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-28T12:49:08.768 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-28T12:49:08.768 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-28T12:49:08.768 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-28T12:49:08.768 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-28T12:49:08.768 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-28T12:49:08.768 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-28T12:49:08.768 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-28T12:49:08.768 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-28T12:49:08.768 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-28T12:49:08.768 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-28T12:49:08.769 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-28T12:49:08.769 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-28T12:49:08.769 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-28T12:49:08.769 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-28T12:49:08.769 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-28T12:49:08.769 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-28T12:49:08.769 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-28T12:49:08.769 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-28T12:49:08.769 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-28T12:49:08.769 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-28T12:49:08.769 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-28T12:49:08.769 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-28T12:49:08.769 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-28T12:49:08.770 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-28T12:49:08.770 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-28T12:49:08.770 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-28T12:49:08.770 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-28T12:49:08.770 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-28T12:49:08.770 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-28T12:49:08.770 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-28T12:49:08.770 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-28T12:49:08.770 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-28T12:49:08.770 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-28T12:49:08.770 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-28T12:49:08.771 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-28T12:49:08.771 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-28T12:49:08.771 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-28T12:49:08.771 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-28T12:49:08.771 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-28T12:49:08.771 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-28T12:49:08.771 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-28T12:49:08.771 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-28T12:49:08.771 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-28T12:49:08.771 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-28T12:49:08.771 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-28T12:49:08.771 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-28T12:49:08.772 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-28T12:49:08.772 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-28T12:49:08.772 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-28T12:49:08.772 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-28T12:49:08.772 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-28T12:49:08.772 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-28T12:49:08.772 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-28T12:49:08.772 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-28T12:49:08.772 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-28T12:49:08.772 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-28T12:49:08.772 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-28T12:49:08.772 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-28T12:49:08.772 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-28T12:49:08.773 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-28T12:49:08.773 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-28T12:49:08.773 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-28T12:49:08.773 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-28T12:49:08.773 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-28T12:49:08.773 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-28T12:49:08.773 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-28T12:49:08.773 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-28T12:49:08.773 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-28T12:49:08.773 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-28T12:49:08.773 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-28T12:49:08.773 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-28T12:49:08.774 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-28T12:49:08.774 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-28T12:49:08.774 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-28T12:49:08.774 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-28T12:49:08.774 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-28T12:49:08.774 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-28T12:49:08.774 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-28T12:49:08.774 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-28T12:49:08.774 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-28T12:49:08.774 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-28T12:49:08.774 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-28T12:49:08.775 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-28T12:49:08.775 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-28T12:49:08.775 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-28T12:49:08.775 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-28T12:49:08.775 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-28T12:49:08.775 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-28T12:49:08.775 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-28T12:49:08.775 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-28T12:49:08.775 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-28T12:49:08.775 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-28T12:49:08.775 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-28T12:49:08.775 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-28T12:49:08.776 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-28T12:49:08.776 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-28T12:49:08.776 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-28T12:49:08.776 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-28T12:49:08.776 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-28T12:49:08.776 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-28T12:49:08.776 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-28T12:49:08.776 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-28T12:49:08.776 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-28T12:49:08.776 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-28T12:49:08.776 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-28T12:49:08.776 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-28T12:49:08.777 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-28T12:49:08.777 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-28T12:49:08.777 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-28T12:49:08.777 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-28T12:49:08.777 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-28T12:49:08.777 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-28T12:49:08.777 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-28T12:49:08.777 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-28T12:49:08.777 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-28T12:49:08.777 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-28T12:49:08.777 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-28T12:49:08.777 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-28T12:49:08.778 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-28T12:49:08.778 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-28T12:49:08.778 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-28T12:49:08.778 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-28T12:49:08.778 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-28T12:49:08.778 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-28T12:49:08.778 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-28T12:49:08.778 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-28T12:49:08.778 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-28T12:49:08.778 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-28T12:49:08.778 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-28T12:49:08.778 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-28T12:49:08.779 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-28T12:49:08.779 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-28T12:49:08.779 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-28T12:49:08.779 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-28T12:49:08.779 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-28T12:49:08.779 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-28T12:49:08.779 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-28T12:49:08.779 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-28T12:49:08.779 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-28T12:49:08.779 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-28T12:49:08.779 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-28T12:49:08.779 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-28T12:49:08.780 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-28T12:49:08.780 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-28T12:49:08.780 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-28T12:49:08.780 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-28T12:49:08.780 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-28T12:49:08.780 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-28T12:49:08.780 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-28T12:49:08.780 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-28T12:49:08.780 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-28T12:49:08.780 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-28T12:49:08.780 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-28T12:49:08.780 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-28T12:49:08.781 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-28T12:49:08.781 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-28T12:49:08.781 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-28T12:49:08.781 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-28T12:49:08.781 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-28T12:49:08.781 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-28T12:49:08.781 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-28T12:49:08.781 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-28T12:49:08.781 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-28T12:49:08.781 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-28T12:49:08.781 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-28T12:49:08.782 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-28T12:49:08.782 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-28T12:49:08.782 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-28T12:49:08.782 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-28T12:49:08.782 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-28T12:49:08.782 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-28T12:49:08.782 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-28T12:49:08.782 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-28T12:49:08.782 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-28T12:49:08.782 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-28T12:49:08.782 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-28T12:49:08.782 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-28T12:49:08.783 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-28T12:49:08.783 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-28T12:49:08.783 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-28T12:49:08.783 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-28T12:49:08.783 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-28T12:49:08.783 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-28T12:49:08.783 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-28T12:49:08.783 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-28T12:49:08.783 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-28T12:49:08.783 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-28T12:49:08.783 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-28T12:49:08.783 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-28T12:49:08.784 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-28T12:49:08.784 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-28T12:49:08.784 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-28T12:49:08.784 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-28T12:49:08.784 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-28T12:49:08.784 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-28T12:49:08.784 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-28T12:49:08.784 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-28T12:49:08.784 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-28T12:49:08.784 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-28T12:49:08.784 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-28T12:49:08.784 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-28T12:49:08.785 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-28T12:49:08.785 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-28T12:49:08.785 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-28T12:49:08.785 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-28T12:49:08.785 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-28T12:49:08.785 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-28T12:49:08.785 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-28T12:49:08.785 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-28T12:49:08.785 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-28T12:49:08.785 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-28T12:49:08.785 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-28T12:49:08.785 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-28T12:49:08.786 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-28T12:49:08.786 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-28T12:49:08.786 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-28T12:49:08.786 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-28T12:49:08.786 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-28T12:49:08.786 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-28T12:49:08.786 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-28T12:49:08.786 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-28T12:49:08.786 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-28T12:49:08.786 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-28T12:49:08.786 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-28T12:49:08.786 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-28T12:49:08.787 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-28T12:49:08.787 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-28T12:49:08.787 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-28T12:49:08.787 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-28T12:49:08.787 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-28T12:49:08.787 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-28T12:49:08.787 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-28T12:49:08.787 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-28T12:49:08.787 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-28T12:49:08.787 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-28T12:49:08.787 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-28T12:49:08.787 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-28T12:49:08.788 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-28T12:49:08.788 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-28T12:49:08.788 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-28T12:49:08.788 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-28T12:49:08.788 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-28T12:49:08.788 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-28T12:49:08.788 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-28T12:49:08.788 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-28T12:49:08.788 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-28T12:49:08.788 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-28T12:49:08.788 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-28T12:49:08.788 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-28T12:49:08.788 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-28T12:49:08.789 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-28T12:49:08.789 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-28T12:49:08.789 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-28T12:49:08.789 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-28T12:49:08.789 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-28T12:49:08.789 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-28T12:49:08.789 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-28T12:49:08.789 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-28T12:49:08.789 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-28T12:49:08.789 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-28T12:49:08.789 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-28T12:49:08.789 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-28T12:49:08.790 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-28T12:49:08.790 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-28T12:49:08.790 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-28T12:49:08.790 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-28T12:49:08.790 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-28T12:49:08.790 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-28T12:49:08.790 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-28T12:49:08.790 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-28T12:49:08.790 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-28T12:49:08.790 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-28T12:49:08.790 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-28T12:49:08.790 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-28T12:49:08.791 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-28T12:49:08.791 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-28T12:49:08.791 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-28T12:49:08.791 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-28T12:49:08.791 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-28T12:49:08.791 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-28T12:49:08.791 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-28T12:49:08.791 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-28T12:49:08.791 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-28T12:49:08.791 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-28T12:49:08.791 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-28T12:49:08.791 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-28T12:49:08.792 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-28T12:49:08.792 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-28T12:49:08.792 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-28T12:49:08.792 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-28T12:49:08.792 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-28T12:49:08.792 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-28T12:49:08.792 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-28T12:49:08.792 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-28T12:49:08.792 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-28T12:49:08.792 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-28T12:49:08.792 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-28T12:49:08.792 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-28T12:49:08.793 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-28T12:49:08.793 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-28T12:49:08.793 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-28T12:49:08.793 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-28T12:49:08.793 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-28T12:49:08.793 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-28T12:49:08.793 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-28T12:49:08.793 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-28T12:49:08.793 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-28T12:49:08.793 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-28T12:49:08.793 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-28T12:49:08.793 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-28T12:49:08.793 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-28T12:49:08.794 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-28T12:49:08.794 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-28T12:49:08.794 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-28T12:49:08.794 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-28T12:49:08.794 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-28T12:49:08.794 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-28T12:49:08.794 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-28T12:49:08.794 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-28T12:49:08.794 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-28T12:49:08.794 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-28T12:49:08.794 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-28T12:49:08.794 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-28T12:49:08.795 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-28T12:49:08.795 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-28T12:49:08.795 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-28T12:49:08.795 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-28T12:49:08.795 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-28T12:49:08.795 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-28T12:49:08.795 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-28T12:49:08.795 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-28T12:49:08.795 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-28T12:49:08.795 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-28T12:49:08.795 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-28T12:49:08.795 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-28T12:49:08.795 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-28T12:49:08.796 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-28T12:49:08.796 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-28T12:49:08.796 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-28T12:49:08.796 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-28T12:49:08.796 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-28T12:49:08.796 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-28T12:49:08.796 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-28T12:49:08.797 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-28T12:49:08.797 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-28T12:49:08.797 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-28T12:49:08.797 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-28T12:49:08.797 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-28T12:49:08.797 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-28T12:49:08.797 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-28T12:49:08.797 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-28T12:49:08.797 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-28T12:49:08.797 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-28T12:49:08.797 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-28T12:49:08.797 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-28T12:49:08.797 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-28T12:49:08.798 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-28T12:49:08.798 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-28T12:49:08.798 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-28T12:49:08.798 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-28T12:49:08.798 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-28T12:49:08.798 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-28T12:49:08.798 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-28T12:49:08.798 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-28T12:49:08.798 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-28T12:49:08.798 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-28T12:49:08.798 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-28T12:49:08.799 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-28T12:49:08.799 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-28T12:49:08.799 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-28T12:49:08.799 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-28T12:49:08.799 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-28T12:49:08.799 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-28T12:49:08.799 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-28T12:49:08.799 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-28T12:49:08.799 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-28T12:49:08.799 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-28T12:49:08.799 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-28T12:49:08.799 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-28T12:49:08.800 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-28T12:49:08.800 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-28T12:49:08.800 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-28T12:49:08.800 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-28T12:49:08.800 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-28T12:49:08.800 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-28T12:49:08.800 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-28T12:49:08.800 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-28T12:49:08.800 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-28T12:49:08.800 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-28T12:49:08.800 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-28T12:49:08.800 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-28T12:49:08.801 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-28T12:49:08.801 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-28T12:49:08.801 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-28T12:49:08.801 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-28T12:49:08.801 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-28T12:49:08.801 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-28T12:49:08.801 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-28T12:49:08.801 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-28T12:49:08.801 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-28T12:49:08.801 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-28T12:49:08.801 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-28T12:49:08.801 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-28T12:49:08.802 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-28T12:49:08.802 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-28T12:49:08.802 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-28T12:49:08.802 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-28T12:49:08.802 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-28T12:49:08.802 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-28T12:49:08.802 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-28T12:49:08.802 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-28T12:49:08.802 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-28T12:49:08.802 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-28T12:49:08.802 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-28T12:49:08.802 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-28T12:49:08.803 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-28T12:49:08.803 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-28T12:49:08.803 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-28T12:49:08.803 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-28T12:49:08.803 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-28T12:49:08.803 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-28T12:49:08.803 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-28T12:49:08.803 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-28T12:49:08.803 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-28T12:49:08.803 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-28T12:49:08.803 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-28T12:49:08.803 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-28T12:49:08.804 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-28T12:49:08.804 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-28T12:49:08.804 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-28T12:49:08.804 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-28T12:49:08.804 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-28T12:49:08.804 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-28T12:49:08.804 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-28T12:49:08.804 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-28T12:49:08.804 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-28T12:49:08.804 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-28T12:49:08.804 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-28T12:49:08.804 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-28T12:49:08.805 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-28T12:49:08.805 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-28T12:49:08.805 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-28T12:49:08.805 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-28T12:49:08.805 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-28T12:49:08.805 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-28T12:49:08.805 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-28T12:49:08.805 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-28T12:49:08.805 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-28T12:49:08.805 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-28T12:49:08.805 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-28T12:49:08.805 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-28T12:49:08.806 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-28T12:49:08.806 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-28T12:49:08.806 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-28T12:49:08.806 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-28T12:49:08.806 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-28T12:49:08.806 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-28T12:49:08.806 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-28T12:49:08.806 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-28T12:49:08.806 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-28T12:49:08.806 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-28T12:49:08.806 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-28T12:49:08.806 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-28T12:49:08.807 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-28T12:49:08.807 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-28T12:49:08.807 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-28T12:49:08.807 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-28T12:49:08.807 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-28T12:49:08.807 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-28T12:49:08.807 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-28T12:49:08.807 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-28T12:49:08.807 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-28T12:49:08.807 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-28T12:49:08.807 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-28T12:49:08.807 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-28T12:49:08.807 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-28T12:49:08.808 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-28T12:49:08.808 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-28T12:49:08.808 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-28T12:49:08.808 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-28T12:49:08.808 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-28T12:49:08.808 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-28T12:49:08.808 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-28T12:49:08.808 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-28T12:49:08.808 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-28T12:49:08.808 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-28T12:49:08.808 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-28T12:49:08.808 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-28T12:49:08.809 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-28T12:49:08.809 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-28T12:49:08.809 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-28T12:49:08.809 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-28T12:49:08.809 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-28T12:49:08.809 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-28T12:49:08.809 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-28T12:49:08.809 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-28T12:49:08.809 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-28T12:49:08.809 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-28T12:49:08.809 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-28T12:49:08.809 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-28T12:49:08.809 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-28T12:49:08.810 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-28T12:49:08.810 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-28T12:49:08.810 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-28T12:49:08.810 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-28T12:49:08.810 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-28T12:49:08.810 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-28T12:49:08.810 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-28T12:49:08.810 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-28T12:49:08.810 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-28T12:49:08.810 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-28T12:49:08.810 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-28T12:49:08.810 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-28T12:49:08.811 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-28T12:49:08.811 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-28T12:49:08.811 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-28T12:49:08.811 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-28T12:49:08.811 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-28T12:49:08.811 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-28T12:49:08.811 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-28T12:49:08.811 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-28T12:49:08.811 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-28T12:49:08.811 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-28T12:49:08.811 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-28T12:49:08.812 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-28T12:49:08.812 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-28T12:49:08.812 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-28T12:49:08.812 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-28T12:49:08.812 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-28T12:49:08.812 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-28T12:49:08.812 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-28T12:49:08.812 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-28T12:49:08.812 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-28T12:49:08.812 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-28T12:49:08.812 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-28T12:49:08.812 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-28T12:49:08.812 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-28T12:49:08.813 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-28T12:49:08.813 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-28T12:49:08.813 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-28T12:49:08.813 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-28T12:49:08.813 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-28T12:49:08.813 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-28T12:49:08.813 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-28T12:49:08.813 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-28T12:49:08.813 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-28T12:49:08.813 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-28T12:49:08.813 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-28T12:49:08.813 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-28T12:49:08.814 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-28T12:49:08.814 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-28T12:49:08.814 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-28T12:49:08.814 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-28T12:49:08.814 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-28T12:49:08.814 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-28T12:49:08.814 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-28T12:49:08.814 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-28T12:49:08.814 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-28T12:49:08.814 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-28T12:49:08.814 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-28T12:49:08.815 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-28T12:49:08.815 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-28T12:49:08.815 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-28T12:49:08.815 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-28T12:49:08.815 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-28T12:49:08.815 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-28T12:49:08.815 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-28T12:49:08.815 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-28T12:49:08.815 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-28T12:49:08.815 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-28T12:49:08.815 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-28T12:49:08.815 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-28T12:49:08.815 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-28T12:49:08.816 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-28T12:49:08.816 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-28T12:49:08.816 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-28T12:49:08.816 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-28T12:49:08.816 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-28T12:49:08.816 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-28T12:49:08.816 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-28T12:49:08.816 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-28T12:49:08.816 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-28T12:49:08.816 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-28T12:49:08.816 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-28T12:49:08.816 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-28T12:49:08.817 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-28T12:49:08.817 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-28T12:49:08.817 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-28T12:49:08.817 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-28T12:49:08.817 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-28T12:49:08.817 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-28T12:49:08.817 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-28T12:49:08.817 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-28T12:49:08.817 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-28T12:49:08.817 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-28T12:49:08.817 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-28T12:49:08.817 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-28T12:49:08.818 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-28T12:49:08.818 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-28T12:49:08.818 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-28T12:49:08.818 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-28T12:49:08.818 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-28T12:49:08.818 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-28T12:49:08.818 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-28T12:49:08.818 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-28T12:49:08.818 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-28T12:49:08.818 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-28T12:49:08.818 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-28T12:49:08.818 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-28T12:49:08.819 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-28T12:49:08.819 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-28T12:49:08.819 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-28T12:49:08.819 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-28T12:49:08.819 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-28T12:49:08.819 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-28T12:49:08.820 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-28T12:49:08.820 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-28T12:49:08.820 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-28T12:49:08.820 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-28T12:49:08.820 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-28T12:49:08.820 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-28T12:49:08.821 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-28T12:49:08.821 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-28T12:49:08.821 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-28T12:49:08.821 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-28T12:49:08.821 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-28T12:49:08.821 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-28T12:49:08.821 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-28T12:49:08.821 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-28T12:49:08.821 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-28T12:49:08.821 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-28T12:49:08.821 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-28T12:49:08.821 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-28T12:49:08.822 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-28T12:49:08.822 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-28T12:49:08.822 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-28T12:49:08.822 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-28T12:49:08.822 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-28T12:49:08.822 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-28T12:49:08.822 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-28T12:49:08.822 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-28T12:49:08.822 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-28T12:49:08.822 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-28T12:49:08.822 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-28T12:49:08.822 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-28T12:49:08.823 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-28T12:49:08.823 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-28T12:49:08.823 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-28T12:49:08.823 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-28T12:49:08.823 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-28T12:49:08.823 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-28T12:49:08.823 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-28T12:49:08.823 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-28T12:49:08.823 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-28T12:49:08.823 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-28T12:49:08.823 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-28T12:49:08.823 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-28T12:49:08.824 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-28T12:49:08.824 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-28T12:49:08.824 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-28T12:49:08.824 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-28T12:49:08.824 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-28T12:49:08.824 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-28T12:49:08.824 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-28T12:49:08.824 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-28T12:49:08.824 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-28T12:49:08.824 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-28T12:49:08.824 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-28T12:49:08.824 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-28T12:49:08.825 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-28T12:49:08.825 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-28T12:49:08.825 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-28T12:49:08.825 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-28T12:49:08.825 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-28T12:49:08.825 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-28T12:49:08.825 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-28T12:49:08.825 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-28T12:49:08.825 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-28T12:49:08.825 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-28T12:49:08.825 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-28T12:49:08.825 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-28T12:49:08.826 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-28T12:49:08.826 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-28T12:49:08.826 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-28T12:49:08.826 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-28T12:49:08.826 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-28T12:49:08.826 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-28T12:49:08.826 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-28T12:49:08.826 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-28T12:49:08.826 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-28T12:49:08.826 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-28T12:49:08.826 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-28T12:49:08.826 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-28T12:49:08.827 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-28T12:49:08.827 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-28T12:49:08.827 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-28T12:49:08.827 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-28T12:49:08.827 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-28T12:49:08.827 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-28T12:49:08.827 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-28T12:49:08.827 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-28T12:49:08.827 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-28T12:49:08.827 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-28T12:49:08.827 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-28T12:49:08.827 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-28T12:49:08.828 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-28T12:49:08.828 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-28T12:49:08.828 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-28T12:49:08.828 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-28T12:49:08.828 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-28T12:49:08.828 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-28T12:49:08.828 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-28T12:49:08.828 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-28T12:49:08.828 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-28T12:49:08.828 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-28T12:49:08.828 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-28T12:49:08.828 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-28T12:49:08.829 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-28T12:49:08.829 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-28T12:49:08.829 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-28T12:49:08.829 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-28T12:49:08.829 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-28T12:49:08.829 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-28T12:49:08.829 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-28T12:49:08.829 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-28T12:49:08.829 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-28T12:49:08.829 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-28T12:49:08.829 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-28T12:49:08.829 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-28T12:49:08.830 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-28T12:49:08.830 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-28T12:49:08.830 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-28T12:49:08.830 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-28T12:49:08.830 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-28T12:49:08.830 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-28T12:49:08.830 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-28T12:49:08.830 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-28T12:49:08.830 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-28T12:49:08.830 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-28T12:49:08.830 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-28T12:49:08.830 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-28T12:49:08.831 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-28T12:49:08.831 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-28T12:49:08.831 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-28T12:49:08.831 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-28T12:49:08.831 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-28T12:49:08.831 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-28T12:49:08.831 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-28T12:49:08.831 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-28T12:49:08.831 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-28T12:49:08.831 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-28T12:49:08.831 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-28T12:49:08.831 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-28T12:49:08.831 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-28T12:49:08.832 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-28T12:49:08.832 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-28T12:49:08.832 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-28T12:49:08.832 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-28T12:49:08.832 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-28T12:49:08.832 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-28T12:49:08.832 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-28T12:49:08.832 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-28T12:49:08.832 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-28T12:49:08.832 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-28T12:49:08.832 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-28T12:49:08.832 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-28T12:49:08.833 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-28T12:49:08.833 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-28T12:49:08.833 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-28T12:49:08.833 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-28T12:49:08.833 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-28T12:49:08.833 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-28T12:49:08.833 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-28T12:49:08.833 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-28T12:49:08.833 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-28T12:49:08.833 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-28T12:49:08.833 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-28T12:49:08.833 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-28T12:49:08.834 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-28T12:49:08.834 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-28T12:49:08.834 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-28T12:49:08.834 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-28T12:49:08.834 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-28T12:49:08.834 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-28T12:49:08.834 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-28T12:49:08.834 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-28T12:49:08.834 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-28T12:49:08.834 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-28T12:49:08.834 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-28T12:49:08.834 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-28T12:49:08.835 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-28T12:49:08.835 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-28T12:49:08.835 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-28T12:49:08.835 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-28T12:49:08.835 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-28T12:49:08.835 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-28T12:49:08.835 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-28T12:49:08.835 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-28T12:49:08.835 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-28T12:49:08.835 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-28T12:49:08.835 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-28T12:49:08.835 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-28T12:49:08.835 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-28T12:49:08.836 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-28T12:49:08.836 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-28T12:49:08.836 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-28T12:49:08.836 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-28T12:49:08.836 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-28T12:49:08.836 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-28T12:49:08.836 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-28T12:49:08.836 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-28T12:49:08.836 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-28T12:49:08.836 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-28T12:49:08.836 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-28T12:49:08.836 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-28T12:49:08.837 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-28T12:49:08.837 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-28T12:49:08.837 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-28T12:49:08.837 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-28T12:49:08.837 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-28T12:49:08.837 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-28T12:49:08.837 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-28T12:49:08.837 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-28T12:49:08.837 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-28T12:49:08.837 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-28T12:49:08.837 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-28T12:49:08.837 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-28T12:49:08.838 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-28T12:49:08.838 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-28T12:49:08.838 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-28T12:49:08.838 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-28T12:49:08.838 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-28T12:49:08.838 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-28T12:49:08.838 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-28T12:49:08.838 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-28T12:49:08.838 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-28T12:49:08.838 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-28T12:49:08.838 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-28T12:49:08.838 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-28T12:49:08.839 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-28T12:49:08.839 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-28T12:49:08.839 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-28T12:49:08.839 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-28T12:49:08.839 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-28T12:49:08.839 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-28T12:49:08.839 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-28T12:49:08.839 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-28T12:49:08.839 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-28T12:49:08.839 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-28T12:49:08.839 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-28T12:49:08.839 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-28T12:49:08.840 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-28T12:49:08.840 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-28T12:49:08.840 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-28T12:49:08.840 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-28T12:49:08.840 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-28T12:49:08.840 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-28T12:49:08.840 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-28T12:49:08.840 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-28T12:49:08.840 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-28T12:49:08.840 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-28T12:49:08.840 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-28T12:49:08.840 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-28T12:49:08.841 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-28T12:49:08.841 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-28T12:49:08.841 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-28T12:49:08.841 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-28T12:49:08.841 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-28T12:49:08.841 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-28T12:49:08.841 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-28T12:49:08.841 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-28T12:49:08.841 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-28T12:49:08.841 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-28T12:49:08.841 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-28T12:49:08.841 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-28T12:49:08.842 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-28T12:49:08.842 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-28T12:49:08.842 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-28T12:49:08.842 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-28T12:49:08.842 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-28T12:49:08.842 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-28T12:49:08.842 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-28T12:49:08.842 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-28T12:49:08.842 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-28T12:49:08.842 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-28T12:49:08.842 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-28T12:49:08.842 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-28T12:49:08.843 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-28T12:49:08.843 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-28T12:49:08.843 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-28T12:49:08.843 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-28T12:49:08.843 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-28T12:49:08.843 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-28T12:49:08.843 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-28T12:49:08.843 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-28T12:49:08.843 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-28T12:49:08.843 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-28T12:49:08.843 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-28T12:49:08.843 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-28T12:49:08.843 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-28T12:49:08.844 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-28T12:49:08.844 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-28T12:49:08.844 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-28T12:49:08.844 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-28T12:49:08.844 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-28T12:49:08.844 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-28T12:49:08.844 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-28T12:49:08.844 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-28T12:49:08.844 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-28T12:49:08.844 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-28T12:49:08.844 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-28T12:49:08.845 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-28T12:49:08.845 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-28T12:49:08.845 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-28T12:49:08.845 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-28T12:49:08.845 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-28T12:49:08.845 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-28T12:49:08.845 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-28T12:49:08.845 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-28T12:49:08.845 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-28T12:49:08.845 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-28T12:49:08.845 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-28T12:49:08.845 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-28T12:49:08.845 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-28T12:49:08.846 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-28T12:49:08.846 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-28T12:49:08.846 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-28T12:49:08.846 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-28T12:49:08.846 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-28T12:49:08.846 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-28T12:49:08.846 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-28T12:49:08.846 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-28T12:49:08.846 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-28T12:49:08.846 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-28T12:49:08.846 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-28T12:49:08.846 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-28T12:49:08.847 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-28T12:49:08.847 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-28T12:49:08.847 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-28T12:49:08.847 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-28T12:49:08.847 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-28T12:49:08.847 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-28T12:49:08.847 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-28T12:49:08.847 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-28T12:49:08.847 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-28T12:49:08.847 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-28T12:49:08.847 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-28T12:49:08.847 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-28T12:49:08.848 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-28T12:49:08.848 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-28T12:49:08.848 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-28T12:49:08.848 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-28T12:49:08.848 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-28T12:49:08.848 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-28T12:49:08.848 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-28T12:49:08.848 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-28T12:49:08.848 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-28T12:49:08.848 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-28T12:49:08.848 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-28T12:49:08.848 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-28T12:49:08.848 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-28T12:49:08.849 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-28T12:49:08.849 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-28T12:49:08.849 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-28T12:49:08.849 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-28T12:49:08.849 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-28T12:49:08.849 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-28T12:49:08.849 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-28T12:49:08.849 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-28T12:49:08.849 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-28T12:49:08.849 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-28T12:49:08.849 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-28T12:49:08.849 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-28T12:49:08.850 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-28T12:49:08.850 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-28T12:49:08.850 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-28T12:49:08.850 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-28T12:49:08.850 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-28T12:49:08.850 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-28T12:49:08.850 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-28T12:49:08.850 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-28T12:49:08.850 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-28T12:49:08.850 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-28T12:49:08.850 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-28T12:49:08.851 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-28T12:49:08.851 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-28T12:49:08.851 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-28T12:49:08.851 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-28T12:49:08.851 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-28T12:49:08.851 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-28T12:49:08.851 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-28T12:49:08.851 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-28T12:49:08.851 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-28T12:49:08.851 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-28T12:49:08.851 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-28T12:49:08.851 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-28T12:49:08.852 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-28T12:49:08.852 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-28T12:49:08.852 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-28T12:49:08.852 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-28T12:49:08.852 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-28T12:49:08.852 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-28T12:49:08.852 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-28T12:49:08.852 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-28T12:49:08.852 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-28T12:49:08.852 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-28T12:49:08.852 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-28T12:49:08.852 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-28T12:49:08.852 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-28T12:49:08.853 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-28T12:49:08.853 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-28T12:49:08.853 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-28T12:49:08.853 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-28T12:49:08.853 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-28T12:49:08.853 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-28T12:49:08.853 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-28T12:49:08.853 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-28T12:49:08.853 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-28T12:49:08.853 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-28T12:49:08.853 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-28T12:49:08.853 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-28T12:49:08.854 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-28T12:49:08.854 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-28T12:49:08.854 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-28T12:49:08.854 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-28T12:49:08.854 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-28T12:49:08.854 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-28T12:49:08.854 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-28T12:49:08.854 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-28T12:49:08.854 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-28T12:49:08.854 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-28T12:49:08.854 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-28T12:49:08.855 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-28T12:49:08.855 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-28T12:49:08.855 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-28T12:49:08.855 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-28T12:49:08.855 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-28T12:49:08.855 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-28T12:49:08.855 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-28T12:49:08.855 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-28T12:49:08.855 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-28T12:49:08.855 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-28T12:49:08.855 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-28T12:49:08.855 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-28T12:49:08.856 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-28T12:49:08.856 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-28T12:49:08.856 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-28T12:49:08.856 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-28T12:49:08.856 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-28T12:49:08.856 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-28T12:49:08.856 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-28T12:49:08.856 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-28T12:49:08.856 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-28T12:49:08.856 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-28T12:49:08.856 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-28T12:49:08.856 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-28T12:49:08.857 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-28T12:49:08.857 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-28T12:49:08.857 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-28T12:49:08.857 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-28T12:49:08.857 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-28T12:49:08.857 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-28T12:49:08.857 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-28T12:49:08.857 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-28T12:49:08.857 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-28T12:49:08.857 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-28T12:49:08.857 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-28T12:49:08.857 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-28T12:49:08.857 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-28T12:49:08.858 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-28T12:49:08.858 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-28T12:49:08.858 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-28T12:49:08.858 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-28T12:49:08.858 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-28T12:49:08.858 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-28T12:49:08.858 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-28T12:49:08.858 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-28T12:49:08.858 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-28T12:49:08.858 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-28T12:49:08.858 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-28T12:49:08.858 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-28T12:49:08.859 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-28T12:49:08.859 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-28T12:49:08.859 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-28T12:49:08.859 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-28T12:49:08.859 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-28T12:49:08.859 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-28T12:49:08.859 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-28T12:49:08.859 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-28T12:49:08.859 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-28T12:49:08.859 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-28T12:49:08.859 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-28T12:49:08.859 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-28T12:49:08.860 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-28T12:49:08.860 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-28T12:49:08.860 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-28T12:49:08.860 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-28T12:49:08.860 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-28T12:49:08.860 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-28T12:49:08.860 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-28T12:49:08.860 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-28T12:49:08.860 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-28T12:49:08.860 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-28T12:49:08.860 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-28T12:49:08.860 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-28T12:49:08.861 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-28T12:49:08.861 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-28T12:49:08.861 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-28T12:49:08.861 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-28T12:49:08.861 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-28T12:49:08.861 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-28T12:49:08.861 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-28T12:49:08.861 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-28T12:49:08.861 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-28T12:49:08.861 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-28T12:49:08.861 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-28T12:49:08.861 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-28T12:49:08.862 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-28T12:49:08.862 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-28T12:49:08.862 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-28T12:49:08.862 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-28T12:49:08.862 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-28T12:49:08.862 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-28T12:49:08.862 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-28T12:49:08.862 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-28T12:49:08.862 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-28T12:49:08.862 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-28T12:49:08.862 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-28T12:49:08.862 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-28T12:49:08.863 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-28T12:49:08.863 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-28T12:49:08.863 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-28T12:49:08.863 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-28T12:49:08.863 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-28T12:49:08.863 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-28T12:49:08.863 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-28T12:49:08.863 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-28T12:49:08.863 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-28T12:49:08.863 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-28T12:49:08.863 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-28T12:49:08.863 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-28T12:49:08.863 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-28T12:49:08.864 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-28T12:49:08.864 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-28T12:49:08.864 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-28T12:49:08.864 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-28T12:49:08.864 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-28T12:49:08.864 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-28T12:49:08.864 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-28T12:49:08.864 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-28T12:49:08.864 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-28T12:49:08.864 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-28T12:49:08.864 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-28T12:49:08.864 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-28T12:49:08.865 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-28T12:49:08.865 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-28T12:49:08.865 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-28T12:49:08.865 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-28T12:49:08.865 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-28T12:49:08.865 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-28T12:49:08.865 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-28T12:49:08.865 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-28T12:49:08.865 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-28T12:49:08.865 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-28T12:49:08.865 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-28T12:49:08.865 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-28T12:49:08.865 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-28T12:49:08.866 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-28T12:49:08.866 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-28T12:49:08.866 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-28T12:49:08.866 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-28T12:49:08.866 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-28T12:49:08.866 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-28T12:49:08.866 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-28T12:49:08.866 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-28T12:49:08.866 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-28T12:49:08.866 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-28T12:49:08.866 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-28T12:49:08.867 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-28T12:49:08.867 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-28T12:49:08.867 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-28T12:49:08.867 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-28T12:49:08.867 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-28T12:49:08.867 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-28T12:49:08.867 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-28T12:49:08.867 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-28T12:49:08.867 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-28T12:49:08.867 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-28T12:49:08.867 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-28T12:49:08.867 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-28T12:49:08.867 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-28T12:49:08.868 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-28T12:49:08.868 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-28T12:49:08.868 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-28T12:49:08.868 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-28T12:49:08.868 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-28T12:49:08.868 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-28T12:49:08.868 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-28T12:49:08.868 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-28T12:49:08.868 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-28T12:49:08.868 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-28T12:49:08.868 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-28T12:49:08.868 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-28T12:49:08.869 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-28T12:49:08.869 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-28T12:49:08.869 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-28T12:49:08.869 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-28T12:49:08.869 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-28T12:49:08.869 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-28T12:49:08.869 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-28T12:49:08.869 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-28T12:49:08.869 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-28T12:49:08.869 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-28T12:49:08.869 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-28T12:49:08.869 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-28T12:49:08.869 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-28T12:49:08.870 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-28T12:49:08.870 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-28T12:49:08.870 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-28T12:49:08.870 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-28T12:49:08.870 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-28T12:49:08.870 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-28T12:49:08.870 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-28T12:49:08.870 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-28T12:49:08.870 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-28T12:49:08.870 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-28T12:49:08.870 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-28T12:49:08.870 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-28T12:49:08.871 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-28T12:49:08.871 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-28T12:49:08.871 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-28T12:49:08.871 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-28T12:49:08.871 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-28T12:49:08.871 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-28T12:49:08.871 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-28T12:49:08.871 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-28T12:49:08.871 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-28T12:49:08.871 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-28T12:49:08.871 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-28T12:49:08.871 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-28T12:49:08.872 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-28T12:49:08.872 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-28T12:49:08.872 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-28T12:49:08.872 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-28T12:49:08.872 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-28T12:49:08.872 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-28T12:49:08.872 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-28T12:49:08.872 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-28T12:49:08.872 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-28T12:49:08.872 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-28T12:49:08.872 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-28T12:49:08.872 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-28T12:49:08.872 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-28T12:49:08.873 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-28T12:49:08.873 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-28T12:49:08.873 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-28T12:49:08.873 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-28T12:49:08.873 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-28T12:49:08.873 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-28T12:49:08.873 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-28T12:49:08.873 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-28T12:49:08.873 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-28T12:49:08.873 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-28T12:49:08.873 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-28T12:49:08.874 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-28T12:49:08.874 DEBUG:teuthology.orchestra.run.smithi156:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-28T12:49:08.902 INFO:teuthology.orchestra.run.smithi156.stdout:postinstall scriptlet (using /bin/sh): 2023-11-28T12:49:08.902 INFO:teuthology.orchestra.run.smithi156.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-28T12:49:08.902 INFO:teuthology.orchestra.run.smithi156.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-28T12:49:08.902 INFO:teuthology.orchestra.run.smithi156.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T12:49:08.902 INFO:teuthology.orchestra.run.smithi156.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-28T12:49:08.902 INFO:teuthology.orchestra.run.smithi156.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T12:49:08.902 INFO:teuthology.orchestra.run.smithi156.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T12:49:08.902 INFO:teuthology.orchestra.run.smithi156.stdout:fi 2023-11-28T12:49:08.903 INFO:teuthology.orchestra.run.smithi156.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-28T12:49:08.903 INFO:teuthology.orchestra.run.smithi156.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-28T12:49:08.903 INFO:teuthology.orchestra.run.smithi156.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-28T12:49:08.903 INFO:teuthology.orchestra.run.smithi156.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-28T12:49:08.903 INFO:teuthology.orchestra.run.smithi156.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-28T12:49:08.903 INFO:teuthology.orchestra.run.smithi156.stdout:fi 2023-11-28T12:49:08.903 INFO:teuthology.orchestra.run.smithi156.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-28T12:49:08.903 INFO:teuthology.orchestra.run.smithi156.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-28T12:49:08.903 INFO:teuthology.orchestra.run.smithi156.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-28T12:49:08.903 INFO:teuthology.orchestra.run.smithi156.stdout:fi 2023-11-28T12:49:08.904 DEBUG:teuthology.orchestra.run.smithi156:> sudo rpm -qi grub2-tools 2023-11-28T12:49:09.077 INFO:teuthology.orchestra.run.smithi156.stdout:Name : grub2-tools 2023-11-28T12:49:09.077 INFO:teuthology.orchestra.run.smithi156.stdout:Epoch : 1 2023-11-28T12:49:09.077 INFO:teuthology.orchestra.run.smithi156.stdout:Version : 2.02 2023-11-28T12:49:09.078 INFO:teuthology.orchestra.run.smithi156.stdout:Release : 123.el8 2023-11-28T12:49:09.078 INFO:teuthology.orchestra.run.smithi156.stdout:Architecture: x86_64 2023-11-28T12:49:09.078 INFO:teuthology.orchestra.run.smithi156.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-11-28T12:49:09.078 INFO:teuthology.orchestra.run.smithi156.stdout:Group : System Environment/Base 2023-11-28T12:49:09.078 INFO:teuthology.orchestra.run.smithi156.stdout:Size : 9265450 2023-11-28T12:49:09.078 INFO:teuthology.orchestra.run.smithi156.stdout:License : GPLv3+ 2023-11-28T12:49:09.078 INFO:teuthology.orchestra.run.smithi156.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-11-28T12:49:09.078 INFO:teuthology.orchestra.run.smithi156.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-11-28T12:49:09.078 INFO:teuthology.orchestra.run.smithi156.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-11-28T12:49:09.078 INFO:teuthology.orchestra.run.smithi156.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-11-28T12:49:09.078 INFO:teuthology.orchestra.run.smithi156.stdout:Relocations : (not relocatable) 2023-11-28T12:49:09.079 INFO:teuthology.orchestra.run.smithi156.stdout:Packager : Red Hat, Inc. 2023-11-28T12:49:09.079 INFO:teuthology.orchestra.run.smithi156.stdout:Vendor : Red Hat, Inc. 2023-11-28T12:49:09.079 INFO:teuthology.orchestra.run.smithi156.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-28T12:49:09.079 INFO:teuthology.orchestra.run.smithi156.stdout:Summary : Support tools for GRUB. 2023-11-28T12:49:09.079 INFO:teuthology.orchestra.run.smithi156.stdout:Description : 2023-11-28T12:49:09.079 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T12:49:09.079 INFO:teuthology.orchestra.run.smithi156.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-28T12:49:09.079 INFO:teuthology.orchestra.run.smithi156.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-28T12:49:09.079 INFO:teuthology.orchestra.run.smithi156.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-28T12:49:09.079 INFO:teuthology.orchestra.run.smithi156.stdout:hardware devices. 2023-11-28T12:49:09.079 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T12:49:09.079 INFO:teuthology.orchestra.run.smithi156.stdout:This subpackage provides tools for support of all platforms. 2023-11-28T12:49:09.080 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-28T12:49:09.080 INFO:teuthology.task.kernel:Updating grub on smithi156 to boot 6.6.0-g8247bbb5b470 2023-11-28T12:49:09.081 DEBUG:teuthology.orchestra.run.smithi156:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-28T12:49:11.172 INFO:teuthology.orchestra.run.smithi156.stderr:Generating grub configuration file ... 2023-11-28T12:49:13.361 INFO:teuthology.orchestra.run.smithi156.stderr:done 2023-11-28T12:49:13.363 DEBUG:teuthology.orchestra.run.smithi156:> mktemp 2023-11-28T12:49:13.387 INFO:teuthology.orchestra.run.smithi156.stdout:/tmp/tmp.SKG1iYKDfn 2023-11-28T12:49:13.388 DEBUG:teuthology.orchestra.run.smithi156:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.SKG1iYKDfn 2023-11-28T12:49:13.461 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0666 /tmp/tmp.SKG1iYKDfn 2023-11-28T12:49:13.611 DEBUG:teuthology.orchestra.remote:smithi156:/tmp/tmp.SKG1iYKDfn is 6KB 2023-11-28T12:49:13.660 DEBUG:teuthology.orchestra.run.smithi156:> rm -fr /tmp/tmp.SKG1iYKDfn 2023-11-28T12:49:13.677 DEBUG:teuthology.orchestra.run.smithi156:> sudo /bin/ls /boot/loader/entries || true 2023-11-28T12:49:13.744 INFO:teuthology.orchestra.run.smithi156.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-11-28T12:49:13.744 INFO:teuthology.orchestra.run.smithi156.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-11-28T12:49:13.744 INFO:teuthology.orchestra.run.smithi156.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.6.0-g8247bbb5b470.conf 2023-11-28T12:49:13.745 DEBUG:teuthology.orchestra.run.smithi156:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.6.0-g8247bbb5b470 2023-11-28T12:49:13.895 DEBUG:teuthology.orchestra.run.smithi156:> sudo shutdown -r now 2023-11-28T12:49:35.918 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-11-28T12:49:35.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-11-28T12:49:35.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T12:49:43.906 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-11-28T12:49:43.906 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-11-28T12:49:43.907 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T12:50:10.406 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.186 2023-11-28T12:50:19.414 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-11-28T12:50:19.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T12:50:43.932 DEBUG:teuthology.orchestra.remote:timed out 2023-11-28T12:50:52.942 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-11-28T12:50:52.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T12:51:08.615 DEBUG:teuthology.orchestra.run.smithi156:> true 2023-11-28T12:51:11.665 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-11-28T12:51:11.665 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-g8247bbb5b470"... 2023-11-28T12:51:11.665 DEBUG:teuthology.orchestra.run.smithi156:> uname -r 2023-11-28T12:51:11.708 INFO:teuthology.orchestra.run.smithi156.stdout:6.6.0-g8247bbb5b470 2023-11-28T12:51:11.708 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-28T12:51:11.708 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-28T12:51:11.708 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-11-28T12:51:12.709 DEBUG:teuthology.parallel:result is None 2023-11-28T12:51:19.417 DEBUG:teuthology.orchestra.remote:timed out 2023-11-28T12:51:31.418 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-11-28T12:51:31.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T12:51:31.792 DEBUG:teuthology.orchestra.run.smithi186:> true 2023-11-28T12:51:32.244 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-11-28T12:51:32.244 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-g8247bbb5b470"... 2023-11-28T12:51:32.245 DEBUG:teuthology.orchestra.run.smithi186:> uname -r 2023-11-28T12:51:32.287 INFO:teuthology.orchestra.run.smithi186.stdout:6.6.0-g8247bbb5b470 2023-11-28T12:51:32.288 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-28T12:51:32.288 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-28T12:51:32.288 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-11-28T12:51:33.289 DEBUG:teuthology.parallel:result is None 2023-11-28T12:51:33.290 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-28T12:51:33.300 INFO:teuthology.task.internal:Creating test directory... 2023-11-28T12:51:33.300 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T12:51:33.305 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T12:51:33.353 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-28T12:51:33.361 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-28T12:51:33.369 INFO:teuthology.task.internal:Creating archive directory... 2023-11-28T12:51:33.369 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T12:51:33.374 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T12:51:33.443 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-28T12:51:33.450 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-28T12:51:33.450 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T12:51:33.455 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T12:51:33.571 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T12:51:33.589 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T12:51:33.603 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T12:51:33.619 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T12:51:33.623 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-28T12:51:33.630 INFO:teuthology.task.internal:Configuring sudo... 2023-11-28T12:51:33.631 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T12:51:33.635 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T12:51:33.703 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-28T12:51:33.713 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-28T12:51:33.713 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T12:51:33.717 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T12:51:33.760 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T12:51:33.871 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T12:51:33.954 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T12:51:34.028 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T12:51:34.096 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T12:51:34.097 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T12:51:34.176 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T12:51:34.248 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T12:51:34.317 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T12:51:34.393 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T12:51:34.473 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T12:51:34.473 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T12:51:34.541 DEBUG:teuthology.orchestra.run.smithi156:> sudo service rsyslog restart 2023-11-28T12:51:34.545 DEBUG:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2023-11-28T12:51:34.664 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T12:51:34.698 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T12:51:35.014 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-28T12:51:35.022 INFO:teuthology.task.internal:Starting timer... 2023-11-28T12:51:35.022 INFO:teuthology.run_tasks:Running task pcp... 2023-11-28T12:51:35.033 INFO:teuthology.run_tasks:Running task selinux... 2023-11-28T12:51:35.047 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-28T12:51:35.048 DEBUG:teuthology.orchestra.run.smithi156:> sudo service auditd rotate 2023-11-28T12:51:35.159 INFO:teuthology.orchestra.run.smithi156.stdout:Rotating logs: 2023-11-28T12:51:35.163 DEBUG:teuthology.orchestra.run.smithi186:> sudo service auditd rotate 2023-11-28T12:51:35.292 INFO:teuthology.orchestra.run.smithi186.stdout:Rotating logs: 2023-11-28T12:51:35.295 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-28T12:51:35.296 DEBUG:teuthology.orchestra.run.smithi156:> /usr/sbin/getenforce 2023-11-28T12:51:35.348 INFO:teuthology.orchestra.run.smithi156.stdout:Permissive 2023-11-28T12:51:35.349 DEBUG:teuthology.orchestra.run.smithi186:> /usr/sbin/getenforce 2023-11-28T12:51:35.405 INFO:teuthology.orchestra.run.smithi186.stdout:Permissive 2023-11-28T12:51:35.406 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi156.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi186.front.sepia.ceph.com': 'permissive'} 2023-11-28T12:51:35.406 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T12:51:35.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:51:35.482 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T12:51:35.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:51:35.553 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-28T12:51:35.553 DEBUG:teuthology.orchestra.run.smithi156:> sudo /usr/sbin/setenforce permissive 2023-11-28T12:51:35.624 DEBUG:teuthology.orchestra.run.smithi186:> sudo /usr/sbin/setenforce permissive 2023-11-28T12:51:35.695 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-28T12:51:35.706 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-28T12:51:35.706 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-28T12:51:35.723 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-28T12:51:35.724 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi156.front.sepia.ceph.com,smithi186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-28T13:04:32.106 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi156.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com')] 2023-11-28T13:04:32.107 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-11-28T13:04:32.108 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T13:04:32.195 DEBUG:teuthology.orchestra.run.smithi156:> true 2023-11-28T13:04:32.308 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-11-28T13:04:32.308 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-11-28T13:04:32.309 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T13:04:32.398 DEBUG:teuthology.orchestra.run.smithi186:> true 2023-11-28T13:04:32.511 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-11-28T13:04:32.511 INFO:teuthology.run_tasks:Running task clock... 2023-11-28T13:04:32.526 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-28T13:04:32.526 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T13:04:32.527 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T13:04:32.531 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T13:04:32.531 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T13:04:32.601 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T13:04:32.626 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T13:04:32.636 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T13:04:32.661 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T13:04:32.729 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: ntpd: command not found 2023-11-28T13:04:32.755 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: ntpd: command not found 2023-11-28T13:04:32.759 INFO:teuthology.orchestra.run.smithi156.stdout:506 Cannot talk to daemon 2023-11-28T13:04:32.785 INFO:teuthology.orchestra.run.smithi186.stdout:506 Cannot talk to daemon 2023-11-28T13:04:32.793 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T13:04:32.819 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T13:04:32.828 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T13:04:32.853 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T13:04:32.919 INFO:teuthology.orchestra.run.smithi156.stderr:bash: ntpq: command not found 2023-11-28T13:04:32.926 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T13:04:32.926 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2023-11-28T13:04:32.926 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T13:04:32.926 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T13:04:32.926 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T13:04:32.927 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T13:04:32.945 INFO:teuthology.orchestra.run.smithi186.stderr:bash: ntpq: command not found 2023-11-28T13:04:32.953 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T13:04:32.953 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2023-11-28T13:04:32.953 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T13:04:32.953 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T13:04:32.953 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T13:04:32.953 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T13:04:32.954 INFO:teuthology.run_tasks:Running task install... 2023-11-28T13:04:32.965 DEBUG:teuthology.task.install:project ceph 2023-11-28T13:04:32.965 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}} 2023-11-28T13:04:32.965 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'} 2023-11-28T13:04:32.965 INFO:teuthology.task.install:Using flavor: default 2023-11-28T13:04:32.970 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-28T13:04:32.970 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-28T13:04:32.971 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T13:04:32.971 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T13:04:32.973 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T13:04:32.973 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T13:04:33.123 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T13:04:33.123 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T13:04:33.215 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T13:04:33.215 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T13:04:33.310 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T13:04:33.310 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:04:33.310 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T13:04:33.381 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T13:04:33.382 DEBUG:teuthology.orchestra.run.smithi186:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T13:04:33.385 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T13:04:33.385 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:04:33.385 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T13:04:33.455 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T13:04:33.455 DEBUG:teuthology.orchestra.run.smithi156:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T13:04:33.492 DEBUG:teuthology.orchestra.run.smithi186:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T13:04:33.566 DEBUG:teuthology.orchestra.run.smithi156:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T13:04:33.588 DEBUG:teuthology.orchestra.run.smithi186:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T13:04:33.655 INFO:teuthology.orchestra.run.smithi186.stdout:check_obsoletes = 1 2023-11-28T13:04:33.658 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum clean all 2023-11-28T13:04:33.664 DEBUG:teuthology.orchestra.run.smithi156:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T13:04:33.733 INFO:teuthology.orchestra.run.smithi156.stdout:check_obsoletes = 1 2023-11-28T13:04:33.736 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum clean all 2023-11-28T13:04:34.110 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-11-28T13:04:34.186 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-11-28T13:04:34.469 INFO:teuthology.orchestra.run.smithi186.stdout:44 files removed 2023-11-28T13:04:34.502 INFO:teuthology.orchestra.run.smithi156.stdout:44 files removed 2023-11-28T13:04:34.530 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T13:04:34.558 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T13:04:34.981 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-11-28T13:04:35.007 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-11-28T13:04:35.786 INFO:teuthology.orchestra.run.smithi156.stdout:ceph packages for x86_64 219 kB/s | 80 kB 00:00 2023-11-28T13:04:35.796 INFO:teuthology.orchestra.run.smithi186.stdout:ceph packages for x86_64 206 kB/s | 80 kB 00:00 2023-11-28T13:04:36.106 INFO:teuthology.orchestra.run.smithi186.stdout:ceph noarch packages 34 kB/s | 9.4 kB 00:00 2023-11-28T13:04:36.162 INFO:teuthology.orchestra.run.smithi156.stdout:ceph noarch packages 27 kB/s | 9.4 kB 00:00 2023-11-28T13:04:36.499 INFO:teuthology.orchestra.run.smithi186.stdout:ceph source packages 4.0 kB/s | 1.5 kB 00:00 2023-11-28T13:04:36.512 INFO:teuthology.orchestra.run.smithi156.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2023-11-28T13:04:36.674 INFO:teuthology.orchestra.run.smithi186.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-11-28T13:04:36.679 INFO:teuthology.orchestra.run.smithi156.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-11-28T13:04:37.683 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:00 2023-11-28T13:04:38.633 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 8.3 MB/s | 16 MB 00:01 2023-11-28T13:04:41.884 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 324 kB/s | 24 kB 00:00 2023-11-28T13:04:42.830 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 395 kB/s | 24 kB 00:00 2023-11-28T13:04:42.977 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 52 MB/s | 53 MB 00:01 2023-11-28T13:04:44.007 INFO:teuthology.orchestra.run.smithi156.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 53 MB 00:01 2023-11-28T13:04:51.594 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 51 MB/s | 47 MB 00:00 2023-11-28T13:04:52.858 INFO:teuthology.orchestra.run.smithi156.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 MB/s | 47 MB 00:01 2023-11-28T13:05:03.071 INFO:teuthology.orchestra.run.smithi186.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T13:05:03.072 INFO:teuthology.orchestra.run.smithi186.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T13:05:03.088 INFO:teuthology.orchestra.run.smithi186.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-28T13:05:03.089 INFO:teuthology.orchestra.run.smithi186.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-28T13:05:03.090 INFO:teuthology.orchestra.run.smithi186.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-28T13:05:03.092 INFO:teuthology.orchestra.run.smithi186.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T13:05:03.094 INFO:teuthology.orchestra.run.smithi186.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T13:05:03.097 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T13:05:03.099 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T13:05:03.242 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-11-28T13:05:03.255 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-11-28T13:05:03.255 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-11-28T13:05:03.255 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-11-28T13:05:03.255 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-11-28T13:05:03.255 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-28T13:05:03.256 INFO:teuthology.orchestra.run.smithi186.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T13:05:03.256 INFO:teuthology.orchestra.run.smithi186.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T13:05:03.256 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T13:05:03.256 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T13:05:03.256 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T13:05:03.256 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T13:05:03.256 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T13:05:03.257 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T13:05:03.257 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T13:05:03.257 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T13:05:03.257 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T13:05:03.257 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T13:05:03.257 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T13:05:03.258 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T13:05:03.258 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T13:05:03.258 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T13:05:03.258 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-28T13:05:03.258 INFO:teuthology.orchestra.run.smithi186.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T13:05:03.258 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T13:05:03.258 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T13:05:03.258 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T13:05:03.259 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T13:05:03.259 INFO:teuthology.orchestra.run.smithi186.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T13:05:03.259 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-28T13:05:03.259 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T13:05:03.259 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-28T13:05:03.259 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-28T13:05:03.259 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-28T13:05:03.259 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T13:05:03.259 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T13:05:03.259 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T13:05:03.260 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T13:05:03.260 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T13:05:03.260 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T13:05:03.260 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T13:05:03.260 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-28T13:05:03.260 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrading: 2023-11-28T13:05:03.260 INFO:teuthology.orchestra.run.smithi186.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-28T13:05:03.260 INFO:teuthology.orchestra.run.smithi186.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T13:05:03.260 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T13:05:03.260 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-11-28T13:05:03.260 INFO:teuthology.orchestra.run.smithi186.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T13:05:03.260 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T13:05:03.261 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T13:05:03.261 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T13:05:03.261 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T13:05:03.261 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T13:05:03.261 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T13:05:03.261 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T13:05:03.261 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T13:05:03.261 INFO:teuthology.orchestra.run.smithi186.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T13:05:03.261 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T13:05:03.261 INFO:teuthology.orchestra.run.smithi186.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-28T13:05:03.261 INFO:teuthology.orchestra.run.smithi186.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T13:05:03.261 INFO:teuthology.orchestra.run.smithi186.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-28T13:05:03.262 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T13:05:03.262 INFO:teuthology.orchestra.run.smithi186.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-28T13:05:03.262 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-28T13:05:03.262 INFO:teuthology.orchestra.run.smithi186.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T13:05:03.262 INFO:teuthology.orchestra.run.smithi186.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-28T13:05:03.262 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-28T13:05:03.262 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T13:05:03.262 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T13:05:03.262 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-28T13:05:03.262 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T13:05:03.263 INFO:teuthology.orchestra.run.smithi186.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T13:05:03.263 INFO:teuthology.orchestra.run.smithi186.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-28T13:05:03.263 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T13:05:03.263 INFO:teuthology.orchestra.run.smithi186.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T13:05:03.263 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T13:05:03.263 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T13:05:03.263 INFO:teuthology.orchestra.run.smithi186.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T13:05:03.264 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-28T13:05:03.264 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T13:05:03.264 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T13:05:03.264 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T13:05:03.264 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T13:05:03.265 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T13:05:03.265 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T13:05:03.265 INFO:teuthology.orchestra.run.smithi186.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T13:05:03.265 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T13:05:03.265 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T13:05:03.265 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T13:05:03.266 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T13:05:03.266 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-28T13:05:03.266 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T13:05:03.266 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T13:05:03.266 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T13:05:03.266 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T13:05:03.266 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T13:05:03.267 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T13:05:03.267 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T13:05:03.267 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T13:05:03.267 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T13:05:03.267 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T13:05:03.267 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T13:05:03.267 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T13:05:03.268 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T13:05:03.268 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T13:05:03.268 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T13:05:03.268 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T13:05:03.268 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T13:05:03.268 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T13:05:03.269 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T13:05:03.269 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T13:05:03.269 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T13:05:03.269 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T13:05:03.269 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-28T13:05:03.269 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T13:05:03.269 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T13:05:03.270 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-28T13:05:03.270 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T13:05:03.270 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T13:05:03.270 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T13:05:03.270 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T13:05:03.270 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T13:05:03.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-28T13:05:03.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T13:05:03.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T13:05:03.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T13:05:03.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T13:05:03.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T13:05:03.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T13:05:03.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T13:05:03.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T13:05:03.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T13:05:03.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T13:05:03.272 INFO:teuthology.orchestra.run.smithi186.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-28T13:05:03.272 INFO:teuthology.orchestra.run.smithi186.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T13:05:03.272 INFO:teuthology.orchestra.run.smithi186.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-28T13:05:03.272 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T13:05:03.272 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T13:05:03.272 INFO:teuthology.orchestra.run.smithi186.stdout:Enabling module streams: 2023-11-28T13:05:03.272 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql 10 2023-11-28T13:05:03.272 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:05:03.272 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-11-28T13:05:03.272 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-11-28T13:05:03.272 INFO:teuthology.orchestra.run.smithi186.stdout:Install 128 Packages 2023-11-28T13:05:03.272 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade 3 Packages 2023-11-28T13:05:03.272 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:05:03.273 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 203 M 2023-11-28T13:05:03.273 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-11-28T13:05:03.678 INFO:teuthology.orchestra.run.smithi186.stdout:(1/131): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 30 kB/s | 6.4 kB 00:00 2023-11-28T13:05:04.053 INFO:teuthology.orchestra.run.smithi186.stdout:(2/131): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 9.2 MB/s | 5.4 MB 00:00 2023-11-28T13:05:04.104 INFO:teuthology.orchestra.run.smithi186.stdout:(3/131): ceph-immutable-object-cache-18.0.0-269 3.3 MB/s | 165 kB 00:00 2023-11-28T13:05:04.162 INFO:teuthology.orchestra.run.smithi186.stdout:(4/131): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.7 MB/s | 845 kB 00:00 2023-11-28T13:05:04.279 INFO:teuthology.orchestra.run.smithi186.stdout:(5/131): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 13 MB/s | 2.3 MB 00:00 2023-11-28T13:05:04.550 INFO:teuthology.orchestra.run.smithi156.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T13:05:04.551 INFO:teuthology.orchestra.run.smithi156.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T13:05:04.566 INFO:teuthology.orchestra.run.smithi156.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-28T13:05:04.568 INFO:teuthology.orchestra.run.smithi156.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-28T13:05:04.569 INFO:teuthology.orchestra.run.smithi156.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-28T13:05:04.570 INFO:teuthology.orchestra.run.smithi156.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T13:05:04.572 INFO:teuthology.orchestra.run.smithi156.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T13:05:04.575 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T13:05:04.577 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T13:05:04.719 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-11-28T13:05:04.732 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-11-28T13:05:04.732 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2023-11-28T13:05:04.732 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-11-28T13:05:04.732 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2023-11-28T13:05:04.732 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-28T13:05:04.732 INFO:teuthology.orchestra.run.smithi156.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T13:05:04.732 INFO:teuthology.orchestra.run.smithi156.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T13:05:04.732 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T13:05:04.732 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T13:05:04.732 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T13:05:04.733 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T13:05:04.733 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T13:05:04.733 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T13:05:04.733 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T13:05:04.733 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T13:05:04.733 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T13:05:04.733 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T13:05:04.733 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T13:05:04.733 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T13:05:04.733 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T13:05:04.733 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T13:05:04.734 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-28T13:05:04.734 INFO:teuthology.orchestra.run.smithi156.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T13:05:04.734 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T13:05:04.734 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T13:05:04.734 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T13:05:04.734 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T13:05:04.734 INFO:teuthology.orchestra.run.smithi156.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T13:05:04.734 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-28T13:05:04.734 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T13:05:04.734 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-28T13:05:04.734 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-28T13:05:04.734 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-28T13:05:04.735 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T13:05:04.735 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T13:05:04.735 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T13:05:04.735 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T13:05:04.735 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T13:05:04.735 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T13:05:04.735 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T13:05:04.735 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-28T13:05:04.735 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrading: 2023-11-28T13:05:04.735 INFO:teuthology.orchestra.run.smithi156.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-28T13:05:04.735 INFO:teuthology.orchestra.run.smithi156.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T13:05:04.735 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T13:05:04.736 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2023-11-28T13:05:04.736 INFO:teuthology.orchestra.run.smithi156.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T13:05:04.736 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T13:05:04.736 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T13:05:04.736 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T13:05:04.736 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T13:05:04.736 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T13:05:04.736 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T13:05:04.736 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T13:05:04.736 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T13:05:04.736 INFO:teuthology.orchestra.run.smithi156.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T13:05:04.736 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T13:05:04.737 INFO:teuthology.orchestra.run.smithi156.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-28T13:05:04.737 INFO:teuthology.orchestra.run.smithi156.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T13:05:04.737 INFO:teuthology.orchestra.run.smithi156.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-28T13:05:04.737 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T13:05:04.737 INFO:teuthology.orchestra.run.smithi156.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-28T13:05:04.737 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-28T13:05:04.737 INFO:teuthology.orchestra.run.smithi156.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T13:05:04.737 INFO:teuthology.orchestra.run.smithi156.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-28T13:05:04.737 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-28T13:05:04.737 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T13:05:04.737 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T13:05:04.737 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-28T13:05:04.738 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T13:05:04.738 INFO:teuthology.orchestra.run.smithi156.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T13:05:04.738 INFO:teuthology.orchestra.run.smithi156.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-28T13:05:04.738 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T13:05:04.738 INFO:teuthology.orchestra.run.smithi156.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T13:05:04.738 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T13:05:04.738 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T13:05:04.738 INFO:teuthology.orchestra.run.smithi156.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T13:05:04.738 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-28T13:05:04.738 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T13:05:04.738 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T13:05:04.738 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T13:05:04.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T13:05:04.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T13:05:04.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T13:05:04.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T13:05:04.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T13:05:04.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T13:05:04.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T13:05:04.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T13:05:04.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-28T13:05:04.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T13:05:04.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T13:05:04.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T13:05:04.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T13:05:04.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T13:05:04.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T13:05:04.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T13:05:04.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T13:05:04.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T13:05:04.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T13:05:04.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T13:05:04.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T13:05:04.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T13:05:04.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T13:05:04.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T13:05:04.741 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T13:05:04.741 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T13:05:04.741 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T13:05:04.741 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T13:05:04.741 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T13:05:04.741 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T13:05:04.741 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T13:05:04.741 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-28T13:05:04.741 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T13:05:04.741 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T13:05:04.741 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-28T13:05:04.741 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T13:05:04.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T13:05:04.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T13:05:04.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T13:05:04.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T13:05:04.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-28T13:05:04.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T13:05:04.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T13:05:04.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T13:05:04.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T13:05:04.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T13:05:04.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T13:05:04.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T13:05:04.743 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T13:05:04.743 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T13:05:04.743 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T13:05:04.743 INFO:teuthology.orchestra.run.smithi156.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-28T13:05:04.743 INFO:teuthology.orchestra.run.smithi156.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T13:05:04.743 INFO:teuthology.orchestra.run.smithi156.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-28T13:05:04.743 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T13:05:04.743 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T13:05:04.743 INFO:teuthology.orchestra.run.smithi156.stdout:Enabling module streams: 2023-11-28T13:05:04.743 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql 10 2023-11-28T13:05:04.743 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:05:04.743 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-11-28T13:05:04.744 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-11-28T13:05:04.744 INFO:teuthology.orchestra.run.smithi156.stdout:Install 128 Packages 2023-11-28T13:05:04.744 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrade 3 Packages 2023-11-28T13:05:04.744 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:05:04.744 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 203 M 2023-11-28T13:05:04.744 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2023-11-28T13:05:04.792 INFO:teuthology.orchestra.run.smithi186.stdout:(6/131): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 9.0 MB/s | 4.6 MB 00:00 2023-11-28T13:05:04.838 INFO:teuthology.orchestra.run.smithi186.stdout:(7/131): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 2.3 MB/s | 1.5 MB 00:00 2023-11-28T13:05:05.650 INFO:teuthology.orchestra.run.smithi156.stdout:(1/131): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 14 kB/s | 6.4 kB 00:00 2023-11-28T13:05:06.038 INFO:teuthology.orchestra.run.smithi186.stdout:(8/131): ceph-osd-18.0.0-2697.g2f5f6028.el8.x86 14 MB/s | 18 MB 00:01 2023-11-28T13:05:06.105 INFO:teuthology.orchestra.run.smithi186.stdout:(9/131): ceph-selinux-18.0.0-2697.g2f5f6028.el8 368 kB/s | 24 kB 00:00 2023-11-28T13:05:06.167 INFO:teuthology.orchestra.run.smithi156.stdout:(2/131): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.6 MB/s | 845 kB 00:00 2023-11-28T13:05:06.259 INFO:teuthology.orchestra.run.smithi156.stdout:(3/131): ceph-immutable-object-cache-18.0.0-269 1.8 MB/s | 165 kB 00:00 2023-11-28T13:05:06.339 INFO:teuthology.orchestra.run.smithi186.stdout:(10/131): ceph-common-18.0.0-2697.g2f5f6028.el8 7.5 MB/s | 22 MB 00:02 2023-11-28T13:05:06.439 INFO:teuthology.orchestra.run.smithi186.stdout:(11/131): cephfs-mirror-18.0.0-2697.g2f5f6028.e 2.5 MB/s | 257 kB 00:00 2023-11-28T13:05:06.539 INFO:teuthology.orchestra.run.smithi186.stdout:(12/131): libcephfs-devel-18.0.0-2697.g2f5f6028 305 kB/s | 30 kB 00:00 2023-11-28T13:05:06.831 INFO:teuthology.orchestra.run.smithi186.stdout:(13/131): libcephfs2-18.0.0-2697.g2f5f6028.el8. 2.6 MB/s | 767 kB 00:00 2023-11-28T13:05:06.923 INFO:teuthology.orchestra.run.smithi186.stdout:(14/131): libcephsqlite-18.0.0-2697.g2f5f6028.e 1.5 MB/s | 137 kB 00:00 2023-11-28T13:05:07.032 INFO:teuthology.orchestra.run.smithi186.stdout:(15/131): librados-devel-18.0.0-2697.g2f5f6028. 1.3 MB/s | 139 kB 00:00 2023-11-28T13:05:07.134 INFO:teuthology.orchestra.run.smithi156.stdout:(4/131): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 2.6 MB/s | 2.3 MB 00:00 2023-11-28T13:05:07.149 INFO:teuthology.orchestra.run.smithi186.stdout:(16/131): libradosstriper1-18.0.0-2697.g2f5f602 4.0 MB/s | 474 kB 00:00 2023-11-28T13:05:07.851 INFO:teuthology.orchestra.run.smithi156.stdout:(5/131): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 2.2 MB/s | 1.5 MB 00:00 2023-11-28T13:05:08.299 INFO:teuthology.orchestra.run.smithi186.stdout:(17/131): librgw2-18.0.0-2697.g2f5f6028.el8.x86 5.6 MB/s | 6.5 MB 00:01 2023-11-28T13:05:08.416 INFO:teuthology.orchestra.run.smithi186.stdout:(18/131): python3-ceph-argparse-18.0.0-2697.g2f 383 kB/s | 45 kB 00:00 2023-11-28T13:05:08.533 INFO:teuthology.orchestra.run.smithi186.stdout:(19/131): python3-ceph-common-18.0.0-2697.g2f5f 1.0 MB/s | 117 kB 00:00 2023-11-28T13:05:08.625 INFO:teuthology.orchestra.run.smithi186.stdout:(20/131): python3-cephfs-18.0.0-2697.g2f5f6028. 2.0 MB/s | 190 kB 00:00 2023-11-28T13:05:08.708 INFO:teuthology.orchestra.run.smithi186.stdout:(21/131): python3-rados-18.0.0-2697.g2f5f6028.e 4.4 MB/s | 371 kB 00:00 2023-11-28T13:05:08.743 INFO:teuthology.orchestra.run.smithi156.stdout:(6/131): ceph-common-18.0.0-2697.g2f5f6028.el8. 6.0 MB/s | 22 MB 00:03 2023-11-28T13:05:08.800 INFO:teuthology.orchestra.run.smithi186.stdout:(22/131): python3-rbd-18.0.0-2697.g2f5f6028.el8 3.9 MB/s | 364 kB 00:00 2023-11-28T13:05:08.893 INFO:teuthology.orchestra.run.smithi186.stdout:(23/131): python3-rgw-18.0.0-2697.g2f5f6028.el8 1.1 MB/s | 107 kB 00:00 2023-11-28T13:05:09.001 INFO:teuthology.orchestra.run.smithi186.stdout:(24/131): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 874 kB/s | 94 kB 00:00 2023-11-28T13:05:09.535 INFO:teuthology.orchestra.run.smithi156.stdout:(7/131): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 2.7 MB/s | 4.6 MB 00:01 2023-11-28T13:05:09.576 INFO:teuthology.orchestra.run.smithi186.stdout:(25/131): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 6.0 MB/s | 3.4 MB 00:00 2023-11-28T13:05:09.767 INFO:teuthology.orchestra.run.smithi186.stdout:(26/131): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 950 kB/s | 180 kB 00:00 2023-11-28T13:05:09.793 INFO:teuthology.orchestra.run.smithi156.stdout:(8/131): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 1.2 MB/s | 5.4 MB 00:04 2023-11-28T13:05:09.844 INFO:teuthology.orchestra.run.smithi186.stdout:(27/131): ceph-grafana-dashboards-18.0.0-2697.g 300 kB/s | 23 kB 00:00 2023-11-28T13:05:09.868 INFO:teuthology.orchestra.run.smithi156.stdout:(9/131): ceph-selinux-18.0.0-2697.g2f5f6028.el8 325 kB/s | 24 kB 00:00 2023-11-28T13:05:09.927 INFO:teuthology.orchestra.run.smithi186.stdout:(28/131): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 1.4 MB/s | 123 kB 00:00 2023-11-28T13:05:10.461 INFO:teuthology.orchestra.run.smithi186.stdout:(29/131): ceph-test-18.0.0-2697.g2f5f6028.el8.x 11 MB/s | 47 MB 00:04 2023-11-28T13:05:10.511 INFO:teuthology.orchestra.run.smithi186.stdout:(30/131): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 3.0 MB/s | 1.7 MB 00:00 2023-11-28T13:05:10.611 INFO:teuthology.orchestra.run.smithi186.stdout:(31/131): ceph-mgr-modules-core-18.0.0-2697.g2f 2.3 MB/s | 236 kB 00:00 2023-11-28T13:05:10.704 INFO:teuthology.orchestra.run.smithi186.stdout:(32/131): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 516 kB/s | 47 kB 00:00 2023-11-28T13:05:10.770 INFO:teuthology.orchestra.run.smithi186.stdout:(33/131): ceph-prometheus-alerts-18.0.0-2697.g2 219 kB/s | 15 kB 00:00 2023-11-28T13:05:10.852 INFO:teuthology.orchestra.run.smithi156.stdout:(10/131): ceph-osd-18.0.0-2697.g2f5f6028.el8.x8 8.4 MB/s | 18 MB 00:02 2023-11-28T13:05:10.944 INFO:teuthology.orchestra.run.smithi156.stdout:(11/131): cephfs-mirror-18.0.0-2697.g2f5f6028.e 2.8 MB/s | 257 kB 00:00 2023-11-28T13:05:10.971 INFO:teuthology.orchestra.run.smithi186.stdout:(34/131): ceph-radosgw-18.0.0-2697.g2f5f6028.el 2.1 MB/s | 13 MB 00:06 2023-11-28T13:05:11.004 INFO:teuthology.orchestra.run.smithi186.stdout:(35/131): ceph-volume-18.0.0-2697.g2f5f6028.el8 1.1 MB/s | 263 kB 00:00 2023-11-28T13:05:11.019 INFO:teuthology.orchestra.run.smithi156.stdout:(12/131): libcephfs-devel-18.0.0-2697.g2f5f6028 407 kB/s | 30 kB 00:00 2023-11-28T13:05:11.038 INFO:teuthology.orchestra.run.smithi186.stdout:(36/131): cephadm-18.0.0-2697.g2f5f6028.el8.noa 1.2 MB/s | 83 kB 00:00 2023-11-28T13:05:11.055 INFO:teuthology.orchestra.run.smithi186.stdout:(37/131): cephfs-top-18.0.0-2697.g2f5f6028.el8. 359 kB/s | 18 kB 00:00 2023-11-28T13:05:11.128 INFO:teuthology.orchestra.run.smithi156.stdout:(13/131): libcephfs2-18.0.0-2697.g2f5f6028.el8. 6.9 MB/s | 767 kB 00:00 2023-11-28T13:05:11.147 INFO:teuthology.orchestra.run.smithi186.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 11 MB/s | 7.4 MB 00:00 2023-11-28T13:05:11.195 INFO:teuthology.orchestra.run.smithi156.stdout:(14/131): libcephsqlite-18.0.0-2697.g2f5f6028.e 2.0 MB/s | 137 kB 00:00 2023-11-28T13:05:11.253 INFO:teuthology.orchestra.run.smithi156.stdout:(15/131): librados-devel-18.0.0-2697.g2f5f6028. 2.3 MB/s | 139 kB 00:00 2023-11-28T13:05:11.346 INFO:teuthology.orchestra.run.smithi156.stdout:(16/131): libradosstriper1-18.0.0-2697.g2f5f602 5.1 MB/s | 474 kB 00:00 2023-11-28T13:05:11.589 INFO:teuthology.orchestra.run.smithi186.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 110 kB/s | 59 kB 00:00 2023-11-28T13:05:11.756 INFO:teuthology.orchestra.run.smithi186.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 427 kB/s | 306 kB 00:00 2023-11-28T13:05:11.773 INFO:teuthology.orchestra.run.smithi186.stdout:(41/131): libunwind-1.3.1-3.el8.x86_64.rpm 122 kB/s | 75 kB 00:00 2023-11-28T13:05:11.879 INFO:teuthology.orchestra.run.smithi156.stdout:(17/131): librgw2-18.0.0-2697.g2f5f6028.el8.x86 12 MB/s | 6.5 MB 00:00 2023-11-28T13:05:11.897 INFO:teuthology.orchestra.run.smithi186.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 309 kB/s | 44 kB 00:00 2023-11-28T13:05:11.938 INFO:teuthology.orchestra.run.smithi156.stdout:(18/131): python3-ceph-argparse-18.0.0-2697.g2f 766 kB/s | 45 kB 00:00 2023-11-28T13:05:11.965 INFO:teuthology.orchestra.run.smithi186.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 967 kB/s | 185 kB 00:00 2023-11-28T13:05:11.990 INFO:teuthology.orchestra.run.smithi186.stdout:(44/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.2 MB/s | 476 kB 00:00 2023-11-28T13:05:12.005 INFO:teuthology.orchestra.run.smithi156.stdout:(19/131): python3-ceph-common-18.0.0-2697.g2f5f 1.7 MB/s | 117 kB 00:00 2023-11-28T13:05:12.015 INFO:teuthology.orchestra.run.smithi186.stdout:(45/131): python3-cachetools-3.1.1-4.el8.noarch 280 kB/s | 33 kB 00:00 2023-11-28T13:05:12.066 INFO:teuthology.orchestra.run.smithi186.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 162 kB/s | 16 kB 00:00 2023-11-28T13:05:12.080 INFO:teuthology.orchestra.run.smithi156.stdout:(20/131): python3-cephfs-18.0.0-2697.g2f5f6028. 2.5 MB/s | 190 kB 00:00 2023-11-28T13:05:12.099 INFO:teuthology.orchestra.run.smithi186.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.6 MB/s | 173 kB 00:00 2023-11-28T13:05:12.200 INFO:teuthology.orchestra.run.smithi186.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 609 kB/s | 81 kB 00:00 2023-11-28T13:05:12.214 INFO:teuthology.orchestra.run.smithi156.stdout:(21/131): ceph-radosgw-18.0.0-2697.g2f5f6028.el 4.9 MB/s | 13 MB 00:02 2023-11-28T13:05:12.225 INFO:teuthology.orchestra.run.smithi186.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 89 kB/s | 11 kB 00:00 2023-11-28T13:05:12.247 INFO:teuthology.orchestra.run.smithi156.stdout:(22/131): python3-rados-18.0.0-2697.g2f5f6028.e 2.2 MB/s | 371 kB 00:00 2023-11-28T13:05:12.250 INFO:teuthology.orchestra.run.smithi186.stdout:(50/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.6 MB/s | 384 kB 00:00 2023-11-28T13:05:12.322 INFO:teuthology.orchestra.run.smithi156.stdout:(23/131): python3-rgw-18.0.0-2697.g2f5f6028.el8 1.4 MB/s | 107 kB 00:00 2023-11-28T13:05:12.356 INFO:teuthology.orchestra.run.smithi156.stdout:(24/131): python3-rbd-18.0.0-2697.g2f5f6028.el8 2.5 MB/s | 364 kB 00:00 2023-11-28T13:05:12.376 INFO:teuthology.orchestra.run.smithi186.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 100 kB/s | 18 kB 00:00 2023-11-28T13:05:12.381 INFO:teuthology.orchestra.run.smithi156.stdout:(25/131): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.6 MB/s | 94 kB 00:00 2023-11-28T13:05:12.401 INFO:teuthology.orchestra.run.smithi186.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 323 kB/s | 49 kB 00:00 2023-11-28T13:05:12.443 INFO:teuthology.orchestra.run.smithi186.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 5.4 MB/s | 1.2 MB 00:00 2023-11-28T13:05:12.457 INFO:teuthology.orchestra.run.smithi156.stdout:(26/131): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 2.4 MB/s | 180 kB 00:00 2023-11-28T13:05:12.485 INFO:teuthology.orchestra.run.smithi186.stdout:(54/131): python3-more-itertools-7.2.0-3.el8.no 540 kB/s | 59 kB 00:00 2023-11-28T13:05:12.518 INFO:teuthology.orchestra.run.smithi186.stdout:(55/131): python3-natsort-7.1.1-2.el8.noarch.rp 516 kB/s | 60 kB 00:00 2023-11-28T13:05:12.557 INFO:teuthology.orchestra.run.smithi156.stdout:(27/131): ceph-grafana-dashboards-18.0.0-2697.g 231 kB/s | 23 kB 00:00 2023-11-28T13:05:12.610 INFO:teuthology.orchestra.run.smithi186.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 124 kB/s | 16 kB 00:00 2023-11-28T13:05:12.636 INFO:teuthology.orchestra.run.smithi186.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 287 kB/s | 34 kB 00:00 2023-11-28T13:05:12.656 INFO:teuthology.orchestra.run.smithi156.stdout:(28/131): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 1.2 MB/s | 123 kB 00:00 2023-11-28T13:05:12.728 INFO:teuthology.orchestra.run.smithi186.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.6 MB/s | 196 kB 00:00 2023-11-28T13:05:12.753 INFO:teuthology.orchestra.run.smithi186.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 550 kB/s | 64 kB 00:00 2023-11-28T13:05:12.849 INFO:teuthology.orchestra.run.smithi156.stdout:(29/131): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 9.0 MB/s | 1.7 MB 00:00 2023-11-28T13:05:12.870 INFO:teuthology.orchestra.run.smithi186.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 132 kB/s | 19 kB 00:00 2023-11-28T13:05:12.903 INFO:teuthology.orchestra.run.smithi186.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 159 kB/s | 24 kB 00:00 2023-11-28T13:05:12.929 INFO:teuthology.orchestra.run.smithi186.stdout:(62/131): python3-pecan-1.3.2-9.el8.noarch.rpm 582 kB/s | 283 kB 00:00 2023-11-28T13:05:12.979 INFO:teuthology.orchestra.run.smithi186.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 269 kB/s | 29 kB 00:00 2023-11-28T13:05:13.004 INFO:teuthology.orchestra.run.smithi186.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 274 kB/s | 27 kB 00:00 2023-11-28T13:05:13.038 INFO:teuthology.orchestra.run.smithi186.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.3 MB/s | 255 kB 00:00 2023-11-28T13:05:13.088 INFO:teuthology.orchestra.run.smithi186.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2023-11-28T13:05:13.099 INFO:teuthology.orchestra.run.smithi156.stdout:(30/131): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 4.6 MB/s | 3.4 MB 00:00 2023-11-28T13:05:13.113 INFO:teuthology.orchestra.run.smithi186.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 564 kB/s | 61 kB 00:00 2023-11-28T13:05:13.147 INFO:teuthology.orchestra.run.smithi186.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 785 kB/s | 85 kB 00:00 2023-11-28T13:05:13.172 INFO:teuthology.orchestra.run.smithi186.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-11-28T13:05:13.183 INFO:teuthology.orchestra.run.smithi156.stdout:(31/131): ceph-mgr-modules-core-18.0.0-2697.g2f 2.8 MB/s | 236 kB 00:00 2023-11-28T13:05:13.197 INFO:teuthology.orchestra.run.smithi186.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 210 kB/s | 23 kB 00:00 2023-11-28T13:05:13.256 INFO:teuthology.orchestra.run.smithi186.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 3.4 MB/s | 200 kB 00:00 2023-11-28T13:05:13.266 INFO:teuthology.orchestra.run.smithi156.stdout:(32/131): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 570 kB/s | 47 kB 00:00 2023-11-28T13:05:13.281 INFO:teuthology.orchestra.run.smithi186.stdout:(72/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 431 kB/s | 47 kB 00:00 2023-11-28T13:05:13.306 INFO:teuthology.orchestra.run.smithi186.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 851 kB/s | 43 kB 00:00 2023-11-28T13:05:13.334 INFO:teuthology.orchestra.run.smithi156.stdout:(33/131): ceph-prometheus-alerts-18.0.0-2697.g2 218 kB/s | 15 kB 00:00 2023-11-28T13:05:13.365 INFO:teuthology.orchestra.run.smithi186.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.6 MB/s | 220 kB 00:00 2023-11-28T13:05:13.382 INFO:teuthology.orchestra.run.smithi186.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 642 kB/s | 48 kB 00:00 2023-11-28T13:05:13.424 INFO:teuthology.orchestra.run.smithi186.stdout:(76/131): thrift-0.13.0-2.el8.x86_64.rpm 5.6 MB/s | 1.7 MB 00:00 2023-11-28T13:05:13.449 INFO:teuthology.orchestra.run.smithi186.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.0 MB/s | 87 kB 00:00 2023-11-28T13:05:13.475 INFO:teuthology.orchestra.run.smithi186.stdout:(78/131): python3-requests-oauthlib-1.0.0-1.el8 464 kB/s | 43 kB 00:00 2023-11-28T13:05:13.500 INFO:teuthology.orchestra.run.smithi186.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2023-11-28T13:05:13.526 INFO:teuthology.orchestra.run.smithi186.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.2 MB/s | 229 kB 00:00 2023-11-28T13:05:13.551 INFO:teuthology.orchestra.run.smithi186.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 371 kB/s | 19 kB 00:00 2023-11-28T13:05:13.576 INFO:teuthology.orchestra.run.smithi186.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-11-28T13:05:13.618 INFO:teuthology.orchestra.run.smithi186.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.8 MB/s | 560 kB 00:00 2023-11-28T13:05:13.643 INFO:teuthology.orchestra.run.smithi186.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 268 kB/s | 18 kB 00:00 2023-11-28T13:05:13.677 INFO:teuthology.orchestra.run.smithi186.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.3 MB/s | 559 kB 00:00 2023-11-28T13:05:13.693 INFO:teuthology.orchestra.run.smithi156.stdout:(34/131): ceph-mgr-diskprediction-local-18.0.0- 8.8 MB/s | 7.4 MB 00:00 2023-11-28T13:05:13.710 INFO:teuthology.orchestra.run.smithi186.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-11-28T13:05:13.718 INFO:teuthology.orchestra.run.smithi156.stdout:(35/131): ceph-volume-18.0.0-2697.g2f5f6028.el8 685 kB/s | 263 kB 00:00 2023-11-28T13:05:13.744 INFO:teuthology.orchestra.run.smithi186.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2023-11-28T13:05:13.769 INFO:teuthology.orchestra.run.smithi186.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.2 MB/s | 72 kB 00:00 2023-11-28T13:05:13.777 INFO:teuthology.orchestra.run.smithi156.stdout:(36/131): cephadm-18.0.0-2697.g2f5f6028.el8.noa 1.4 MB/s | 83 kB 00:00 2023-11-28T13:05:13.794 INFO:teuthology.orchestra.run.smithi186.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-11-28T13:05:13.802 INFO:teuthology.orchestra.run.smithi156.stdout:(37/131): cephfs-top-18.0.0-2697.g2f5f6028.el8. 217 kB/s | 18 kB 00:00 2023-11-28T13:05:13.845 INFO:teuthology.orchestra.run.smithi186.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 798 kB/s | 60 kB 00:00 2023-11-28T13:05:13.870 INFO:teuthology.orchestra.run.smithi186.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.3 MB/s | 98 kB 00:00 2023-11-28T13:05:13.895 INFO:teuthology.orchestra.run.smithi186.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-11-28T13:05:13.921 INFO:teuthology.orchestra.run.smithi186.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 770 kB/s | 39 kB 00:00 2023-11-28T13:05:13.963 INFO:teuthology.orchestra.run.smithi186.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-11-28T13:05:13.977 INFO:teuthology.orchestra.run.smithi156.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 337 kB/s | 59 kB 00:00 2023-11-28T13:05:13.988 INFO:teuthology.orchestra.run.smithi186.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-11-28T13:05:14.013 INFO:teuthology.orchestra.run.smithi186.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-11-28T13:05:14.047 INFO:teuthology.orchestra.run.smithi186.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 3.3 MB/s | 110 kB 00:00 2023-11-28T13:05:14.077 INFO:teuthology.orchestra.run.smithi156.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 753 kB/s | 75 kB 00:00 2023-11-28T13:05:14.103 INFO:teuthology.orchestra.run.smithi156.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.0 MB/s | 306 kB 00:00 2023-11-28T13:05:14.105 INFO:teuthology.orchestra.run.smithi186.stdout:(98/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 5.4 MB/s | 2.3 MB 00:00 2023-11-28T13:05:14.131 INFO:teuthology.orchestra.run.smithi186.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 650 kB/s | 54 kB 00:00 2023-11-28T13:05:14.164 INFO:teuthology.orchestra.run.smithi186.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2023-11-28T13:05:14.170 INFO:teuthology.orchestra.run.smithi156.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 654 kB/s | 44 kB 00:00 2023-11-28T13:05:14.198 INFO:teuthology.orchestra.run.smithi186.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-11-28T13:05:14.223 INFO:teuthology.orchestra.run.smithi186.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 666 kB/s | 39 kB 00:00 2023-11-28T13:05:14.245 INFO:teuthology.orchestra.run.smithi156.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.8 MB/s | 476 kB 00:00 2023-11-28T13:05:14.265 INFO:teuthology.orchestra.run.smithi186.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-11-28T13:05:14.270 INFO:teuthology.orchestra.run.smithi156.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2023-11-28T13:05:14.304 INFO:teuthology.orchestra.run.smithi156.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 564 kB/s | 33 kB 00:00 2023-11-28T13:05:14.329 INFO:teuthology.orchestra.run.smithi156.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 279 kB/s | 16 kB 00:00 2023-11-28T13:05:14.332 INFO:teuthology.orchestra.run.smithi186.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 4.8 MB/s | 538 kB 00:00 2023-11-28T13:05:14.357 INFO:teuthology.orchestra.run.smithi186.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.7 MB/s | 157 kB 00:00 2023-11-28T13:05:14.371 INFO:teuthology.orchestra.run.smithi156.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2023-11-28T13:05:14.391 INFO:teuthology.orchestra.run.smithi186.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-11-28T13:05:14.430 INFO:teuthology.orchestra.run.smithi156.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.7 MB/s | 384 kB 00:00 2023-11-28T13:05:14.441 INFO:teuthology.orchestra.run.smithi186.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 8.4 MB/s | 713 kB 00:00 2023-11-28T13:05:14.455 INFO:teuthology.orchestra.run.smithi156.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 976 kB/s | 81 kB 00:00 2023-11-28T13:05:14.475 INFO:teuthology.orchestra.run.smithi186.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 799 kB/s | 27 kB 00:00 2023-11-28T13:05:14.489 INFO:teuthology.orchestra.run.smithi156.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-11-28T13:05:14.514 INFO:teuthology.orchestra.run.smithi156.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 301 kB/s | 18 kB 00:00 2023-11-28T13:05:14.525 INFO:teuthology.orchestra.run.smithi186.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-11-28T13:05:14.567 INFO:teuthology.orchestra.run.smithi186.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-11-28T13:05:14.573 INFO:teuthology.orchestra.run.smithi156.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 836 kB/s | 49 kB 00:00 2023-11-28T13:05:14.631 INFO:teuthology.orchestra.run.smithi156.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 8.2 MB/s | 1.2 MB 00:00 2023-11-28T13:05:14.657 INFO:teuthology.orchestra.run.smithi156.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 704 kB/s | 59 kB 00:00 2023-11-28T13:05:14.698 INFO:teuthology.orchestra.run.smithi156.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 904 kB/s | 60 kB 00:00 2023-11-28T13:05:14.709 INFO:teuthology.orchestra.run.smithi186.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2023-11-28T13:05:14.732 INFO:teuthology.orchestra.run.smithi156.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2023-11-28T13:05:14.792 INFO:teuthology.orchestra.run.smithi156.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 576 kB/s | 34 kB 00:00 2023-11-28T13:05:14.823 INFO:teuthology.orchestra.run.smithi186.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 3.9 MB/s | 457 kB 00:00 2023-11-28T13:05:14.858 INFO:teuthology.orchestra.run.smithi156.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 2.9 MB/s | 196 kB 00:00 2023-11-28T13:05:14.925 INFO:teuthology.orchestra.run.smithi156.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 967 kB/s | 64 kB 00:00 2023-11-28T13:05:14.950 INFO:teuthology.orchestra.run.smithi156.stdout:(59/131): python3-portend-2.6-1.el8.noarch.rpm 62 kB/s | 16 kB 00:00 2023-11-28T13:05:14.984 INFO:teuthology.orchestra.run.smithi156.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 318 kB/s | 19 kB 00:00 2023-11-28T13:05:15.010 INFO:teuthology.orchestra.run.smithi156.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 407 kB/s | 24 kB 00:00 2023-11-28T13:05:15.010 INFO:teuthology.orchestra.run.smithi186.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 8.4 MB/s | 3.7 MB 00:00 2023-11-28T13:05:15.043 INFO:teuthology.orchestra.run.smithi186.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.0 MB/s | 225 kB 00:00 2023-11-28T13:05:15.044 INFO:teuthology.orchestra.run.smithi156.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 500 kB/s | 29 kB 00:00 2023-11-28T13:05:15.068 INFO:teuthology.orchestra.run.smithi186.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2023-11-28T13:05:15.086 INFO:teuthology.orchestra.run.smithi156.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 363 kB/s | 27 kB 00:00 2023-11-28T13:05:15.094 INFO:teuthology.orchestra.run.smithi186.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-11-28T13:05:15.113 INFO:teuthology.orchestra.run.smithi186.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 478 kB/s | 20 kB 00:00 2023-11-28T13:05:15.118 INFO:teuthology.orchestra.run.smithi156.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2023-11-28T13:05:15.136 INFO:teuthology.orchestra.run.smithi186.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-11-28T13:05:15.161 INFO:teuthology.orchestra.run.smithi156.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.3 MB/s | 251 kB 00:00 2023-11-28T13:05:15.178 INFO:teuthology.orchestra.run.smithi186.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-11-28T13:05:15.186 INFO:teuthology.orchestra.run.smithi156.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 914 kB/s | 61 kB 00:00 2023-11-28T13:05:15.219 INFO:teuthology.orchestra.run.smithi156.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2023-11-28T13:05:15.220 INFO:teuthology.orchestra.run.smithi186.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-11-28T13:05:15.237 INFO:teuthology.orchestra.run.smithi156.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 459 kB/s | 23 kB 00:00 2023-11-28T13:05:15.262 INFO:teuthology.orchestra.run.smithi156.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-28T13:05:15.312 INFO:teuthology.orchestra.run.smithi156.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 941 kB/s | 47 kB 00:00 2023-11-28T13:05:15.362 INFO:teuthology.orchestra.run.smithi156.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 3.9 MB/s | 200 kB 00:00 2023-11-28T13:05:15.395 INFO:teuthology.orchestra.run.smithi186.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 9.7 MB/s | 14 MB 00:01 2023-11-28T13:05:15.404 INFO:teuthology.orchestra.run.smithi156.stdout:(72/131): thrift-0.13.0-2.el8.x86_64.rpm 9.4 MB/s | 1.7 MB 00:00 2023-11-28T13:05:15.421 INFO:teuthology.orchestra.run.smithi156.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 730 kB/s | 43 kB 00:00 2023-11-28T13:05:15.437 INFO:teuthology.orchestra.run.smithi186.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-11-28T13:05:15.463 INFO:teuthology.orchestra.run.smithi156.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.3 MB/s | 48 kB 00:00 2023-11-28T13:05:15.497 INFO:teuthology.orchestra.run.smithi156.stdout:(75/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.3 MB/s | 220 kB 00:00 2023-11-28T13:05:15.504 INFO:teuthology.orchestra.run.smithi186.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 5.2 MB/s | 354 kB 00:00 2023-11-28T13:05:15.522 INFO:teuthology.orchestra.run.smithi156.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.5 MB/s | 87 kB 00:00 2023-11-28T13:05:15.548 INFO:teuthology.orchestra.run.smithi156.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 855 kB/s | 43 kB 00:00 2023-11-28T13:05:15.573 INFO:teuthology.orchestra.run.smithi156.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-11-28T13:05:15.579 INFO:teuthology.orchestra.run.smithi186.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 11 MB/s | 4.9 MB 00:00 2023-11-28T13:05:15.607 INFO:teuthology.orchestra.run.smithi156.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-11-28T13:05:15.640 INFO:teuthology.orchestra.run.smithi156.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 8.2 MB/s | 560 kB 00:00 2023-11-28T13:05:15.666 INFO:teuthology.orchestra.run.smithi156.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 321 kB/s | 19 kB 00:00 2023-11-28T13:05:15.671 INFO:teuthology.orchestra.run.smithi186.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 10 MB/s | 4.7 MB 00:00 2023-11-28T13:05:15.705 INFO:teuthology.orchestra.run.smithi186.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.3 MB/s | 808 kB 00:00 2023-11-28T13:05:15.707 INFO:teuthology.orchestra.run.smithi156.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 2.3 MB/s | 155 kB 00:00 2023-11-28T13:05:15.741 INFO:teuthology.orchestra.run.smithi156.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 7.3 MB/s | 559 kB 00:00 2023-11-28T13:05:15.763 INFO:teuthology.orchestra.run.smithi186.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 16 MB/s | 1.5 MB 00:00 2023-11-28T13:05:15.766 INFO:teuthology.orchestra.run.smithi156.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-11-28T13:05:15.817 INFO:teuthology.orchestra.run.smithi156.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.8 MB/s | 605 kB 00:00 2023-11-28T13:05:15.850 INFO:teuthology.orchestra.run.smithi156.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2023-11-28T13:05:15.855 INFO:teuthology.orchestra.run.smithi186.stdout:(128/131): postgresql-server-10.21-2.module+el8 14 MB/s | 5.1 MB 00:00 2023-11-28T13:05:15.893 INFO:teuthology.orchestra.run.smithi156.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-11-28T13:05:15.914 INFO:teuthology.orchestra.run.smithi186.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-11-28T13:05:15.935 INFO:teuthology.orchestra.run.smithi156.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-11-28T13:05:15.968 INFO:teuthology.orchestra.run.smithi156.stdout:(89/131): elfutils-libelf-devel-0.186-1.el8.x86 1.8 MB/s | 60 kB 00:00 2023-11-28T13:05:16.018 INFO:teuthology.orchestra.run.smithi156.stdout:(90/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 12 MB/s | 2.3 MB 00:00 2023-11-28T13:05:16.044 INFO:teuthology.orchestra.run.smithi156.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.3 MB/s | 98 kB 00:00 2023-11-28T13:05:16.069 INFO:teuthology.orchestra.run.smithi156.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-11-28T13:05:16.094 INFO:teuthology.orchestra.run.smithi156.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 770 kB/s | 39 kB 00:00 2023-11-28T13:05:16.131 INFO:teuthology.orchestra.run.smithi186.stdout:(130/131): librbd1-18.0.0-2697.g2f5f6028.el8.x8 9.7 MB/s | 3.6 MB 00:00 2023-11-28T13:05:16.145 INFO:teuthology.orchestra.run.smithi156.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 8.4 MB/s | 644 kB 00:00 2023-11-28T13:05:16.170 INFO:teuthology.orchestra.run.smithi156.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.2 MB/s | 171 kB 00:00 2023-11-28T13:05:16.195 INFO:teuthology.orchestra.run.smithi156.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-11-28T13:05:16.229 INFO:teuthology.orchestra.run.smithi156.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 3.3 MB/s | 110 kB 00:00 2023-11-28T13:05:16.263 INFO:teuthology.orchestra.run.smithi156.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-11-28T13:05:16.321 INFO:teuthology.orchestra.run.smithi156.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2023-11-28T13:05:16.363 INFO:teuthology.orchestra.run.smithi156.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-11-28T13:05:16.397 INFO:teuthology.orchestra.run.smithi156.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 1.1 MB/s | 39 kB 00:00 2023-11-28T13:05:16.455 INFO:teuthology.orchestra.run.smithi156.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-11-28T13:05:16.506 INFO:teuthology.orchestra.run.smithi156.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 11 MB/s | 538 kB 00:00 2023-11-28T13:05:16.547 INFO:teuthology.orchestra.run.smithi156.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-11-28T13:05:16.589 INFO:teuthology.orchestra.run.smithi156.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-11-28T13:05:16.640 INFO:teuthology.orchestra.run.smithi156.stdout:(106/131): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-11-28T13:05:16.714 INFO:teuthology.orchestra.run.smithi186.stdout:(131/131): librados2-18.0.0-2697.g2f5f6028.el8. 3.6 MB/s | 3.7 MB 00:01 2023-11-28T13:05:16.718 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-11-28T13:05:16.718 INFO:teuthology.orchestra.run.smithi186.stdout:Total 15 MB/s | 203 MB 00:13 2023-11-28T13:05:16.832 INFO:teuthology.orchestra.run.smithi156.stdout:(107/131): python3-scipy-1.0.0-21.module+el8.5. 21 MB/s | 14 MB 00:00 2023-11-28T13:05:16.874 INFO:teuthology.orchestra.run.smithi156.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 643 kB/s | 27 kB 00:00 2023-11-28T13:05:16.916 INFO:teuthology.orchestra.run.smithi156.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.6 MB/s | 320 kB 00:00 2023-11-28T13:05:16.958 INFO:teuthology.orchestra.run.smithi156.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-11-28T13:05:17.025 INFO:teuthology.orchestra.run.smithi156.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 12 MB/s | 4.8 MB 00:00 2023-11-28T13:05:17.075 INFO:teuthology.orchestra.run.smithi156.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-11-28T13:05:17.117 INFO:teuthology.orchestra.run.smithi156.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-11-28T13:05:17.142 INFO:teuthology.orchestra.run.smithi156.stdout:(114/131): uuid-1.6.2-43.el8.x86_64.rpm 2.5 MB/s | 64 kB 00:00 2023-11-28T13:05:17.218 INFO:teuthology.orchestra.run.smithi156.stdout:(115/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-11-28T13:05:17.243 INFO:teuthology.orchestra.run.smithi156.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.2 MB/s | 126 kB 00:00 2023-11-28T13:05:17.268 INFO:teuthology.orchestra.run.smithi156.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-11-28T13:05:17.294 INFO:teuthology.orchestra.run.smithi156.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-11-28T13:05:17.344 INFO:teuthology.orchestra.run.smithi156.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2023-11-28T13:05:17.386 INFO:teuthology.orchestra.run.smithi156.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-11-28T13:05:17.545 INFO:teuthology.orchestra.run.smithi156.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 19 MB/s | 4.9 MB 00:00 2023-11-28T13:05:17.586 INFO:teuthology.orchestra.run.smithi156.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-11-28T13:05:17.662 INFO:teuthology.orchestra.run.smithi156.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-11-28T13:05:18.087 INFO:teuthology.orchestra.run.smithi156.stdout:(124/131): ceph-test-18.0.0-2697.g2f5f6028.el8. 5.7 MB/s | 47 MB 00:08 2023-11-28T13:05:18.121 INFO:teuthology.orchestra.run.smithi156.stdout:(125/131): librdkafka-0.11.4-3.el8.x86_64.rpm 663 kB/s | 354 kB 00:00 2023-11-28T13:05:18.196 INFO:teuthology.orchestra.run.smithi156.stdout:(126/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-11-28T13:05:18.246 INFO:teuthology.orchestra.run.smithi156.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-11-28T13:05:18.346 INFO:teuthology.orchestra.run.smithi156.stdout:(128/131): postgresql-server-10.21-2.module+el8 7.4 MB/s | 5.1 MB 00:00 2023-11-28T13:05:18.377 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-11-28T13:05:18.397 INFO:teuthology.orchestra.run.smithi156.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-11-28T13:05:18.515 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-11-28T13:05:18.515 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-11-28T13:05:18.680 INFO:teuthology.orchestra.run.smithi156.stdout:(130/131): librbd1-18.0.0-2697.g2f5f6028.el8.x8 8.2 MB/s | 3.6 MB 00:00 2023-11-28T13:05:19.005 INFO:teuthology.orchestra.run.smithi156.stdout:(131/131): librados2-18.0.0-2697.g2f5f6028.el8. 4.5 MB/s | 3.7 MB 00:00 2023-11-28T13:05:19.008 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2023-11-28T13:05:19.008 INFO:teuthology.orchestra.run.smithi156.stdout:Total 14 MB/s | 203 MB 00:14 2023-11-28T13:05:20.427 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-11-28T13:05:20.432 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-11-28T13:05:20.694 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2023-11-28T13:05:20.829 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2023-11-28T13:05:20.830 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2023-11-28T13:05:22.634 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2023-11-28T13:05:22.638 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2023-11-28T13:05:25.527 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-11-28T13:05:25.745 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-28T13:05:25.955 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-28T13:05:26.162 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T13:05:26.336 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T13:05:26.554 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T13:05:27.032 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T13:05:27.304 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T13:05:27.680 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2023-11-28T13:05:27.754 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T13:05:27.852 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-28T13:05:27.929 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T13:05:28.045 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-28T13:05:28.089 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T13:05:28.220 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T13:05:28.287 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T13:05:28.388 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T13:05:28.407 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T13:05:28.618 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T13:05:28.637 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T13:05:28.833 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-28T13:05:29.104 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T13:05:29.137 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-28T13:05:29.342 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-28T13:05:29.344 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T13:05:29.779 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T13:05:29.820 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-28T13:05:29.978 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T13:05:30.108 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-28T13:05:30.120 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T13:05:30.328 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T13:05:30.441 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T13:05:30.620 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T13:05:30.642 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T13:05:30.742 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T13:05:30.850 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-28T13:05:30.945 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T13:05:31.133 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T13:05:31.137 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-28T13:05:31.351 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-28T13:05:31.367 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-28T13:05:31.571 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-28T13:05:31.774 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-28T13:05:31.832 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-28T13:05:31.920 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T13:05:32.058 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-28T13:05:32.124 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T13:05:32.262 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T13:05:32.269 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T13:05:32.398 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T13:05:32.586 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T13:05:32.772 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T13:05:32.946 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T13:05:32.993 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-28T13:05:33.112 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T13:05:33.204 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-28T13:05:33.425 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-28T13:05:33.561 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T13:05:33.734 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T13:05:33.778 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T13:05:33.902 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T13:05:34.028 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T13:05:34.863 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T13:05:35.117 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T13:05:35.145 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-28T13:05:35.294 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T13:05:35.624 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-28T13:05:35.875 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-28T13:05:35.942 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T13:05:36.069 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 26/134 2023-11-28T13:05:36.127 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T13:05:36.215 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 27/134 2023-11-28T13:05:36.465 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 28/134 2023-11-28T13:05:36.678 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T13:05:36.973 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T13:05:37.254 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-28T13:05:37.470 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-28T13:05:37.738 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-28T13:05:37.926 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 26/134 2023-11-28T13:05:38.056 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 27/134 2023-11-28T13:05:38.331 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 28/134 2023-11-28T13:05:38.535 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T13:05:38.714 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T13:05:39.896 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-28T13:05:40.257 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-28T13:05:40.560 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-28T13:05:40.608 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T13:05:40.780 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-28T13:05:41.022 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-28T13:05:41.224 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-28T13:05:41.427 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 36/134 2023-11-28T13:05:41.618 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 37/134 2023-11-28T13:05:41.773 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-28T13:05:41.811 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-28T13:05:41.972 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-28T13:05:42.143 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-28T13:05:42.152 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T13:05:42.238 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T13:05:42.401 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-28T13:05:42.516 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-28T13:05:42.648 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-28T13:05:42.736 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-28T13:05:42.881 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-28T13:05:42.893 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T13:05:42.984 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T13:05:43.115 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-28T13:05:43.216 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-28T13:05:43.291 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 36/134 2023-11-28T13:05:43.409 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-28T13:05:43.492 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 37/134 2023-11-28T13:05:43.598 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-28T13:05:43.652 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-28T13:05:43.813 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-28T13:05:43.827 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-28T13:05:43.975 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T13:05:44.006 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-28T13:05:44.066 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T13:05:44.209 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-28T13:05:44.341 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-28T13:05:44.449 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-28T13:05:44.548 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-28T13:05:44.709 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T13:05:44.713 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-28T13:05:44.812 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T13:05:45.044 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-28T13:05:45.124 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-28T13:05:45.239 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-28T13:05:45.413 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-28T13:05:45.433 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-28T13:05:45.646 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-28T13:05:45.691 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-28T13:05:45.820 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-28T13:05:45.947 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-28T13:05:46.007 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-28T13:05:46.187 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-28T13:05:46.196 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-28T13:05:46.423 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-28T13:05:46.520 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-28T13:05:46.794 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-28T13:05:46.819 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-28T13:05:47.007 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-28T13:05:47.099 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-28T13:05:47.279 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-28T13:05:47.332 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-28T13:05:47.468 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-28T13:05:47.596 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-28T13:05:47.698 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-28T13:05:47.810 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-28T13:05:48.035 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-28T13:05:48.177 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-28T13:05:48.356 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-28T13:05:48.418 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-28T13:05:48.594 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-28T13:05:48.649 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-28T13:05:48.811 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-28T13:05:48.896 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-28T13:05:49.009 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-28T13:05:49.119 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-28T13:05:49.230 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-28T13:05:49.342 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-28T13:05:49.574 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-28T13:05:49.645 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-28T13:05:49.871 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-28T13:05:49.875 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-28T13:05:50.073 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-28T13:05:50.148 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-28T13:05:50.188 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-28T13:05:50.364 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-28T13:05:50.378 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-28T13:05:50.568 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-28T13:05:50.608 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-28T13:05:50.783 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-28T13:05:50.831 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-28T13:05:51.004 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-28T13:05:51.058 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-28T13:05:51.132 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-28T13:05:51.283 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-28T13:05:51.371 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-28T13:05:51.446 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-28T13:05:51.596 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-28T13:05:51.606 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-28T13:05:51.799 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-28T13:05:51.830 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-28T13:05:52.048 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-28T13:05:52.105 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-28T13:05:52.238 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-28T13:05:52.343 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-28T13:05:52.386 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-28T13:05:52.506 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-28T13:05:52.527 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-28T13:05:52.738 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-28T13:05:52.739 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-28T13:05:52.972 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-28T13:05:53.156 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-28T13:05:53.437 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-28T13:05:53.709 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-28T13:05:53.909 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-28T13:05:54.071 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-28T13:05:54.124 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-28T13:05:54.345 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-28T13:05:54.597 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-28T13:05:54.905 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-28T13:05:55.197 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-28T13:05:55.442 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-28T13:05:55.482 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-28T13:05:55.762 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-28T13:05:56.038 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-28T13:05:56.276 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-28T13:05:56.383 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-28T13:05:56.466 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T13:05:56.564 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T13:05:56.703 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-28T13:05:56.759 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 93/134 2023-11-28T13:05:56.916 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-28T13:05:57.761 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-28T13:05:57.948 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T13:05:58.046 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T13:05:58.224 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 93/134 2023-11-28T13:06:01.679 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T13:06:01.900 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T13:06:02.574 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T13:06:02.895 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T13:06:03.051 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T13:06:03.207 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T13:06:03.475 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T13:06:03.682 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T13:06:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T13:06:04.637 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T13:06:04.773 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T13:06:04.897 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T13:06:23.765 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T13:06:23.766 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /sys 2023-11-28T13:06:23.766 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /proc 2023-11-28T13:06:23.766 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /dev 2023-11-28T13:06:23.766 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /run 2023-11-28T13:06:23.766 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /mnt 2023-11-28T13:06:23.766 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /var/tmp 2023-11-28T13:06:23.766 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /home 2023-11-28T13:06:23.766 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /tmp 2023-11-28T13:06:23.766 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /dev 2023-11-28T13:06:23.767 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:06:24.130 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T13:06:25.438 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T13:06:25.438 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /sys 2023-11-28T13:06:25.438 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /proc 2023-11-28T13:06:25.438 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /dev 2023-11-28T13:06:25.438 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /run 2023-11-28T13:06:25.438 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /mnt 2023-11-28T13:06:25.438 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /var/tmp 2023-11-28T13:06:25.438 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /home 2023-11-28T13:06:25.439 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /tmp 2023-11-28T13:06:25.439 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /dev 2023-11-28T13:06:25.439 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:06:25.828 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T13:06:25.835 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T13:06:26.009 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T13:06:26.275 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T13:06:26.413 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T13:06:26.943 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T13:06:27.163 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T13:06:27.205 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T13:06:27.537 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T13:06:27.725 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T13:06:27.993 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T13:06:28.145 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T13:06:28.674 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T13:06:28.894 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T13:06:28.937 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T13:06:30.042 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T13:06:30.203 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T13:06:30.370 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 102/134 2023-11-28T13:06:30.607 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 103/134 2023-11-28T13:06:30.795 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T13:06:31.055 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T13:06:31.266 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T13:06:31.769 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T13:06:31.944 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T13:06:32.110 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 102/134 2023-11-28T13:06:32.332 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 103/134 2023-11-28T13:06:32.359 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T13:06:32.534 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T13:06:32.569 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T13:06:32.696 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T13:06:32.799 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T13:06:32.990 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T13:06:33.121 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 107/134 2023-11-28T13:06:33.308 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T13:06:33.373 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T13:06:33.569 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T13:06:33.598 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T13:06:34.079 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T13:06:34.259 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T13:06:34.389 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T13:06:34.813 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 107/134 2023-11-28T13:06:34.949 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 110/134 2023-11-28T13:06:35.007 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T13:06:35.074 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T13:06:35.193 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T13:06:35.243 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T13:06:35.269 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T13:06:35.272 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T13:06:35.445 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T13:06:35.516 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T13:06:35.645 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T13:06:36.049 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T13:06:36.203 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T13:06:36.610 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 110/134 2023-11-28T13:06:36.862 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T13:06:36.930 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T13:06:37.109 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T13:06:37.181 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T13:06:37.326 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T13:06:37.730 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T13:06:37.901 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T13:06:41.025 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T13:06:42.284 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 115/134 2023-11-28T13:06:42.688 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-28T13:06:42.727 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T13:06:42.875 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 117/134 2023-11-28T13:06:43.202 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-28T13:06:43.376 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T13:06:43.626 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T13:06:43.848 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-28T13:06:43.934 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 115/134 2023-11-28T13:06:44.073 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 121/134 2023-11-28T13:06:44.236 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T13:06:44.279 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-28T13:06:44.482 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 117/134 2023-11-28T13:06:44.751 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-28T13:06:44.932 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T13:06:45.189 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T13:06:45.413 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-28T13:06:45.613 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 121/134 2023-11-28T13:06:45.759 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T13:06:47.687 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T13:06:47.985 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T13:06:48.025 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T13:06:48.297 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 124/134 2023-11-28T13:06:48.544 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/134 2023-11-28T13:06:48.905 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/134 2023-11-28T13:06:49.188 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T13:06:49.284 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-28T13:06:49.474 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T13:06:49.492 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T13:06:49.514 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T13:06:49.598 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T13:06:49.759 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 124/134 2023-11-28T13:06:49.784 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T13:06:49.813 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T13:06:49.950 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/134 2023-11-28T13:06:50.042 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T13:06:50.065 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T13:06:50.221 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-28T13:06:50.222 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T13:06:50.258 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/134 2023-11-28T13:06:50.482 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T13:06:50.482 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T13:06:50.653 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-28T13:06:50.742 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T13:06:50.840 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T13:06:50.946 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T13:06:50.950 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T13:06:51.148 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T13:06:51.177 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T13:06:51.207 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T13:06:51.207 INFO:teuthology.orchestra.run.smithi186.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-28T13:06:51.207 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:06:51.365 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T13:06:51.387 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T13:06:51.552 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-28T13:06:51.552 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T13:06:51.813 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T13:06:51.813 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T13:06:51.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/134 2023-11-28T13:06:51.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/134 2023-11-28T13:06:51.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/134 2023-11-28T13:06:51.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/134 2023-11-28T13:06:51.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/134 2023-11-28T13:06:51.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T13:06:51.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T13:06:51.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T13:06:51.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T13:06:51.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/134 2023-11-28T13:06:51.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/134 2023-11-28T13:06:51.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/134 2023-11-28T13:06:51.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/134 2023-11-28T13:06:51.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/134 2023-11-28T13:06:51.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/134 2023-11-28T13:06:51.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/134 2023-11-28T13:06:51.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/134 2023-11-28T13:06:51.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/134 2023-11-28T13:06:51.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/134 2023-11-28T13:06:51.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/134 2023-11-28T13:06:51.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/134 2023-11-28T13:06:51.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/134 2023-11-28T13:06:51.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/134 2023-11-28T13:06:51.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/134 2023-11-28T13:06:51.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/134 2023-11-28T13:06:51.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/134 2023-11-28T13:06:51.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/134 2023-11-28T13:06:51.820 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/134 2023-11-28T13:06:51.820 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/134 2023-11-28T13:06:51.829 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/134 2023-11-28T13:06:51.829 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/134 2023-11-28T13:06:51.829 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/134 2023-11-28T13:06:51.829 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/134 2023-11-28T13:06:51.830 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/134 2023-11-28T13:06:51.830 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/134 2023-11-28T13:06:51.830 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/134 2023-11-28T13:06:51.830 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/134 2023-11-28T13:06:51.830 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/134 2023-11-28T13:06:51.830 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-28T13:06:51.830 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-28T13:06:51.830 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-28T13:06:51.830 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-28T13:06:51.830 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-28T13:06:51.830 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-28T13:06:51.831 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-28T13:06:51.831 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-28T13:06:51.831 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-28T13:06:51.831 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-28T13:06:51.831 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-28T13:06:51.831 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-28T13:06:51.831 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-28T13:06:51.831 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-28T13:06:51.831 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-28T13:06:51.831 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-28T13:06:51.831 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-28T13:06:51.832 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-28T13:06:51.832 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-28T13:06:51.832 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-28T13:06:51.832 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-28T13:06:51.832 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-28T13:06:51.832 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-28T13:06:51.832 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-28T13:06:51.832 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-28T13:06:51.832 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-28T13:06:51.832 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-28T13:06:51.832 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-28T13:06:51.832 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-28T13:06:51.834 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-28T13:06:51.834 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-28T13:06:51.834 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-28T13:06:51.834 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-28T13:06:51.834 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-28T13:06:51.834 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-28T13:06:51.834 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-28T13:06:51.834 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-28T13:06:51.834 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-28T13:06:51.834 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-28T13:06:51.835 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-28T13:06:51.835 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-28T13:06:51.835 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-28T13:06:51.835 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-28T13:06:51.835 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-28T13:06:51.835 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-28T13:06:51.835 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-28T13:06:51.835 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-28T13:06:51.835 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-28T13:06:51.835 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-28T13:06:51.835 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-28T13:06:51.837 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-28T13:06:51.837 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-28T13:06:51.837 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-28T13:06:51.837 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-28T13:06:51.837 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-28T13:06:51.837 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-28T13:06:51.837 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-28T13:06:51.837 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-28T13:06:51.837 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-28T13:06:51.838 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-28T13:06:51.838 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-28T13:06:51.838 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-28T13:06:51.838 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-28T13:06:51.838 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-28T13:06:51.838 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-28T13:06:51.838 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-28T13:06:51.838 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-28T13:06:51.838 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-28T13:06:51.838 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-28T13:06:51.838 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/134 2023-11-28T13:06:51.839 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-28T13:06:51.839 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/134 2023-11-28T13:06:51.839 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T13:06:51.839 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-28T13:06:52.106 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T13:06:52.315 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T13:06:52.605 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T13:06:52.605 INFO:teuthology.orchestra.run.smithi156.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-28T13:06:52.605 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:06:53.214 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/134 2023-11-28T13:06:53.214 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/134 2023-11-28T13:06:53.215 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/134 2023-11-28T13:06:53.215 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/134 2023-11-28T13:06:53.215 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/134 2023-11-28T13:06:53.215 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T13:06:53.215 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T13:06:53.215 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T13:06:53.215 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T13:06:53.215 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/134 2023-11-28T13:06:53.215 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/134 2023-11-28T13:06:53.215 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/134 2023-11-28T13:06:53.215 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/134 2023-11-28T13:06:53.215 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/134 2023-11-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/134 2023-11-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/134 2023-11-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/134 2023-11-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/134 2023-11-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/134 2023-11-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/134 2023-11-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/134 2023-11-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/134 2023-11-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/134 2023-11-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/134 2023-11-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/134 2023-11-28T13:06:53.217 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/134 2023-11-28T13:06:53.217 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/134 2023-11-28T13:06:53.217 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/134 2023-11-28T13:06:53.217 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/134 2023-11-28T13:06:53.217 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/134 2023-11-28T13:06:53.217 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/134 2023-11-28T13:06:53.226 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/134 2023-11-28T13:06:53.226 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/134 2023-11-28T13:06:53.226 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/134 2023-11-28T13:06:53.226 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/134 2023-11-28T13:06:53.226 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/134 2023-11-28T13:06:53.227 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/134 2023-11-28T13:06:53.227 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/134 2023-11-28T13:06:53.227 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-28T13:06:53.227 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-28T13:06:53.227 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-28T13:06:53.227 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-28T13:06:53.227 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-28T13:06:53.227 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-28T13:06:53.227 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-28T13:06:53.227 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-28T13:06:53.227 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-28T13:06:53.227 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-28T13:06:53.228 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-28T13:06:53.228 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-28T13:06:53.228 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-28T13:06:53.228 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-28T13:06:53.228 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-28T13:06:53.228 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-28T13:06:53.228 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-28T13:06:53.228 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-28T13:06:53.228 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-28T13:06:53.228 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-28T13:06:53.228 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-28T13:06:53.228 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-28T13:06:53.229 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-28T13:06:53.229 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-28T13:06:53.229 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-28T13:06:53.229 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-28T13:06:53.229 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-28T13:06:53.229 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-28T13:06:53.229 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-28T13:06:53.229 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-28T13:06:53.229 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-28T13:06:53.229 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-28T13:06:53.229 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-28T13:06:53.229 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-28T13:06:53.230 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-28T13:06:53.230 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-28T13:06:53.230 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-28T13:06:53.230 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-28T13:06:53.230 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-28T13:06:53.230 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-28T13:06:53.230 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-28T13:06:53.230 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-28T13:06:53.230 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-28T13:06:53.230 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-28T13:06:53.230 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-28T13:06:53.230 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-28T13:06:53.231 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-28T13:06:53.231 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-28T13:06:53.231 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-28T13:06:53.231 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-28T13:06:53.231 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-28T13:06:53.231 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-28T13:06:53.231 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-28T13:06:53.231 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-28T13:06:53.231 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-28T13:06:53.231 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-28T13:06:53.231 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-28T13:06:53.231 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-28T13:06:53.232 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-28T13:06:53.232 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-28T13:06:53.232 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-28T13:06:53.232 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-28T13:06:53.232 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-28T13:06:53.232 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-28T13:06:53.232 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-28T13:06:53.232 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-28T13:06:53.232 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-28T13:06:53.232 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-28T13:06:53.232 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-28T13:06:53.233 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-28T13:06:53.233 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-28T13:06:53.233 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-28T13:06:53.233 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-28T13:06:53.233 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-28T13:06:53.233 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-28T13:06:53.233 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-28T13:06:53.233 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-28T13:06:53.233 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-28T13:06:53.233 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-28T13:06:53.233 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-28T13:06:53.233 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-28T13:06:53.234 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-28T13:06:53.234 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-28T13:06:53.234 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-28T13:06:53.234 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-28T13:06:53.234 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-28T13:06:53.234 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-28T13:06:53.234 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-28T13:06:53.234 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-28T13:06:53.234 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-28T13:06:53.234 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/134 2023-11-28T13:06:53.234 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-28T13:06:53.235 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/134 2023-11-28T13:06:53.235 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T13:06:53.235 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-28T13:06:54.622 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T13:06:54.623 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:03 ago on Tue 28 Nov 2023 01:04:51 PM UTC. 2023-11-28T13:06:55.965 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T13:06:55.965 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:02:03 ago on Tue 28 Nov 2023 01:04:52 PM UTC. 2023-11-28T13:06:57.258 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-11-28T13:06:57.826 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:06:57.826 INFO:teuthology.orchestra.run.smithi186.stdout:Upgraded: 2023-11-28T13:06:57.827 INFO:teuthology.orchestra.run.smithi186.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-28T13:06:57.827 INFO:teuthology.orchestra.run.smithi186.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.827 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.827 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-11-28T13:06:57.827 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-28T13:06:57.827 INFO:teuthology.orchestra.run.smithi186.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T13:06:57.827 INFO:teuthology.orchestra.run.smithi186.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T13:06:57.827 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.827 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.827 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.827 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.827 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:57.828 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.828 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.828 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.828 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:57.828 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:57.828 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:57.828 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:57.828 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:57.828 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.828 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.828 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:57.828 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.829 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.829 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.829 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:57.829 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:57.829 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.829 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:57.829 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-28T13:06:57.829 INFO:teuthology.orchestra.run.smithi186.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T13:06:57.829 INFO:teuthology.orchestra.run.smithi186.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-28T13:06:57.829 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T13:06:57.829 INFO:teuthology.orchestra.run.smithi186.stdout: jq-1.6-3.el8.x86_64 2023-11-28T13:06:57.830 INFO:teuthology.orchestra.run.smithi186.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T13:06:57.830 INFO:teuthology.orchestra.run.smithi186.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-28T13:06:57.830 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T13:06:57.830 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.830 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.830 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.830 INFO:teuthology.orchestra.run.smithi186.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-28T13:06:57.830 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-28T13:06:57.830 INFO:teuthology.orchestra.run.smithi186.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T13:06:57.830 INFO:teuthology.orchestra.run.smithi186.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T13:06:57.830 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-28T13:06:57.830 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T13:06:57.831 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.831 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.831 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T13:06:57.831 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.831 INFO:teuthology.orchestra.run.smithi186.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T13:06:57.831 INFO:teuthology.orchestra.run.smithi186.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T13:06:57.831 INFO:teuthology.orchestra.run.smithi186.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T13:06:57.831 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T13:06:57.831 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-28T13:06:57.831 INFO:teuthology.orchestra.run.smithi186.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T13:06:57.833 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T13:06:57.833 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T13:06:57.833 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-28T13:06:57.833 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-28T13:06:57.833 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-28T13:06:57.833 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T13:06:57.833 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T13:06:57.833 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T13:06:57.833 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T13:06:57.834 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T13:06:57.834 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T13:06:57.834 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T13:06:57.834 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T13:06:57.834 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T13:06:57.834 INFO:teuthology.orchestra.run.smithi186.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T13:06:57.834 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T13:06:57.834 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.834 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.834 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.835 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T13:06:57.835 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-28T13:06:57.835 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T13:06:57.835 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T13:06:57.835 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-28T13:06:57.835 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T13:06:57.835 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T13:06:57.835 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T13:06:57.835 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T13:06:57.835 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T13:06:57.835 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T13:06:57.835 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T13:06:57.836 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T13:06:57.836 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T13:06:57.836 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T13:06:57.836 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T13:06:57.836 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T13:06:57.836 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T13:06:57.836 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T13:06:57.836 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T13:06:57.836 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T13:06:57.836 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T13:06:57.836 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T13:06:57.837 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T13:06:57.837 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T13:06:57.837 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T13:06:57.837 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T13:06:57.837 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T13:06:57.837 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T13:06:57.837 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.837 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.837 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T13:06:57.837 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T13:06:57.837 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.837 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T13:06:57.838 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T13:06:57.838 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-28T13:06:57.838 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T13:06:57.838 INFO:teuthology.orchestra.run.smithi186.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T13:06:57.838 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T13:06:57.838 INFO:teuthology.orchestra.run.smithi186.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T13:06:57.838 INFO:teuthology.orchestra.run.smithi186.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T13:06:57.838 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T13:06:57.839 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T13:06:57.839 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T13:06:57.839 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T13:06:57.839 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T13:06:57.839 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.839 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.839 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:57.839 INFO:teuthology.orchestra.run.smithi186.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T13:06:57.839 INFO:teuthology.orchestra.run.smithi186.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T13:06:57.839 INFO:teuthology.orchestra.run.smithi186.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T13:06:57.839 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-28T13:06:57.839 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T13:06:57.840 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-28T13:06:57.840 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:06:57.840 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-11-28T13:06:58.325 DEBUG:teuthology.parallel:result is None 2023-11-28T13:06:58.600 INFO:teuthology.orchestra.run.smithi156.stdout:Installed products updated. 2023-11-28T13:06:59.198 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:06:59.199 INFO:teuthology.orchestra.run.smithi156.stdout:Upgraded: 2023-11-28T13:06:59.199 INFO:teuthology.orchestra.run.smithi156.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-28T13:06:59.199 INFO:teuthology.orchestra.run.smithi156.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.199 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.199 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2023-11-28T13:06:59.199 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-28T13:06:59.199 INFO:teuthology.orchestra.run.smithi156.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T13:06:59.199 INFO:teuthology.orchestra.run.smithi156.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T13:06:59.200 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.200 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.200 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.200 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.200 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:59.200 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.200 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.200 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.200 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:59.200 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:59.200 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:59.200 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:59.201 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:59.201 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.201 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.201 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:59.201 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.201 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.201 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.201 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:59.202 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:59.202 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.202 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T13:06:59.202 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-28T13:06:59.202 INFO:teuthology.orchestra.run.smithi156.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T13:06:59.202 INFO:teuthology.orchestra.run.smithi156.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-28T13:06:59.202 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T13:06:59.202 INFO:teuthology.orchestra.run.smithi156.stdout: jq-1.6-3.el8.x86_64 2023-11-28T13:06:59.202 INFO:teuthology.orchestra.run.smithi156.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T13:06:59.203 INFO:teuthology.orchestra.run.smithi156.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-28T13:06:59.203 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T13:06:59.203 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.203 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.203 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.203 INFO:teuthology.orchestra.run.smithi156.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-28T13:06:59.203 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-28T13:06:59.203 INFO:teuthology.orchestra.run.smithi156.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T13:06:59.203 INFO:teuthology.orchestra.run.smithi156.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T13:06:59.203 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-28T13:06:59.203 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T13:06:59.203 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.204 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.204 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T13:06:59.204 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.204 INFO:teuthology.orchestra.run.smithi156.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T13:06:59.204 INFO:teuthology.orchestra.run.smithi156.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T13:06:59.204 INFO:teuthology.orchestra.run.smithi156.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T13:06:59.204 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T13:06:59.204 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-28T13:06:59.204 INFO:teuthology.orchestra.run.smithi156.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T13:06:59.204 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T13:06:59.205 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T13:06:59.205 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-28T13:06:59.205 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-28T13:06:59.205 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-28T13:06:59.205 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T13:06:59.205 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T13:06:59.205 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T13:06:59.205 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T13:06:59.205 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T13:06:59.206 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T13:06:59.206 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T13:06:59.206 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T13:06:59.206 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T13:06:59.206 INFO:teuthology.orchestra.run.smithi156.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T13:06:59.206 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T13:06:59.206 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.206 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.206 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.207 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T13:06:59.207 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-28T13:06:59.207 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T13:06:59.207 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T13:06:59.207 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-28T13:06:59.207 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T13:06:59.207 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T13:06:59.208 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T13:06:59.208 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T13:06:59.208 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T13:06:59.208 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T13:06:59.208 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T13:06:59.208 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T13:06:59.208 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T13:06:59.208 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T13:06:59.208 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T13:06:59.209 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T13:06:59.209 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T13:06:59.209 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T13:06:59.209 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T13:06:59.209 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T13:06:59.209 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T13:06:59.209 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T13:06:59.209 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T13:06:59.209 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T13:06:59.209 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T13:06:59.209 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T13:06:59.210 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T13:06:59.210 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T13:06:59.210 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.210 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.210 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T13:06:59.210 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T13:06:59.210 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.210 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T13:06:59.210 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T13:06:59.211 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-28T13:06:59.211 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T13:06:59.211 INFO:teuthology.orchestra.run.smithi156.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T13:06:59.211 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T13:06:59.211 INFO:teuthology.orchestra.run.smithi156.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T13:06:59.211 INFO:teuthology.orchestra.run.smithi156.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T13:06:59.211 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T13:06:59.212 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T13:06:59.212 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T13:06:59.212 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T13:06:59.212 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T13:06:59.212 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.213 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.213 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T13:06:59.213 INFO:teuthology.orchestra.run.smithi156.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T13:06:59.213 INFO:teuthology.orchestra.run.smithi156.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T13:06:59.213 INFO:teuthology.orchestra.run.smithi156.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T13:06:59.214 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-28T13:06:59.214 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T13:06:59.215 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-28T13:06:59.215 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:06:59.215 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-11-28T13:06:59.731 DEBUG:teuthology.parallel:result is None 2023-11-28T13:06:59.732 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T13:06:59.954 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T13:07:00.010 INFO:teuthology.orchestra.run.smithi156.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T13:07:00.010 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T13:07:00.010 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T13:07:00.011 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T13:07:00.225 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T13:07:00.281 INFO:teuthology.orchestra.run.smithi186.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T13:07:00.281 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T13:07:00.281 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T13:07:00.282 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-28T13:07:00.282 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:00.282 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T13:07:00.372 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:00.373 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T13:07:00.465 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-28T13:07:00.465 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:00.465 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T13:07:00.539 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T13:07:00.649 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:00.649 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T13:07:00.721 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T13:07:00.831 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-28T13:07:00.831 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:00.831 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T13:07:00.899 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T13:07:01.010 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:01.011 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T13:07:01.079 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T13:07:01.189 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-28T13:07:01.189 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:01.190 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T13:07:01.259 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T13:07:01.370 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:01.370 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T13:07:01.439 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T13:07:01.547 INFO:teuthology.run_tasks:Running task ceph... 2023-11-28T13:07:01.606 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-28T13:07:01.606 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 777 /var/log/ceph 2023-11-28T13:07:01.611 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 777 /var/log/ceph 2023-11-28T13:07:01.681 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-28T13:07:01.682 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T13:07:01.686 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T13:07:01.755 INFO:tasks.ceph:Creating extra log directories... 2023-11-28T13:07:01.756 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T13:07:01.760 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T13:07:01.835 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-28T13:07:01.835 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-28T13:07:01.835 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-28T13:07:01.835 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:01.836 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T13:07:01.880 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T13:07:02.113 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T13:07:02.183 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:02.183 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T13:07:02.227 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T13:07:02.431 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T13:07:02.509 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-28T13:07:02.509 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-28T13:07:02.509 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469974', 'branch': 'wip-neesingh-testing-231123', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7469974', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-231123'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'issue_54557', 'sha1': '3ae99413970476692dfab1fe38dadb056700a61d'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 10, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6754, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'issue_54557', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': '3ae99413970476692dfab1fe38dadb056700a61d', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi156.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi186.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '5ae09a5cc7064ca2bf994df19f3006b79794789a', 'timestamp': '2023-11-28_12:38:30', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-28T13:07:02.510 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T13:07:02.514 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T13:07:02.558 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T13:07:02.600 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T13:07:02.672 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:02.672 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T13:07:02.715 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T13:07:02.716 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_1 2023-11-28T13:07:02.802 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T13:07:02.802 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T13:07:02.802 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-11-28T13:07:02.802 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T13:07:02.802 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T13:07:02.803 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-11-28 13:06:52.734174097 +0000 2023-11-28T13:07:02.803 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-11-28 13:03:51.688789256 +0000 2023-11-28T13:07:02.803 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-11-28 13:03:51.688789256 +0000 2023-11-28T13:07:02.803 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2023-11-28 13:03:51.664789602 +0000 2023-11-28T13:07:02.803 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T13:07:02.910 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-11-28T13:07:02.911 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-11-28T13:07:02.911 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000240006 s, 2.1 MB/s 2023-11-28T13:07:02.913 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T13:07:02.959 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_2 2023-11-28T13:07:03.046 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T13:07:03.046 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T13:07:03.046 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-11-28T13:07:03.046 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T13:07:03.046 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T13:07:03.046 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-11-28 13:06:52.734174097 +0000 2023-11-28T13:07:03.046 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-11-28 13:03:52.512777382 +0000 2023-11-28T13:07:03.046 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-11-28 13:03:52.512777382 +0000 2023-11-28T13:07:03.046 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2023-11-28 13:03:52.488777728 +0000 2023-11-28T13:07:03.047 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T13:07:03.153 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-11-28T13:07:03.153 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-11-28T13:07:03.153 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000171404 s, 3.0 MB/s 2023-11-28T13:07:03.155 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T13:07:03.203 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_3 2023-11-28T13:07:03.254 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T13:07:03.255 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T13:07:03.256 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-11-28T13:07:03.256 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T13:07:03.256 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T13:07:03.256 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-11-28 13:06:52.734174097 +0000 2023-11-28T13:07:03.256 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-11-28 13:03:53.335765522 +0000 2023-11-28T13:07:03.256 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-11-28 13:03:53.335765522 +0000 2023-11-28T13:07:03.256 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2023-11-28 13:03:53.312765853 +0000 2023-11-28T13:07:03.257 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T13:07:03.323 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-11-28T13:07:03.323 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-11-28T13:07:03.323 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000179177 s, 2.9 MB/s 2023-11-28T13:07:03.326 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T13:07:03.373 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_4 2023-11-28T13:07:03.418 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T13:07:03.418 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T13:07:03.419 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-11-28T13:07:03.419 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T13:07:03.419 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T13:07:03.419 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-11-28 13:06:52.735174083 +0000 2023-11-28T13:07:03.419 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-11-28 13:03:54.168753518 +0000 2023-11-28T13:07:03.419 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-11-28 13:03:54.168753518 +0000 2023-11-28T13:07:03.419 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2023-11-28 13:03:54.145753849 +0000 2023-11-28T13:07:03.420 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T13:07:03.526 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-11-28T13:07:03.526 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-11-28T13:07:03.526 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000185755 s, 2.8 MB/s 2023-11-28T13:07:03.529 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T13:07:03.576 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T13:07:03.576 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:03.576 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T13:07:03.619 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T13:07:03.620 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 2023-11-28T13:07:03.705 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T13:07:03.705 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T13:07:03.705 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-11-28T13:07:03.705 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T13:07:03.706 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T13:07:03.706 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-11-28 13:06:51.331871751 +0000 2023-11-28T13:07:03.706 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-11-28 13:03:46.296935421 +0000 2023-11-28T13:07:03.706 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-11-28 13:03:46.296935421 +0000 2023-11-28T13:07:03.706 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2023-11-28 13:03:46.266935952 +0000 2023-11-28T13:07:03.706 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T13:07:03.812 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-11-28T13:07:03.812 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-11-28T13:07:03.813 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000404833 s, 1.3 MB/s 2023-11-28T13:07:03.815 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T13:07:03.863 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 2023-11-28T13:07:03.908 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T13:07:03.908 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T13:07:03.908 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-11-28T13:07:03.909 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T13:07:03.909 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T13:07:03.909 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-11-28 13:06:51.331871751 +0000 2023-11-28T13:07:03.909 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-11-28 13:03:47.119920869 +0000 2023-11-28T13:07:03.909 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-11-28 13:03:47.119920869 +0000 2023-11-28T13:07:03.910 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2023-11-28 13:03:47.090921382 +0000 2023-11-28T13:07:03.910 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T13:07:04.017 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-11-28T13:07:04.017 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-11-28T13:07:04.018 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000298028 s, 1.7 MB/s 2023-11-28T13:07:04.020 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T13:07:04.070 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 2023-11-28T13:07:04.114 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T13:07:04.114 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T13:07:04.114 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-11-28T13:07:04.115 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T13:07:04.115 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T13:07:04.115 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-11-28 13:06:51.331871751 +0000 2023-11-28T13:07:04.115 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-11-28 13:03:47.977905699 +0000 2023-11-28T13:07:04.115 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-11-28 13:03:47.977905699 +0000 2023-11-28T13:07:04.115 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2023-11-28 13:03:47.949906194 +0000 2023-11-28T13:07:04.115 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T13:07:04.224 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-11-28T13:07:04.224 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-11-28T13:07:04.224 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000278676 s, 1.8 MB/s 2023-11-28T13:07:04.227 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T13:07:04.275 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 2023-11-28T13:07:04.359 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T13:07:04.359 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T13:07:04.359 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-11-28T13:07:04.359 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T13:07:04.359 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T13:07:04.360 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-11-28 13:06:51.331871751 +0000 2023-11-28T13:07:04.360 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-11-28 13:03:48.817890846 +0000 2023-11-28T13:07:04.360 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-11-28 13:03:48.817890846 +0000 2023-11-28T13:07:04.360 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2023-11-28 13:03:48.791891306 +0000 2023-11-28T13:07:04.360 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T13:07:04.467 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-11-28T13:07:04.467 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-11-28T13:07:04.467 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.00030312 s, 1.7 MB/s 2023-11-28T13:07:04.470 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T13:07:04.515 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T13:07:04.515 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi156.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi186.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T13:07:04.515 INFO:tasks.ceph:Generating config... 2023-11-28T13:07:04.516 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-28T13:07:04.516 INFO:tasks.ceph:[client] debug client = 20 2023-11-28T13:07:04.517 INFO:tasks.ceph:[client] debug ms = 1 2023-11-28T13:07:04.517 INFO:tasks.ceph:[client] pid file = 2023-11-28T13:07:04.517 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-28T13:07:04.517 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-28T13:07:04.517 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-28T13:07:04.517 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-28T13:07:04.517 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-28T13:07:04.517 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-28T13:07:04.517 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-28T13:07:04.517 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-28T13:07:04.517 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-28T13:07:04.518 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-28T13:07:04.518 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-28T13:07:04.518 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-28T13:07:04.518 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-28T13:07:04.518 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-28T13:07:04.518 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-28T13:07:04.518 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-28T13:07:04.518 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-28T13:07:04.518 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-28T13:07:04.518 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-28T13:07:04.518 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-28T13:07:04.518 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-28T13:07:04.519 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-28T13:07:04.519 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-28T13:07:04.519 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-28T13:07:04.519 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-28T13:07:04.519 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-28T13:07:04.519 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-28T13:07:04.519 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-28T13:07:04.519 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-28T13:07:04.519 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-28T13:07:04.519 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-28T13:07:04.519 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-28T13:07:04.519 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-28T13:07:04.520 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-28T13:07:04.520 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-28T13:07:04.520 INFO:tasks.ceph:Setting up mon.a... 2023-11-28T13:07:04.520 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-28T13:07:04.635 INFO:teuthology.orchestra.run.smithi156.stdout:creating /etc/ceph/ceph.keyring 2023-11-28T13:07:04.641 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-28T13:07:04.744 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T13:07:04.814 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.156'), ('mon.b', '172.21.15.186'), ('mon.c', '[v2:172.21.15.186:3301,v1:172.21.15.186:6790]')] 2023-11-28T13:07:04.814 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.156,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-28T13:07:04.815 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:04.815 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T13:07:04.859 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.156 --add b 172.21.15.186 --addv c '[v2:172.21.15.186:3301,v1:172.21.15.186:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-28T13:07:04.989 INFO:teuthology.orchestra.run.smithi156.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-28T13:07:04.989 INFO:teuthology.orchestra.run.smithi156.stdout:monmaptool: generated fsid 12d9e849-dc60-46f0-8e8e-9e291633f61b 2023-11-28T13:07:04.989 INFO:teuthology.orchestra.run.smithi156.stdout:setting min_mon_release = pacific 2023-11-28T13:07:04.989 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 0 2023-11-28T13:07:04.990 INFO:teuthology.orchestra.run.smithi156.stdout:fsid 12d9e849-dc60-46f0-8e8e-9e291633f61b 2023-11-28T13:07:04.990 INFO:teuthology.orchestra.run.smithi156.stdout:last_changed 2023-11-28T13:07:04.989279+0000 2023-11-28T13:07:04.990 INFO:teuthology.orchestra.run.smithi156.stdout:created 2023-11-28T13:07:04.989279+0000 2023-11-28T13:07:04.990 INFO:teuthology.orchestra.run.smithi156.stdout:min_mon_release 16 (pacific) 2023-11-28T13:07:04.990 INFO:teuthology.orchestra.run.smithi156.stdout:election_strategy: 1 2023-11-28T13:07:04.990 INFO:teuthology.orchestra.run.smithi156.stdout:0: [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon.a 2023-11-28T13:07:04.990 INFO:teuthology.orchestra.run.smithi156.stdout:1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.b 2023-11-28T13:07:04.990 INFO:teuthology.orchestra.run.smithi156.stdout:2: [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] mon.c 2023-11-28T13:07:04.990 INFO:teuthology.orchestra.run.smithi156.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-28T13:07:04.992 DEBUG:teuthology.orchestra.run.smithi156:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T13:07:05.036 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 12d9e849-dc60-46f0-8e8e-9e291633f61b... 2023-11-28T13:07:05.037 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T13:07:05.081 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T13:07:05.199 INFO:teuthology.orchestra.run.smithi156.stdout:[global] 2023-11-28T13:07:05.199 INFO:teuthology.orchestra.run.smithi156.stdout: chdir = "" 2023-11-28T13:07:05.199 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T13:07:05.200 INFO:teuthology.orchestra.run.smithi156.stdout: auth supported = cephx 2023-11-28T13:07:05.200 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.200 INFO:teuthology.orchestra.run.smithi156.stdout: filestore xattr use omap = true 2023-11-28T13:07:05.200 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.200 INFO:teuthology.orchestra.run.smithi156.stdout: mon clock drift allowed = 1.000 2023-11-28T13:07:05.200 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.200 INFO:teuthology.orchestra.run.smithi156.stdout: osd crush chooseleaf type = 0 2023-11-28T13:07:05.200 INFO:teuthology.orchestra.run.smithi156.stdout: auth debug = true 2023-11-28T13:07:05.200 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.200 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on old message = true 2023-11-28T13:07:05.200 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on bug = true 2023-11-28T13:07:05.200 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.201 INFO:teuthology.orchestra.run.smithi156.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T13:07:05.201 INFO:teuthology.orchestra.run.smithi156.stdout: mon pg warn max object skew = 0 2023-11-28T13:07:05.201 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.201 INFO:teuthology.orchestra.run.smithi156.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T13:07:05.201 INFO:teuthology.orchestra.run.smithi156.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T13:07:05.201 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.201 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default size = 2 2023-11-28T13:07:05.201 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.201 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow primary affinity = true 2023-11-28T13:07:05.201 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow pg remap = true 2023-11-28T13:07:05.202 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on legacy crush tunables = false 2023-11-28T13:07:05.202 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on crush straw calc version zero = false 2023-11-28T13:07:05.202 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on no sortbitwise = false 2023-11-28T13:07:05.202 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on osd down out interval zero = false 2023-11-28T13:07:05.202 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on too few osds = false 2023-11-28T13:07:05.202 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T13:07:05.202 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T13:07:05.202 INFO:teuthology.orchestra.run.smithi156.stdout: mon_allow_pool_size_one = true 2023-11-28T13:07:05.202 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.202 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T13:07:05.202 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.203 INFO:teuthology.orchestra.run.smithi156.stdout: osd default data pool replay window = 5 2023-11-28T13:07:05.203 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.203 INFO:teuthology.orchestra.run.smithi156.stdout: mon allow pool delete = true 2023-11-28T13:07:05.203 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.203 INFO:teuthology.orchestra.run.smithi156.stdout: mon cluster log file level = debug 2023-11-28T13:07:05.203 INFO:teuthology.orchestra.run.smithi156.stdout: debug asserts on shutdown = true 2023-11-28T13:07:05.203 INFO:teuthology.orchestra.run.smithi156.stdout: mon health detail to clog = false 2023-11-28T13:07:05.203 INFO:teuthology.orchestra.run.smithi156.stdout: mon host = "172.21.15.156,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]" 2023-11-28T13:07:05.203 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on skipped message = False 2023-11-28T13:07:05.203 INFO:teuthology.orchestra.run.smithi156.stdout: fsid = 12d9e849-dc60-46f0-8e8e-9e291633f61b 2023-11-28T13:07:05.203 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.203 INFO:teuthology.orchestra.run.smithi156.stdout:[osd] 2023-11-28T13:07:05.204 INFO:teuthology.orchestra.run.smithi156.stdout: osd journal size = 100 2023-11-28T13:07:05.204 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.204 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub load threshold = 5.0 2023-11-28T13:07:05.204 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub max interval = 600 2023-11-28T13:07:05.204 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.204 INFO:teuthology.orchestra.run.smithi156.stdout: osd recover clone overlap = true 2023-11-28T13:07:05.204 INFO:teuthology.orchestra.run.smithi156.stdout: osd recovery max chunk = 1048576 2023-11-28T13:07:05.204 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.204 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug shutdown = true 2023-11-28T13:07:05.204 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug op order = true 2023-11-28T13:07:05.204 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug verify stray on activate = true 2023-11-28T13:07:05.205 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.205 INFO:teuthology.orchestra.run.smithi156.stdout: osd open classes on start = true 2023-11-28T13:07:05.205 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug pg log writeout = true 2023-11-28T13:07:05.205 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.205 INFO:teuthology.orchestra.run.smithi156.stdout: osd deep scrub update digest min age = 30 2023-11-28T13:07:05.205 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.205 INFO:teuthology.orchestra.run.smithi156.stdout: osd map max advance = 10 2023-11-28T13:07:05.205 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.205 INFO:teuthology.orchestra.run.smithi156.stdout: journal zero on create = true 2023-11-28T13:07:05.205 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.205 INFO:teuthology.orchestra.run.smithi156.stdout: filestore ondisk finisher threads = 3 2023-11-28T13:07:05.206 INFO:teuthology.orchestra.run.smithi156.stdout: filestore apply finisher threads = 3 2023-11-28T13:07:05.206 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.206 INFO:teuthology.orchestra.run.smithi156.stdout: bdev debug aio = true 2023-11-28T13:07:05.206 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug misdirected ops = true 2023-11-28T13:07:05.206 INFO:teuthology.orchestra.run.smithi156.stdout: bdev async discard = True 2023-11-28T13:07:05.206 INFO:teuthology.orchestra.run.smithi156.stdout: bdev enable discard = True 2023-11-28T13:07:05.206 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore allocator = bitmap 2023-11-28T13:07:05.206 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore block size = 96636764160 2023-11-28T13:07:05.206 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore fsck on mount = True 2023-11-28T13:07:05.206 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluefs = 1/20 2023-11-28T13:07:05.206 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluestore = 1/20 2023-11-28T13:07:05.206 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-11-28T13:07:05.207 INFO:teuthology.orchestra.run.smithi156.stdout: debug osd = 20 2023-11-28T13:07:05.207 INFO:teuthology.orchestra.run.smithi156.stdout: debug rocksdb = 4/10 2023-11-28T13:07:05.207 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T13:07:05.207 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd full ratio = 0.9 2023-11-28T13:07:05.207 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd nearfull ratio = 0.8 2023-11-28T13:07:05.207 INFO:teuthology.orchestra.run.smithi156.stdout: osd failsafe full ratio = 0.95 2023-11-28T13:07:05.207 INFO:teuthology.orchestra.run.smithi156.stdout: osd objectstore = bluestore 2023-11-28T13:07:05.207 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2023-11-28T13:07:05.207 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.207 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr] 2023-11-28T13:07:05.207 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-11-28T13:07:05.207 INFO:teuthology.orchestra.run.smithi156.stdout: debug mgr = 20 2023-11-28T13:07:05.208 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2023-11-28T13:07:05.208 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2023-11-28T13:07:05.208 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2023-11-28T13:07:05.208 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2023-11-28T13:07:05.208 INFO:teuthology.orchestra.run.smithi156.stdout: mgr/telemetry/nag = false 2023-11-28T13:07:05.208 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.208 INFO:teuthology.orchestra.run.smithi156.stdout:[mon] 2023-11-28T13:07:05.208 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-11-28T13:07:05.208 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2023-11-28T13:07:05.208 INFO:teuthology.orchestra.run.smithi156.stdout: debug paxos = 20 2023-11-28T13:07:05.208 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2023-11-28T13:07:05.208 INFO:teuthology.orchestra.run.smithi156.stdout: mon data avail warn = 5 2023-11-28T13:07:05.208 INFO:teuthology.orchestra.run.smithi156.stdout: mon mgr mkfs grace = 240 2023-11-28T13:07:05.209 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2023-11-28T13:07:05.209 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd reporter subtree level = osd 2023-11-28T13:07:05.209 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd prime pg temp = true 2023-11-28T13:07:05.209 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2023-11-28T13:07:05.209 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.209 INFO:teuthology.orchestra.run.smithi156.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T13:07:05.209 INFO:teuthology.orchestra.run.smithi156.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T13:07:05.209 INFO:teuthology.orchestra.run.smithi156.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T13:07:05.209 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.209 INFO:teuthology.orchestra.run.smithi156.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T13:07:05.209 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T13:07:05.210 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T13:07:05.210 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.210 INFO:teuthology.orchestra.run.smithi156.stdout: # 1m isn't quite enough 2023-11-28T13:07:05.210 INFO:teuthology.orchestra.run.smithi156.stdout: mon_down_mkfs_grace = 2m 2023-11-28T13:07:05.210 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.210 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_filestore_osds = false 2023-11-28T13:07:05.210 INFO:teuthology.orchestra.run.smithi156.stdout: mon op complaint time = 120 2023-11-28T13:07:05.210 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:05.210 INFO:teuthology.orchestra.run.smithi156.stdout:[client] 2023-11-28T13:07:05.210 INFO:teuthology.orchestra.run.smithi156.stdout: rgw cache enabled = true 2023-11-28T13:07:05.210 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable ops log = true 2023-11-28T13:07:05.211 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable usage log = true 2023-11-28T13:07:05.211 INFO:teuthology.orchestra.run.smithi156.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T13:07:05.211 INFO:teuthology.orchestra.run.smithi156.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T13:07:05.211 INFO:teuthology.orchestra.run.smithi156.stdout: client mount timeout = 600 2023-11-28T13:07:05.211 INFO:teuthology.orchestra.run.smithi156.stdout: debug client = 20 2023-11-28T13:07:05.211 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-11-28T13:07:05.211 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = "" 2023-11-28T13:07:05.211 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2023-11-28T13:07:05.211 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2023-11-28T13:07:05.211 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.a] 2023-11-28T13:07:05.211 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.b] 2023-11-28T13:07:05.212 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.c] 2023-11-28T13:07:05.212 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.a] 2023-11-28T13:07:05.212 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.c] 2023-11-28T13:07:05.212 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.b] 2023-11-28T13:07:05.212 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.d] 2023-11-28T13:07:05.212 INFO:teuthology.orchestra.run.smithi156.stdout:[mds] 2023-11-28T13:07:05.212 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds = 20 2023-11-28T13:07:05.212 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds balancer = 20 2023-11-28T13:07:05.212 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-11-28T13:07:05.212 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug frag = True 2023-11-28T13:07:05.212 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug scatterstat = True 2023-11-28T13:07:05.212 INFO:teuthology.orchestra.run.smithi156.stdout: mds op complaint time = 180 2023-11-28T13:07:05.213 INFO:teuthology.orchestra.run.smithi156.stdout: mds verify scatter = True 2023-11-28T13:07:05.213 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2023-11-28T13:07:05.213 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2023-11-28T13:07:05.213 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2023-11-28T13:07:05.213 INFO:teuthology.orchestra.run.smithi186.stdout:[global] 2023-11-28T13:07:05.213 INFO:teuthology.orchestra.run.smithi186.stdout: chdir = "" 2023-11-28T13:07:05.213 INFO:teuthology.orchestra.run.smithi186.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T13:07:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: auth supported = cephx 2023-11-28T13:07:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: filestore xattr use omap = true 2023-11-28T13:07:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: mon clock drift allowed = 1.000 2023-11-28T13:07:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: osd crush chooseleaf type = 0 2023-11-28T13:07:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: auth debug = true 2023-11-28T13:07:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on old message = true 2023-11-28T13:07:05.214 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on bug = true 2023-11-28T13:07:05.215 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.215 INFO:teuthology.orchestra.run.smithi186.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T13:07:05.215 INFO:teuthology.orchestra.run.smithi186.stdout: mon pg warn max object skew = 0 2023-11-28T13:07:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T13:07:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T13:07:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default size = 2 2023-11-28T13:07:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow primary affinity = true 2023-11-28T13:07:05.218 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow pg remap = true 2023-11-28T13:07:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on legacy crush tunables = false 2023-11-28T13:07:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on crush straw calc version zero = false 2023-11-28T13:07:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on no sortbitwise = false 2023-11-28T13:07:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on osd down out interval zero = false 2023-11-28T13:07:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on too few osds = false 2023-11-28T13:07:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T13:07:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T13:07:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: mon_allow_pool_size_one = true 2023-11-28T13:07:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T13:07:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.219 INFO:teuthology.orchestra.run.smithi186.stdout: osd default data pool replay window = 5 2023-11-28T13:07:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: mon allow pool delete = true 2023-11-28T13:07:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: mon cluster log file level = debug 2023-11-28T13:07:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: debug asserts on shutdown = true 2023-11-28T13:07:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: mon health detail to clog = false 2023-11-28T13:07:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: mon host = "172.21.15.156,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]" 2023-11-28T13:07:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on skipped message = False 2023-11-28T13:07:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: fsid = 12d9e849-dc60-46f0-8e8e-9e291633f61b 2023-11-28T13:07:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.220 INFO:teuthology.orchestra.run.smithi186.stdout:[osd] 2023-11-28T13:07:05.220 INFO:teuthology.orchestra.run.smithi186.stdout: osd journal size = 100 2023-11-28T13:07:05.221 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.221 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub load threshold = 5.0 2023-11-28T13:07:05.221 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub max interval = 600 2023-11-28T13:07:05.221 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.221 INFO:teuthology.orchestra.run.smithi186.stdout: osd recover clone overlap = true 2023-11-28T13:07:05.221 INFO:teuthology.orchestra.run.smithi186.stdout: osd recovery max chunk = 1048576 2023-11-28T13:07:05.221 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.221 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug shutdown = true 2023-11-28T13:07:05.221 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug op order = true 2023-11-28T13:07:05.221 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug verify stray on activate = true 2023-11-28T13:07:05.221 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.221 INFO:teuthology.orchestra.run.smithi186.stdout: osd open classes on start = true 2023-11-28T13:07:05.221 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug pg log writeout = true 2023-11-28T13:07:05.222 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.222 INFO:teuthology.orchestra.run.smithi186.stdout: osd deep scrub update digest min age = 30 2023-11-28T13:07:05.222 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.222 INFO:teuthology.orchestra.run.smithi186.stdout: osd map max advance = 10 2023-11-28T13:07:05.222 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.222 INFO:teuthology.orchestra.run.smithi186.stdout: journal zero on create = true 2023-11-28T13:07:05.222 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.222 INFO:teuthology.orchestra.run.smithi186.stdout: filestore ondisk finisher threads = 3 2023-11-28T13:07:05.222 INFO:teuthology.orchestra.run.smithi186.stdout: filestore apply finisher threads = 3 2023-11-28T13:07:05.222 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.222 INFO:teuthology.orchestra.run.smithi186.stdout: bdev debug aio = true 2023-11-28T13:07:05.222 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug misdirected ops = true 2023-11-28T13:07:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: bdev async discard = True 2023-11-28T13:07:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: bdev enable discard = True 2023-11-28T13:07:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore allocator = bitmap 2023-11-28T13:07:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore block size = 96636764160 2023-11-28T13:07:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore fsck on mount = True 2023-11-28T13:07:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluefs = 1/20 2023-11-28T13:07:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluestore = 1/20 2023-11-28T13:07:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-11-28T13:07:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: debug osd = 20 2023-11-28T13:07:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: debug rocksdb = 4/10 2023-11-28T13:07:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T13:07:05.223 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd full ratio = 0.9 2023-11-28T13:07:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd nearfull ratio = 0.8 2023-11-28T13:07:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: osd failsafe full ratio = 0.95 2023-11-28T13:07:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: osd objectstore = bluestore 2023-11-28T13:07:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: osd op complaint time = 180 2023-11-28T13:07:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.224 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr] 2023-11-28T13:07:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-11-28T13:07:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: debug mgr = 20 2023-11-28T13:07:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2023-11-28T13:07:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2023-11-28T13:07:05.224 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2023-11-28T13:07:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2023-11-28T13:07:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: mgr/telemetry/nag = false 2023-11-28T13:07:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.225 INFO:teuthology.orchestra.run.smithi186.stdout:[mon] 2023-11-28T13:07:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-11-28T13:07:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2023-11-28T13:07:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: debug paxos = 20 2023-11-28T13:07:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2023-11-28T13:07:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: mon data avail warn = 5 2023-11-28T13:07:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: mon mgr mkfs grace = 240 2023-11-28T13:07:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2023-11-28T13:07:05.225 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd reporter subtree level = osd 2023-11-28T13:07:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd prime pg temp = true 2023-11-28T13:07:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2023-11-28T13:07:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T13:07:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T13:07:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T13:07:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T13:07:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T13:07:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T13:07:05.226 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: # 1m isn't quite enough 2023-11-28T13:07:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: mon_down_mkfs_grace = 2m 2023-11-28T13:07:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_filestore_osds = false 2023-11-28T13:07:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: mon op complaint time = 120 2023-11-28T13:07:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-28T13:07:05.227 INFO:teuthology.orchestra.run.smithi186.stdout:[client] 2023-11-28T13:07:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: rgw cache enabled = true 2023-11-28T13:07:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable ops log = true 2023-11-28T13:07:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable usage log = true 2023-11-28T13:07:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T13:07:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T13:07:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: client mount timeout = 600 2023-11-28T13:07:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: debug client = 20 2023-11-28T13:07:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-11-28T13:07:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: pid file = "" 2023-11-28T13:07:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: rados mon op timeout = 900 2023-11-28T13:07:05.228 INFO:teuthology.orchestra.run.smithi186.stdout: rados osd op timeout = 900 2023-11-28T13:07:05.228 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.a] 2023-11-28T13:07:05.228 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.b] 2023-11-28T13:07:05.228 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.c] 2023-11-28T13:07:05.229 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.a] 2023-11-28T13:07:05.229 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.c] 2023-11-28T13:07:05.229 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.b] 2023-11-28T13:07:05.229 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.d] 2023-11-28T13:07:05.229 INFO:teuthology.orchestra.run.smithi186.stdout:[mds] 2023-11-28T13:07:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: debug mds = 20 2023-11-28T13:07:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: debug mds balancer = 20 2023-11-28T13:07:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-11-28T13:07:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: mds debug frag = True 2023-11-28T13:07:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: mds debug scatterstat = True 2023-11-28T13:07:05.229 INFO:teuthology.orchestra.run.smithi186.stdout: mds op complaint time = 180 2023-11-28T13:07:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: mds verify scatter = True 2023-11-28T13:07:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: osd op complaint time = 180 2023-11-28T13:07:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: rados mon op timeout = 900 2023-11-28T13:07:05.230 INFO:teuthology.orchestra.run.smithi186.stdout: rados osd op timeout = 900 2023-11-28T13:07:05.233 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-28T13:07:05.234 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-28T13:07:05.347 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-28T13:07:05.347 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:05.347 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-28T13:07:05.391 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:05.391 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-28T13:07:05.476 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T13:07:05.476 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:05.476 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T13:07:05.477 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T13:07:05.613 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:05.613 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T13:07:05.697 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T13:07:05.697 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:05.697 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T13:07:05.698 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T13:07:05.795 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:05.795 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T13:07:05.880 INFO:tasks.ceph:Setting up mon nodes... 2023-11-28T13:07:05.881 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-28T13:07:05.881 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T13:07:06.009 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T13:07:06.015 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T13:07:06.152 INFO:teuthology.orchestra.run.smithi186.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T13:07:06.161 INFO:tasks.ceph:Setting up mds nodes... 2023-11-28T13:07:06.161 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T13:07:06.288 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T13:07:06.293 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-28T13:07:06.365 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T13:07:06.493 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T13:07:06.499 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-28T13:07:06.569 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T13:07:06.697 INFO:teuthology.orchestra.run.smithi186.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T13:07:06.703 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-28T13:07:06.779 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T13:07:06.905 INFO:teuthology.orchestra.run.smithi186.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T13:07:06.910 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-28T13:07:06.982 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-28T13:07:06.983 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-28T13:07:07.080 INFO:teuthology.orchestra.run.smithi156.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-28T13:07:07.113 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-28T13:07:07.212 INFO:teuthology.orchestra.run.smithi156.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-28T13:07:07.249 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-28T13:07:07.347 INFO:teuthology.orchestra.run.smithi186.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-28T13:07:07.381 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-28T13:07:07.479 INFO:teuthology.orchestra.run.smithi186.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-28T13:07:07.513 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-28T13:07:07.513 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi156.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi186.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T13:07:07.514 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-28T13:07:07.584 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T13:07:07.585 INFO:tasks.ceph:role: osd.0 2023-11-28T13:07:07.585 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T13:07:07.585 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T13:07:07.655 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T13:07:07.655 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T13:07:07.655 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T13:07:07.655 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T13:07:07.655 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T13:07:07.655 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-11-28T13:07:07.655 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T13:07:07.655 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T13:07:07.656 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T13:07:07.656 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T13:07:07.678 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2023-11-28T13:07:07.683 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-11-28T13:07:07.683 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-28T13:07:07.795 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-28T13:07:07.878 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-28T13:07:07.952 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T13:07:07.952 INFO:tasks.ceph:role: osd.1 2023-11-28T13:07:07.952 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T13:07:07.952 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T13:07:08.019 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T13:07:08.020 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T13:07:08.020 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T13:07:08.020 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T13:07:08.020 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T13:07:08.020 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-11-28T13:07:08.020 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T13:07:08.020 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T13:07:08.020 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T13:07:08.020 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T13:07:08.043 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2023-11-28T13:07:08.048 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-11-28T13:07:08.048 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-28T13:07:08.158 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-28T13:07:08.236 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-28T13:07:08.308 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T13:07:08.308 INFO:tasks.ceph:role: osd.2 2023-11-28T13:07:08.308 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T13:07:08.309 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T13:07:08.378 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T13:07:08.378 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T13:07:08.378 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T13:07:08.378 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T13:07:08.378 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T13:07:08.378 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-11-28T13:07:08.378 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T13:07:08.379 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T13:07:08.379 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T13:07:08.379 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T13:07:08.402 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2023-11-28T13:07:08.406 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-11-28T13:07:08.406 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-28T13:07:08.513 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-28T13:07:08.589 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-28T13:07:08.659 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T13:07:08.660 INFO:tasks.ceph:role: osd.3 2023-11-28T13:07:08.660 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T13:07:08.660 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T13:07:08.730 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T13:07:08.730 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T13:07:08.730 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T13:07:08.730 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T13:07:08.730 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T13:07:08.731 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-11-28T13:07:08.731 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T13:07:08.731 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T13:07:08.731 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T13:07:08.731 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T13:07:08.753 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2023-11-28T13:07:08.757 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-11-28T13:07:08.757 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-28T13:07:08.864 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-28T13:07:08.942 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T13:07:09.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:09.069+0000 7f02e80ec640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-28T13:07:09.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:09.069+0000 7f02e80ec640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-28T13:07:09.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:09.069+0000 7f02e80ec640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T13:07:09.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:09.069+0000 7f02e80ec640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T13:07:09.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:09.069+0000 7f02e80ec640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T13:07:09.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:09.071+0000 7f02e80ec640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-28T13:07:12.001 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-28T13:07:12.070 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T13:07:12.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:12.176+0000 7fefc5f50640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-28T13:07:12.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:12.176+0000 7fefc5f50640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-28T13:07:12.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:12.176+0000 7fefc5f50640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T13:07:12.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:12.176+0000 7fefc5f50640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T13:07:12.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:12.176+0000 7fefc5f50640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T13:07:12.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:12.178+0000 7fefc5f50640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-28T13:07:15.118 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-28T13:07:15.190 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T13:07:15.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:15.297+0000 7f7ba7613640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-28T13:07:15.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:15.297+0000 7f7ba7613640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-28T13:07:15.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:15.297+0000 7f7ba7613640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T13:07:15.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:15.297+0000 7f7ba7613640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T13:07:15.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:15.297+0000 7f7ba7613640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T13:07:15.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:15.299+0000 7f7ba7613640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-28T13:07:18.233 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-28T13:07:18.304 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T13:07:18.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:18.412+0000 7fa5bc128640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-28T13:07:18.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:18.412+0000 7fa5bc128640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-28T13:07:18.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:18.412+0000 7fa5bc128640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T13:07:18.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:18.412+0000 7fa5bc128640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T13:07:18.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:18.412+0000 7fa5bc128640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T13:07:18.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:18.414+0000 7fa5bc128640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-28T13:07:21.365 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-28T13:07:21.436 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-28T13:07:21.508 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T13:07:21.508 INFO:tasks.ceph:role: osd.4 2023-11-28T13:07:21.508 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T13:07:21.508 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T13:07:21.584 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T13:07:21.585 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T13:07:21.585 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T13:07:21.585 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T13:07:21.585 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T13:07:21.585 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2023-11-28T13:07:21.585 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T13:07:21.585 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T13:07:21.585 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T13:07:21.585 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T13:07:22.027 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2023-11-28T13:07:22.030 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2023-11-28T13:07:22.031 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-28T13:07:22.239 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-28T13:07:22.322 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-28T13:07:22.393 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T13:07:22.393 INFO:tasks.ceph:role: osd.5 2023-11-28T13:07:22.393 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T13:07:22.393 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T13:07:22.471 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T13:07:22.471 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T13:07:22.471 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T13:07:22.471 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T13:07:22.471 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T13:07:22.471 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2023-11-28T13:07:22.471 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T13:07:22.471 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T13:07:22.471 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T13:07:22.471 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T13:07:22.879 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2023-11-28T13:07:22.884 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2023-11-28T13:07:22.884 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-28T13:07:23.031 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-28T13:07:23.112 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-28T13:07:23.230 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T13:07:23.230 INFO:tasks.ceph:role: osd.6 2023-11-28T13:07:23.230 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T13:07:23.230 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T13:07:23.311 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T13:07:23.311 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T13:07:23.311 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T13:07:23.311 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T13:07:23.311 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T13:07:23.311 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2023-11-28T13:07:23.311 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T13:07:23.311 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T13:07:23.311 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T13:07:23.312 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T13:07:23.764 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2023-11-28T13:07:23.768 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2023-11-28T13:07:23.768 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-28T13:07:23.958 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-28T13:07:24.037 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-28T13:07:24.111 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T13:07:24.111 INFO:tasks.ceph:role: osd.7 2023-11-28T13:07:24.111 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T13:07:24.111 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T13:07:24.189 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T13:07:24.189 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T13:07:24.189 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T13:07:24.189 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T13:07:24.189 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T13:07:24.189 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2023-11-28T13:07:24.189 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T13:07:24.189 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T13:07:24.190 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T13:07:24.190 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T13:07:24.618 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2023-11-28T13:07:24.623 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2023-11-28T13:07:24.623 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-28T13:07:24.739 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-28T13:07:24.816 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T13:07:24.969 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:24.968+0000 7f88b7b7a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-28T13:07:24.969 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:24.968+0000 7f88b7b7a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-28T13:07:24.969 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:24.968+0000 7f88b7b7a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T13:07:24.969 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:24.968+0000 7f88b7b7a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T13:07:24.969 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:24.968+0000 7f88b7b7a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T13:07:24.971 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:24.970+0000 7f88b7b7a640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-28T13:07:27.905 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-28T13:07:27.979 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T13:07:28.086 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:28.085+0000 7f5b181c6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-28T13:07:28.087 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:28.085+0000 7f5b181c6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-28T13:07:28.087 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:28.085+0000 7f5b181c6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T13:07:28.087 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:28.085+0000 7f5b181c6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T13:07:28.087 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:28.085+0000 7f5b181c6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T13:07:28.088 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:28.087+0000 7f5b181c6640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-28T13:07:31.039 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-28T13:07:31.111 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T13:07:31.220 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:31.219+0000 7ff5ab3c6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-28T13:07:31.221 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:31.219+0000 7ff5ab3c6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-28T13:07:31.221 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:31.219+0000 7ff5ab3c6640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T13:07:31.221 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:31.219+0000 7ff5ab3c6640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T13:07:31.221 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:31.219+0000 7ff5ab3c6640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T13:07:31.222 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:31.221+0000 7ff5ab3c6640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-28T13:07:32.511 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:07:32.516 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:07:34.137 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-28T13:07:34.209 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T13:07:34.319 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:34.317+0000 7fac38ce7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-28T13:07:34.319 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:34.318+0000 7fac38ce7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-28T13:07:34.319 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:34.318+0000 7fac38ce7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T13:07:34.319 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:34.318+0000 7fac38ce7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T13:07:34.319 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:34.318+0000 7fac38ce7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T13:07:34.320 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:34.319+0000 7fac38ce7640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-28T13:07:37.250 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-28T13:07:37.320 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-28T13:07:37.320 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:37.321 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-28T13:07:37.392 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:37.392 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-28T13:07:37.503 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:37.503 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-28T13:07:37.614 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:37.615 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-28T13:07:37.726 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:37.726 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-28T13:07:37.839 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:37.839 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-28T13:07:37.950 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:37.951 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-28T13:07:38.061 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:38.061 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-28T13:07:38.135 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:38.135 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-28T13:07:38.243 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:38.244 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-28T13:07:38.354 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:38.354 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-28T13:07:38.465 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:38.465 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-28T13:07:38.580 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:38.580 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-28T13:07:38.690 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:38.690 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-28T13:07:38.800 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:38.800 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-28T13:07:38.845 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:38.845 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-28T13:07:38.929 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:38.930 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-28T13:07:38.972 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:38.973 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-28T13:07:39.058 INFO:tasks.ceph:Adding keys to all mons... 2023-11-28T13:07:39.058 DEBUG:teuthology.orchestra.run.smithi156:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T13:07:39.062 DEBUG:teuthology.orchestra.run.smithi186:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T13:07:39.124 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr.y] 2023-11-28T13:07:39.124 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQD65WVlhmWPABAA5F0d66op3eUWwWPKJ6Mc6w== 2023-11-28T13:07:39.124 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.a] 2023-11-28T13:07:39.124 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQD65WVl34w0ERAA9H2i6sMS60YUKvNCit0idg== 2023-11-28T13:07:39.124 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.c] 2023-11-28T13:07:39.124 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQD65WVl++RtHRAAsenmAb/Y3eRvWw22940lkw== 2023-11-28T13:07:39.124 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.0] 2023-11-28T13:07:39.125 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQD95WVlF1g2BBAAM1AkiYFhmzcy6f2xOUHuSA== 2023-11-28T13:07:39.125 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.1] 2023-11-28T13:07:39.125 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAA5mVlMP+VChAAvBK9dslAIznG9cJCnmarfg== 2023-11-28T13:07:39.125 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.2] 2023-11-28T13:07:39.125 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAD5mVlY5TMERAAH2W59d+/mWQ7vVOFV0uH5g== 2023-11-28T13:07:39.125 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.3] 2023-11-28T13:07:39.125 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAG5mVlKreoGBAAHwbSd1vt4sF/Ahmz97lv5A== 2023-11-28T13:07:39.125 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr.x] 2023-11-28T13:07:39.125 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQD65WVl3oYaCRAALjx/GgP3kX/gyD4ML7Af7Q== 2023-11-28T13:07:39.125 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.b] 2023-11-28T13:07:39.125 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQD65WVlzVSSKRAA3APxxYZXNOOOnG1t7DIIDg== 2023-11-28T13:07:39.125 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.d] 2023-11-28T13:07:39.126 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQD65WVlFDP1NRAASlTZM2P0RL+lcFskrr08uQ== 2023-11-28T13:07:39.126 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.4] 2023-11-28T13:07:39.126 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAM5mVlVb7BORAA2/l64tOgJl6WRDwI1Ee+YA== 2023-11-28T13:07:39.126 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.5] 2023-11-28T13:07:39.126 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAQ5mVlIl0jBRAAYrEb4PGL8amZxfsGUhE0BQ== 2023-11-28T13:07:39.126 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.6] 2023-11-28T13:07:39.126 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAT5mVlicggDRAANowLXao2cG7JHDi+ytdREg== 2023-11-28T13:07:39.126 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.7] 2023-11-28T13:07:39.126 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQAW5mVlPTz9EhAAyyq2SF/TYYStii1bExbDew== 2023-11-28T13:07:39.126 INFO:teuthology.orchestra.run.smithi156.stdout:[client.0] 2023-11-28T13:07:39.126 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQD75WVlhwXEBBAAu/NSgck5Joi7bKQoBne7MA== 2023-11-28T13:07:39.127 INFO:teuthology.orchestra.run.smithi156.stdout:[client.2] 2023-11-28T13:07:39.127 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQD75WVl49+oDBAA/rsj8joMrNDo2FOlvvLvWQ== 2023-11-28T13:07:39.127 INFO:teuthology.orchestra.run.smithi156.stdout:[client.1] 2023-11-28T13:07:39.127 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQD75WVlSLGyFBAATUac8v+U0A55jCLv6SCsAA== 2023-11-28T13:07:39.127 INFO:teuthology.orchestra.run.smithi156.stdout:[client.3] 2023-11-28T13:07:39.127 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQD75WVlmfuWHBAAbf1HvrNxzm3zJwTF6De2Kg== 2023-11-28T13:07:39.165 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr.y] 2023-11-28T13:07:39.166 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQD65WVlhmWPABAA5F0d66op3eUWwWPKJ6Mc6w== 2023-11-28T13:07:39.166 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.a] 2023-11-28T13:07:39.166 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQD65WVl34w0ERAA9H2i6sMS60YUKvNCit0idg== 2023-11-28T13:07:39.166 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.c] 2023-11-28T13:07:39.166 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQD65WVl++RtHRAAsenmAb/Y3eRvWw22940lkw== 2023-11-28T13:07:39.166 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.0] 2023-11-28T13:07:39.166 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQD95WVlF1g2BBAAM1AkiYFhmzcy6f2xOUHuSA== 2023-11-28T13:07:39.166 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.1] 2023-11-28T13:07:39.166 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAA5mVlMP+VChAAvBK9dslAIznG9cJCnmarfg== 2023-11-28T13:07:39.167 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.2] 2023-11-28T13:07:39.167 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAD5mVlY5TMERAAH2W59d+/mWQ7vVOFV0uH5g== 2023-11-28T13:07:39.167 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.3] 2023-11-28T13:07:39.167 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAG5mVlKreoGBAAHwbSd1vt4sF/Ahmz97lv5A== 2023-11-28T13:07:39.167 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr.x] 2023-11-28T13:07:39.167 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQD65WVl3oYaCRAALjx/GgP3kX/gyD4ML7Af7Q== 2023-11-28T13:07:39.167 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.b] 2023-11-28T13:07:39.167 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQD65WVlzVSSKRAA3APxxYZXNOOOnG1t7DIIDg== 2023-11-28T13:07:39.167 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.d] 2023-11-28T13:07:39.167 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQD65WVlFDP1NRAASlTZM2P0RL+lcFskrr08uQ== 2023-11-28T13:07:39.167 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.4] 2023-11-28T13:07:39.167 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAM5mVlVb7BORAA2/l64tOgJl6WRDwI1Ee+YA== 2023-11-28T13:07:39.167 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.5] 2023-11-28T13:07:39.168 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAQ5mVlIl0jBRAAYrEb4PGL8amZxfsGUhE0BQ== 2023-11-28T13:07:39.168 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.6] 2023-11-28T13:07:39.168 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAT5mVlicggDRAANowLXao2cG7JHDi+ytdREg== 2023-11-28T13:07:39.168 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.7] 2023-11-28T13:07:39.168 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAW5mVlPTz9EhAAyyq2SF/TYYStii1bExbDew== 2023-11-28T13:07:39.168 INFO:teuthology.orchestra.run.smithi186.stdout:[client.0] 2023-11-28T13:07:39.168 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQD75WVlhwXEBBAAu/NSgck5Joi7bKQoBne7MA== 2023-11-28T13:07:39.168 INFO:teuthology.orchestra.run.smithi186.stdout:[client.2] 2023-11-28T13:07:39.168 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQD75WVl49+oDBAA/rsj8joMrNDo2FOlvvLvWQ== 2023-11-28T13:07:39.168 INFO:teuthology.orchestra.run.smithi186.stdout:[client.1] 2023-11-28T13:07:39.168 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQD75WVlSLGyFBAATUac8v+U0A55jCLv6SCsAA== 2023-11-28T13:07:39.168 INFO:teuthology.orchestra.run.smithi186.stdout:[client.3] 2023-11-28T13:07:39.169 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQD75WVlmfuWHBAAbf1HvrNxzm3zJwTF6De2Kg== 2023-11-28T13:07:39.169 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T13:07:39.174 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T13:07:39.321 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T13:07:39.325 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T13:07:39.442 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T13:07:39.446 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T13:07:39.560 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:07:39.564 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:07:39.673 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:07:39.677 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:07:39.792 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:07:39.797 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:07:39.911 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:07:39.915 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:07:40.024 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T13:07:40.029 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T13:07:40.137 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T13:07:40.142 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T13:07:40.255 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T13:07:40.260 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T13:07:40.376 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:07:40.380 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:07:40.491 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:07:40.496 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:07:40.607 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:07:40.611 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:07:40.722 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:07:40.727 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:07:40.839 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T13:07:40.843 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T13:07:40.956 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T13:07:40.960 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T13:07:41.074 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T13:07:41.078 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T13:07:41.195 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T13:07:41.200 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T13:07:41.315 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-28T13:07:41.315 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-28T13:07:41.385 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T13:07:41.511 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-28T13:07:41.579 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-28T13:07:41.648 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T13:07:41.777 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-28T13:07:41.849 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-28T13:07:41.919 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T13:07:42.046 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-28T13:07:42.118 DEBUG:teuthology.orchestra.run.smithi156:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T13:07:42.122 DEBUG:teuthology.orchestra.run.smithi186:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T13:07:42.166 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-28T13:07:42.166 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-28T13:07:42.166 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-28T13:07:42.206 INFO:tasks.ceph.mon.a:Started 2023-11-28T13:07:42.206 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-28T13:07:42.206 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-28T13:07:42.211 INFO:tasks.ceph.mon.b:Started 2023-11-28T13:07:42.212 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-28T13:07:42.212 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-28T13:07:42.216 INFO:tasks.ceph.mon.c:Started 2023-11-28T13:07:42.216 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-28T13:07:42.216 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-28T13:07:42.217 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-28T13:07:42.222 INFO:tasks.ceph.mgr.y:Started 2023-11-28T13:07:42.223 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-28T13:07:42.223 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-28T13:07:42.227 INFO:tasks.ceph.mgr.x:Started 2023-11-28T13:07:42.228 INFO:tasks.ceph:Setting crush tunables to default 2023-11-28T13:07:42.228 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd crush tunables default 2023-11-28T13:07:42.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.438+0000 7fbd2d7ff700 1 Processor -- start 2023-11-28T13:07:42.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.438+0000 7fbd2d7ff700 1 -- start start 2023-11-28T13:07:42.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.440+0000 7fbd2d7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd2815c9b0 0x7fbd280b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.440+0000 7fbd2d7ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbd28161340 0x7fbd280d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.440+0000 7fbd2d7ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbd280d2320 0x7fbd280cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.440+0000 7fbd2d7ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd280b5800 con 0x7fbd28161340 2023-11-28T13:07:42.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.440+0000 7fbd2d7ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd280d1950 con 0x7fbd280d2320 2023-11-28T13:07:42.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.440+0000 7fbd2d7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd280b5520 con 0x7fbd2815c9b0 2023-11-28T13:07:42.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.440+0000 7fbd267fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbd28161340 0x7fbd280d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.440+0000 7fbd267fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbd28161340 0x7fbd280d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:51436/0 (socket says 172.21.15.156:51436) 2023-11-28T13:07:42.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.440+0000 7fbd267fc700 1 -- 172.21.15.156:0/2623824825 learned_addr learned my addr 172.21.15.156:0/2623824825 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:42.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.440+0000 7fbd277fe700 1 --2- 172.21.15.156:0/2623824825 >> v2:172.21.15.186:3300/0 conn(0x7fbd280d2320 0x7fbd280cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.440+0000 7fbd267fc700 1 -- 172.21.15.156:0/2623824825 >> v2:172.21.15.156:3300/0 conn(0x7fbd28161340 msgr2=0x7fbd280d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T13:07:42.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.440+0000 7fbd267fc700 1 -- 172.21.15.156:0/2623824825 >> v2:172.21.15.156:3300/0 conn(0x7fbd28161340 msgr2=0x7fbd280d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T13:07:42.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.440+0000 7fbd26ffd700 1 --2- 172.21.15.156:0/2623824825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd2815c9b0 0x7fbd280b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.440+0000 7fbd267fc700 1 --2- 172.21.15.156:0/2623824825 >> v2:172.21.15.156:3300/0 conn(0x7fbd28161340 0x7fbd280d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T13:07:42.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.440+0000 7fbd267fc700 1 --2- 172.21.15.156:0/2623824825 >> v2:172.21.15.156:3300/0 conn(0x7fbd28161340 0x7fbd280d1de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T13:07:42.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.441+0000 7fbd277fe700 1 -- 172.21.15.156:0/2623824825 >> v2:172.21.15.186:3300/0 conn(0x7fbd280d2320 msgr2=0x7fbd280cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T13:07:42.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.441+0000 7fbd277fe700 1 -- 172.21.15.156:0/2623824825 >> v2:172.21.15.186:3300/0 conn(0x7fbd280d2320 msgr2=0x7fbd280cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T13:07:42.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.441+0000 7fbd277fe700 1 --2- 172.21.15.156:0/2623824825 >> v2:172.21.15.186:3300/0 conn(0x7fbd280d2320 0x7fbd280cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T13:07:42.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.441+0000 7fbd26ffd700 1 -- 172.21.15.156:0/2623824825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd2815c9b0 msgr2=0x7fbd280b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-28T13:07:42.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.441+0000 7fbd26ffd700 1 -- 172.21.15.156:0/2623824825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd2815c9b0 msgr2=0x7fbd280b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T13:07:42.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.441+0000 7fbd26ffd700 1 --2- 172.21.15.156:0/2623824825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd2815c9b0 0x7fbd280b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T13:07:42.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.441+0000 7fbd277fe700 1 --2- 172.21.15.156:0/2623824825 >> v2:172.21.15.186:3300/0 conn(0x7fbd280d2320 0x7fbd280cd7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T13:07:42.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.441+0000 7fbd26ffd700 1 --2- 172.21.15.156:0/2623824825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd2815c9b0 0x7fbd280b4bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T13:07:42.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.641+0000 7fbd267fc700 1 --2- 172.21.15.156:0/2623824825 >> v2:172.21.15.156:3300/0 conn(0x7fbd28161340 0x7fbd280d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.641+0000 7fbd267fc700 1 -- 172.21.15.156:0/2623824825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd2815c9b0 msgr2=0x7fbd280b4bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T13:07:42.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.641+0000 7fbd267fc700 1 --2- 172.21.15.156:0/2623824825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd2815c9b0 0x7fbd280b4bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.641+0000 7fbd267fc700 1 -- 172.21.15.156:0/2623824825 >> v2:172.21.15.186:3300/0 conn(0x7fbd280d2320 msgr2=0x7fbd280cd7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T13:07:42.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.641+0000 7fbd267fc700 1 --2- 172.21.15.156:0/2623824825 >> v2:172.21.15.186:3300/0 conn(0x7fbd280d2320 0x7fbd280cd7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.641+0000 7fbd267fc700 1 -- 172.21.15.156:0/2623824825 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd280d1300 con 0x7fbd28161340 2023-11-28T13:07:42.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.641+0000 7fbd267fc700 1 --2- 172.21.15.156:0/2623824825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd28161340 0x7fbd280d1de0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbd2808c4d0 tx=0x7fbd1c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6667722889ebbe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:42.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.642+0000 7fbd25ffb700 1 -- 172.21.15.156:0/2623824825 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c004230 con 0x7fbd28161340 2023-11-28T13:07:42.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.642+0000 7fbd25ffb700 1 -- 172.21.15.156:0/2623824825 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1c004390 con 0x7fbd28161340 2023-11-28T13:07:42.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.642+0000 7fbd25ffb700 1 -- 172.21.15.156:0/2623824825 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c0038e0 con 0x7fbd28161340 2023-11-28T13:07:42.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.642+0000 7fbd2d7ff700 1 -- 172.21.15.156:0/2623824825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd28161340 msgr2=0x7fbd280d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.642+0000 7fbd2d7ff700 1 --2- 172.21.15.156:0/2623824825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd28161340 0x7fbd280d1de0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbd2808c4d0 tx=0x7fbd1c009c30 comp rx=0 tx=0).stop 2023-11-28T13:07:42.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.643+0000 7fbd2d7ff700 1 -- 172.21.15.156:0/2623824825 shutdown_connections 2023-11-28T13:07:42.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.643+0000 7fbd2d7ff700 1 --2- 172.21.15.156:0/2623824825 >> v2:172.21.15.186:3300/0 conn(0x7fbd280d2320 0x7fbd280cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.643+0000 7fbd2d7ff700 1 --2- 172.21.15.156:0/2623824825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd2815c9b0 0x7fbd280b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.643+0000 7fbd2d7ff700 1 --2- 172.21.15.156:0/2623824825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd28161340 0x7fbd280d1de0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.643+0000 7fbd2d7ff700 1 -- 172.21.15.156:0/2623824825 >> 172.21.15.156:0/2623824825 conn(0x7fbd28072600 msgr2=0x7fbd280cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:42.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.643+0000 7fbd2d7ff700 1 -- 172.21.15.156:0/2623824825 shutdown_connections 2023-11-28T13:07:42.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.643+0000 7fbd2d7ff700 1 -- 172.21.15.156:0/2623824825 wait complete. 2023-11-28T13:07:42.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.644+0000 7fbd2d7ff700 1 Processor -- start 2023-11-28T13:07:42.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.645+0000 7fbd2d7ff700 1 -- start start 2023-11-28T13:07:42.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.646+0000 7fbd2d7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd280d2320 0x7fbd28150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.646+0000 7fbd2d7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd2815c9b0 0x7fbd281e6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.646+0000 7fbd2d7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd28161340 0x7fbd280c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.646+0000 7fbd2d7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd281516c0 con 0x7fbd28161340 2023-11-28T13:07:42.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.646+0000 7fbd2d7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd281e7990 con 0x7fbd280d2320 2023-11-28T13:07:42.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.646+0000 7fbd2d7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd281e7c70 con 0x7fbd2815c9b0 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.646+0000 7fbd277fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd28161340 0x7fbd280c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.646+0000 7fbd277fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd28161340 0x7fbd280c7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:51450/0 (socket says 172.21.15.156:51450) 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.646+0000 7fbd277fe700 1 -- 172.21.15.156:0/3648525676 learned_addr learned my addr 172.21.15.156:0/3648525676 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.646+0000 7fbd267fc700 1 --2- 172.21.15.156:0/3648525676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd2815c9b0 0x7fbd281e6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.646+0000 7fbd26ffd700 1 --2- 172.21.15.156:0/3648525676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd280d2320 0x7fbd28150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.647+0000 7fbd277fe700 1 -- 172.21.15.156:0/3648525676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd2815c9b0 msgr2=0x7fbd281e6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.647+0000 7fbd277fe700 1 --2- 172.21.15.156:0/3648525676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd2815c9b0 0x7fbd281e6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.647+0000 7fbd277fe700 1 -- 172.21.15.156:0/3648525676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd280d2320 msgr2=0x7fbd28150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.647+0000 7fbd277fe700 1 --2- 172.21.15.156:0/3648525676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd280d2320 0x7fbd28150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.647+0000 7fbd277fe700 1 -- 172.21.15.156:0/3648525676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1c0095b0 con 0x7fbd28161340 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.647+0000 7fbd277fe700 1 --2- 172.21.15.156:0/3648525676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd28161340 0x7fbd280c7ee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbd20007e30 tx=0x7fbd2000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:42.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.647+0000 7fbd0f7fe700 1 -- 172.21.15.156:0/3648525676 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2000ed30 con 0x7fbd28161340 2023-11-28T13:07:42.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.647+0000 7fbd0f7fe700 1 -- 172.21.15.156:0/3648525676 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2000ee90 con 0x7fbd28161340 2023-11-28T13:07:42.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.647+0000 7fbd0f7fe700 1 -- 172.21.15.156:0/3648525676 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd20014640 con 0x7fbd28161340 2023-11-28T13:07:42.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.647+0000 7fbd2d7ff700 1 -- 172.21.15.156:0/3648525676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd280b5130 con 0x7fbd28161340 2023-11-28T13:07:42.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.647+0000 7fbd2d7ff700 1 -- 172.21.15.156:0/3648525676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd280d15e0 con 0x7fbd28161340 2023-11-28T13:07:42.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.648+0000 7fbd0f7fe700 1 -- 172.21.15.156:0/3648525676 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fbd20012050 con 0x7fbd28161340 2023-11-28T13:07:42.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.648+0000 7fbd0f7fe700 1 -- 172.21.15.156:0/3648525676 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbd2001d980 con 0x7fbd28161340 2023-11-28T13:07:42.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.648+0000 7fbd2d7ff700 1 -- 172.21.15.156:0/3648525676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd280b52c0 con 0x7fbd28161340 2023-11-28T13:07:42.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.652+0000 7fbd0f7fe700 1 -- 172.21.15.156:0/3648525676 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fbd2001d410 con 0x7fbd28161340 2023-11-28T13:07:42.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.731+0000 7fbd2d7ff700 1 -- 172.21.15.156:0/3648525676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fbd28151850 con 0x7fbd28161340 2023-11-28T13:07:42.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.735+0000 7fbd0f7fe700 1 -- 172.21.15.156:0/3648525676 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fbd20014c70 con 0x7fbd28161340 2023-11-28T13:07:42.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.736+0000 7fbd2d7ff700 1 -- 172.21.15.156:0/3648525676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd28161340 msgr2=0x7fbd280c7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.736+0000 7fbd2d7ff700 1 --2- 172.21.15.156:0/3648525676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd28161340 0x7fbd280c7ee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbd20007e30 tx=0x7fbd2000e3c0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.736+0000 7fbd2d7ff700 1 -- 172.21.15.156:0/3648525676 shutdown_connections 2023-11-28T13:07:42.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.736+0000 7fbd2d7ff700 1 --2- 172.21.15.156:0/3648525676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd28161340 0x7fbd280c7ee0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.736+0000 7fbd2d7ff700 1 --2- 172.21.15.156:0/3648525676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd280d2320 0x7fbd28150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.736+0000 7fbd2d7ff700 1 --2- 172.21.15.156:0/3648525676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd2815c9b0 0x7fbd281e6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.736+0000 7fbd2d7ff700 1 -- 172.21.15.156:0/3648525676 >> 172.21.15.156:0/3648525676 conn(0x7fbd28072600 msgr2=0x7fbd28160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:42.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.736+0000 7fbd2d7ff700 1 -- 172.21.15.156:0/3648525676 shutdown_connections 2023-11-28T13:07:42.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:42.736+0000 7fbd2d7ff700 1 -- 172.21.15.156:0/3648525676 wait complete. 2023-11-28T13:07:42.739 INFO:teuthology.orchestra.run.smithi156.stderr:adjusted tunables profile to default 2023-11-28T13:07:42.757 INFO:tasks.ceph:check_enable_crimson: False 2023-11-28T13:07:42.757 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-28T13:07:42.757 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:42.757 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-28T13:07:42.784 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:42.782+0000 7fe7d950f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T13:07:42.825 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:42.826 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-28T13:07:42.880 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:42.878+0000 7fe7d950f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T13:07:42.935 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:42.935 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-28T13:07:42.989 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:42.987+0000 7fe7d950f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T13:07:43.047 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:07:43.047 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-28T13:07:43.111 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:43.109+0000 7fe7d950f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T13:07:43.149 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:43.148+0000 7fe509d39200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T13:07:43.163 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:43.163 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-28T13:07:43.239 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:43.239 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-28T13:07:43.240 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:43.238+0000 7fe509d39200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T13:07:43.309 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:43.307+0000 7fe7d950f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T13:07:43.339 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:43.338+0000 7fe509d39200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T13:07:43.353 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:43.353 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-28T13:07:43.446 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:43.444+0000 7fe509d39200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T13:07:43.463 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:07:43.463 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-28T13:07:43.571 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 220c17ef-9e08-4a62-b062-f68eec1bd446 0 2023-11-28T13:07:43.636 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:43.635+0000 7fe509d39200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T13:07:43.813 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.811+0000 7fa818fff700 1 Processor -- start 2023-11-28T13:07:43.813 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.811+0000 7fa818fff700 1 -- start start 2023-11-28T13:07:43.814 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.812+0000 7fa818fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa81415c9b0 0x7fa8140b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:43.815 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.812+0000 7fa818fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa8140cd7c0 0x7fa8140cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:43.815 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.812+0000 7fa818fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8140ce160 con 0x7fa814161340 2023-11-28T13:07:43.815 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.812+0000 7fa818fff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8140b5800 con 0x7fa8140cd7c0 2023-11-28T13:07:43.815 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.812+0000 7fa818fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140d1950 con 0x7fa81415c9b0 2023-11-28T13:07:43.815 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.812+0000 7fa81259c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa81415c9b0 0x7fa8140b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:43.815 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.812+0000 7fa81259c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa81415c9b0 0x7fa8140b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:46692/0 (socket says 172.21.15.186:46692) 2023-11-28T13:07:43.816 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.812+0000 7fa81259c700 1 -- 172.21.15.186:0/2259519624 learned_addr learned my addr 172.21.15.186:0/2259519624 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T13:07:43.816 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.812+0000 7fa812d9d700 1 --2- 172.21.15.186:0/2259519624 >> v2:172.21.15.186:3300/0 conn(0x7fa8140cd7c0 0x7fa8140cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:43.818 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.813+0000 7fa812d9d700 1 -- 172.21.15.186:0/2259519624 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa81415c9b0 msgr2=0x7fa8140b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:43.818 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.813+0000 7fa812d9d700 1 --2- 172.21.15.186:0/2259519624 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa81415c9b0 0x7fa8140b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:43.818 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.813+0000 7fa812d9d700 1 -- 172.21.15.186:0/2259519624 >> v1:172.21.15.156:6789/0 conn(0x7fa814161340 legacy=0x7fa8140d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:43.819 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.813+0000 7fa81159a700 1 -- 172.21.15.186:0/2259519624 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030770824 0 0) 0x7fa8140ce160 con 0x7fa814161340 2023-11-28T13:07:43.819 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.813+0000 7fa812d9d700 1 -- 172.21.15.186:0/2259519624 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8140d24f0 con 0x7fa8140cd7c0 2023-11-28T13:07:43.819 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.813+0000 7fa81259c700 1 --2- 172.21.15.186:0/2259519624 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa81415c9b0 0x7fa8140b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:43.819 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.813+0000 7fa812d9d700 1 --2- 172.21.15.186:0/2259519624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8140cd7c0 0x7fa8140cdc20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009d90 tx=0x7fa80c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2c75995b07a2a49 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:43.819 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.814+0000 7fa81159a700 1 -- 172.21.15.186:0/2259519624 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c00e070 con 0x7fa8140cd7c0 2023-11-28T13:07:43.819 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.814+0000 7fa81159a700 1 -- 172.21.15.186:0/2259519624 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c00bb20 con 0x7fa8140cd7c0 2023-11-28T13:07:43.819 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.814+0000 7fa81159a700 1 -- 172.21.15.186:0/2259519624 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c012430 con 0x7fa8140cd7c0 2023-11-28T13:07:43.820 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.814+0000 7fa818fff700 1 -- 172.21.15.186:0/2259519624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8140cd7c0 msgr2=0x7fa8140cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:43.820 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.814+0000 7fa818fff700 1 --2- 172.21.15.186:0/2259519624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8140cd7c0 0x7fa8140cdc20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009d90 tx=0x7fa80c00b330 comp rx=0 tx=0).stop 2023-11-28T13:07:43.820 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.814+0000 7fa818fff700 1 -- 172.21.15.186:0/2259519624 shutdown_connections 2023-11-28T13:07:43.820 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.814+0000 7fa818fff700 1 --2- 172.21.15.186:0/2259519624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8140cd7c0 0x7fa8140cdc20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:43.820 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.814+0000 7fa818fff700 1 --2- 172.21.15.186:0/2259519624 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa81415c9b0 0x7fa8140b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:43.820 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.814+0000 7fa818fff700 1 -- 172.21.15.186:0/2259519624 >> 172.21.15.186:0/2259519624 conn(0x7fa814072600 msgr2=0x7fa8140cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:43.821 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.815+0000 7fa818fff700 1 -- 172.21.15.186:0/2259519624 shutdown_connections 2023-11-28T13:07:43.821 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.815+0000 7fa818fff700 1 -- 172.21.15.186:0/2259519624 wait complete. 2023-11-28T13:07:43.824 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.817+0000 7fa818fff700 1 Processor -- start 2023-11-28T13:07:43.825 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.817+0000 7fa818fff700 1 -- start start 2023-11-28T13:07:43.825 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.819+0000 7fa818fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8140cd7c0 0x7fa8141f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:43.825 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.819+0000 7fa818fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa81415c9b0 0x7fa81415e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:43.825 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.819+0000 7fa818fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa814161340 0x7fa81415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:43.825 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.819+0000 7fa818fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81415f440 con 0x7fa814161340 2023-11-28T13:07:43.825 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.819+0000 7fa818fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81415eba0 con 0x7fa81415c9b0 2023-11-28T13:07:43.826 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.819+0000 7fa818fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81415ee80 con 0x7fa8140cd7c0 2023-11-28T13:07:43.826 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.819+0000 7fa811d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa81415c9b0 0x7fa81415e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:43.826 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.819+0000 7fa81259c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8140cd7c0 0x7fa8141f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:43.826 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.819+0000 7fa81259c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8140cd7c0 0x7fa8141f52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:46704/0 (socket says 172.21.15.186:46704) 2023-11-28T13:07:43.826 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.819+0000 7fa811d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa81415c9b0 0x7fa81415e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58600/0 (socket says 172.21.15.186:58600) 2023-11-28T13:07:43.826 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.819+0000 7fa81259c700 1 -- 172.21.15.186:0/207497861 learned_addr learned my addr 172.21.15.186:0/207497861 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T13:07:43.826 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.819+0000 7fa812d9d700 1 --2- 172.21.15.186:0/207497861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa814161340 0x7fa81415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:43.827 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.820+0000 7fa811d9b700 1 -- 172.21.15.186:0/207497861 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8140cd7c0 msgr2=0x7fa8141f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:43.827 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.820+0000 7fa811d9b700 1 --2- 172.21.15.186:0/207497861 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8140cd7c0 0x7fa8141f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:43.827 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.820+0000 7fa811d9b700 1 -- 172.21.15.186:0/207497861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa814161340 msgr2=0x7fa81415a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:43.827 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.820+0000 7fa811d9b700 1 --2- 172.21.15.186:0/207497861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa814161340 0x7fa81415a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:43.827 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.820+0000 7fa811d9b700 1 -- 172.21.15.186:0/207497861 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa804009550 con 0x7fa81415c9b0 2023-11-28T13:07:43.827 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.820+0000 7fa81259c700 1 --2- 172.21.15.186:0/207497861 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8140cd7c0 0x7fa8141f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:43.827 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.820+0000 7fa811d9b700 1 --2- 172.21.15.186:0/207497861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa81415c9b0 0x7fa81415e2c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa80800c3e0 tx=0x7fa80800c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:43.827 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.820+0000 7fa802ffd700 1 -- 172.21.15.186:0/207497861 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8080052d0 con 0x7fa81415c9b0 2023-11-28T13:07:43.827 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.820+0000 7fa802ffd700 1 -- 172.21.15.186:0/207497861 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa808005430 con 0x7fa81415c9b0 2023-11-28T13:07:43.827 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.820+0000 7fa818fff700 1 -- 172.21.15.186:0/207497861 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80c009a70 con 0x7fa81415c9b0 2023-11-28T13:07:43.827 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.820+0000 7fa802ffd700 1 -- 172.21.15.186:0/207497861 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8080038a0 con 0x7fa81415c9b0 2023-11-28T13:07:43.827 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.820+0000 7fa818fff700 1 -- 172.21.15.186:0/207497861 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8140d1300 con 0x7fa81415c9b0 2023-11-28T13:07:43.828 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.821+0000 7fa802ffd700 1 -- 172.21.15.186:0/207497861 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fa808019030 con 0x7fa81415c9b0 2023-11-28T13:07:43.828 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.821+0000 7fa802ffd700 1 -- 172.21.15.186:0/207497861 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa80801cc00 con 0x7fa81415c9b0 2023-11-28T13:07:43.828 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.822+0000 7fa818fff700 1 -- 172.21.15.186:0/207497861 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8140b5520 con 0x7fa81415c9b0 2023-11-28T13:07:43.829 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.825+0000 7fa802ffd700 1 -- 172.21.15.186:0/207497861 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fa808017070 con 0x7fa81415c9b0 2023-11-28T13:07:43.914 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.912+0000 7fa818fff700 1 -- 172.21.15.186:0/207497861 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd new", "uuid": "220c17ef-9e08-4a62-b062-f68eec1bd446", "id": 0} v 0) v1 -- 0x7fa8140d15e0 con 0x7fa81415c9b0 2023-11-28T13:07:43.920 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.918+0000 7fa802ffd700 1 -- 172.21.15.186:0/207497861 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "220c17ef-9e08-4a62-b062-f68eec1bd446", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa808003aa0 con 0x7fa81415c9b0 2023-11-28T13:07:43.921 INFO:teuthology.orchestra.run.smithi186.stdout:0 2023-11-28T13:07:43.923 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.919+0000 7fa818fff700 1 -- 172.21.15.186:0/207497861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa81415c9b0 msgr2=0x7fa81415e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:43.923 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.919+0000 7fa818fff700 1 --2- 172.21.15.186:0/207497861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa81415c9b0 0x7fa81415e2c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa80800c3e0 tx=0x7fa80800c6f0 comp rx=0 tx=0).stop 2023-11-28T13:07:43.923 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.920+0000 7fa818fff700 1 -- 172.21.15.186:0/207497861 shutdown_connections 2023-11-28T13:07:43.923 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.920+0000 7fa818fff700 1 --2- 172.21.15.186:0/207497861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa814161340 0x7fa81415a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:43.924 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.920+0000 7fa818fff700 1 --2- 172.21.15.186:0/207497861 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8140cd7c0 0x7fa8141f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:43.924 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.920+0000 7fa818fff700 1 --2- 172.21.15.186:0/207497861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa81415c9b0 0x7fa81415e2c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:43.924 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.920+0000 7fa818fff700 1 -- 172.21.15.186:0/207497861 >> 172.21.15.186:0/207497861 conn(0x7fa814072600 msgr2=0x7fa8140d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:43.924 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.920+0000 7fa818fff700 1 -- 172.21.15.186:0/207497861 shutdown_connections 2023-11-28T13:07:43.924 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:43.920+0000 7fa818fff700 1 -- 172.21.15.186:0/207497861 wait complete. 2023-11-28T13:07:43.943 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 69f34e6a-bf4b-4a4f-baff-8d6e41022f60 1 2023-11-28T13:07:44.140 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.138+0000 7efe01fff700 1 Processor -- start 2023-11-28T13:07:44.141 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.138+0000 7efe01fff700 1 -- start start 2023-11-28T13:07:44.142 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.139+0000 7efe01fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efdfc0d1330 0x7efdfc0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.143 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.140+0000 7efe01fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7efdfc0b20b0 0x7efdfc0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.143 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.140+0000 7efe01fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdfc0b0230 con 0x7efdfc0d01b0 2023-11-28T13:07:44.143 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.140+0000 7efe01fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdfc15cec0 con 0x7efdfc0d1330 2023-11-28T13:07:44.143 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.140+0000 7efe01fff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdfc0b4570 con 0x7efdfc0b20b0 2023-11-28T13:07:44.143 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.140+0000 7efdfbfff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7efdfc0b20b0 0x7efdfc0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.143 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.140+0000 7efdfbfff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7efdfc0b20b0 0x7efdfc0b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58614/0 (socket says 172.21.15.186:58614) 2023-11-28T13:07:44.143 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.140+0000 7efdfbfff700 1 -- 172.21.15.186:0/2409580823 learned_addr learned my addr 172.21.15.186:0/2409580823 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T13:07:44.143 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.140+0000 7efdfaffd700 1 --2- 172.21.15.186:0/2409580823 >> v2:172.21.15.156:3300/0 conn(0x7efdfc0d1330 0x7efdfc0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.143 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.140+0000 7efdfbfff700 1 -- 172.21.15.186:0/2409580823 >> v2:172.21.15.156:3300/0 conn(0x7efdfc0d1330 msgr2=0x7efdfc0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:44.144 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.140+0000 7efdfbfff700 1 --2- 172.21.15.186:0/2409580823 >> v2:172.21.15.156:3300/0 conn(0x7efdfc0d1330 0x7efdfc0b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.144 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.140+0000 7efdfbfff700 1 -- 172.21.15.186:0/2409580823 >> v1:172.21.15.156:6789/0 conn(0x7efdfc0d01b0 legacy=0x7efdfc0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.144 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.140+0000 7efdfbfff700 1 -- 172.21.15.186:0/2409580823 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc0b4850 con 0x7efdfc0b20b0 2023-11-28T13:07:44.144 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.141+0000 7efdfbfff700 1 --2- 172.21.15.186:0/2409580823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdfc0b20b0 0x7efdfc0b0c00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efdfc066e80 tx=0x7efdf400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b5a5c0a853a5505 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:44.144 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.141+0000 7efdfa7fc700 1 -- 172.21.15.186:0/2409580823 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf400e070 con 0x7efdfc0b20b0 2023-11-28T13:07:44.144 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.141+0000 7efdfa7fc700 1 -- 172.21.15.186:0/2409580823 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf400b910 con 0x7efdfc0b20b0 2023-11-28T13:07:44.144 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.141+0000 7efdfa7fc700 1 -- 172.21.15.186:0/2409580823 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf400bcc0 con 0x7efdfc0b20b0 2023-11-28T13:07:44.145 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.141+0000 7efe01fff700 1 -- 172.21.15.186:0/2409580823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdfc0b20b0 msgr2=0x7efdfc0b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.145 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.141+0000 7efe01fff700 1 --2- 172.21.15.186:0/2409580823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdfc0b20b0 0x7efdfc0b0c00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efdfc066e80 tx=0x7efdf400b330 comp rx=0 tx=0).stop 2023-11-28T13:07:44.145 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.141+0000 7efe01fff700 1 -- 172.21.15.186:0/2409580823 shutdown_connections 2023-11-28T13:07:44.145 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.141+0000 7efe01fff700 1 --2- 172.21.15.186:0/2409580823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdfc0b20b0 0x7efdfc0b0c00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.146 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.141+0000 7efe01fff700 1 --2- 172.21.15.186:0/2409580823 >> v2:172.21.15.156:3300/0 conn(0x7efdfc0d1330 0x7efdfc0b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.146 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.141+0000 7efe01fff700 1 -- 172.21.15.186:0/2409580823 >> 172.21.15.186:0/2409580823 conn(0x7efdfc09c3f0 msgr2=0x7efdfc0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:44.146 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.142+0000 7efe01fff700 1 -- 172.21.15.186:0/2409580823 shutdown_connections 2023-11-28T13:07:44.146 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.142+0000 7efe01fff700 1 -- 172.21.15.186:0/2409580823 wait complete. 2023-11-28T13:07:44.146 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.143+0000 7efe01fff700 1 Processor -- start 2023-11-28T13:07:44.146 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.143+0000 7efe01fff700 1 -- start start 2023-11-28T13:07:44.150 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.144+0000 7efe01fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdfc0b20b0 0x7efdfc155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.150 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.144+0000 7efe01fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efdfc0d01b0 0x7efdfc15f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.150 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.144+0000 7efe01fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc0d1330 0x7efdfc158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.150 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.144+0000 7efe01fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc15d9c0 con 0x7efdfc0d1330 2023-11-28T13:07:44.150 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.144+0000 7efe01fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc15dc60 con 0x7efdfc0b20b0 2023-11-28T13:07:44.151 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.144+0000 7efe01fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc15df40 con 0x7efdfc0d01b0 2023-11-28T13:07:44.151 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.145+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdfc0b20b0 0x7efdfc155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.151 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.145+0000 7efdfaffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efdfc0d01b0 0x7efdfc15f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.151 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.145+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdfc0b20b0 0x7efdfc155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58622/0 (socket says 172.21.15.186:58622) 2023-11-28T13:07:44.151 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.145+0000 7efdfb7fe700 1 -- 172.21.15.186:0/60565862 learned_addr learned my addr 172.21.15.186:0/60565862 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T13:07:44.151 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.145+0000 7efdfbfff700 1 --2- 172.21.15.186:0/60565862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc0d1330 0x7efdfc158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.151 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.145+0000 7efdfb7fe700 1 -- 172.21.15.186:0/60565862 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efdfc0d01b0 msgr2=0x7efdfc15f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.151 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.145+0000 7efdfb7fe700 1 --2- 172.21.15.186:0/60565862 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efdfc0d01b0 0x7efdfc15f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.151 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.145+0000 7efdfb7fe700 1 -- 172.21.15.186:0/60565862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc0d1330 msgr2=0x7efdfc158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.151 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.145+0000 7efdfb7fe700 1 --2- 172.21.15.186:0/60565862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc0d1330 0x7efdfc158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.151 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.145+0000 7efdfb7fe700 1 -- 172.21.15.186:0/60565862 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf4009b30 con 0x7efdfc0b20b0 2023-11-28T13:07:44.152 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.145+0000 7efdfaffd700 1 --2- 172.21.15.186:0/60565862 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efdfc0d01b0 0x7efdfc15f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:44.152 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.145+0000 7efdfb7fe700 1 --2- 172.21.15.186:0/60565862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdfc0b20b0 0x7efdfc155ad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efdec00c4e0 tx=0x7efdec00c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:44.152 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.146+0000 7efde3fff700 1 -- 172.21.15.186:0/60565862 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0052d0 con 0x7efdfc0b20b0 2023-11-28T13:07:44.152 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.146+0000 7efde3fff700 1 -- 172.21.15.186:0/60565862 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec005430 con 0x7efdfc0b20b0 2023-11-28T13:07:44.152 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.146+0000 7efde3fff700 1 -- 172.21.15.186:0/60565862 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0038c0 con 0x7efdfc0b20b0 2023-11-28T13:07:44.152 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.146+0000 7efe01fff700 1 -- 172.21.15.186:0/60565862 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdfc0afec0 con 0x7efdfc0b20b0 2023-11-28T13:07:44.152 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.146+0000 7efe01fff700 1 -- 172.21.15.186:0/60565862 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc0afbe0 con 0x7efdfc0b20b0 2023-11-28T13:07:44.152 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.146+0000 7efde3fff700 1 -- 172.21.15.186:0/60565862 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7efdec019030 con 0x7efdfc0b20b0 2023-11-28T13:07:44.152 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.147+0000 7efde3fff700 1 -- 172.21.15.186:0/60565862 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7efdec0118a0 con 0x7efdfc0b20b0 2023-11-28T13:07:44.152 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.147+0000 7efe01fff700 1 -- 172.21.15.186:0/60565862 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdfc0b2600 con 0x7efdfc0b20b0 2023-11-28T13:07:44.153 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.150+0000 7efde3fff700 1 -- 172.21.15.186:0/60565862 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7efdec017070 con 0x7efdfc0b20b0 2023-11-28T13:07:44.241 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.239+0000 7efe01fff700 1 -- 172.21.15.186:0/60565862 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd new", "uuid": "69f34e6a-bf4b-4a4f-baff-8d6e41022f60", "id": 1} v 0) v1 -- 0x7efdfc068690 con 0x7efdfc0b20b0 2023-11-28T13:07:44.244 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.242+0000 7efde3fff700 1 -- 172.21.15.186:0/60565862 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "69f34e6a-bf4b-4a4f-baff-8d6e41022f60", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7efdec011a60 con 0x7efdfc0b20b0 2023-11-28T13:07:44.245 INFO:teuthology.orchestra.run.smithi186.stdout:1 2023-11-28T13:07:44.246 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.244+0000 7efe01fff700 1 -- 172.21.15.186:0/60565862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdfc0b20b0 msgr2=0x7efdfc155ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.246 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.244+0000 7efe01fff700 1 --2- 172.21.15.186:0/60565862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdfc0b20b0 0x7efdfc155ad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efdec00c4e0 tx=0x7efdec00c7f0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.246 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.244+0000 7efe01fff700 1 -- 172.21.15.186:0/60565862 shutdown_connections 2023-11-28T13:07:44.246 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.244+0000 7efe01fff700 1 --2- 172.21.15.186:0/60565862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdfc0d1330 0x7efdfc158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.246 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.244+0000 7efe01fff700 1 --2- 172.21.15.186:0/60565862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdfc0b20b0 0x7efdfc155ad0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.246 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.244+0000 7efe01fff700 1 --2- 172.21.15.186:0/60565862 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efdfc0d01b0 0x7efdfc15f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.246 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.244+0000 7efe01fff700 1 -- 172.21.15.186:0/60565862 >> 172.21.15.186:0/60565862 conn(0x7efdfc09c3f0 msgr2=0x7efdfc15c490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:44.246 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.244+0000 7efe01fff700 1 -- 172.21.15.186:0/60565862 shutdown_connections 2023-11-28T13:07:44.247 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.244+0000 7efe01fff700 1 -- 172.21.15.186:0/60565862 wait complete. 2023-11-28T13:07:44.264 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 89cd6662-def1-41b7-9cf4-4abb8d377ea0 2 2023-11-28T13:07:44.277 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:44.275+0000 7fe7d950f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T13:07:44.390 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:44.388+0000 7fe7d950f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T13:07:44.463 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.461+0000 7f83399ff700 1 Processor -- start 2023-11-28T13:07:44.463 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.461+0000 7f83399ff700 1 -- start start 2023-11-28T13:07:44.464 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.462+0000 7f83399ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f83340d1330 0x7f83340b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.464 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.462+0000 7f83399ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83340b20b0 0x7f83340b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.464 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.462+0000 7f83399ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83340b0230 con 0x7f83340d01b0 2023-11-28T13:07:44.464 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.462+0000 7f83399ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833415cec0 con 0x7f83340d1330 2023-11-28T13:07:44.464 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.462+0000 7f83399ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83340b4570 con 0x7f83340b20b0 2023-11-28T13:07:44.465 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.462+0000 7f83327fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f83340d1330 0x7f83340b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.465 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.462+0000 7f8332ffd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f83340d01b0 0x7f83340d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.186:40528/0 (socket says 172.21.15.186:40528) 2023-11-28T13:07:44.466 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.462+0000 7f83337fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83340b20b0 0x7f83340b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.466 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.462+0000 7f8332ffd700 1 -- 172.21.15.186:0/3198247840 learned_addr learned my addr 172.21.15.186:0/3198247840 (peer_addr_for_me v1:172.21.15.186:0/0) 2023-11-28T13:07:44.466 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.463+0000 7f8331ffb700 1 -- 172.21.15.186:0/3198247840 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3572375212 0 0) 0x7f83340b0230 con 0x7f83340d01b0 2023-11-28T13:07:44.466 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.463+0000 7f83327fc700 1 -- 172.21.15.186:0/3198247840 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83340b20b0 msgr2=0x7f83340b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:44.466 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.463+0000 7f83327fc700 1 --2- 172.21.15.186:0/3198247840 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83340b20b0 0x7f83340b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.466 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.463+0000 7f83327fc700 1 -- 172.21.15.186:0/3198247840 >> v1:172.21.15.156:6789/0 conn(0x7f83340d01b0 legacy=0x7f83340d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.466 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.463+0000 7f83327fc700 1 -- 172.21.15.186:0/3198247840 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83340b4850 con 0x7f83340d1330 2023-11-28T13:07:44.466 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.463+0000 7f83337fe700 1 --2- 172.21.15.186:0/3198247840 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83340b20b0 0x7f83340b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:44.466 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.463+0000 7f83327fc700 1 --2- 172.21.15.186:0/3198247840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83340d1330 0x7f83340b06c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8328009840 tx=0x7f8328009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9230de1bc3cae998 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:44.466 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.463+0000 7f8331ffb700 1 -- 172.21.15.186:0/3198247840 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328011830 con 0x7f83340d1330 2023-11-28T13:07:44.466 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.463+0000 7f8331ffb700 1 -- 172.21.15.186:0/3198247840 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8328011990 con 0x7f83340d1330 2023-11-28T13:07:44.466 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.463+0000 7f8331ffb700 1 -- 172.21.15.186:0/3198247840 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328011d20 con 0x7f83340d1330 2023-11-28T13:07:44.467 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.464+0000 7f83399ff700 1 -- 172.21.15.186:0/3198247840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83340d1330 msgr2=0x7f83340b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.467 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.464+0000 7f83399ff700 1 --2- 172.21.15.186:0/3198247840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83340d1330 0x7f83340b06c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8328009840 tx=0x7f8328009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:44.467 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.464+0000 7f83399ff700 1 -- 172.21.15.186:0/3198247840 shutdown_connections 2023-11-28T13:07:44.467 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.464+0000 7f83399ff700 1 --2- 172.21.15.186:0/3198247840 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83340b20b0 0x7f83340b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.468 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.464+0000 7f83399ff700 1 --2- 172.21.15.186:0/3198247840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83340d1330 0x7f83340b06c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.468 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.464+0000 7f83399ff700 1 -- 172.21.15.186:0/3198247840 >> 172.21.15.186:0/3198247840 conn(0x7f833409c3f0 msgr2=0x7f83340d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:44.468 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.464+0000 7f83399ff700 1 -- 172.21.15.186:0/3198247840 shutdown_connections 2023-11-28T13:07:44.468 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.464+0000 7f83399ff700 1 -- 172.21.15.186:0/3198247840 wait complete. 2023-11-28T13:07:44.468 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.466+0000 7f83399ff700 1 Processor -- start 2023-11-28T13:07:44.468 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.466+0000 7f83399ff700 1 -- start start 2023-11-28T13:07:44.469 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.467+0000 7f83399ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83340b20b0 0x7f83341584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.469 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.467+0000 7f83399ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83340d01b0 0x7f833415a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.469 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.467+0000 7f83399ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83340d1330 0x7f83340cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.469 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.467+0000 7f83399ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8334158a10 con 0x7f83340b20b0 2023-11-28T13:07:44.470 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.467+0000 7f83399ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833415b000 con 0x7f83340d1330 2023-11-28T13:07:44.470 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.467+0000 7f83399ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83340b6440 con 0x7f83340d01b0 2023-11-28T13:07:44.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.468+0000 7f83327fc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83340d01b0 0x7f833415a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.468+0000 7f8332ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83340b20b0 0x7f83341584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.468+0000 7f83327fc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83340d01b0 0x7f833415a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:46732/0 (socket says 172.21.15.186:46732) 2023-11-28T13:07:44.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.468+0000 7f83327fc700 1 -- 172.21.15.186:0/148470893 learned_addr learned my addr 172.21.15.186:0/148470893 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T13:07:44.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.468+0000 7f83337fe700 1 --2- 172.21.15.186:0/148470893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83340d1330 0x7f83340cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.468+0000 7f83327fc700 1 -- 172.21.15.186:0/148470893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83340d1330 msgr2=0x7f83340cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.468+0000 7f83327fc700 1 --2- 172.21.15.186:0/148470893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83340d1330 0x7f83340cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.468+0000 7f83327fc700 1 -- 172.21.15.186:0/148470893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83340b20b0 msgr2=0x7f83341584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.468+0000 7f83327fc700 1 --2- 172.21.15.186:0/148470893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83340b20b0 0x7f83341584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.468+0000 7f83327fc700 1 -- 172.21.15.186:0/148470893 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c009a70 con 0x7f83340d01b0 2023-11-28T13:07:44.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.468+0000 7f8332ffd700 1 --2- 172.21.15.186:0/148470893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83340b20b0 0x7f83341584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:44.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.468+0000 7f83327fc700 1 --2- 172.21.15.186:0/148470893 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83340d01b0 0x7f833415a7d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f83280090b0 tx=0x7f8328004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:44.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.469+0000 7f831bfff700 1 -- 172.21.15.186:0/148470893 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328004550 con 0x7f83340d01b0 2023-11-28T13:07:44.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.469+0000 7f831bfff700 1 -- 172.21.15.186:0/148470893 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832801f070 con 0x7f83340d01b0 2023-11-28T13:07:44.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.469+0000 7f831bfff700 1 -- 172.21.15.186:0/148470893 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83280038a0 con 0x7f83340d01b0 2023-11-28T13:07:44.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.469+0000 7f83399ff700 1 -- 172.21.15.186:0/148470893 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83280094a0 con 0x7f83340d01b0 2023-11-28T13:07:44.473 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.469+0000 7f83399ff700 1 -- 172.21.15.186:0/148470893 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83340afec0 con 0x7f83340d01b0 2023-11-28T13:07:44.473 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.470+0000 7f831bfff700 1 -- 172.21.15.186:0/148470893 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f832800f050 con 0x7f83340d01b0 2023-11-28T13:07:44.473 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.470+0000 7f831bfff700 1 -- 172.21.15.186:0/148470893 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f8328021070 con 0x7f83340d01b0 2023-11-28T13:07:44.473 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.471+0000 7f83399ff700 1 -- 172.21.15.186:0/148470893 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83340afbe0 con 0x7f83340d01b0 2023-11-28T13:07:44.475 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.474+0000 7f831bfff700 1 -- 172.21.15.186:0/148470893 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f832802b020 con 0x7f83340d01b0 2023-11-28T13:07:44.526 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:44.524+0000 7fe7d950f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T13:07:44.595 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.593+0000 7f83399ff700 1 -- 172.21.15.186:0/148470893 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd new", "uuid": "89cd6662-def1-41b7-9cf4-4abb8d377ea0", "id": 2} v 0) v1 -- 0x7f83340b2600 con 0x7f83340d01b0 2023-11-28T13:07:44.598 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.597+0000 7f831bfff700 1 -- 172.21.15.186:0/148470893 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "89cd6662-def1-41b7-9cf4-4abb8d377ea0", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8328015020 con 0x7f83340d01b0 2023-11-28T13:07:44.599 INFO:teuthology.orchestra.run.smithi186.stdout:2 2023-11-28T13:07:44.600 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.598+0000 7f83399ff700 1 -- 172.21.15.186:0/148470893 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83340d01b0 msgr2=0x7f833415a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.600 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.598+0000 7f83399ff700 1 --2- 172.21.15.186:0/148470893 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83340d01b0 0x7f833415a7d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f83280090b0 tx=0x7f8328004750 comp rx=0 tx=0).stop 2023-11-28T13:07:44.600 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.598+0000 7f83399ff700 1 -- 172.21.15.186:0/148470893 shutdown_connections 2023-11-28T13:07:44.601 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.598+0000 7f83399ff700 1 --2- 172.21.15.186:0/148470893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83340d1330 0x7f83340cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.601 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.598+0000 7f83399ff700 1 --2- 172.21.15.186:0/148470893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83340b20b0 0x7f83341584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.601 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.598+0000 7f83399ff700 1 --2- 172.21.15.186:0/148470893 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83340d01b0 0x7f833415a7d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.601 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.598+0000 7f83399ff700 1 -- 172.21.15.186:0/148470893 >> 172.21.15.186:0/148470893 conn(0x7f833409c3f0 msgr2=0x7f83340b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:44.601 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.598+0000 7f83399ff700 1 -- 172.21.15.186:0/148470893 shutdown_connections 2023-11-28T13:07:44.601 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.598+0000 7f83399ff700 1 -- 172.21.15.186:0/148470893 wait complete. 2023-11-28T13:07:44.614 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:44.613+0000 7fe509d39200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T13:07:44.621 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 9ac08dfd-f39c-4929-a7ae-eff60f1df906 3 2023-11-28T13:07:44.709 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:44.707+0000 7fe7d950f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T13:07:44.714 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:44.713+0000 7fe509d39200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T13:07:44.826 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:44.825+0000 7fe509d39200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T13:07:44.856 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.854+0000 7f5f7739d700 1 Processor -- start 2023-11-28T13:07:44.857 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.854+0000 7f5f7739d700 1 -- start start 2023-11-28T13:07:44.858 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.856+0000 7f5f7739d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5f700cffa0 0x7f5f70160af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.858 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.856+0000 7f5f7739d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5f700cf170 0x7f5f700d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.858 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.856+0000 7f5f7739d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f700d0d20 con 0x7f5f7015c450 2023-11-28T13:07:44.858 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.856+0000 7f5f7739d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f700cdd80 con 0x7f5f700cf170 2023-11-28T13:07:44.858 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.856+0000 7f5f7739d700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f700d0a80 con 0x7f5f700cffa0 2023-11-28T13:07:44.858 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.856+0000 7f5f75b9a700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5f700cffa0 0x7f5f70160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.858 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.856+0000 7f5f7639b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f5f7015c450 0x7f5f700cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.186:40534/0 (socket says 172.21.15.186:40534) 2023-11-28T13:07:44.859 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.856+0000 7f5f7639b700 1 -- 172.21.15.186:0/3446366821 learned_addr learned my addr 172.21.15.186:0/3446366821 (peer_addr_for_me v1:172.21.15.186:0/0) 2023-11-28T13:07:44.859 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.856+0000 7f5f76b9c700 1 --2- 172.21.15.186:0/3446366821 >> v2:172.21.15.156:3300/0 conn(0x7f5f700cf170 0x7f5f700d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.860 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.856+0000 7f5f75b9a700 1 -- 172.21.15.186:0/3446366821 >> v2:172.21.15.156:3300/0 conn(0x7f5f700cf170 msgr2=0x7f5f700d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:44.860 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.857+0000 7f5f75b9a700 1 --2- 172.21.15.186:0/3446366821 >> v2:172.21.15.156:3300/0 conn(0x7f5f700cf170 0x7f5f700d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.860 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.856+0000 7f5f75399700 1 -- 172.21.15.186:0/3446366821 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251053794 0 0) 0x7f5f700d0d20 con 0x7f5f7015c450 2023-11-28T13:07:44.860 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.857+0000 7f5f75b9a700 1 -- 172.21.15.186:0/3446366821 >> v1:172.21.15.156:6789/0 conn(0x7f5f7015c450 legacy=0x7f5f700cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.860 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.857+0000 7f5f75b9a700 1 -- 172.21.15.186:0/3446366821 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f700cd500 con 0x7f5f700cffa0 2023-11-28T13:07:44.861 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.857+0000 7f5f75b9a700 1 --2- 172.21.15.186:0/3446366821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f700cffa0 0x7f5f70160af0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5f700d0510 tx=0x7f5f60009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fc0e582e335c9af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:44.861 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.857+0000 7f5f75399700 1 -- 172.21.15.186:0/3446366821 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60011830 con 0x7f5f700cffa0 2023-11-28T13:07:44.861 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.857+0000 7f5f75399700 1 -- 172.21.15.186:0/3446366821 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f60011990 con 0x7f5f700cffa0 2023-11-28T13:07:44.861 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.857+0000 7f5f75399700 1 -- 172.21.15.186:0/3446366821 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60011d20 con 0x7f5f700cffa0 2023-11-28T13:07:44.861 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.858+0000 7f5f7739d700 1 -- 172.21.15.186:0/3446366821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f700cffa0 msgr2=0x7f5f70160af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.861 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.858+0000 7f5f7739d700 1 --2- 172.21.15.186:0/3446366821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f700cffa0 0x7f5f70160af0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5f700d0510 tx=0x7f5f60009b20 comp rx=0 tx=0).stop 2023-11-28T13:07:44.861 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.858+0000 7f5f7739d700 1 -- 172.21.15.186:0/3446366821 shutdown_connections 2023-11-28T13:07:44.861 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.858+0000 7f5f7739d700 1 --2- 172.21.15.186:0/3446366821 >> v2:172.21.15.156:3300/0 conn(0x7f5f700cf170 0x7f5f700d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.861 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.858+0000 7f5f7739d700 1 --2- 172.21.15.186:0/3446366821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f700cffa0 0x7f5f70160af0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.861 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.858+0000 7f5f7739d700 1 -- 172.21.15.186:0/3446366821 >> 172.21.15.186:0/3446366821 conn(0x7f5f70072630 msgr2=0x7f5f7006a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:44.861 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.858+0000 7f5f7739d700 1 -- 172.21.15.186:0/3446366821 shutdown_connections 2023-11-28T13:07:44.861 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.858+0000 7f5f7739d700 1 -- 172.21.15.186:0/3446366821 wait complete. 2023-11-28T13:07:44.863 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.861+0000 7f5f7739d700 1 Processor -- start 2023-11-28T13:07:44.863 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.861+0000 7f5f7739d700 1 -- start start 2023-11-28T13:07:44.865 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.863+0000 7f5f7739d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f700cf170 0x7f5f7015d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.865 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.863+0000 7f5f7739d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f700cffa0 0x7f5f700cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.865 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.863+0000 7f5f7739d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f7015c450 0x7f5f700b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.865 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.863+0000 7f5f7739d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7015c820 con 0x7f5f700cf170 2023-11-28T13:07:44.865 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.863+0000 7f5f7739d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7015da90 con 0x7f5f7015c450 2023-11-28T13:07:44.865 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.863+0000 7f5f7739d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700cae00 con 0x7f5f700cffa0 2023-11-28T13:07:44.866 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.863+0000 7f5f75b9a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f700cffa0 0x7f5f700cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.866 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.863+0000 7f5f76b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f7015c450 0x7f5f700b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.866 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.863+0000 7f5f75b9a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f700cffa0 0x7f5f700cb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:46744/0 (socket says 172.21.15.186:46744) 2023-11-28T13:07:44.866 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.863+0000 7f5f75b9a700 1 -- 172.21.15.186:0/1444755623 learned_addr learned my addr 172.21.15.186:0/1444755623 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T13:07:44.866 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.863+0000 7f5f7639b700 1 --2- 172.21.15.186:0/1444755623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f700cf170 0x7f5f7015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.867 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.863+0000 7f5f75b9a700 1 -- 172.21.15.186:0/1444755623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f7015c450 msgr2=0x7f5f700b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.868 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.863+0000 7f5f75b9a700 1 --2- 172.21.15.186:0/1444755623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f7015c450 0x7f5f700b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.868 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.864+0000 7f5f75b9a700 1 -- 172.21.15.186:0/1444755623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f700cf170 msgr2=0x7f5f7015d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.868 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.864+0000 7f5f75b9a700 1 --2- 172.21.15.186:0/1444755623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f700cf170 0x7f5f7015d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.868 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.864+0000 7f5f75b9a700 1 -- 172.21.15.186:0/1444755623 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f600094a0 con 0x7f5f700cffa0 2023-11-28T13:07:44.868 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.864+0000 7f5f75b9a700 1 --2- 172.21.15.186:0/1444755623 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f700cffa0 0x7f5f700cb7b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f600090b0 tx=0x7f5f60004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:44.868 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.864+0000 7f5f66ffd700 1 -- 172.21.15.186:0/1444755623 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f600036a0 con 0x7f5f700cffa0 2023-11-28T13:07:44.868 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.864+0000 7f5f66ffd700 1 -- 172.21.15.186:0/1444755623 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6001f070 con 0x7f5f700cffa0 2023-11-28T13:07:44.868 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.864+0000 7f5f7739d700 1 -- 172.21.15.186:0/1444755623 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f700cdaa0 con 0x7f5f700cffa0 2023-11-28T13:07:44.868 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.864+0000 7f5f66ffd700 1 -- 172.21.15.186:0/1444755623 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60003a20 con 0x7f5f700cffa0 2023-11-28T13:07:44.868 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.864+0000 7f5f7739d700 1 -- 172.21.15.186:0/1444755623 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f700cd7c0 con 0x7f5f700cffa0 2023-11-28T13:07:44.868 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.865+0000 7f5f66ffd700 1 -- 172.21.15.186:0/1444755623 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f5f6000f050 con 0x7f5f700cffa0 2023-11-28T13:07:44.868 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.865+0000 7f5f66ffd700 1 -- 172.21.15.186:0/1444755623 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f5f60021070 con 0x7f5f700cffa0 2023-11-28T13:07:44.869 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.866+0000 7f5f7739d700 1 -- 172.21.15.186:0/1444755623 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f700cf5f0 con 0x7f5f700cffa0 2023-11-28T13:07:44.870 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.868+0000 7f5f66ffd700 1 -- 172.21.15.186:0/1444755623 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f5f60015020 con 0x7f5f700cffa0 2023-11-28T13:07:44.971 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.969+0000 7f5f7739d700 1 -- 172.21.15.186:0/1444755623 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9ac08dfd-f39c-4929-a7ae-eff60f1df906", "id": 3} v 0) v1 -- 0x7f5f70061e80 con 0x7f5f700cffa0 2023-11-28T13:07:44.975 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.973+0000 7f5f66ffd700 1 -- 172.21.15.186:0/1444755623 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9ac08dfd-f39c-4929-a7ae-eff60f1df906", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5f60003b80 con 0x7f5f700cffa0 2023-11-28T13:07:44.976 INFO:teuthology.orchestra.run.smithi186.stdout:3 2023-11-28T13:07:44.977 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.975+0000 7f5f7739d700 1 -- 172.21.15.186:0/1444755623 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f700cffa0 msgr2=0x7f5f700cb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.977 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.975+0000 7f5f7739d700 1 --2- 172.21.15.186:0/1444755623 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f700cffa0 0x7f5f700cb7b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f600090b0 tx=0x7f5f60004750 comp rx=0 tx=0).stop 2023-11-28T13:07:44.977 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.975+0000 7f5f7739d700 1 -- 172.21.15.186:0/1444755623 shutdown_connections 2023-11-28T13:07:44.977 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.975+0000 7f5f7739d700 1 --2- 172.21.15.186:0/1444755623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f7015c450 0x7f5f700b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.977 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.975+0000 7f5f7739d700 1 --2- 172.21.15.186:0/1444755623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f700cf170 0x7f5f7015d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.977 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.975+0000 7f5f7739d700 1 --2- 172.21.15.186:0/1444755623 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5f700cffa0 0x7f5f700cb7b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.977 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.975+0000 7f5f7739d700 1 -- 172.21.15.186:0/1444755623 >> 172.21.15.186:0/1444755623 conn(0x7f5f70072630 msgr2=0x7f5f700c9570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:44.977 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.975+0000 7f5f7739d700 1 -- 172.21.15.186:0/1444755623 shutdown_connections 2023-11-28T13:07:44.978 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:44.975+0000 7f5f7739d700 1 -- 172.21.15.186:0/1444755623 wait complete. 2023-11-28T13:07:44.995 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 1b0b7457-5a57-456a-b22d-4751c0004593 4 2023-11-28T13:07:45.038 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:45.036+0000 7fe509d39200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T13:07:45.215 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.213+0000 7f3e809ff700 1 Processor -- start 2023-11-28T13:07:45.216 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.214+0000 7f3e809ff700 1 -- start start 2023-11-28T13:07:45.218 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.215+0000 7f3e809ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3e7c158ce0 0x7f3e7c15d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.218 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.215+0000 7f3e809ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3e7c06fb30 0x7f3e7c0cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.218 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.215+0000 7f3e809ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e7c0ce2e0 0x7f3e7c06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.218 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.215+0000 7f3e809ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7c06bf10 con 0x7f3e7c06fb30 2023-11-28T13:07:45.219 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.215+0000 7f3e809ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7c15d730 con 0x7f3e7c158ce0 2023-11-28T13:07:45.219 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.215+0000 7f3e809ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0ce6f0 con 0x7f3e7c0ce2e0 2023-11-28T13:07:45.219 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.216+0000 7f3e7a59c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3e7c158ce0 0x7f3e7c15d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.219 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.216+0000 7f3e7a59c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3e7c158ce0 0x7f3e7c15d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58664/0 (socket says 172.21.15.186:58664) 2023-11-28T13:07:45.219 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.216+0000 7f3e7a59c700 1 -- 172.21.15.186:0/1164807455 learned_addr learned my addr 172.21.15.186:0/1164807455 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T13:07:45.219 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.216+0000 7f3e79d9b700 1 --2- 172.21.15.186:0/1164807455 >> v2:172.21.15.156:3300/0 conn(0x7f3e7c06fb30 0x7f3e7c0cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.219 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.216+0000 7f3e7ad9d700 1 --2- 172.21.15.186:0/1164807455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e7c0ce2e0 0x7f3e7c06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.219 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.216+0000 7f3e7a59c700 1 -- 172.21.15.186:0/1164807455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e7c0ce2e0 msgr2=0x7f3e7c06c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:45.219 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.216+0000 7f3e7a59c700 1 --2- 172.21.15.186:0/1164807455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e7c0ce2e0 0x7f3e7c06c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.219 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.216+0000 7f3e7a59c700 1 -- 172.21.15.186:0/1164807455 >> v2:172.21.15.156:3300/0 conn(0x7f3e7c06fb30 msgr2=0x7f3e7c0cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:45.219 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.216+0000 7f3e7a59c700 1 --2- 172.21.15.186:0/1164807455 >> v2:172.21.15.156:3300/0 conn(0x7f3e7c06fb30 0x7f3e7c0cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.219 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.216+0000 7f3e7a59c700 1 -- 172.21.15.186:0/1164807455 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c1590b0 con 0x7f3e7c158ce0 2023-11-28T13:07:45.220 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.216+0000 7f3e79d9b700 1 --2- 172.21.15.186:0/1164807455 >> v2:172.21.15.156:3300/0 conn(0x7f3e7c06fb30 0x7f3e7c0cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:45.220 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.216+0000 7f3e7a59c700 1 --2- 172.21.15.186:0/1164807455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c158ce0 0x7f3e7c15d1f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c06fa30 tx=0x7f3e6c009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c81444fe354149c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:45.221 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.217+0000 7f3e7959a700 1 -- 172.21.15.186:0/1164807455 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c011830 con 0x7f3e7c158ce0 2023-11-28T13:07:45.221 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.217+0000 7f3e7959a700 1 -- 172.21.15.186:0/1164807455 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c011990 con 0x7f3e7c158ce0 2023-11-28T13:07:45.221 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.217+0000 7f3e7959a700 1 -- 172.21.15.186:0/1164807455 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c011d20 con 0x7f3e7c158ce0 2023-11-28T13:07:45.221 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.218+0000 7f3e809ff700 1 -- 172.21.15.186:0/1164807455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c158ce0 msgr2=0x7f3e7c15d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.221 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.218+0000 7f3e809ff700 1 --2- 172.21.15.186:0/1164807455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c158ce0 0x7f3e7c15d1f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c06fa30 tx=0x7f3e6c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.221 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.218+0000 7f3e809ff700 1 -- 172.21.15.186:0/1164807455 shutdown_connections 2023-11-28T13:07:45.221 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.218+0000 7f3e809ff700 1 --2- 172.21.15.186:0/1164807455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e7c0ce2e0 0x7f3e7c06c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.221 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.218+0000 7f3e809ff700 1 --2- 172.21.15.186:0/1164807455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c158ce0 0x7f3e7c15d1f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.221 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.218+0000 7f3e809ff700 1 --2- 172.21.15.186:0/1164807455 >> v2:172.21.15.156:3300/0 conn(0x7f3e7c06fb30 0x7f3e7c0cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.221 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.218+0000 7f3e809ff700 1 -- 172.21.15.186:0/1164807455 >> 172.21.15.186:0/1164807455 conn(0x7f3e7c0cb670 msgr2=0x7f3e7c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:45.222 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.218+0000 7f3e809ff700 1 -- 172.21.15.186:0/1164807455 shutdown_connections 2023-11-28T13:07:45.222 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.218+0000 7f3e809ff700 1 -- 172.21.15.186:0/1164807455 wait complete. 2023-11-28T13:07:45.222 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.220+0000 7f3e809ff700 1 Processor -- start 2023-11-28T13:07:45.222 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.220+0000 7f3e809ff700 1 -- start start 2023-11-28T13:07:45.225 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.221+0000 7f3e809ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e7c06fb30 0x7f3e7c0af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.225 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.221+0000 7f3e809ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c0ce2e0 0x7f3e7c147df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.225 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.221+0000 7f3e809ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e7c158ce0 0x7f3e7c14a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.225 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.221+0000 7f3e809ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0b35d0 con 0x7f3e7c158ce0 2023-11-28T13:07:45.225 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.221+0000 7f3e809ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0b2530 con 0x7f3e7c0ce2e0 2023-11-28T13:07:45.225 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.221+0000 7f3e809ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0b2810 con 0x7f3e7c06fb30 2023-11-28T13:07:45.225 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.222+0000 7f3e7a59c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e7c06fb30 0x7f3e7c0af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.225 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.222+0000 7f3e79d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c0ce2e0 0x7f3e7c147df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.225 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.222+0000 7f3e7a59c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e7c06fb30 0x7f3e7c0af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:46760/0 (socket says 172.21.15.186:46760) 2023-11-28T13:07:45.226 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.222+0000 7f3e7a59c700 1 -- 172.21.15.186:0/1759306700 learned_addr learned my addr 172.21.15.186:0/1759306700 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T13:07:45.226 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.222+0000 7f3e79d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c0ce2e0 0x7f3e7c147df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58668/0 (socket says 172.21.15.186:58668) 2023-11-28T13:07:45.226 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.222+0000 7f3e7ad9d700 1 --2- 172.21.15.186:0/1759306700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e7c158ce0 0x7f3e7c14a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.226 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.222+0000 7f3e7a59c700 1 -- 172.21.15.186:0/1759306700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c0ce2e0 msgr2=0x7f3e7c147df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.226 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.222+0000 7f3e7a59c700 1 --2- 172.21.15.186:0/1759306700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c0ce2e0 0x7f3e7c147df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.226 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.222+0000 7f3e7a59c700 1 -- 172.21.15.186:0/1759306700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e7c158ce0 msgr2=0x7f3e7c14a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.226 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.222+0000 7f3e7a59c700 1 --2- 172.21.15.186:0/1759306700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e7c158ce0 0x7f3e7c14a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.226 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.222+0000 7f3e7a59c700 1 -- 172.21.15.186:0/1759306700 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e700094a0 con 0x7f3e7c06fb30 2023-11-28T13:07:45.226 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.222+0000 7f3e79d9b700 1 --2- 172.21.15.186:0/1759306700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c0ce2e0 0x7f3e7c147df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:45.226 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.222+0000 7f3e7ad9d700 1 --2- 172.21.15.186:0/1759306700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e7c158ce0 0x7f3e7c14a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:45.226 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.222+0000 7f3e7a59c700 1 --2- 172.21.15.186:0/1759306700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e7c06fb30 0x7f3e7c0af710 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009c10 tx=0x7f3e6c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:45.227 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.223+0000 7f3e6affd700 1 -- 172.21.15.186:0/1759306700 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c004450 con 0x7f3e7c06fb30 2023-11-28T13:07:45.227 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.223+0000 7f3e6affd700 1 -- 172.21.15.186:0/1759306700 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c01e070 con 0x7f3e7c06fb30 2023-11-28T13:07:45.227 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.223+0000 7f3e809ff700 1 -- 172.21.15.186:0/1759306700 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6c009550 con 0x7f3e7c06fb30 2023-11-28T13:07:45.227 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.223+0000 7f3e6affd700 1 -- 172.21.15.186:0/1759306700 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c0038e0 con 0x7f3e7c06fb30 2023-11-28T13:07:45.227 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.223+0000 7f3e809ff700 1 -- 172.21.15.186:0/1759306700 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c0cb3b0 con 0x7f3e7c06fb30 2023-11-28T13:07:45.227 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.223+0000 7f3e6affd700 1 -- 172.21.15.186:0/1759306700 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f3e6c00f050 con 0x7f3e7c06fb30 2023-11-28T13:07:45.227 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.223+0000 7f3e6affd700 1 -- 172.21.15.186:0/1759306700 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f3e6c02db40 con 0x7f3e7c06fb30 2023-11-28T13:07:45.227 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.224+0000 7f3e809ff700 1 -- 172.21.15.186:0/1759306700 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7c0cb0d0 con 0x7f3e7c06fb30 2023-11-28T13:07:45.229 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.227+0000 7f3e6affd700 1 -- 172.21.15.186:0/1759306700 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f3e6c020070 con 0x7f3e7c06fb30 2023-11-28T13:07:45.259 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:45.257+0000 7fe7d950f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T13:07:45.329 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.327+0000 7f3e809ff700 1 -- 172.21.15.186:0/1759306700 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1b0b7457-5a57-456a-b22d-4751c0004593", "id": 4} v 0) v1 -- 0x7f3e7c06b8c0 con 0x7f3e7c06fb30 2023-11-28T13:07:45.338 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.336+0000 7f3e6affd700 1 -- 172.21.15.186:0/1759306700 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1b0b7457-5a57-456a-b22d-4751c0004593", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3e6c015020 con 0x7f3e7c06fb30 2023-11-28T13:07:45.338 INFO:teuthology.orchestra.run.smithi186.stdout:4 2023-11-28T13:07:45.339 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.337+0000 7f3e809ff700 1 -- 172.21.15.186:0/1759306700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e7c06fb30 msgr2=0x7f3e7c0af710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.340 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.337+0000 7f3e809ff700 1 --2- 172.21.15.186:0/1759306700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e7c06fb30 0x7f3e7c0af710 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009c10 tx=0x7f3e6c004750 comp rx=0 tx=0).stop 2023-11-28T13:07:45.340 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.337+0000 7f3e809ff700 1 -- 172.21.15.186:0/1759306700 shutdown_connections 2023-11-28T13:07:45.340 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.337+0000 7f3e809ff700 1 --2- 172.21.15.186:0/1759306700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e7c158ce0 0x7f3e7c14a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.340 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.337+0000 7f3e809ff700 1 --2- 172.21.15.186:0/1759306700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e7c06fb30 0x7f3e7c0af710 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.340 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.337+0000 7f3e809ff700 1 --2- 172.21.15.186:0/1759306700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e7c0ce2e0 0x7f3e7c147df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.340 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.337+0000 7f3e809ff700 1 -- 172.21.15.186:0/1759306700 >> 172.21.15.186:0/1759306700 conn(0x7f3e7c0cb670 msgr2=0x7f3e7c071360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:45.340 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.337+0000 7f3e809ff700 1 -- 172.21.15.186:0/1759306700 shutdown_connections 2023-11-28T13:07:45.340 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.338+0000 7f3e809ff700 1 -- 172.21.15.186:0/1759306700 wait complete. 2023-11-28T13:07:45.357 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 13b142fc-4857-4039-bee3-1d616041b4ab 5 2023-11-28T13:07:45.557 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.555+0000 7fd1921ff700 1 Processor -- start 2023-11-28T13:07:45.557 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.555+0000 7fd1921ff700 1 -- start start 2023-11-28T13:07:45.559 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.557+0000 7fd1921ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd18c0d1330 0x7fd18c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.559 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.557+0000 7fd1921ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18c0b0230 con 0x7fd18c0d01b0 2023-11-28T13:07:45.559 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.557+0000 7fd1921ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18c0b4570 con 0x7fd18c0b20b0 2023-11-28T13:07:45.559 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.557+0000 7fd1921ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18c15cec0 con 0x7fd18c0d1330 2023-11-28T13:07:45.560 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.557+0000 7fd18bfff700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fd18c0b20b0 0x7fd18c0b0c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.186:39764/0 (socket says 172.21.15.186:39764) 2023-11-28T13:07:45.560 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.557+0000 7fd18bfff700 1 -- 172.21.15.186:0/3376421720 learned_addr learned my addr 172.21.15.186:0/3376421720 (peer_addr_for_me v1:172.21.15.186:0/0) 2023-11-28T13:07:45.560 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.557+0000 7fd18affd700 1 --2- 172.21.15.186:0/3376421720 >> v2:172.21.15.156:3300/0 conn(0x7fd18c0d1330 0x7fd18c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.561 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.558+0000 7fd18a7fc700 1 -- 172.21.15.186:0/3376421720 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766097321 0 0) 0x7fd18c0b4570 con 0x7fd18c0b20b0 2023-11-28T13:07:45.561 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.558+0000 7fd18a7fc700 1 -- 172.21.15.186:0/3376421720 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1740035b0 con 0x7fd18c0b20b0 2023-11-28T13:07:45.561 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.558+0000 7fd18affd700 1 -- 172.21.15.186:0/3376421720 >> v1:172.21.15.186:6789/0 conn(0x7fd18c0b20b0 legacy=0x7fd18c0b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.561 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.558+0000 7fd18affd700 1 -- 172.21.15.186:0/3376421720 >> v1:172.21.15.156:6789/0 conn(0x7fd18c0d01b0 legacy=0x7fd18c0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.561 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.558+0000 7fd18a7fc700 1 -- 172.21.15.186:0/3376421720 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346684404 0 0) 0x7fd18c0b0230 con 0x7fd18c0d01b0 2023-11-28T13:07:45.561 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.558+0000 7fd18affd700 1 -- 172.21.15.186:0/3376421720 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c0b4850 con 0x7fd18c0d1330 2023-11-28T13:07:45.561 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.558+0000 7fd18affd700 1 --2- 172.21.15.186:0/3376421720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd18c0d1330 0x7fd18c0b06c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd180009840 tx=0x7fd180009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e366fc137076ef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:45.562 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.559+0000 7fd18a7fc700 1 -- 172.21.15.186:0/3376421720 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180011830 con 0x7fd18c0d1330 2023-11-28T13:07:45.562 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.559+0000 7fd18a7fc700 1 -- 172.21.15.186:0/3376421720 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd180011990 con 0x7fd18c0d1330 2023-11-28T13:07:45.562 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.559+0000 7fd18a7fc700 1 -- 172.21.15.186:0/3376421720 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180011d20 con 0x7fd18c0d1330 2023-11-28T13:07:45.563 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.560+0000 7fd1921ff700 1 -- 172.21.15.186:0/3376421720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd18c0d1330 msgr2=0x7fd18c0b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.563 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.560+0000 7fd1921ff700 1 --2- 172.21.15.186:0/3376421720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd18c0d1330 0x7fd18c0b06c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd180009840 tx=0x7fd180009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:45.563 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.560+0000 7fd1921ff700 1 -- 172.21.15.186:0/3376421720 shutdown_connections 2023-11-28T13:07:45.563 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.560+0000 7fd1921ff700 1 --2- 172.21.15.186:0/3376421720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd18c0d1330 0x7fd18c0b06c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.563 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.560+0000 7fd1921ff700 1 -- 172.21.15.186:0/3376421720 >> 172.21.15.186:0/3376421720 conn(0x7fd18c09c3f0 msgr2=0x7fd18c0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:45.563 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.560+0000 7fd1921ff700 1 -- 172.21.15.186:0/3376421720 shutdown_connections 2023-11-28T13:07:45.563 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.560+0000 7fd1921ff700 1 -- 172.21.15.186:0/3376421720 wait complete. 2023-11-28T13:07:45.564 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.562+0000 7fd1921ff700 1 Processor -- start 2023-11-28T13:07:45.564 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.563+0000 7fd1921ff700 1 -- start start 2023-11-28T13:07:45.568 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.564+0000 7fd1921ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd18c0b20b0 0x7fd18c0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.568 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.564+0000 7fd1921ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c0d01b0 0x7fd18c1e65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.568 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.564+0000 7fd1921ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd18c0d1330 0x7fd18c1e6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.568 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.564+0000 7fd1921ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c15fe60 con 0x7fd18c0d1330 2023-11-28T13:07:45.568 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.564+0000 7fd1921ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c15f5c0 con 0x7fd18c0b20b0 2023-11-28T13:07:45.568 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.564+0000 7fd1921ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c15f8a0 con 0x7fd18c0d01b0 2023-11-28T13:07:45.568 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.564+0000 7fd18b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd18c0b20b0 0x7fd18c0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.568 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.564+0000 7fd18affd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c0d01b0 0x7fd18c1e65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.568 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.564+0000 7fd18affd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c0d01b0 0x7fd18c1e65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:46776/0 (socket says 172.21.15.186:46776) 2023-11-28T13:07:45.568 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.564+0000 7fd18affd700 1 -- 172.21.15.186:0/1158877527 learned_addr learned my addr 172.21.15.186:0/1158877527 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T13:07:45.569 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.564+0000 7fd18bfff700 1 --2- 172.21.15.186:0/1158877527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd18c0d1330 0x7fd18c1e6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.569 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.565+0000 7fd18b7fe700 1 -- 172.21.15.186:0/1158877527 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c0d01b0 msgr2=0x7fd18c1e65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.569 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.565+0000 7fd18b7fe700 1 --2- 172.21.15.186:0/1158877527 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c0d01b0 0x7fd18c1e65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.569 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.565+0000 7fd18b7fe700 1 -- 172.21.15.186:0/1158877527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd18c0d1330 msgr2=0x7fd18c1e6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.569 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.565+0000 7fd18b7fe700 1 --2- 172.21.15.186:0/1158877527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd18c0d1330 0x7fd18c1e6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.569 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.565+0000 7fd18b7fe700 1 -- 172.21.15.186:0/1158877527 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1800094a0 con 0x7fd18c0b20b0 2023-11-28T13:07:45.569 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.565+0000 7fd18affd700 1 --2- 172.21.15.186:0/1158877527 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c0d01b0 0x7fd18c1e65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:45.569 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.565+0000 7fd18b7fe700 1 --2- 172.21.15.186:0/1158877527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd18c0b20b0 0x7fd18c0c9100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00c520 tx=0x7fd17c00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:45.570 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.565+0000 7fd173fff700 1 -- 172.21.15.186:0/1158877527 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0054e0 con 0x7fd18c0b20b0 2023-11-28T13:07:45.570 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.565+0000 7fd173fff700 1 -- 172.21.15.186:0/1158877527 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c003680 con 0x7fd18c0b20b0 2023-11-28T13:07:45.570 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.565+0000 7fd1921ff700 1 -- 172.21.15.186:0/1158877527 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18c0afec0 con 0x7fd18c0b20b0 2023-11-28T13:07:45.570 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.565+0000 7fd173fff700 1 -- 172.21.15.186:0/1158877527 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c003a30 con 0x7fd18c0b20b0 2023-11-28T13:07:45.570 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.565+0000 7fd1921ff700 1 -- 172.21.15.186:0/1158877527 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18c0afbe0 con 0x7fd18c0b20b0 2023-11-28T13:07:45.570 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.566+0000 7fd173fff700 1 -- 172.21.15.186:0/1158877527 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fd17c019030 con 0x7fd18c0b20b0 2023-11-28T13:07:45.570 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.566+0000 7fd173fff700 1 -- 172.21.15.186:0/1158877527 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fd17c01c440 con 0x7fd18c0b20b0 2023-11-28T13:07:45.571 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:45.566+0000 7fe509d39200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T13:07:45.571 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.566+0000 7fd1921ff700 1 -- 172.21.15.186:0/1158877527 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18c0b2600 con 0x7fd18c0b20b0 2023-11-28T13:07:45.572 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.570+0000 7fd173fff700 1 -- 172.21.15.186:0/1158877527 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fd17c021060 con 0x7fd18c0b20b0 2023-11-28T13:07:45.664 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.662+0000 7fd1921ff700 1 -- 172.21.15.186:0/1158877527 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd new", "uuid": "13b142fc-4857-4039-bee3-1d616041b4ab", "id": 5} v 0) v1 -- 0x7fd18c068690 con 0x7fd18c0b20b0 2023-11-28T13:07:45.669 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.668+0000 7fd173fff700 1 -- 172.21.15.186:0/1158877527 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "13b142fc-4857-4039-bee3-1d616041b4ab", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd17c017070 con 0x7fd18c0b20b0 2023-11-28T13:07:45.670 INFO:teuthology.orchestra.run.smithi186.stdout:5 2023-11-28T13:07:45.671 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.669+0000 7fd1921ff700 1 -- 172.21.15.186:0/1158877527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd18c0b20b0 msgr2=0x7fd18c0c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.671 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.669+0000 7fd1921ff700 1 --2- 172.21.15.186:0/1158877527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd18c0b20b0 0x7fd18c0c9100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00c520 tx=0x7fd17c00c830 comp rx=0 tx=0).stop 2023-11-28T13:07:45.671 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.669+0000 7fd1921ff700 1 -- 172.21.15.186:0/1158877527 shutdown_connections 2023-11-28T13:07:45.671 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.669+0000 7fd1921ff700 1 --2- 172.21.15.186:0/1158877527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd18c0d1330 0x7fd18c1e6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.671 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.669+0000 7fd1921ff700 1 --2- 172.21.15.186:0/1158877527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd18c0b20b0 0x7fd18c0c9100 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.672 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.669+0000 7fd1921ff700 1 --2- 172.21.15.186:0/1158877527 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c0d01b0 0x7fd18c1e65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.672 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.669+0000 7fd1921ff700 1 -- 172.21.15.186:0/1158877527 >> 172.21.15.186:0/1158877527 conn(0x7fd18c09c3f0 msgr2=0x7fd18c06b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:45.672 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.669+0000 7fd1921ff700 1 -- 172.21.15.186:0/1158877527 shutdown_connections 2023-11-28T13:07:45.672 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.669+0000 7fd1921ff700 1 -- 172.21.15.186:0/1158877527 wait complete. 2023-11-28T13:07:45.689 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 568bcbd6-4435-4c87-aada-29a5b9499432 6 2023-11-28T13:07:45.909 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.907+0000 7fa8cbdff700 1 Processor -- start 2023-11-28T13:07:45.910 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.908+0000 7fa8cbdff700 1 -- start start 2023-11-28T13:07:45.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.909+0000 7fa8cbdff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa8c415c9b0 0x7fa8c40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.909+0000 7fa8cbdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8c4161340 0x7fa8c40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.909+0000 7fa8cbdff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c40d1950 con 0x7fa8c40d2320 2023-11-28T13:07:45.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.909+0000 7fa8cbdff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c40b5800 con 0x7fa8c415c9b0 2023-11-28T13:07:45.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.909+0000 7fa8cbdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40b5520 con 0x7fa8c4161340 2023-11-28T13:07:45.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.910+0000 7fa8c9b9b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa8c415c9b0 0x7fa8c40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.910+0000 7fa8c9b9b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa8c415c9b0 0x7fa8c40b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58692/0 (socket says 172.21.15.186:58692) 2023-11-28T13:07:45.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.910+0000 7fa8c9b9b700 1 -- 172.21.15.186:0/1395076243 learned_addr learned my addr 172.21.15.186:0/1395076243 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T13:07:45.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.910+0000 7fa8c939a700 1 --2- 172.21.15.186:0/1395076243 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8c4161340 0x7fa8c40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.913 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.910+0000 7fa8c9b9b700 1 -- 172.21.15.186:0/1395076243 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8c4161340 msgr2=0x7fa8c40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:45.913 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.910+0000 7fa8c9b9b700 1 --2- 172.21.15.186:0/1395076243 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8c4161340 0x7fa8c40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.913 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.910+0000 7fa8c9b9b700 1 -- 172.21.15.186:0/1395076243 >> v1:172.21.15.186:6789/0 conn(0x7fa8c40d2320 legacy=0x7fa8c40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.910+0000 7fa8c9b9b700 1 -- 172.21.15.186:0/1395076243 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c40d1300 con 0x7fa8c415c9b0 2023-11-28T13:07:45.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.910+0000 7fa8c939a700 1 --2- 172.21.15.186:0/1395076243 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8c4161340 0x7fa8c40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:45.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.911+0000 7fa8c9b9b700 1 --2- 172.21.15.186:0/1395076243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8c415c9b0 0x7fa8c40b4bf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00098f0 tx=0x7fa8c0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26ba205ff05f26c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:45.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.911+0000 7fa8c8b99700 1 -- 172.21.15.186:0/1395076243 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c000b8a0 con 0x7fa8c415c9b0 2023-11-28T13:07:45.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.911+0000 7fa8c8b99700 1 -- 172.21.15.186:0/1395076243 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c000ba00 con 0x7fa8c415c9b0 2023-11-28T13:07:45.916 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.911+0000 7fa8c8b99700 1 -- 172.21.15.186:0/1395076243 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c000bd90 con 0x7fa8c415c9b0 2023-11-28T13:07:45.917 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.912+0000 7fa8cbdff700 1 -- 172.21.15.186:0/1395076243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8c415c9b0 msgr2=0x7fa8c40b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.917 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.912+0000 7fa8cbdff700 1 --2- 172.21.15.186:0/1395076243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8c415c9b0 0x7fa8c40b4bf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00098f0 tx=0x7fa8c0009c00 comp rx=0 tx=0).stop 2023-11-28T13:07:45.917 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.912+0000 7fa8cbdff700 1 -- 172.21.15.186:0/1395076243 shutdown_connections 2023-11-28T13:07:45.917 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.912+0000 7fa8cbdff700 1 --2- 172.21.15.186:0/1395076243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8c415c9b0 0x7fa8c40b4bf0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.917 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.912+0000 7fa8cbdff700 1 --2- 172.21.15.186:0/1395076243 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8c4161340 0x7fa8c40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.918 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.912+0000 7fa8cbdff700 1 -- 172.21.15.186:0/1395076243 >> 172.21.15.186:0/1395076243 conn(0x7fa8c4072600 msgr2=0x7fa8c40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:45.918 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.912+0000 7fa8cbdff700 1 -- 172.21.15.186:0/1395076243 shutdown_connections 2023-11-28T13:07:45.918 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.912+0000 7fa8cbdff700 1 -- 172.21.15.186:0/1395076243 wait complete. 2023-11-28T13:07:45.918 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.915+0000 7fa8cbdff700 1 Processor -- start 2023-11-28T13:07:45.918 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.915+0000 7fa8cbdff700 1 -- start start 2023-11-28T13:07:45.920 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.916+0000 7fa8cbdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8c40d2320 0x7fa8c4151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.920 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.916+0000 7fa8cbdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8c415c9b0 0x7fa8c4155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.920 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.916+0000 7fa8cbdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8c4161340 0x7fa8c4157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.920 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.916+0000 7fa8cbdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c4157160 con 0x7fa8c415c9b0 2023-11-28T13:07:45.920 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.916+0000 7fa8cbdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c4151ba0 con 0x7fa8c4161340 2023-11-28T13:07:45.920 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.916+0000 7fa8cbdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41564e0 con 0x7fa8c40d2320 2023-11-28T13:07:45.920 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.916+0000 7fa8c9b9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8c40d2320 0x7fa8c4151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.920 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.916+0000 7fa8ca39c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8c4161340 0x7fa8c4157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.916+0000 7fa8c9b9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8c40d2320 0x7fa8c4151660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:46786/0 (socket says 172.21.15.186:46786) 2023-11-28T13:07:45.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.916+0000 7fa8c9b9b700 1 -- 172.21.15.186:0/1363191922 learned_addr learned my addr 172.21.15.186:0/1363191922 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T13:07:45.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.916+0000 7fa8c939a700 1 --2- 172.21.15.186:0/1363191922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8c415c9b0 0x7fa8c4155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.917+0000 7fa8c9b9b700 1 -- 172.21.15.186:0/1363191922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8c4161340 msgr2=0x7fa8c4157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.917+0000 7fa8c9b9b700 1 --2- 172.21.15.186:0/1363191922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8c4161340 0x7fa8c4157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.917+0000 7fa8c9b9b700 1 -- 172.21.15.186:0/1363191922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8c415c9b0 msgr2=0x7fa8c4155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.917+0000 7fa8c9b9b700 1 --2- 172.21.15.186:0/1363191922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8c415c9b0 0x7fa8c4155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.917+0000 7fa8c9b9b700 1 -- 172.21.15.186:0/1363191922 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c0009550 con 0x7fa8c40d2320 2023-11-28T13:07:45.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.917+0000 7fa8ca39c700 1 --2- 172.21.15.186:0/1363191922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8c4161340 0x7fa8c4157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:45.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.917+0000 7fa8c9b9b700 1 --2- 172.21.15.186:0/1363191922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8c40d2320 0x7fa8c4151660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0014040 tx=0x7fa8c0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:45.922 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.917+0000 7fa8ba7fc700 1 -- 172.21.15.186:0/1363191922 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0004510 con 0x7fa8c40d2320 2023-11-28T13:07:45.922 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.917+0000 7fa8ba7fc700 1 -- 172.21.15.186:0/1363191922 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c001e070 con 0x7fa8c40d2320 2023-11-28T13:07:45.922 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.917+0000 7fa8ba7fc700 1 -- 172.21.15.186:0/1363191922 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c00038e0 con 0x7fa8c40d2320 2023-11-28T13:07:45.922 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.917+0000 7fa8cbdff700 1 -- 172.21.15.186:0/1363191922 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c40b5130 con 0x7fa8c40d2320 2023-11-28T13:07:45.922 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.917+0000 7fa8cbdff700 1 -- 172.21.15.186:0/1363191922 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c40d15e0 con 0x7fa8c40d2320 2023-11-28T13:07:45.922 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.918+0000 7fa8ba7fc700 1 -- 172.21.15.186:0/1363191922 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fa8c000f050 con 0x7fa8c40d2320 2023-11-28T13:07:45.922 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.918+0000 7fa8ba7fc700 1 -- 172.21.15.186:0/1363191922 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fa8c0020070 con 0x7fa8c40d2320 2023-11-28T13:07:45.923 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.919+0000 7fa8cbdff700 1 -- 172.21.15.186:0/1363191922 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c40b52c0 con 0x7fa8c40d2320 2023-11-28T13:07:45.923 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:45.921+0000 7fa8ba7fc700 1 -- 172.21.15.186:0/1363191922 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fa8c002a020 con 0x7fa8c40d2320 2023-11-28T13:07:46.032 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.030+0000 7fa8cbdff700 1 -- 172.21.15.186:0/1363191922 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd new", "uuid": "568bcbd6-4435-4c87-aada-29a5b9499432", "id": 6} v 0) v1 -- 0x7fa8c406d330 con 0x7fa8c40d2320 2023-11-28T13:07:46.040 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.038+0000 7fa8ba7fc700 1 -- 172.21.15.186:0/1363191922 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "568bcbd6-4435-4c87-aada-29a5b9499432", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa8c0015020 con 0x7fa8c40d2320 2023-11-28T13:07:46.040 INFO:teuthology.orchestra.run.smithi186.stdout:6 2023-11-28T13:07:46.042 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.040+0000 7fa8cbdff700 1 -- 172.21.15.186:0/1363191922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8c40d2320 msgr2=0x7fa8c4151660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.042 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.040+0000 7fa8cbdff700 1 --2- 172.21.15.186:0/1363191922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8c40d2320 0x7fa8c4151660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0014040 tx=0x7fa8c0004750 comp rx=0 tx=0).stop 2023-11-28T13:07:46.042 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.040+0000 7fa8cbdff700 1 -- 172.21.15.186:0/1363191922 shutdown_connections 2023-11-28T13:07:46.042 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.040+0000 7fa8cbdff700 1 --2- 172.21.15.186:0/1363191922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8c4161340 0x7fa8c4157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.042 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.040+0000 7fa8cbdff700 1 --2- 172.21.15.186:0/1363191922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8c40d2320 0x7fa8c4151660 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.042 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.040+0000 7fa8cbdff700 1 --2- 172.21.15.186:0/1363191922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8c415c9b0 0x7fa8c4155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.043 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.040+0000 7fa8cbdff700 1 -- 172.21.15.186:0/1363191922 >> 172.21.15.186:0/1363191922 conn(0x7fa8c4072600 msgr2=0x7fa8c40b89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:46.043 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.040+0000 7fa8cbdff700 1 -- 172.21.15.186:0/1363191922 shutdown_connections 2023-11-28T13:07:46.043 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.040+0000 7fa8cbdff700 1 -- 172.21.15.186:0/1363191922 wait complete. 2023-11-28T13:07:46.068 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9 7 2023-11-28T13:07:46.074 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:46.071+0000 7fe7d950f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T13:07:46.174 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:46.172+0000 7fe7d950f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T13:07:46.281 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.279+0000 7f8246dff700 1 Processor -- start 2023-11-28T13:07:46.282 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.280+0000 7f8246dff700 1 -- start start 2023-11-28T13:07:46.283 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.281+0000 7f8246dff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f82400d1330 0x7f82400b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:46.283 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.281+0000 7f8246dff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82400b0230 con 0x7f82400b20b0 2023-11-28T13:07:46.283 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.281+0000 7f8246dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82400b4570 con 0x7f82400d01b0 2023-11-28T13:07:46.283 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.281+0000 7f8246dff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f824015cec0 con 0x7f82400d1330 2023-11-28T13:07:46.284 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.282+0000 7f8244b9b700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f82400d01b0 0x7f82400d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.186:39776/0 (socket says 172.21.15.186:39776) 2023-11-28T13:07:46.284 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.282+0000 7f8244b9b700 1 -- 172.21.15.186:0/1155139124 learned_addr learned my addr 172.21.15.186:0/1155139124 (peer_addr_for_me v1:172.21.15.186:0/0) 2023-11-28T13:07:46.284 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.282+0000 7f823ffff700 1 --2- 172.21.15.186:0/1155139124 >> v2:172.21.15.156:3300/0 conn(0x7f82400d1330 0x7f82400b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:46.285 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.282+0000 7f823f7fe700 1 -- 172.21.15.186:0/1155139124 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1145023582 0 0) 0x7f82400b4570 con 0x7f82400d01b0 2023-11-28T13:07:46.285 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.282+0000 7f823f7fe700 1 -- 172.21.15.186:0/1155139124 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82300035b0 con 0x7f82400d01b0 2023-11-28T13:07:46.285 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.282+0000 7f823ffff700 1 -- 172.21.15.186:0/1155139124 >> v1:172.21.15.186:6789/0 conn(0x7f82400d01b0 legacy=0x7f82400d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.285 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.282+0000 7f823ffff700 1 -- 172.21.15.186:0/1155139124 >> v1:172.21.15.156:6789/0 conn(0x7f82400b20b0 legacy=0x7f82400b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.285 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.282+0000 7f823f7fe700 1 -- 172.21.15.186:0/1155139124 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3031885309 0 0) 0x7f82400b0230 con 0x7f82400b20b0 2023-11-28T13:07:46.285 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.282+0000 7f823ffff700 1 -- 172.21.15.186:0/1155139124 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82400b4850 con 0x7f82400d1330 2023-11-28T13:07:46.285 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.283+0000 7f823ffff700 1 --2- 172.21.15.186:0/1155139124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82400d1330 0x7f82400b06c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f822c009840 tx=0x7f822c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bacb4e1807bda7bb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:46.285 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.283+0000 7f823f7fe700 1 -- 172.21.15.186:0/1155139124 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c011830 con 0x7f82400d1330 2023-11-28T13:07:46.285 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.283+0000 7f823f7fe700 1 -- 172.21.15.186:0/1155139124 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c011990 con 0x7f82400d1330 2023-11-28T13:07:46.285 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.283+0000 7f823f7fe700 1 -- 172.21.15.186:0/1155139124 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c011d20 con 0x7f82400d1330 2023-11-28T13:07:46.286 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.284+0000 7f8246dff700 1 -- 172.21.15.186:0/1155139124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82400d1330 msgr2=0x7f82400b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.286 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.284+0000 7f8246dff700 1 --2- 172.21.15.186:0/1155139124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82400d1330 0x7f82400b06c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f822c009840 tx=0x7f822c009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:46.286 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.284+0000 7f8246dff700 1 -- 172.21.15.186:0/1155139124 shutdown_connections 2023-11-28T13:07:46.286 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.284+0000 7f8246dff700 1 --2- 172.21.15.186:0/1155139124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82400d1330 0x7f82400b06c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.287 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.284+0000 7f8246dff700 1 -- 172.21.15.186:0/1155139124 >> 172.21.15.186:0/1155139124 conn(0x7f824009c3f0 msgr2=0x7f82400d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:46.287 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.284+0000 7f8246dff700 1 -- 172.21.15.186:0/1155139124 shutdown_connections 2023-11-28T13:07:46.287 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.284+0000 7f8246dff700 1 -- 172.21.15.186:0/1155139124 wait complete. 2023-11-28T13:07:46.288 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.286+0000 7f8246dff700 1 Processor -- start 2023-11-28T13:07:46.288 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.286+0000 7f8246dff700 1 -- start start 2023-11-28T13:07:46.289 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.287+0000 7f8246dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82400b20b0 0x7f824015d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:46.289 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.287+0000 7f8246dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82400d01b0 0x7f824015a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:46.289 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.287+0000 7f8246dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82400d1330 0x7f8240148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:46.289 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.287+0000 7f8246dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824015fab0 con 0x7f82400d01b0 2023-11-28T13:07:46.289 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.287+0000 7f8246dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82400c9bc0 con 0x7f82400d1330 2023-11-28T13:07:46.289 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.287+0000 7f8246dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824015ff40 con 0x7f82400b20b0 2023-11-28T13:07:46.291 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.287+0000 7f823ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82400d01b0 0x7f824015a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:46.291 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.287+0000 7f8244b9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82400b20b0 0x7f824015d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:46.291 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.287+0000 7f824539c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82400d1330 0x7f8240148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:46.291 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.287+0000 7f823ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82400d01b0 0x7f824015a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.186:48332/0 (socket says 172.21.15.186:48332) 2023-11-28T13:07:46.291 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.287+0000 7f823ffff700 1 -- 172.21.15.186:0/2511826884 learned_addr learned my addr 172.21.15.186:0/2511826884 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-28T13:07:46.291 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.288+0000 7f823ffff700 1 -- 172.21.15.186:0/2511826884 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82400b20b0 msgr2=0x7f824015d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.291 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.288+0000 7f823ffff700 1 --2- 172.21.15.186:0/2511826884 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82400b20b0 0x7f824015d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.292 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.288+0000 7f823ffff700 1 -- 172.21.15.186:0/2511826884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82400d1330 msgr2=0x7f8240148e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.292 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.288+0000 7f823ffff700 1 --2- 172.21.15.186:0/2511826884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82400d1330 0x7f8240148e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.292 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.288+0000 7f823ffff700 1 -- 172.21.15.186:0/2511826884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c0094a0 con 0x7f82400d01b0 2023-11-28T13:07:46.292 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.288+0000 7f823ffff700 1 --2- 172.21.15.186:0/2511826884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82400d01b0 0x7f824015a4c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f822c0090b0 tx=0x7f822c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:46.292 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.289+0000 7f823d7fa700 1 -- 172.21.15.186:0/2511826884 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c01a7e0 con 0x7f82400d01b0 2023-11-28T13:07:46.292 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.289+0000 7f823d7fa700 1 -- 172.21.15.186:0/2511826884 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c01a940 con 0x7f82400d01b0 2023-11-28T13:07:46.292 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.289+0000 7f823d7fa700 1 -- 172.21.15.186:0/2511826884 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c01acd0 con 0x7f82400d01b0 2023-11-28T13:07:46.292 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.289+0000 7f8246dff700 1 -- 172.21.15.186:0/2511826884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82400afec0 con 0x7f82400d01b0 2023-11-28T13:07:46.292 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.289+0000 7f8246dff700 1 -- 172.21.15.186:0/2511826884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82400afbe0 con 0x7f82400d01b0 2023-11-28T13:07:46.292 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.289+0000 7f823d7fa700 1 -- 172.21.15.186:0/2511826884 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f822c00f050 con 0x7f82400d01b0 2023-11-28T13:07:46.292 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.289+0000 7f823d7fa700 1 -- 172.21.15.186:0/2511826884 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f822c01ae30 con 0x7f82400d01b0 2023-11-28T13:07:46.293 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.290+0000 7f8246dff700 1 -- 172.21.15.186:0/2511826884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82400b2600 con 0x7f82400d01b0 2023-11-28T13:07:46.296 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.294+0000 7f823d7fa700 1 -- 172.21.15.186:0/2511826884 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f822c029070 con 0x7f82400d01b0 2023-11-28T13:07:46.306 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:46.304+0000 7fe509d39200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T13:07:46.384 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.382+0000 7f8246dff700 1 -- 172.21.15.186:0/2511826884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9", "id": 7} v 0) v1 -- 0x7f8240068690 con 0x7f82400d01b0 2023-11-28T13:07:46.387 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.385+0000 7f823d7fa700 1 -- 172.21.15.186:0/2511826884 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f822c015020 con 0x7f82400d01b0 2023-11-28T13:07:46.387 INFO:teuthology.orchestra.run.smithi186.stdout:7 2023-11-28T13:07:46.388 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.386+0000 7f8246dff700 1 -- 172.21.15.186:0/2511826884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82400d01b0 msgr2=0x7f824015a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.388 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.386+0000 7f8246dff700 1 --2- 172.21.15.186:0/2511826884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82400d01b0 0x7f824015a4c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f822c0090b0 tx=0x7f822c004750 comp rx=0 tx=0).stop 2023-11-28T13:07:46.388 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.386+0000 7f8246dff700 1 -- 172.21.15.186:0/2511826884 shutdown_connections 2023-11-28T13:07:46.388 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.386+0000 7f8246dff700 1 --2- 172.21.15.186:0/2511826884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82400d1330 0x7f8240148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.388 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.386+0000 7f8246dff700 1 --2- 172.21.15.186:0/2511826884 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82400b20b0 0x7f824015d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.389 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.386+0000 7f8246dff700 1 --2- 172.21.15.186:0/2511826884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82400d01b0 0x7f824015a4c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.389 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.386+0000 7f8246dff700 1 -- 172.21.15.186:0/2511826884 >> 172.21.15.186:0/2511826884 conn(0x7f824009c3f0 msgr2=0x7f82400b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:46.389 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.386+0000 7f8246dff700 1 -- 172.21.15.186:0/2511826884 shutdown_connections 2023-11-28T13:07:46.389 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-28T13:07:46.386+0000 7f8246dff700 1 -- 172.21.15.186:0/2511826884 wait complete. 2023-11-28T13:07:46.401 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:46.400+0000 7fe509d39200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T13:07:46.406 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-28T13:07:46.406 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-28T13:07:46.410 INFO:tasks.ceph.osd.0:Started 2023-11-28T13:07:46.410 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-28T13:07:46.410 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-28T13:07:46.417 INFO:tasks.ceph.osd.1:Started 2023-11-28T13:07:46.417 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-28T13:07:46.417 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-28T13:07:46.421 INFO:tasks.ceph.osd.2:Started 2023-11-28T13:07:46.421 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-28T13:07:46.421 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-28T13:07:46.427 INFO:tasks.ceph.osd.3:Started 2023-11-28T13:07:46.427 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-28T13:07:46.427 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-28T13:07:46.431 INFO:tasks.ceph.osd.4:Started 2023-11-28T13:07:46.432 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-28T13:07:46.432 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-28T13:07:46.436 INFO:tasks.ceph.osd.5:Started 2023-11-28T13:07:46.436 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-28T13:07:46.436 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-28T13:07:46.442 INFO:tasks.ceph.osd.6:Started 2023-11-28T13:07:46.442 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-28T13:07:46.442 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-28T13:07:46.447 INFO:tasks.ceph.osd.7:Started 2023-11-28T13:07:46.448 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:07:46.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.731+0000 7f4bdb1ff700 1 Processor -- start 2023-11-28T13:07:46.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.731+0000 7f4bdb1ff700 1 -- start start 2023-11-28T13:07:46.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.732+0000 7f4bdb1ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4bd40d01b0 0x7f4bd40d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:46.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.732+0000 7f4bdb1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bd40b20b0 0x7f4bd40d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:46.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.733+0000 7f4bdb1ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd40b0230 con 0x7f4bd40b29b0 2023-11-28T13:07:46.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.733+0000 7f4bdb1ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd415cec0 con 0x7f4bd40d01b0 2023-11-28T13:07:46.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.733+0000 7f4bdb1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd40b4570 con 0x7f4bd40b20b0 2023-11-28T13:07:46.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.733+0000 7f4bd8f9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4bd40d01b0 0x7f4bd40d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:46.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.733+0000 7f4bd8f9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4bd40d01b0 0x7f4bd40d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45268/0 (socket says 172.21.15.156:45268) 2023-11-28T13:07:46.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.733+0000 7f4bd8f9b700 1 -- 172.21.15.156:0/4171719544 learned_addr learned my addr 172.21.15.156:0/4171719544 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:46.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.733+0000 7f4bd3fff700 1 --2- 172.21.15.156:0/4171719544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bd40b20b0 0x7f4bd40d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:46.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.733+0000 7f4bd8f9b700 1 -- 172.21.15.156:0/4171719544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bd40b20b0 msgr2=0x7f4bd40d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:46.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.733+0000 7f4bd8f9b700 1 --2- 172.21.15.156:0/4171719544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bd40b20b0 0x7f4bd40d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.733+0000 7f4bd8f9b700 1 -- 172.21.15.156:0/4171719544 >> v1:172.21.15.186:6789/0 conn(0x7f4bd40b29b0 legacy=0x7f4bd40b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.733+0000 7f4bd8f9b700 1 -- 172.21.15.156:0/4171719544 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd40b4850 con 0x7f4bd40d01b0 2023-11-28T13:07:46.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.733+0000 7f4bd8f9b700 1 --2- 172.21.15.156:0/4171719544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd40d01b0 0x7f4bd40d0c20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80098f0 tx=0x7f4bc8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82cad263987dccd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:46.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.734+0000 7f4bd37fe700 1 -- 172.21.15.156:0/4171719544 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc800b8a0 con 0x7f4bd40d01b0 2023-11-28T13:07:46.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.734+0000 7f4bd37fe700 1 -- 172.21.15.156:0/4171719544 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc800ba00 con 0x7f4bd40d01b0 2023-11-28T13:07:46.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.734+0000 7f4bd37fe700 1 -- 172.21.15.156:0/4171719544 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc800bd90 con 0x7f4bd40d01b0 2023-11-28T13:07:46.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.734+0000 7f4bdb1ff700 1 -- 172.21.15.156:0/4171719544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd40d01b0 msgr2=0x7f4bd40d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.734+0000 7f4bdb1ff700 1 --2- 172.21.15.156:0/4171719544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd40d01b0 0x7f4bd40d0c20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80098f0 tx=0x7f4bc8009c00 comp rx=0 tx=0).stop 2023-11-28T13:07:46.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.735+0000 7f4bdb1ff700 1 -- 172.21.15.156:0/4171719544 shutdown_connections 2023-11-28T13:07:46.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.735+0000 7f4bdb1ff700 1 --2- 172.21.15.156:0/4171719544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd40d01b0 0x7f4bd40d0c20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.735+0000 7f4bdb1ff700 1 --2- 172.21.15.156:0/4171719544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bd40b20b0 0x7f4bd40d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.735+0000 7f4bdb1ff700 1 -- 172.21.15.156:0/4171719544 >> 172.21.15.156:0/4171719544 conn(0x7f4bd409c3f0 msgr2=0x7f4bd40d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:46.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.735+0000 7f4bdb1ff700 1 -- 172.21.15.156:0/4171719544 shutdown_connections 2023-11-28T13:07:46.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.735+0000 7f4bdb1ff700 1 -- 172.21.15.156:0/4171719544 wait complete. 2023-11-28T13:07:46.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.736+0000 7f4bdb1ff700 1 Processor -- start 2023-11-28T13:07:46.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.736+0000 7f4bdb1ff700 1 -- start start 2023-11-28T13:07:46.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.737+0000 7f4bdb1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd40b20b0 0x7f4bd41523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:46.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.737+0000 7f4bdb1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bd40b29b0 0x7f4bd40b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:46.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.737+0000 7f4bdb1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd40d01b0 0x7f4bd4151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:46.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.737+0000 7f4bdb1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd40b6890 con 0x7f4bd40d01b0 2023-11-28T13:07:46.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.737+0000 7f4bdb1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd40b7af0 con 0x7f4bd40b20b0 2023-11-28T13:07:46.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.737+0000 7f4bdb1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd40b6ae0 con 0x7f4bd40b29b0 2023-11-28T13:07:46.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.738+0000 7f4bd8f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd40b20b0 0x7f4bd41523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:46.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.738+0000 7f4bd3fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bd40b29b0 0x7f4bd40b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:46.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.738+0000 7f4bd8f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd40b20b0 0x7f4bd41523d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:45840/0 (socket says 172.21.15.156:45840) 2023-11-28T13:07:46.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.738+0000 7f4bd3fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bd40b29b0 0x7f4bd40b75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:48010/0 (socket says 172.21.15.156:48010) 2023-11-28T13:07:46.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.738+0000 7f4bd8f9b700 1 -- 172.21.15.156:0/2376393642 learned_addr learned my addr 172.21.15.156:0/2376393642 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:46.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.738+0000 7f4bd979c700 1 --2- 172.21.15.156:0/2376393642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd40d01b0 0x7f4bd4151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:46.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.738+0000 7f4bd3fff700 1 -- 172.21.15.156:0/2376393642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd40b20b0 msgr2=0x7f4bd41523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.738+0000 7f4bd3fff700 1 --2- 172.21.15.156:0/2376393642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd40b20b0 0x7f4bd41523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.738+0000 7f4bd3fff700 1 -- 172.21.15.156:0/2376393642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd40d01b0 msgr2=0x7f4bd4151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.738+0000 7f4bd3fff700 1 --2- 172.21.15.156:0/2376393642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd40d01b0 0x7f4bd4151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.738+0000 7f4bd3fff700 1 -- 172.21.15.156:0/2376393642 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc8009550 con 0x7f4bd40b29b0 2023-11-28T13:07:46.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.738+0000 7f4bd8f9b700 1 --2- 172.21.15.156:0/2376393642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd40b20b0 0x7f4bd41523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:46.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.738+0000 7f4bd979c700 1 --2- 172.21.15.156:0/2376393642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd40d01b0 0x7f4bd4151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:46.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.738+0000 7f4bd3fff700 1 --2- 172.21.15.156:0/2376393642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bd40b29b0 0x7f4bd40b75b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000e810 tx=0x7f4bc000eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:46.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.739+0000 7f4bd17fa700 1 -- 172.21.15.156:0/2376393642 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc000c070 con 0x7f4bd40b29b0 2023-11-28T13:07:46.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.739+0000 7f4bd17fa700 1 -- 172.21.15.156:0/2376393642 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc0013610 con 0x7f4bd40b29b0 2023-11-28T13:07:46.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.739+0000 7f4bdb1ff700 1 -- 172.21.15.156:0/2376393642 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd40afec0 con 0x7f4bd40b29b0 2023-11-28T13:07:46.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.739+0000 7f4bd17fa700 1 -- 172.21.15.156:0/2376393642 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc00139a0 con 0x7f4bd40b29b0 2023-11-28T13:07:46.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.739+0000 7f4bdb1ff700 1 -- 172.21.15.156:0/2376393642 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd40afbe0 con 0x7f4bd40b29b0 2023-11-28T13:07:46.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.739+0000 7f4bd17fa700 1 -- 172.21.15.156:0/2376393642 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f4bc0023070 con 0x7f4bd40b29b0 2023-11-28T13:07:46.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.739+0000 7f4bd17fa700 1 -- 172.21.15.156:0/2376393642 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4bc001c400 con 0x7f4bd40b29b0 2023-11-28T13:07:46.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.740+0000 7f4bdb1ff700 1 -- 172.21.15.156:0/2376393642 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd40b2600 con 0x7f4bd40b29b0 2023-11-28T13:07:46.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.744+0000 7f4bd17fa700 1 -- 172.21.15.156:0/2376393642 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f4bc003bb00 con 0x7f4bd40b29b0 2023-11-28T13:07:46.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.818+0000 7f4bdb1ff700 1 -- 172.21.15.156:0/2376393642 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bd4068690 con 0x7f4bd40b29b0 2023-11-28T13:07:46.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.819+0000 7f4bd17fa700 1 -- 172.21.15.156:0/2376393642 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4bc002a070 con 0x7f4bd40b29b0 2023-11-28T13:07:46.820 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:46.820 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":10,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:07:46.384751+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:07:46.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.820+0000 7f4bdb1ff700 1 -- 172.21.15.156:0/2376393642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bd40b29b0 msgr2=0x7f4bd40b75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.820+0000 7f4bdb1ff700 1 --2- 172.21.15.156:0/2376393642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bd40b29b0 0x7f4bd40b75b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000e810 tx=0x7f4bc000eb20 comp rx=0 tx=0).stop 2023-11-28T13:07:46.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.820+0000 7f4bdb1ff700 1 -- 172.21.15.156:0/2376393642 shutdown_connections 2023-11-28T13:07:46.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.820+0000 7f4bdb1ff700 1 --2- 172.21.15.156:0/2376393642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd40d01b0 0x7f4bd4151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.820+0000 7f4bdb1ff700 1 --2- 172.21.15.156:0/2376393642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd40b20b0 0x7f4bd41523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.820+0000 7f4bdb1ff700 1 --2- 172.21.15.156:0/2376393642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bd40b29b0 0x7f4bd40b75b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.820+0000 7f4bdb1ff700 1 -- 172.21.15.156:0/2376393642 >> 172.21.15.156:0/2376393642 conn(0x7f4bd409c3f0 msgr2=0x7f4bd415c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:46.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.820+0000 7f4bdb1ff700 1 -- 172.21.15.156:0/2376393642 shutdown_connections 2023-11-28T13:07:46.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:46.820+0000 7f4bdb1ff700 1 -- 172.21.15.156:0/2376393642 wait complete. 2023-11-28T13:07:46.842 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-28T13:07:46.842 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-28T13:07:46.905 INFO:tasks.ceph.osd.1.smithi156.stderr:2023-11-28T13:07:46.903+0000 7f5a6b878640 -1 Falling back to public interface 2023-11-28T13:07:47.063 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:47.061+0000 7fe7d950f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T13:07:47.135 INFO:tasks.ceph.osd.5.smithi186.stderr:2023-11-28T13:07:47.133+0000 7fa43db2c640 -1 Falling back to public interface 2023-11-28T13:07:47.145 INFO:tasks.ceph.osd.7.smithi186.stderr:2023-11-28T13:07:47.143+0000 7f317642b640 -1 Falling back to public interface 2023-11-28T13:07:47.156 INFO:tasks.ceph.osd.0.smithi156.stderr:2023-11-28T13:07:47.155+0000 7fce73a0d640 -1 Falling back to public interface 2023-11-28T13:07:47.160 INFO:tasks.ceph.osd.2.smithi156.stderr:2023-11-28T13:07:47.158+0000 7f73e4cd0640 -1 Falling back to public interface 2023-11-28T13:07:47.160 INFO:tasks.ceph.osd.4.smithi186.stderr:2023-11-28T13:07:47.159+0000 7fd6dfe62640 -1 Falling back to public interface 2023-11-28T13:07:47.166 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:47.164+0000 7fe7d950f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T13:07:47.167 INFO:tasks.ceph.osd.3.smithi156.stderr:2023-11-28T13:07:47.166+0000 7fcced639640 -1 Falling back to public interface 2023-11-28T13:07:47.176 INFO:tasks.ceph.osd.6.smithi186.stderr:2023-11-28T13:07:47.174+0000 7fe7147a2640 -1 Falling back to public interface 2023-11-28T13:07:47.347 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:47.346+0000 7fe7d950f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T13:07:47.418 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:47.416+0000 7fe509d39200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T13:07:47.522 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:47.521+0000 7fe509d39200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T13:07:47.648 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T13:07:47.710 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:47.708+0000 7fe509d39200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T13:07:47.729 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:47.727+0000 7fe7d950f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T13:07:47.827 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.825+0000 7fab403ff700 1 Processor -- start 2023-11-28T13:07:47.827 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.826+0000 7fab403ff700 1 -- start start 2023-11-28T13:07:47.828 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.827+0000 7fab403ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fab38161340 0x7fab380d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:47.828 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.827+0000 7fab403ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab380d1950 con 0x7fab3815c9b0 2023-11-28T13:07:47.828 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.827+0000 7fab403ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab380b5520 con 0x7fab380d2320 2023-11-28T13:07:47.829 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.827+0000 7fab403ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab380b5800 con 0x7fab38161340 2023-11-28T13:07:47.829 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.827+0000 7fab3e19b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fab3815c9b0 0x7fab380b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:44028/0 (socket says 172.21.15.156:44028) 2023-11-28T13:07:47.829 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.827+0000 7fab3e19b700 1 -- 172.21.15.156:0/3512723176 learned_addr learned my addr 172.21.15.156:0/3512723176 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:07:47.829 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.827+0000 7fab3d99a700 1 --2- 172.21.15.156:0/3512723176 >> v2:172.21.15.156:3300/0 conn(0x7fab38161340 0x7fab380d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:47.831 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.828+0000 7fab3d199700 1 -- 172.21.15.156:0/3512723176 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1955742652 0 0) 0x7fab380d1950 con 0x7fab3815c9b0 2023-11-28T13:07:47.831 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.828+0000 7fab3d199700 1 -- 172.21.15.156:0/3512723176 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab240035b0 con 0x7fab3815c9b0 2023-11-28T13:07:47.831 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.828+0000 7fab3d99a700 1 -- 172.21.15.156:0/3512723176 >> v1:172.21.15.186:6789/0 conn(0x7fab380d2320 legacy=0x7fab380cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.831 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.828+0000 7fab3d199700 1 -- 172.21.15.156:0/3512723176 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938343259 0 0) 0x7fab380b5520 con 0x7fab380d2320 2023-11-28T13:07:47.831 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.828+0000 7fab3d99a700 1 -- 172.21.15.156:0/3512723176 >> v1:172.21.15.156:6789/0 conn(0x7fab3815c9b0 legacy=0x7fab380b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.831 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.828+0000 7fab3d99a700 1 -- 172.21.15.156:0/3512723176 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab380d1300 con 0x7fab38161340 2023-11-28T13:07:47.831 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.828+0000 7fab3d99a700 1 --2- 172.21.15.156:0/3512723176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab38161340 0x7fab380d1de0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fab28009840 tx=0x7fab28009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527e8e389b5d610c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:47.831 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.828+0000 7fab3d199700 1 -- 172.21.15.156:0/3512723176 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28011830 con 0x7fab38161340 2023-11-28T13:07:47.831 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.828+0000 7fab3d199700 1 -- 172.21.15.156:0/3512723176 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab28011990 con 0x7fab38161340 2023-11-28T13:07:47.832 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.828+0000 7fab3d199700 1 -- 172.21.15.156:0/3512723176 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28011d20 con 0x7fab38161340 2023-11-28T13:07:47.832 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.829+0000 7fab403ff700 1 -- 172.21.15.156:0/3512723176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab38161340 msgr2=0x7fab380d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.832 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.829+0000 7fab403ff700 1 --2- 172.21.15.156:0/3512723176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab38161340 0x7fab380d1de0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fab28009840 tx=0x7fab28009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:47.832 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.829+0000 7fab403ff700 1 -- 172.21.15.156:0/3512723176 shutdown_connections 2023-11-28T13:07:47.832 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.829+0000 7fab403ff700 1 --2- 172.21.15.156:0/3512723176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab38161340 0x7fab380d1de0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.832 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.829+0000 7fab403ff700 1 -- 172.21.15.156:0/3512723176 >> 172.21.15.156:0/3512723176 conn(0x7fab38072600 msgr2=0x7fab380cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:47.832 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.829+0000 7fab403ff700 1 -- 172.21.15.156:0/3512723176 shutdown_connections 2023-11-28T13:07:47.832 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.829+0000 7fab403ff700 1 -- 172.21.15.156:0/3512723176 wait complete. 2023-11-28T13:07:47.833 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.831+0000 7fab403ff700 1 Processor -- start 2023-11-28T13:07:47.833 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.831+0000 7fab403ff700 1 -- start start 2023-11-28T13:07:47.835 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.832+0000 7fab403ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab380d2320 0x7fab381505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:47.835 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.832+0000 7fab403ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab3815c9b0 0x7fab38151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:47.835 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.832+0000 7fab403ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab38161340 0x7fab3815d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:47.835 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.832+0000 7fab403ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab38150b20 con 0x7fab38161340 2023-11-28T13:07:47.835 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.832+0000 7fab403ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab38151ec0 con 0x7fab3815c9b0 2023-11-28T13:07:47.836 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.832+0000 7fab403ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab381521a0 con 0x7fab380d2320 2023-11-28T13:07:47.836 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.832+0000 7fab3e99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab38161340 0x7fab3815d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:47.836 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.832+0000 7fab3e99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab38161340 0x7fab3815d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45308/0 (socket says 172.21.15.156:45308) 2023-11-28T13:07:47.836 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.832+0000 7fab3e99c700 1 -- 172.21.15.156:0/4067770307 learned_addr learned my addr 172.21.15.156:0/4067770307 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:47.836 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.832+0000 7fab3d99a700 1 --2- 172.21.15.156:0/4067770307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab3815c9b0 0x7fab38151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:47.836 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.832+0000 7fab3e19b700 1 --2- 172.21.15.156:0/4067770307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab380d2320 0x7fab381505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:47.836 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.833+0000 7fab3e99c700 1 -- 172.21.15.156:0/4067770307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab380d2320 msgr2=0x7fab381505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.836 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.833+0000 7fab3e99c700 1 --2- 172.21.15.156:0/4067770307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab380d2320 0x7fab381505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.836 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.833+0000 7fab3e99c700 1 -- 172.21.15.156:0/4067770307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab3815c9b0 msgr2=0x7fab38151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.836 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.833+0000 7fab3e99c700 1 --2- 172.21.15.156:0/4067770307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab3815c9b0 0x7fab38151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.836 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.833+0000 7fab3e99c700 1 -- 172.21.15.156:0/4067770307 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab280094a0 con 0x7fab38161340 2023-11-28T13:07:47.837 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.833+0000 7fab3e99c700 1 --2- 172.21.15.156:0/4067770307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab38161340 0x7fab3815d810 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fab3000c820 tx=0x7fab3000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:47.837 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.833+0000 7fab2effd700 1 -- 172.21.15.156:0/4067770307 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30003e30 con 0x7fab38161340 2023-11-28T13:07:47.837 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.833+0000 7fab2effd700 1 -- 172.21.15.156:0/4067770307 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab300050f0 con 0x7fab38161340 2023-11-28T13:07:47.837 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.833+0000 7fab2effd700 1 -- 172.21.15.156:0/4067770307 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab300054a0 con 0x7fab38161340 2023-11-28T13:07:47.837 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.833+0000 7fab403ff700 1 -- 172.21.15.156:0/4067770307 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3815cd80 con 0x7fab38161340 2023-11-28T13:07:47.837 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.833+0000 7fab403ff700 1 -- 172.21.15.156:0/4067770307 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab380d15e0 con 0x7fab38161340 2023-11-28T13:07:47.837 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.834+0000 7fab2effd700 1 -- 172.21.15.156:0/4067770307 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fab30019030 con 0x7fab38161340 2023-11-28T13:07:47.837 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.834+0000 7fab2effd700 1 -- 172.21.15.156:0/4067770307 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fab30012400 con 0x7fab38161340 2023-11-28T13:07:47.837 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.834+0000 7fab403ff700 1 -- 172.21.15.156:0/4067770307 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3815cf10 con 0x7fab38161340 2023-11-28T13:07:47.838 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.837+0000 7fab2effd700 1 -- 172.21.15.156:0/4067770307 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fab30017070 con 0x7fab38161340 2023-11-28T13:07:47.898 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:47.896+0000 7fe7d950f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T13:07:47.912 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.910+0000 7fab403ff700 1 -- 172.21.15.156:0/4067770307 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab380d26f0 con 0x7fab38161340 2023-11-28T13:07:47.913 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.911+0000 7fab2effd700 1 -- 172.21.15.156:0/4067770307 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fab300125c0 con 0x7fab38161340 2023-11-28T13:07:47.913 INFO:teuthology.misc.health.smithi156.stdout: 2023-11-28T13:07:47.913 INFO:teuthology.misc.health.smithi156.stdout:{"epoch":10,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:07:46.384751+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:07:47.914 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.912+0000 7fab403ff700 1 -- 172.21.15.156:0/4067770307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab38161340 msgr2=0x7fab3815d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.915 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.912+0000 7fab403ff700 1 --2- 172.21.15.156:0/4067770307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab38161340 0x7fab3815d810 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fab3000c820 tx=0x7fab3000cb30 comp rx=0 tx=0).stop 2023-11-28T13:07:47.915 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.912+0000 7fab403ff700 1 -- 172.21.15.156:0/4067770307 shutdown_connections 2023-11-28T13:07:47.915 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.912+0000 7fab403ff700 1 --2- 172.21.15.156:0/4067770307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab38161340 0x7fab3815d810 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.915 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.912+0000 7fab403ff700 1 --2- 172.21.15.156:0/4067770307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab380d2320 0x7fab381505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.915 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.912+0000 7fab403ff700 1 --2- 172.21.15.156:0/4067770307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab3815c9b0 0x7fab38151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.915 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.912+0000 7fab403ff700 1 -- 172.21.15.156:0/4067770307 >> 172.21.15.156:0/4067770307 conn(0x7fab38072600 msgr2=0x7fab3815e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:47.915 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.913+0000 7fab403ff700 1 -- 172.21.15.156:0/4067770307 shutdown_connections 2023-11-28T13:07:47.915 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:47.913+0000 7fab403ff700 1 -- 172.21.15.156:0/4067770307 wait complete. 2023-11-28T13:07:47.931 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-28T13:07:48.103 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:48.102+0000 7fe509d39200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T13:07:48.179 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:48.178+0000 7fe7d950f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T13:07:48.272 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:48.270+0000 7fe509d39200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T13:07:48.382 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:48.380+0000 7fe7d950f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T13:07:48.532 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:48.530+0000 7fe7d950f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T13:07:48.555 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:48.554+0000 7fe509d39200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T13:07:48.758 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:48.757+0000 7fe509d39200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T13:07:48.911 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:48.909+0000 7fe509d39200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T13:07:48.929 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:48.928+0000 7fe7d950f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T13:07:49.054 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:49.053+0000 7fe7d950f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T13:07:49.287 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:49.286+0000 7fe7d950f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T13:07:49.307 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:49.305+0000 7fe509d39200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T13:07:49.440 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:49.439+0000 7fe509d39200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T13:07:49.703 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:49.703+0000 7fe509d39200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T13:07:49.994 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:49.993+0000 7fe7d950f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T13:07:50.482 INFO:tasks.ceph.osd.1.smithi156.stderr:2023-11-28T13:07:50.482+0000 7f5a6b878640 -1 osd.1 0 log_to_monitors true 2023-11-28T13:07:50.504 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-11-28T13:07:50.504+0000 7fe509d39200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T13:07:50.705 INFO:tasks.ceph.osd.0.smithi156.stderr:2023-11-28T13:07:50.703+0000 7fce73a0d640 -1 osd.0 0 log_to_monitors true 2023-11-28T13:07:50.745 INFO:tasks.ceph.osd.2.smithi156.stderr:2023-11-28T13:07:50.745+0000 7f73e4cd0640 -1 osd.2 0 log_to_monitors true 2023-11-28T13:07:50.959 INFO:tasks.ceph.osd.7.smithi186.stderr:2023-11-28T13:07:50.958+0000 7f317642b640 -1 osd.7 0 log_to_monitors true 2023-11-28T13:07:50.985 INFO:tasks.ceph.osd.3.smithi156.stderr:2023-11-28T13:07:50.985+0000 7fcced639640 -1 osd.3 0 log_to_monitors true 2023-11-28T13:07:50.996 INFO:tasks.ceph.osd.5.smithi186.stderr:2023-11-28T13:07:50.995+0000 7fa43db2c640 -1 osd.5 0 log_to_monitors true 2023-11-28T13:07:51.016 INFO:tasks.ceph.osd.6.smithi186.stderr:2023-11-28T13:07:51.015+0000 7fe7147a2640 -1 osd.6 0 log_to_monitors true 2023-11-28T13:07:51.020 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:51.019+0000 7fe78c7fa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-28T13:07:51.021 INFO:tasks.ceph.osd.4.smithi186.stderr:2023-11-28T13:07:51.020+0000 7fd6dfe62640 -1 osd.4 0 log_to_monitors true 2023-11-28T13:07:51.021 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-11-28T13:07:51.021+0000 7fe78c7fa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-28T13:07:53.812 INFO:tasks.ceph.osd.3.smithi156.stderr:2023-11-28T13:07:53.811+0000 7fccdbdf6700 -1 osd.3 0 waiting for initial osdmap 2023-11-28T13:07:53.812 INFO:tasks.ceph.osd.1.smithi156.stderr:2023-11-28T13:07:53.812+0000 7f5a59ff6700 -1 osd.1 0 waiting for initial osdmap 2023-11-28T13:07:53.827 INFO:tasks.ceph.osd.3.smithi156.stderr:2023-11-28T13:07:53.827+0000 7fccd3dfb700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:07:53.828 INFO:tasks.ceph.osd.1.smithi156.stderr:2023-11-28T13:07:53.827+0000 7f5a51ffb700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:07:53.834 INFO:tasks.ceph.osd.2.smithi156.stderr:2023-11-28T13:07:53.834+0000 7f73d33f6700 -1 osd.2 0 waiting for initial osdmap 2023-11-28T13:07:53.845 INFO:tasks.ceph.osd.0.smithi156.stderr:2023-11-28T13:07:53.845+0000 7fce61ff6700 -1 osd.0 0 waiting for initial osdmap 2023-11-28T13:07:53.848 INFO:tasks.ceph.osd.2.smithi156.stderr:2023-11-28T13:07:53.848+0000 7f73cb3fb700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:07:53.857 INFO:tasks.ceph.osd.0.smithi156.stderr:2023-11-28T13:07:53.857+0000 7fce59ffb700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:07:54.412 INFO:tasks.ceph.osd.4.smithi186.stderr:2023-11-28T13:07:54.409+0000 7fd6ce5f6700 -1 osd.4 0 waiting for initial osdmap 2023-11-28T13:07:54.413 INFO:tasks.ceph.osd.7.smithi186.stderr:2023-11-28T13:07:54.411+0000 7f31663f9700 -1 osd.7 0 waiting for initial osdmap 2023-11-28T13:07:54.429 INFO:tasks.ceph.osd.7.smithi186.stderr:2023-11-28T13:07:54.428+0000 7f315cbfb700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:07:54.429 INFO:tasks.ceph.osd.4.smithi186.stderr:2023-11-28T13:07:54.428+0000 7fd6c65fb700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:07:54.430 INFO:tasks.ceph.osd.5.smithi186.stderr:2023-11-28T13:07:54.429+0000 7fa42c1f6700 -1 osd.5 0 waiting for initial osdmap 2023-11-28T13:07:54.447 INFO:tasks.ceph.osd.5.smithi186.stderr:2023-11-28T13:07:54.446+0000 7fa4241fb700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:07:54.479 INFO:tasks.ceph.osd.6.smithi186.stderr:2023-11-28T13:07:54.478+0000 7fe702df6700 -1 osd.6 0 waiting for initial osdmap 2023-11-28T13:07:54.495 INFO:tasks.ceph.osd.6.smithi186.stderr:2023-11-28T13:07:54.494+0000 7fe6fadfb700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:07:54.738 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T13:07:54.923 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.922+0000 7f7d5b9ff700 1 Processor -- start 2023-11-28T13:07:54.923 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.922+0000 7f7d5b9ff700 1 -- start start 2023-11-28T13:07:54.926 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.923+0000 7f7d5b9ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7d540cd7c0 0x7f7d540cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.926 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.923+0000 7f7d5b9ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d540d1950 con 0x7f7d54161340 2023-11-28T13:07:54.926 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.923+0000 7f7d5b9ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d540ce140 con 0x7f7d5415c9b0 2023-11-28T13:07:54.927 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.923+0000 7f7d5b9ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d540b5800 con 0x7f7d540cd7c0 2023-11-28T13:07:54.927 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.924+0000 7f7d58f9a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f7d54161340 0x7f7d540d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:44034/0 (socket says 172.21.15.156:44034) 2023-11-28T13:07:54.927 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.924+0000 7f7d58f9a700 1 -- 172.21.15.156:0/1536615796 learned_addr learned my addr 172.21.15.156:0/1536615796 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:07:54.927 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.924+0000 7f7d59f9c700 1 --2- 172.21.15.156:0/1536615796 >> v2:172.21.15.186:3300/0 conn(0x7f7d540cd7c0 0x7f7d540cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.927 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.924+0000 7f7d4bfff700 1 -- 172.21.15.156:0/1536615796 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330483521 0 0) 0x7f7d540d1950 con 0x7f7d54161340 2023-11-28T13:07:54.927 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.924+0000 7f7d4bfff700 1 -- 172.21.15.156:0/1536615796 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d400035b0 con 0x7f7d54161340 2023-11-28T13:07:54.927 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.924+0000 7f7d59f9c700 1 -- 172.21.15.156:0/1536615796 >> v1:172.21.15.186:6789/0 conn(0x7f7d5415c9b0 legacy=0x7f7d540b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.927 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.924+0000 7f7d4bfff700 1 -- 172.21.15.156:0/1536615796 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324633153 0 0) 0x7f7d540ce140 con 0x7f7d5415c9b0 2023-11-28T13:07:54.927 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.924+0000 7f7d59f9c700 1 -- 172.21.15.156:0/1536615796 >> v1:172.21.15.156:6789/0 conn(0x7f7d54161340 legacy=0x7f7d540d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.927 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.924+0000 7f7d59f9c700 1 -- 172.21.15.156:0/1536615796 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d540d24f0 con 0x7f7d540cd7c0 2023-11-28T13:07:54.927 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.925+0000 7f7d59f9c700 1 --2- 172.21.15.156:0/1536615796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d540cd7c0 0x7f7d540cdc00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009d90 tx=0x7f7d4c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae4d02c0caae4ec5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.927 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.925+0000 7f7d4bfff700 1 -- 172.21.15.156:0/1536615796 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c00e070 con 0x7f7d540cd7c0 2023-11-28T13:07:54.928 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.925+0000 7f7d4bfff700 1 -- 172.21.15.156:0/1536615796 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4c003a20 con 0x7f7d540cd7c0 2023-11-28T13:07:54.928 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.925+0000 7f7d4bfff700 1 -- 172.21.15.156:0/1536615796 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d44001670 con 0x7f7d540cd7c0 2023-11-28T13:07:54.928 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.925+0000 7f7d5b9ff700 1 -- 172.21.15.156:0/1536615796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d540cd7c0 msgr2=0x7f7d540cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.928 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.925+0000 7f7d5b9ff700 1 --2- 172.21.15.156:0/1536615796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d540cd7c0 0x7f7d540cdc00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009d90 tx=0x7f7d4c00b330 comp rx=0 tx=0).stop 2023-11-28T13:07:54.928 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.925+0000 7f7d5b9ff700 1 -- 172.21.15.156:0/1536615796 shutdown_connections 2023-11-28T13:07:54.928 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.925+0000 7f7d5b9ff700 1 --2- 172.21.15.156:0/1536615796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d540cd7c0 0x7f7d540cdc00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.928 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.925+0000 7f7d5b9ff700 1 -- 172.21.15.156:0/1536615796 >> 172.21.15.156:0/1536615796 conn(0x7f7d54072600 msgr2=0x7f7d540cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:54.928 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.925+0000 7f7d5b9ff700 1 -- 172.21.15.156:0/1536615796 shutdown_connections 2023-11-28T13:07:54.928 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.925+0000 7f7d5b9ff700 1 -- 172.21.15.156:0/1536615796 wait complete. 2023-11-28T13:07:54.930 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.927+0000 7f7d5b9ff700 1 Processor -- start 2023-11-28T13:07:54.931 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.927+0000 7f7d5b9ff700 1 -- start start 2023-11-28T13:07:54.931 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.928+0000 7f7d5b9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d540cd7c0 0x7f7d540c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.931 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.928+0000 7f7d5b9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d5415c9b0 0x7f7d54157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.931 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.928+0000 7f7d5b9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d54161340 0x7f7d540cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.931 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.928+0000 7f7d5b9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d540c6ca0 con 0x7f7d54161340 2023-11-28T13:07:54.931 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.928+0000 7f7d5b9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d540ca7a0 con 0x7f7d540cd7c0 2023-11-28T13:07:54.931 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.928+0000 7f7d5b9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d540c6ef0 con 0x7f7d5415c9b0 2023-11-28T13:07:54.931 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.928+0000 7f7d59f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d54161340 0x7f7d540cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.931 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.929+0000 7f7d59f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d54161340 0x7f7d540cac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45366/0 (socket says 172.21.15.156:45366) 2023-11-28T13:07:54.931 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.929+0000 7f7d59f9c700 1 -- 172.21.15.156:0/199556033 learned_addr learned my addr 172.21.15.156:0/199556033 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:54.931 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.929+0000 7f7d58f9a700 1 --2- 172.21.15.156:0/199556033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d5415c9b0 0x7f7d54157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.931 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.929+0000 7f7d5979b700 1 --2- 172.21.15.156:0/199556033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d540cd7c0 0x7f7d540c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.932 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.929+0000 7f7d59f9c700 1 -- 172.21.15.156:0/199556033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d5415c9b0 msgr2=0x7f7d54157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.932 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.929+0000 7f7d59f9c700 1 --2- 172.21.15.156:0/199556033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d5415c9b0 0x7f7d54157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.932 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.929+0000 7f7d59f9c700 1 -- 172.21.15.156:0/199556033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d540cd7c0 msgr2=0x7f7d540c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.932 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.929+0000 7f7d59f9c700 1 --2- 172.21.15.156:0/199556033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d540cd7c0 0x7f7d540c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.932 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.929+0000 7f7d59f9c700 1 -- 172.21.15.156:0/199556033 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4c009a70 con 0x7f7d54161340 2023-11-28T13:07:54.932 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.929+0000 7f7d5979b700 1 --2- 172.21.15.156:0/199556033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d540cd7c0 0x7f7d540c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:54.932 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.929+0000 7f7d59f9c700 1 --2- 172.21.15.156:0/199556033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d54161340 0x7f7d540cac50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c015040 tx=0x7f7d4c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.932 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.929+0000 7f7d49ffb700 1 -- 172.21.15.156:0/199556033 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c004670 con 0x7f7d54161340 2023-11-28T13:07:54.932 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.929+0000 7f7d49ffb700 1 -- 172.21.15.156:0/199556033 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4c0047d0 con 0x7f7d54161340 2023-11-28T13:07:54.932 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.929+0000 7f7d49ffb700 1 -- 172.21.15.156:0/199556033 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c012690 con 0x7f7d54161340 2023-11-28T13:07:54.932 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.929+0000 7f7d5b9ff700 1 -- 172.21.15.156:0/199556033 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d540d1300 con 0x7f7d54161340 2023-11-28T13:07:54.932 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.930+0000 7f7d5b9ff700 1 -- 172.21.15.156:0/199556033 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d540b5520 con 0x7f7d54161340 2023-11-28T13:07:54.933 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.931+0000 7f7d5b9ff700 1 -- 172.21.15.156:0/199556033 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d540d15e0 con 0x7f7d54161340 2023-11-28T13:07:54.933 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.931+0000 7f7d49ffb700 1 -- 172.21.15.156:0/199556033 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f7d4c024030 con 0x7f7d54161340 2023-11-28T13:07:54.933 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.933+0000 7f7d49ffb700 1 --2- 172.21.15.156:0/199556033 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7d380682d0 0x7f7d3806a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.934 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.933+0000 7f7d49ffb700 1 -- 172.21.15.156:0/199556033 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f7d4c087290 con 0x7f7d54161340 2023-11-28T13:07:54.934 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.933+0000 7f7d5979b700 1 --2- 172.21.15.156:0/199556033 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7d380682d0 0x7f7d3806a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.934 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.934+0000 7f7d5979b700 1 --2- 172.21.15.156:0/199556033 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7d380682d0 0x7f7d3806a780 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7d50000f60 tx=0x7f7d50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.938 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:54.938+0000 7f7d49ffb700 1 -- 172.21.15.156:0/199556033 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7d4c054080 con 0x7f7d54161340 2023-11-28T13:07:55.135 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:55.135+0000 7f7d5b9ff700 1 -- 172.21.15.156:0/199556033 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d540745c0 con 0x7f7d54161340 2023-11-28T13:07:55.136 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:55.135+0000 7f7d49ffb700 1 -- 172.21.15.156:0/199556033 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10584 (secure 0 0 0) 0x7f7d4c0576a0 con 0x7f7d54161340 2023-11-28T13:07:55.136 INFO:teuthology.misc.health.smithi156.stdout: 2023-11-28T13:07:55.136 INFO:teuthology.misc.health.smithi156.stdout:{"epoch":13,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:07:54.821228+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:07:55.139 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:55.138+0000 7f7d5b9ff700 1 -- 172.21.15.156:0/199556033 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7d380682d0 msgr2=0x7f7d3806a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.139 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:55.138+0000 7f7d5b9ff700 1 --2- 172.21.15.156:0/199556033 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7d380682d0 0x7f7d3806a780 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7d50000f60 tx=0x7f7d50009040 comp rx=0 tx=0).stop 2023-11-28T13:07:55.139 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:55.138+0000 7f7d5b9ff700 1 -- 172.21.15.156:0/199556033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d54161340 msgr2=0x7f7d540cac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.139 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:55.138+0000 7f7d5b9ff700 1 --2- 172.21.15.156:0/199556033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d54161340 0x7f7d540cac50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c015040 tx=0x7f7d4c004ad0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.139 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:55.138+0000 7f7d5b9ff700 1 -- 172.21.15.156:0/199556033 shutdown_connections 2023-11-28T13:07:55.139 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:55.138+0000 7f7d5b9ff700 1 --2- 172.21.15.156:0/199556033 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7d380682d0 0x7f7d3806a780 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.139 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:55.138+0000 7f7d5b9ff700 1 --2- 172.21.15.156:0/199556033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d54161340 0x7f7d540cac50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.139 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:55.138+0000 7f7d5b9ff700 1 --2- 172.21.15.156:0/199556033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d540cd7c0 0x7f7d540c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.139 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:55.138+0000 7f7d5b9ff700 1 --2- 172.21.15.156:0/199556033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d5415c9b0 0x7f7d54157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.139 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:55.138+0000 7f7d5b9ff700 1 -- 172.21.15.156:0/199556033 >> 172.21.15.156:0/199556033 conn(0x7f7d54072600 msgr2=0x7f7d5415a250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:55.140 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:55.138+0000 7f7d5b9ff700 1 -- 172.21.15.156:0/199556033 shutdown_connections 2023-11-28T13:07:55.140 INFO:teuthology.misc.health.smithi156.stderr:2023-11-28T13:07:55.138+0000 7f7d5b9ff700 1 -- 172.21.15.156:0/199556033 wait complete. 2023-11-28T13:07:55.154 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-28T13:07:55.154 INFO:tasks.ceph:Creating RBD pool 2023-11-28T13:07:55.155 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-28T13:07:55.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.345+0000 7fb2c23ff700 1 Processor -- start 2023-11-28T13:07:55.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.345+0000 7fb2c23ff700 1 -- start start 2023-11-28T13:07:55.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.346+0000 7fb2c23ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb2bc0b0d80 0x7fb2bc15add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.346+0000 7fb2c23ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2bc15e1a0 con 0x7fb2bc070c50 2023-11-28T13:07:55.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.346+0000 7fb2c23ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2bc0cb180 con 0x7fb2bc158200 2023-11-28T13:07:55.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.346+0000 7fb2c23ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2bc15f070 con 0x7fb2bc0b0d80 2023-11-28T13:07:55.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.346+0000 7fb2bb7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb2bc070c50 0x7fb2bc0b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:44038/0 (socket says 172.21.15.156:44038) 2023-11-28T13:07:55.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.347+0000 7fb2bb7fe700 1 -- 172.21.15.156:0/1486508684 learned_addr learned my addr 172.21.15.156:0/1486508684 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:07:55.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.347+0000 7fb2bbfff700 1 --1- 172.21.15.156:0/1486508684 >> v1:172.21.15.186:6789/0 conn(0x7fb2bc158200 0x7fb2bc0caa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:33426/0 (socket says 172.21.15.156:33426) 2023-11-28T13:07:55.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.347+0000 7fb2c099c700 1 --2- 172.21.15.156:0/1486508684 >> v2:172.21.15.186:3300/0 conn(0x7fb2bc0b0d80 0x7fb2bc15add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.347+0000 7fb2baffd700 1 -- 172.21.15.156:0/1486508684 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245337997 0 0) 0x7fb2bc0cb180 con 0x7fb2bc158200 2023-11-28T13:07:55.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.347+0000 7fb2baffd700 1 -- 172.21.15.156:0/1486508684 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2a40035b0 con 0x7fb2bc158200 2023-11-28T13:07:55.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.347+0000 7fb2baffd700 1 -- 172.21.15.156:0/1486508684 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987611459 0 0) 0x7fb2bc15e1a0 con 0x7fb2bc070c50 2023-11-28T13:07:55.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.347+0000 7fb2baffd700 1 -- 172.21.15.156:0/1486508684 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2bc0cb180 con 0x7fb2bc070c50 2023-11-28T13:07:55.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.347+0000 7fb2c099c700 1 -- 172.21.15.156:0/1486508684 >> v1:172.21.15.186:6789/0 conn(0x7fb2bc158200 legacy=0x7fb2bc0caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.347+0000 7fb2c099c700 1 -- 172.21.15.156:0/1486508684 >> v1:172.21.15.156:6789/0 conn(0x7fb2bc070c50 legacy=0x7fb2bc0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.347+0000 7fb2c099c700 1 -- 172.21.15.156:0/1486508684 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc0710f0 con 0x7fb2bc0b0d80 2023-11-28T13:07:55.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.347+0000 7fb2c099c700 1 --2- 172.21.15.156:0/1486508684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2bc0b0d80 0x7fb2bc15add0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009d90 tx=0x7fb2b400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e79b7b9783340000 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:55.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.348+0000 7fb2baffd700 1 -- 172.21.15.156:0/1486508684 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400e070 con 0x7fb2bc0b0d80 2023-11-28T13:07:55.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.348+0000 7fb2baffd700 1 -- 172.21.15.156:0/1486508684 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b400bb20 con 0x7fb2bc0b0d80 2023-11-28T13:07:55.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.348+0000 7fb2baffd700 1 -- 172.21.15.156:0/1486508684 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4012430 con 0x7fb2bc0b0d80 2023-11-28T13:07:55.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.348+0000 7fb2c23ff700 1 -- 172.21.15.156:0/1486508684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2bc0b0d80 msgr2=0x7fb2bc15add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.348+0000 7fb2c23ff700 1 --2- 172.21.15.156:0/1486508684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2bc0b0d80 0x7fb2bc15add0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009d90 tx=0x7fb2b400b330 comp rx=0 tx=0).stop 2023-11-28T13:07:55.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.348+0000 7fb2c23ff700 1 -- 172.21.15.156:0/1486508684 shutdown_connections 2023-11-28T13:07:55.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.348+0000 7fb2c23ff700 1 --2- 172.21.15.156:0/1486508684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2bc0b0d80 0x7fb2bc15add0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.348+0000 7fb2c23ff700 1 -- 172.21.15.156:0/1486508684 >> 172.21.15.156:0/1486508684 conn(0x7fb2bc0cb670 msgr2=0x7fb2bc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:55.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.348+0000 7fb2c23ff700 1 -- 172.21.15.156:0/1486508684 shutdown_connections 2023-11-28T13:07:55.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.348+0000 7fb2c23ff700 1 -- 172.21.15.156:0/1486508684 wait complete. 2023-11-28T13:07:55.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.350+0000 7fb2c23ff700 1 Processor -- start 2023-11-28T13:07:55.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.350+0000 7fb2c23ff700 1 -- start start 2023-11-28T13:07:55.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.351+0000 7fb2c23ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2bc070c50 0x7fb2bc159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.351+0000 7fb2c23ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2bc0b0d80 0x7fb2bc0b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.351+0000 7fb2c23ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2bc158200 0x7fb2bc1f17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.351+0000 7fb2c23ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0b29d0 con 0x7fb2bc158200 2023-11-28T13:07:55.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.351+0000 7fb2c23ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc1f1d20 con 0x7fb2bc0b0d80 2023-11-28T13:07:55.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.351+0000 7fb2c23ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc159830 con 0x7fb2bc070c50 2023-11-28T13:07:55.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.351+0000 7fb2c099c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2bc158200 0x7fb2bc1f17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.351+0000 7fb2bb7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2bc0b0d80 0x7fb2bc0b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.351+0000 7fb2c099c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2bc158200 0x7fb2bc1f17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45374/0 (socket says 172.21.15.156:45374) 2023-11-28T13:07:55.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.351+0000 7fb2c099c700 1 -- 172.21.15.156:0/1258216908 learned_addr learned my addr 172.21.15.156:0/1258216908 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:55.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.351+0000 7fb2bbfff700 1 --2- 172.21.15.156:0/1258216908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2bc070c50 0x7fb2bc159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.352+0000 7fb2c099c700 1 -- 172.21.15.156:0/1258216908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2bc070c50 msgr2=0x7fb2bc159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.352+0000 7fb2c099c700 1 --2- 172.21.15.156:0/1258216908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2bc070c50 0x7fb2bc159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.352+0000 7fb2c099c700 1 -- 172.21.15.156:0/1258216908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2bc0b0d80 msgr2=0x7fb2bc0b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.352+0000 7fb2c099c700 1 --2- 172.21.15.156:0/1258216908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2bc0b0d80 0x7fb2bc0b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.355 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.352+0000 7fb2c099c700 1 -- 172.21.15.156:0/1258216908 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b4009a70 con 0x7fb2bc158200 2023-11-28T13:07:55.355 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.352+0000 7fb2c099c700 1 --2- 172.21.15.156:0/1258216908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2bc158200 0x7fb2bc1f17e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4013000 tx=0x7fb2b4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:55.355 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.352+0000 7fb2b8ff9700 1 -- 172.21.15.156:0/1258216908 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4012430 con 0x7fb2bc158200 2023-11-28T13:07:55.355 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.352+0000 7fb2b8ff9700 1 -- 172.21.15.156:0/1258216908 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b4020070 con 0x7fb2bc158200 2023-11-28T13:07:55.355 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.352+0000 7fb2b8ff9700 1 -- 172.21.15.156:0/1258216908 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b401ba50 con 0x7fb2bc158200 2023-11-28T13:07:55.355 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.352+0000 7fb2c23ff700 1 -- 172.21.15.156:0/1258216908 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2bc15e3f0 con 0x7fb2bc158200 2023-11-28T13:07:55.355 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.352+0000 7fb2c23ff700 1 -- 172.21.15.156:0/1258216908 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2bc1585d0 con 0x7fb2bc158200 2023-11-28T13:07:55.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.353+0000 7fb2c23ff700 1 -- 172.21.15.156:0/1258216908 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2bc15e6d0 con 0x7fb2bc158200 2023-11-28T13:07:55.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.354+0000 7fb2b8ff9700 1 -- 172.21.15.156:0/1258216908 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fb2b402e070 con 0x7fb2bc158200 2023-11-28T13:07:55.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.355+0000 7fb2b8ff9700 1 --2- 172.21.15.156:0/1258216908 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb2a80682d0 0x7fb2a806a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.355+0000 7fb2b8ff9700 1 -- 172.21.15.156:0/1258216908 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fb2b4022070 con 0x7fb2bc158200 2023-11-28T13:07:55.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.355+0000 7fb2bbfff700 1 --2- 172.21.15.156:0/1258216908 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb2a80682d0 0x7fb2a806a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.356+0000 7fb2bbfff700 1 --2- 172.21.15.156:0/1258216908 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb2a80682d0 0x7fb2a806a780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac007fd0 tx=0x7fb2ac007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:55.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.359+0000 7fb2b8ff9700 1 -- 172.21.15.156:0/1258216908 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb2b40543b0 con 0x7fb2bc158200 2023-11-28T13:07:55.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.556+0000 7fb2c23ff700 1 -- 172.21.15.156:0/1258216908 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fb2bc06fd60 con 0x7fb2bc158200 2023-11-28T13:07:55.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.828+0000 7fb2b8ff9700 1 -- 172.21.15.156:0/1258216908 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7fb2b402c0a0 con 0x7fb2bc158200 2023-11-28T13:07:55.831 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.830+0000 7fb2c23ff700 1 -- 172.21.15.156:0/1258216908 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb2a80682d0 msgr2=0x7fb2a806a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.831 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.830+0000 7fb2c23ff700 1 --2- 172.21.15.156:0/1258216908 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb2a80682d0 0x7fb2a806a780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac007fd0 tx=0x7fb2ac007de0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.831 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.830+0000 7fb2c23ff700 1 -- 172.21.15.156:0/1258216908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2bc158200 msgr2=0x7fb2bc1f17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.831 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.830+0000 7fb2c23ff700 1 --2- 172.21.15.156:0/1258216908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2bc158200 0x7fb2bc1f17e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4013000 tx=0x7fb2b4004ad0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.830+0000 7fb2c23ff700 1 -- 172.21.15.156:0/1258216908 shutdown_connections 2023-11-28T13:07:55.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.830+0000 7fb2c23ff700 1 --2- 172.21.15.156:0/1258216908 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb2a80682d0 0x7fb2a806a780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.830+0000 7fb2c23ff700 1 --2- 172.21.15.156:0/1258216908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2bc158200 0x7fb2bc1f17e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.830+0000 7fb2c23ff700 1 --2- 172.21.15.156:0/1258216908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2bc070c50 0x7fb2bc159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.830+0000 7fb2c23ff700 1 --2- 172.21.15.156:0/1258216908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2bc0b0d80 0x7fb2bc0b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.830+0000 7fb2c23ff700 1 -- 172.21.15.156:0/1258216908 >> 172.21.15.156:0/1258216908 conn(0x7fb2bc0cb670 msgr2=0x7fb2bc0ad2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:55.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.830+0000 7fb2c23ff700 1 -- 172.21.15.156:0/1258216908 shutdown_connections 2023-11-28T13:07:55.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:55.830+0000 7fb2c23ff700 1 -- 172.21.15.156:0/1258216908 wait complete. 2023-11-28T13:07:55.833 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'rbd' created 2023-11-28T13:07:55.850 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-28T13:07:56.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.044+0000 7f35d3fff700 1 Processor -- start 2023-11-28T13:07:56.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.044+0000 7f35d3fff700 1 -- start start 2023-11-28T13:07:56.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.045+0000 7f35d3fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f35cc15ca60 0x7f35cc0b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:56.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.045+0000 7f35d3fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35cc0afbe0 con 0x7f35cc0b29d0 2023-11-28T13:07:56.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.045+0000 7f35d3fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35cc0b2750 con 0x7f35cc0d1350 2023-11-28T13:07:56.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.045+0000 7f35d3fff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35cc15ce30 con 0x7f35cc15ca60 2023-11-28T13:07:56.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.046+0000 7f35d1d9b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f35cc15ca60 0x7f35cc0b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:56.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.046+0000 7f35d259c700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f35cc0d1350 0x7f35cc0b0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:47594/0 (socket says 172.21.15.156:47594) 2023-11-28T13:07:56.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.046+0000 7f35d259c700 1 -- 172.21.15.156:0/1540090529 learned_addr learned my addr 172.21.15.156:0/1540090529 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:07:56.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.046+0000 7f35d1d9b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f35cc15ca60 0x7f35cc0b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:40584/0 (socket says 172.21.15.156:40584) 2023-11-28T13:07:56.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.046+0000 7f35d0d99700 1 -- 172.21.15.156:0/1540090529 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42325998 0 0) 0x7f35cc0b2750 con 0x7f35cc0d1350 2023-11-28T13:07:56.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.046+0000 7f35d0d99700 1 -- 172.21.15.156:0/1540090529 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35b80035b0 con 0x7f35cc0d1350 2023-11-28T13:07:56.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.046+0000 7f35d1d9b700 1 -- 172.21.15.156:0/1540090529 >> v1:172.21.15.186:6789/0 conn(0x7f35cc0d1350 legacy=0x7f35cc0b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:56.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.046+0000 7f35d1d9b700 1 -- 172.21.15.156:0/1540090529 >> v1:172.21.15.156:6789/0 conn(0x7f35cc0b29d0 legacy=0x7f35cc0d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:56.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.046+0000 7f35d1d9b700 1 -- 172.21.15.156:0/1540090529 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc0b3f20 con 0x7f35cc15ca60 2023-11-28T13:07:56.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.046+0000 7f35d1d9b700 1 --2- 172.21.15.156:0/1540090529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35cc15ca60 0x7f35cc0b38b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009820 tx=0x7f35c8009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd5f1cf8a9553e8a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:56.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.047+0000 7f35d0d99700 1 -- 172.21.15.156:0/1540090529 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8005e30 con 0x7f35cc15ca60 2023-11-28T13:07:56.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.047+0000 7f35d0d99700 1 -- 172.21.15.156:0/1540090529 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c8006030 con 0x7f35cc15ca60 2023-11-28T13:07:56.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.047+0000 7f35d0d99700 1 -- 172.21.15.156:0/1540090529 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c80063c0 con 0x7f35cc15ca60 2023-11-28T13:07:56.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.047+0000 7f35d3fff700 1 -- 172.21.15.156:0/1540090529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35cc15ca60 msgr2=0x7f35cc0b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:56.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.047+0000 7f35d3fff700 1 --2- 172.21.15.156:0/1540090529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35cc15ca60 0x7f35cc0b38b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009820 tx=0x7f35c8009b30 comp rx=0 tx=0).stop 2023-11-28T13:07:56.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.047+0000 7f35d3fff700 1 -- 172.21.15.156:0/1540090529 shutdown_connections 2023-11-28T13:07:56.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.047+0000 7f35d3fff700 1 --2- 172.21.15.156:0/1540090529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35cc15ca60 0x7f35cc0b38b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:56.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.047+0000 7f35d3fff700 1 -- 172.21.15.156:0/1540090529 >> 172.21.15.156:0/1540090529 conn(0x7f35cc09c3f0 msgr2=0x7f35cc0d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:56.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.048+0000 7f35d3fff700 1 -- 172.21.15.156:0/1540090529 shutdown_connections 2023-11-28T13:07:56.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.048+0000 7f35d3fff700 1 -- 172.21.15.156:0/1540090529 wait complete. 2023-11-28T13:07:56.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.050+0000 7f35d3fff700 1 Processor -- start 2023-11-28T13:07:56.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.050+0000 7f35d3fff700 1 -- start start 2023-11-28T13:07:56.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.052+0000 7f35d3fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35cc0b29d0 0x7f35cc0b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:56.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.052+0000 7f35d3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35cc0d1350 0x7f35cc0b8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:56.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.052+0000 7f35d3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35cc15ca60 0x7f35cc1f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:56.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.052+0000 7f35d3fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc0c9980 con 0x7f35cc15ca60 2023-11-28T13:07:56.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.052+0000 7f35d3fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc0b7870 con 0x7f35cc0d1350 2023-11-28T13:07:56.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.052+0000 7f35d3fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc1f32f0 con 0x7f35cc0b29d0 2023-11-28T13:07:56.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.052+0000 7f35d259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35cc15ca60 0x7f35cc1f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:56.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.052+0000 7f35d259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35cc15ca60 0x7f35cc1f5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43358/0 (socket says 172.21.15.156:43358) 2023-11-28T13:07:56.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.052+0000 7f35d259c700 1 -- 172.21.15.156:0/932956699 learned_addr learned my addr 172.21.15.156:0/932956699 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:56.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.052+0000 7f35d1d9b700 1 --2- 172.21.15.156:0/932956699 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35cc0b29d0 0x7f35cc0b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:56.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.052+0000 7f35d159a700 1 --2- 172.21.15.156:0/932956699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35cc0d1350 0x7f35cc0b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:56.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.053+0000 7f35d1d9b700 1 -- 172.21.15.156:0/932956699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35cc0d1350 msgr2=0x7f35cc0b8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:56.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.053+0000 7f35d1d9b700 1 --2- 172.21.15.156:0/932956699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35cc0d1350 0x7f35cc0b8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:56.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.053+0000 7f35d1d9b700 1 -- 172.21.15.156:0/932956699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35cc15ca60 msgr2=0x7f35cc1f5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:56.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.053+0000 7f35d1d9b700 1 --2- 172.21.15.156:0/932956699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35cc15ca60 0x7f35cc1f5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:56.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.053+0000 7f35d1d9b700 1 -- 172.21.15.156:0/932956699 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c8009430 con 0x7f35cc0b29d0 2023-11-28T13:07:56.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.053+0000 7f35d159a700 1 --2- 172.21.15.156:0/932956699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35cc0d1350 0x7f35cc0b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:56.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.053+0000 7f35d1d9b700 1 --2- 172.21.15.156:0/932956699 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35cc0b29d0 0x7f35cc0b5ea0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0c6af0 tx=0x7f35c8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:56.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.054+0000 7f35c27fc700 1 -- 172.21.15.156:0/932956699 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8004040 con 0x7f35cc0b29d0 2023-11-28T13:07:56.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.054+0000 7f35c27fc700 1 -- 172.21.15.156:0/932956699 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c80041a0 con 0x7f35cc0b29d0 2023-11-28T13:07:56.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.054+0000 7f35d3fff700 1 -- 172.21.15.156:0/932956699 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35cc0b44e0 con 0x7f35cc0b29d0 2023-11-28T13:07:56.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.054+0000 7f35c27fc700 1 -- 172.21.15.156:0/932956699 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8004550 con 0x7f35cc0b29d0 2023-11-28T13:07:56.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.054+0000 7f35d3fff700 1 -- 172.21.15.156:0/932956699 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35cc0b4200 con 0x7f35cc0b29d0 2023-11-28T13:07:56.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.055+0000 7f35d3fff700 1 -- 172.21.15.156:0/932956699 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35cc0b47c0 con 0x7f35cc0b29d0 2023-11-28T13:07:56.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.056+0000 7f35c27fc700 1 -- 172.21.15.156:0/932956699 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f35c800f050 con 0x7f35cc0b29d0 2023-11-28T13:07:56.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.057+0000 7f35c27fc700 1 --2- 172.21.15.156:0/932956699 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35b00682a0 0x7f35b006a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:56.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.057+0000 7f35c27fc700 1 -- 172.21.15.156:0/932956699 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4813+0+0 (secure 0 0 0) 0x7f35c80870a0 con 0x7f35cc0b29d0 2023-11-28T13:07:56.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.057+0000 7f35d159a700 1 --2- 172.21.15.156:0/932956699 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35b00682a0 0x7f35b006a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:56.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.058+0000 7f35d159a700 1 --2- 172.21.15.156:0/932956699 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35b00682a0 0x7f35b006a750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f35bc007fd0 tx=0x7f35bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:56.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.062+0000 7f35c27fc700 1 -- 172.21.15.156:0/932956699 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f35c8053e30 con 0x7f35cc0b29d0 2023-11-28T13:07:56.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.258+0000 7f35d3fff700 1 -- 172.21.15.156:0/932956699 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f35cc0672a0 con 0x7f35cc0b29d0 2023-11-28T13:07:56.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.839+0000 7f35c27fc700 1 -- 172.21.15.156:0/932956699 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f35c802a0c0 con 0x7f35cc0b29d0 2023-11-28T13:07:56.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.842+0000 7f35d3fff700 1 -- 172.21.15.156:0/932956699 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35b00682a0 msgr2=0x7f35b006a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:56.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.842+0000 7f35d3fff700 1 --2- 172.21.15.156:0/932956699 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35b00682a0 0x7f35b006a750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f35bc007fd0 tx=0x7f35bc009040 comp rx=0 tx=0).stop 2023-11-28T13:07:56.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.842+0000 7f35d3fff700 1 -- 172.21.15.156:0/932956699 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35cc0b29d0 msgr2=0x7f35cc0b5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:56.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.842+0000 7f35d3fff700 1 --2- 172.21.15.156:0/932956699 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35cc0b29d0 0x7f35cc0b5ea0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0c6af0 tx=0x7f35c8004750 comp rx=0 tx=0).stop 2023-11-28T13:07:56.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.842+0000 7f35d3fff700 1 -- 172.21.15.156:0/932956699 shutdown_connections 2023-11-28T13:07:56.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.842+0000 7f35d3fff700 1 --2- 172.21.15.156:0/932956699 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35b00682a0 0x7f35b006a750 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:56.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.842+0000 7f35d3fff700 1 --2- 172.21.15.156:0/932956699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35cc15ca60 0x7f35cc1f5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:56.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.842+0000 7f35d3fff700 1 --2- 172.21.15.156:0/932956699 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35cc0b29d0 0x7f35cc0b5ea0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:56.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.842+0000 7f35d3fff700 1 --2- 172.21.15.156:0/932956699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35cc0d1350 0x7f35cc0b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:56.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.842+0000 7f35d3fff700 1 -- 172.21.15.156:0/932956699 >> 172.21.15.156:0/932956699 conn(0x7f35cc09c3f0 msgr2=0x7f35cc0b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:56.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.842+0000 7f35d3fff700 1 -- 172.21.15.156:0/932956699 shutdown_connections 2023-11-28T13:07:56.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:56.842+0000 7f35d3fff700 1 -- 172.21.15.156:0/932956699 wait complete. 2023-11-28T13:07:56.844 INFO:teuthology.orchestra.run.smithi156.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-28T13:07:56.862 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-28T13:07:56.862 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T13:07:56.862 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T13:07:56.867 INFO:tasks.ceph.mds.a:Started 2023-11-28T13:07:56.867 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T13:07:56.867 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T13:07:56.871 INFO:tasks.ceph.mds.c:Started 2023-11-28T13:07:56.871 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T13:07:56.872 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T13:07:56.877 INFO:tasks.ceph.mds.b:Started 2023-11-28T13:07:56.877 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T13:07:56.877 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T13:07:56.881 INFO:tasks.ceph.mds.d:Started 2023-11-28T13:07:56.881 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-28T13:07:56.882 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:07:57.015 INFO:tasks.ceph.mds.a.smithi156.stdout:starting mds.a at 2023-11-28T13:07:57.023 INFO:tasks.ceph.mds.c.smithi156.stdout:starting mds.c at 2023-11-28T13:07:57.036 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2023-11-28T13:07:57.037 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2023-11-28T13:07:57.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.096+0000 7f24345ff700 1 Processor -- start 2023-11-28T13:07:57.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.096+0000 7f24345ff700 1 -- start start 2023-11-28T13:07:57.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.097+0000 7f24345ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f242c0d01b0 0x7f242c0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.097+0000 7f24345ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f242c0b20b0 0x7f242c0d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.097+0000 7f24345ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242c0b0230 con 0x7f242c0b29b0 2023-11-28T13:07:57.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.097+0000 7f24345ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242c15cec0 con 0x7f242c0b20b0 2023-11-28T13:07:57.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.097+0000 7f24345ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242c0b4570 con 0x7f242c0d01b0 2023-11-28T13:07:57.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.097+0000 7f2432b9c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f242c0b29b0 0x7f242c0b2e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:45582/0 (socket says 172.21.15.156:45582) 2023-11-28T13:07:57.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.098+0000 7f2432b9c700 1 -- 172.21.15.156:0/3811644893 learned_addr learned my addr 172.21.15.156:0/3811644893 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:07:57.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.098+0000 7f2431b9a700 1 --2- 172.21.15.156:0/3811644893 >> v2:172.21.15.186:3300/0 conn(0x7f242c0b20b0 0x7f242c0d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.098+0000 7f243239b700 1 --2- 172.21.15.156:0/3811644893 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f242c0d01b0 0x7f242c0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.098+0000 7f2431399700 1 -- 172.21.15.156:0/3811644893 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165296817 0 0) 0x7f242c0b0230 con 0x7f242c0b29b0 2023-11-28T13:07:57.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.098+0000 7f2431399700 1 -- 172.21.15.156:0/3811644893 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24180035b0 con 0x7f242c0b29b0 2023-11-28T13:07:57.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.098+0000 7f2431399700 1 -- 172.21.15.156:0/3811644893 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2147874721 0 0) 0x7f24180035b0 con 0x7f242c0b29b0 2023-11-28T13:07:57.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.098+0000 7f2431b9a700 1 -- 172.21.15.156:0/3811644893 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f242c0d01b0 msgr2=0x7f242c0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:57.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.098+0000 7f2431b9a700 1 --2- 172.21.15.156:0/3811644893 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f242c0d01b0 0x7f242c0d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.098+0000 7f2431b9a700 1 -- 172.21.15.156:0/3811644893 >> v1:172.21.15.156:6789/0 conn(0x7f242c0b29b0 legacy=0x7f242c0b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:57.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.098+0000 7f2431b9a700 1 -- 172.21.15.156:0/3811644893 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c0b4850 con 0x7f242c0b20b0 2023-11-28T13:07:57.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.098+0000 7f243239b700 1 --2- 172.21.15.156:0/3811644893 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f242c0d01b0 0x7f242c0d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:57.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.099+0000 7f2431b9a700 1 --2- 172.21.15.156:0/3811644893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f242c0b20b0 0x7f242c0d1160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f241c009840 tx=0x7f241c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dd7dc45b61deee9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:57.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.099+0000 7f2431399700 1 -- 172.21.15.156:0/3811644893 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24240019f0 con 0x7f242c0b20b0 2023-11-28T13:07:57.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.099+0000 7f2431399700 1 -- 172.21.15.156:0/3811644893 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c004030 con 0x7f242c0b20b0 2023-11-28T13:07:57.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.099+0000 7f2431399700 1 -- 172.21.15.156:0/3811644893 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c0043c0 con 0x7f242c0b20b0 2023-11-28T13:07:57.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.099+0000 7f24345ff700 1 -- 172.21.15.156:0/3811644893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f242c0b20b0 msgr2=0x7f242c0d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:57.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.099+0000 7f24345ff700 1 --2- 172.21.15.156:0/3811644893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f242c0b20b0 0x7f242c0d1160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f241c009840 tx=0x7f241c009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:57.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.099+0000 7f24345ff700 1 -- 172.21.15.156:0/3811644893 shutdown_connections 2023-11-28T13:07:57.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.099+0000 7f24345ff700 1 --2- 172.21.15.156:0/3811644893 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f242c0d01b0 0x7f242c0d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.099+0000 7f24345ff700 1 --2- 172.21.15.156:0/3811644893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f242c0b20b0 0x7f242c0d1160 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.099+0000 7f24345ff700 1 -- 172.21.15.156:0/3811644893 >> 172.21.15.156:0/3811644893 conn(0x7f242c09c3f0 msgr2=0x7f242c0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:57.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.099+0000 7f24345ff700 1 -- 172.21.15.156:0/3811644893 shutdown_connections 2023-11-28T13:07:57.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.100+0000 7f24345ff700 1 -- 172.21.15.156:0/3811644893 wait complete. 2023-11-28T13:07:57.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.102+0000 7f24345ff700 1 Processor -- start 2023-11-28T13:07:57.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.102+0000 7f24345ff700 1 -- start start 2023-11-28T13:07:57.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.103+0000 7f24345ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f242c0b20b0 0x7f242c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.103+0000 7f24345ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f242c0b29b0 0x7f242c1f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.103+0000 7f24345ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f242c0d01b0 0x7f242c155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.103+0000 7f24345ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c1f23c0 con 0x7f242c0d01b0 2023-11-28T13:07:57.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.103+0000 7f24345ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c1f3680 con 0x7f242c0b29b0 2023-11-28T13:07:57.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.103+0000 7f24345ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242c1f33a0 con 0x7f242c0b20b0 2023-11-28T13:07:57.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.103+0000 7f243239b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f242c0b20b0 0x7f242c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.103+0000 7f2431b9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f242c0b29b0 0x7f242c1f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.104+0000 7f243239b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f242c0b20b0 0x7f242c0b7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:52368/0 (socket says 172.21.15.156:52368) 2023-11-28T13:07:57.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.103+0000 7f2432b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f242c0d01b0 0x7f242c155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.104+0000 7f243239b700 1 -- 172.21.15.156:0/2580205864 learned_addr learned my addr 172.21.15.156:0/2580205864 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:57.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.104+0000 7f2431b9a700 1 -- 172.21.15.156:0/2580205864 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f242c0b20b0 msgr2=0x7f242c0b7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:57.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.104+0000 7f2431b9a700 1 --2- 172.21.15.156:0/2580205864 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f242c0b20b0 0x7f242c0b7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.104+0000 7f2431b9a700 1 -- 172.21.15.156:0/2580205864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f242c0d01b0 msgr2=0x7f242c155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:57.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.104+0000 7f2431b9a700 1 --2- 172.21.15.156:0/2580205864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f242c0d01b0 0x7f242c155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.104+0000 7f2431b9a700 1 -- 172.21.15.156:0/2580205864 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24280094a0 con 0x7f242c0b29b0 2023-11-28T13:07:57.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.104+0000 7f243239b700 1 --2- 172.21.15.156:0/2580205864 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f242c0b20b0 0x7f242c0b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:57.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.104+0000 7f2432b9c700 1 --2- 172.21.15.156:0/2580205864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f242c0d01b0 0x7f242c155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:57.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.104+0000 7f2431b9a700 1 --2- 172.21.15.156:0/2580205864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f242c0b29b0 0x7f242c1f1e80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f241c015000 tx=0x7f241c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:57.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.104+0000 7f2422ffd700 1 -- 172.21.15.156:0/2580205864 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c003cf0 con 0x7f242c0b29b0 2023-11-28T13:07:57.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.104+0000 7f2422ffd700 1 -- 172.21.15.156:0/2580205864 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c01f070 con 0x7f242c0b29b0 2023-11-28T13:07:57.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.104+0000 7f2422ffd700 1 -- 172.21.15.156:0/2580205864 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c011870 con 0x7f242c0b29b0 2023-11-28T13:07:57.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.104+0000 7f24345ff700 1 -- 172.21.15.156:0/2580205864 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241c0094a0 con 0x7f242c0b29b0 2023-11-28T13:07:57.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.105+0000 7f24345ff700 1 -- 172.21.15.156:0/2580205864 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242c0afec0 con 0x7f242c0b29b0 2023-11-28T13:07:57.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.105+0000 7f24345ff700 1 -- 172.21.15.156:0/2580205864 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242c0afbe0 con 0x7f242c0b29b0 2023-11-28T13:07:57.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.106+0000 7f2422ffd700 1 -- 172.21.15.156:0/2580205864 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f241c017020 con 0x7f242c0b29b0 2023-11-28T13:07:57.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.107+0000 7f2422ffd700 1 --2- 172.21.15.156:0/2580205864 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f24100682a0 0x7f241006a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.107+0000 7f2422ffd700 1 -- 172.21.15.156:0/2580205864 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f241c021070 con 0x7f242c0b29b0 2023-11-28T13:07:57.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.107+0000 7f243239b700 1 --2- 172.21.15.156:0/2580205864 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f24100682a0 0x7f241006a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.108+0000 7f243239b700 1 --2- 172.21.15.156:0/2580205864 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f24100682a0 0x7f241006a750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2428009fd0 tx=0x7f2428009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:57.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.112+0000 7f2422ffd700 1 -- 172.21.15.156:0/2580205864 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f241c053ba0 con 0x7f242c0b29b0 2023-11-28T13:07:57.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.309+0000 7f24345ff700 1 -- 172.21.15.156:0/2580205864 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f242c0b2600 con 0x7f242c0b29b0 2023-11-28T13:07:57.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.310+0000 7f2422ffd700 1 -- 172.21.15.156:0/2580205864 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+14273 (secure 0 0 0) 0x7f241c02b090 con 0x7f242c0b29b0 2023-11-28T13:07:57.311 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:57.311 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":15,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:07:56.825711+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:07:57.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.313+0000 7f24345ff700 1 -- 172.21.15.156:0/2580205864 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f24100682a0 msgr2=0x7f241006a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:57.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.313+0000 7f24345ff700 1 --2- 172.21.15.156:0/2580205864 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f24100682a0 0x7f241006a750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2428009fd0 tx=0x7f2428009210 comp rx=0 tx=0).stop 2023-11-28T13:07:57.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.313+0000 7f24345ff700 1 -- 172.21.15.156:0/2580205864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f242c0b29b0 msgr2=0x7f242c1f1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:57.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.313+0000 7f24345ff700 1 --2- 172.21.15.156:0/2580205864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f242c0b29b0 0x7f242c1f1e80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f241c015000 tx=0x7f241c004750 comp rx=0 tx=0).stop 2023-11-28T13:07:57.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.313+0000 7f24345ff700 1 -- 172.21.15.156:0/2580205864 shutdown_connections 2023-11-28T13:07:57.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.313+0000 7f24345ff700 1 --2- 172.21.15.156:0/2580205864 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f24100682a0 0x7f241006a750 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.313+0000 7f24345ff700 1 --2- 172.21.15.156:0/2580205864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f242c0d01b0 0x7f242c155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.313+0000 7f24345ff700 1 --2- 172.21.15.156:0/2580205864 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f242c0b20b0 0x7f242c0b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.313+0000 7f24345ff700 1 --2- 172.21.15.156:0/2580205864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f242c0b29b0 0x7f242c1f1e80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.313+0000 7f24345ff700 1 -- 172.21.15.156:0/2580205864 >> 172.21.15.156:0/2580205864 conn(0x7f242c09c3f0 msgr2=0x7f242c0cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:57.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.313+0000 7f24345ff700 1 -- 172.21.15.156:0/2580205864 shutdown_connections 2023-11-28T13:07:57.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.313+0000 7f24345ff700 1 -- 172.21.15.156:0/2580205864 wait complete. 2023-11-28T13:07:57.331 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:07:57.332 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:07:57.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.531+0000 7ff8d6bff700 1 Processor -- start 2023-11-28T13:07:57.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.532+0000 7ff8d6bff700 1 -- start start 2023-11-28T13:07:57.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.533+0000 7ff8d6bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8d015c9b0 0x7ff8d00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.533+0000 7ff8d6bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff8d00cd7c0 0x7ff8d00cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.533+0000 7ff8d6bff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d00ce160 con 0x7ff8d0161340 2023-11-28T13:07:57.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.533+0000 7ff8d6bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d00b5800 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.533+0000 7ff8d6bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d00d1950 con 0x7ff8d015c9b0 2023-11-28T13:07:57.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.533+0000 7ff8d519c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff8d00cd7c0 0x7ff8d00cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.533+0000 7ff8d519c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff8d00cd7c0 0x7ff8d00cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43418/0 (socket says 172.21.15.156:43418) 2023-11-28T13:07:57.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.533+0000 7ff8d519c700 1 -- 172.21.15.156:0/3574357388 learned_addr learned my addr 172.21.15.156:0/3574357388 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:57.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.533+0000 7ff8cffff700 1 --1- 172.21.15.156:0/3574357388 >> v1:172.21.15.156:6789/0 conn(0x7ff8d0161340 0x7ff8d00d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:45592/0 (socket says 172.21.15.156:45592) 2023-11-28T13:07:57.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.533+0000 7ff8d499b700 1 --2- 172.21.15.156:0/3574357388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8d015c9b0 0x7ff8d00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.534+0000 7ff8d519c700 1 -- 172.21.15.156:0/3574357388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8d015c9b0 msgr2=0x7ff8d00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:57.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.534+0000 7ff8d519c700 1 --2- 172.21.15.156:0/3574357388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8d015c9b0 0x7ff8d00b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.534+0000 7ff8d519c700 1 -- 172.21.15.156:0/3574357388 >> v1:172.21.15.156:6789/0 conn(0x7ff8d0161340 legacy=0x7ff8d00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:57.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.534+0000 7ff8d519c700 1 -- 172.21.15.156:0/3574357388 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d00d24f0 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.534+0000 7ff8cf7fe700 1 -- 172.21.15.156:0/3574357388 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158001250 0 0) 0x7ff8d00ce160 con 0x7ff8d0161340 2023-11-28T13:07:57.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.534+0000 7ff8d499b700 1 --2- 172.21.15.156:0/3574357388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8d015c9b0 0x7ff8d00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:57.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.534+0000 7ff8d519c700 1 --2- 172.21.15.156:0/3574357388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8d00cd7c0 0x7ff8d00cdc20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009e50 tx=0x7ff8c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363270627a179edd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:57.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.534+0000 7ff8cf7fe700 1 -- 172.21.15.156:0/3574357388 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c800e070 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.534+0000 7ff8cf7fe700 1 -- 172.21.15.156:0/3574357388 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c800b9c0 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.534+0000 7ff8cf7fe700 1 -- 172.21.15.156:0/3574357388 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c800bd70 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.535+0000 7ff8d6bff700 1 -- 172.21.15.156:0/3574357388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8d00cd7c0 msgr2=0x7ff8d00cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:57.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.535+0000 7ff8d6bff700 1 --2- 172.21.15.156:0/3574357388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8d00cd7c0 0x7ff8d00cdc20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009e50 tx=0x7ff8c800b330 comp rx=0 tx=0).stop 2023-11-28T13:07:57.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.535+0000 7ff8d6bff700 1 -- 172.21.15.156:0/3574357388 shutdown_connections 2023-11-28T13:07:57.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.535+0000 7ff8d6bff700 1 --2- 172.21.15.156:0/3574357388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8d00cd7c0 0x7ff8d00cdc20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.535+0000 7ff8d6bff700 1 --2- 172.21.15.156:0/3574357388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8d015c9b0 0x7ff8d00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.535+0000 7ff8d6bff700 1 -- 172.21.15.156:0/3574357388 >> 172.21.15.156:0/3574357388 conn(0x7ff8d0072600 msgr2=0x7ff8d00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:57.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.535+0000 7ff8d6bff700 1 -- 172.21.15.156:0/3574357388 shutdown_connections 2023-11-28T13:07:57.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.535+0000 7ff8d6bff700 1 -- 172.21.15.156:0/3574357388 wait complete. 2023-11-28T13:07:57.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.536+0000 7ff8d6bff700 1 Processor -- start 2023-11-28T13:07:57.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.537+0000 7ff8d6bff700 1 -- start start 2023-11-28T13:07:57.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.537+0000 7ff8d6bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8d00cd7c0 0x7ff8d00c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8d6bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8d015c9b0 0x7ff8d00b8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8d6bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8d0161340 0x7ff8d00b8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8d6bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d01f72b0 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8d6bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d01f62c0 con 0x7ff8d0161340 2023-11-28T13:07:57.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8d6bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d01f6510 con 0x7ff8d015c9b0 2023-11-28T13:07:57.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8d499b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8d00cd7c0 0x7ff8d00c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8d499b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8d00cd7c0 0x7ff8d00c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43424/0 (socket says 172.21.15.156:43424) 2023-11-28T13:07:57.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8d499b700 1 -- 172.21.15.156:0/3726593254 learned_addr learned my addr 172.21.15.156:0/3726593254 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:57.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8cffff700 1 --2- 172.21.15.156:0/3726593254 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8d015c9b0 0x7ff8d00b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8d499b700 1 -- 172.21.15.156:0/3726593254 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8d015c9b0 msgr2=0x7ff8d00b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:57.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8d519c700 1 --2- 172.21.15.156:0/3726593254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8d0161340 0x7ff8d00b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8d499b700 1 --2- 172.21.15.156:0/3726593254 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8d015c9b0 0x7ff8d00b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8d499b700 1 -- 172.21.15.156:0/3726593254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8d0161340 msgr2=0x7ff8d00b8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:57.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8d499b700 1 --2- 172.21.15.156:0/3726593254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8d0161340 0x7ff8d00b8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8d499b700 1 -- 172.21.15.156:0/3726593254 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c8009b30 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8d519c700 1 --2- 172.21.15.156:0/3726593254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8d0161340 0x7ff8d00b8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:57.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8cffff700 1 --2- 172.21.15.156:0/3726593254 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8d015c9b0 0x7ff8d00b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:57.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.538+0000 7ff8d499b700 1 --2- 172.21.15.156:0/3726593254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8d00cd7c0 0x7ff8d00c9bc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff8c400e790 tx=0x7ff8c400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:57.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.539+0000 7ff8cd7fa700 1 -- 172.21.15.156:0/3726593254 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4013610 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.539+0000 7ff8cd7fa700 1 -- 172.21.15.156:0/3726593254 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c4013770 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.539+0000 7ff8d6bff700 1 -- 172.21.15.156:0/3726593254 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d00d1300 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.539+0000 7ff8cd7fa700 1 -- 172.21.15.156:0/3726593254 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4013b20 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.539+0000 7ff8d6bff700 1 -- 172.21.15.156:0/3726593254 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d00b5520 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.540+0000 7ff8d6bff700 1 -- 172.21.15.156:0/3726593254 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d00d15e0 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.541+0000 7ff8cd7fa700 1 -- 172.21.15.156:0/3726593254 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7ff8c4023070 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.542+0000 7ff8cd7fa700 1 --2- 172.21.15.156:0/3726593254 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff8b40682d0 0x7ff8b406a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.542+0000 7ff8cd7fa700 1 -- 172.21.15.156:0/3726593254 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5199+0+0 (secure 0 0 0) 0x7ff8c4085540 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.542+0000 7ff8cffff700 1 --2- 172.21.15.156:0/3726593254 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff8b40682d0 0x7ff8b406a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.543+0000 7ff8cffff700 1 --2- 172.21.15.156:0/3726593254 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff8b40682d0 0x7ff8b406a780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc007fd0 tx=0x7ff8bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:57.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.546+0000 7ff8cd7fa700 1 -- 172.21.15.156:0/3726593254 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff8c4052060 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.739+0000 7ff8d6bff700 1 -- 172.21.15.156:0/3726593254 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff8d01f7120 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.739+0000 7ff8cd7fa700 1 -- 172.21.15.156:0/3726593254 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff8c4055680 con 0x7ff8d00cd7c0 2023-11-28T13:07:57.740 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:07:57.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.741+0000 7ff8d6bff700 1 -- 172.21.15.156:0/3726593254 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff8b40682d0 msgr2=0x7ff8b406a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:57.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.741+0000 7ff8d6bff700 1 --2- 172.21.15.156:0/3726593254 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff8b40682d0 0x7ff8b406a780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc007fd0 tx=0x7ff8bc009040 comp rx=0 tx=0).stop 2023-11-28T13:07:57.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.741+0000 7ff8d6bff700 1 -- 172.21.15.156:0/3726593254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8d00cd7c0 msgr2=0x7ff8d00c9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:57.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.741+0000 7ff8d6bff700 1 --2- 172.21.15.156:0/3726593254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8d00cd7c0 0x7ff8d00c9bc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff8c400e790 tx=0x7ff8c400eb50 comp rx=0 tx=0).stop 2023-11-28T13:07:57.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.741+0000 7ff8d6bff700 1 -- 172.21.15.156:0/3726593254 shutdown_connections 2023-11-28T13:07:57.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.741+0000 7ff8d6bff700 1 --2- 172.21.15.156:0/3726593254 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff8b40682d0 0x7ff8b406a780 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.741+0000 7ff8d6bff700 1 --2- 172.21.15.156:0/3726593254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8d0161340 0x7ff8d00b8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.741+0000 7ff8d6bff700 1 --2- 172.21.15.156:0/3726593254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8d00cd7c0 0x7ff8d00c9bc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.741+0000 7ff8d6bff700 1 --2- 172.21.15.156:0/3726593254 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8d015c9b0 0x7ff8d00b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.741+0000 7ff8d6bff700 1 -- 172.21.15.156:0/3726593254 >> 172.21.15.156:0/3726593254 conn(0x7ff8d0072600 msgr2=0x7ff8d00d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:57.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.741+0000 7ff8d6bff700 1 -- 172.21.15.156:0/3726593254 shutdown_connections 2023-11-28T13:07:57.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.742+0000 7ff8d6bff700 1 -- 172.21.15.156:0/3726593254 wait complete. 2023-11-28T13:07:57.760 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:07:57.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.969+0000 7f57919ff700 1 Processor -- start 2023-11-28T13:07:57.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.969+0000 7f57919ff700 1 -- start start 2023-11-28T13:07:57.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.970+0000 7f57919ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f578c15c9b0 0x7f578c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.970+0000 7f57919ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f578c161340 0x7f578c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.970+0000 7f57919ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f578c0d1950 con 0x7f578c0d2320 2023-11-28T13:07:57.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.970+0000 7f57919ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f578c0b5800 con 0x7f578c161340 2023-11-28T13:07:57.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.970+0000 7f57919ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f578c0b5520 con 0x7f578c15c9b0 2023-11-28T13:07:57.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.970+0000 7f578a7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f578c161340 0x7f578c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.970+0000 7f578a7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f578c161340 0x7f578c0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43426/0 (socket says 172.21.15.156:43426) 2023-11-28T13:07:57.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.970+0000 7f578b7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f578c0d2320 0x7f578c0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:45598/0 (socket says 172.21.15.156:45598) 2023-11-28T13:07:57.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.970+0000 7f578a7fc700 1 -- 172.21.15.156:0/184652696 learned_addr learned my addr 172.21.15.156:0/184652696 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:57.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.971+0000 7f578affd700 1 --2- 172.21.15.156:0/184652696 >> v2:172.21.15.186:3300/0 conn(0x7f578c15c9b0 0x7f578c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.971+0000 7f578a7fc700 1 -- 172.21.15.156:0/184652696 >> v2:172.21.15.186:3300/0 conn(0x7f578c15c9b0 msgr2=0x7f578c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:57.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.971+0000 7f578a7fc700 1 --2- 172.21.15.156:0/184652696 >> v2:172.21.15.186:3300/0 conn(0x7f578c15c9b0 0x7f578c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.971+0000 7f578a7fc700 1 -- 172.21.15.156:0/184652696 >> v1:172.21.15.156:6789/0 conn(0x7f578c0d2320 legacy=0x7f578c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:57.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.971+0000 7f578a7fc700 1 -- 172.21.15.156:0/184652696 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c0d1300 con 0x7f578c161340 2023-11-28T13:07:57.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.971+0000 7f5789ffb700 1 -- 172.21.15.156:0/184652696 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515718476 0 0) 0x7f578c0d1950 con 0x7f578c0d2320 2023-11-28T13:07:57.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.971+0000 7f578a7fc700 1 --2- 172.21.15.156:0/184652696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f578c161340 0x7f578c0d1de0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f578c067460 tx=0x7f5780009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7745052ecbd708b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:57.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.971+0000 7f5789ffb700 1 -- 172.21.15.156:0/184652696 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578000b870 con 0x7f578c161340 2023-11-28T13:07:57.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.971+0000 7f5789ffb700 1 -- 172.21.15.156:0/184652696 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578000b9d0 con 0x7f578c161340 2023-11-28T13:07:57.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.971+0000 7f5789ffb700 1 -- 172.21.15.156:0/184652696 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578000bd60 con 0x7f578c161340 2023-11-28T13:07:57.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.972+0000 7f57919ff700 1 -- 172.21.15.156:0/184652696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f578c161340 msgr2=0x7f578c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:57.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.972+0000 7f57919ff700 1 --2- 172.21.15.156:0/184652696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f578c161340 0x7f578c0d1de0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f578c067460 tx=0x7f5780009bd0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.972+0000 7f57919ff700 1 -- 172.21.15.156:0/184652696 shutdown_connections 2023-11-28T13:07:57.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.972+0000 7f57919ff700 1 --2- 172.21.15.156:0/184652696 >> v2:172.21.15.186:3300/0 conn(0x7f578c15c9b0 0x7f578c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.972+0000 7f57919ff700 1 --2- 172.21.15.156:0/184652696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f578c161340 0x7f578c0d1de0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.972+0000 7f57919ff700 1 -- 172.21.15.156:0/184652696 >> 172.21.15.156:0/184652696 conn(0x7f578c072600 msgr2=0x7f578c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:57.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.972+0000 7f57919ff700 1 -- 172.21.15.156:0/184652696 shutdown_connections 2023-11-28T13:07:57.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.972+0000 7f57919ff700 1 -- 172.21.15.156:0/184652696 wait complete. 2023-11-28T13:07:57.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.974+0000 7f57919ff700 1 Processor -- start 2023-11-28T13:07:57.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.975+0000 7f57919ff700 1 -- start start 2023-11-28T13:07:57.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.975+0000 7f57919ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f578c0d2320 0x7f578c152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.975+0000 7f57919ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f578c15c9b0 0x7f578c1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.975+0000 7f57919ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f578c161340 0x7f578c157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.975+0000 7f57919ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c1571f0 con 0x7f578c161340 2023-11-28T13:07:57.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.975+0000 7f57919ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c0c6780 con 0x7f578c0d2320 2023-11-28T13:07:57.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.975+0000 7f57919ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578c152e70 con 0x7f578c15c9b0 2023-11-28T13:07:57.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.976+0000 7f578b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f578c161340 0x7f578c157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.976+0000 7f578b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f578c161340 0x7f578c157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43430/0 (socket says 172.21.15.156:43430) 2023-11-28T13:07:57.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.976+0000 7f578b7fe700 1 -- 172.21.15.156:0/579579618 learned_addr learned my addr 172.21.15.156:0/579579618 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:57.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.976+0000 7f578a7fc700 1 --2- 172.21.15.156:0/579579618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f578c15c9b0 0x7f578c1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.976+0000 7f578affd700 1 --2- 172.21.15.156:0/579579618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f578c0d2320 0x7f578c152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.976+0000 7f578b7fe700 1 -- 172.21.15.156:0/579579618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f578c15c9b0 msgr2=0x7f578c1542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:57.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.976+0000 7f578b7fe700 1 --2- 172.21.15.156:0/579579618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f578c15c9b0 0x7f578c1542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.976+0000 7f578b7fe700 1 -- 172.21.15.156:0/579579618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f578c0d2320 msgr2=0x7f578c152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:57.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.976+0000 7f578b7fe700 1 --2- 172.21.15.156:0/579579618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f578c0d2320 0x7f578c152930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:57.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.976+0000 7f578b7fe700 1 -- 172.21.15.156:0/579579618 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5780009550 con 0x7f578c161340 2023-11-28T13:07:57.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.976+0000 7f578b7fe700 1 --2- 172.21.15.156:0/579579618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f578c161340 0x7f578c157610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f578400cb30 tx=0x7f578400ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:57.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.976+0000 7f5773fff700 1 -- 172.21.15.156:0/579579618 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578400a9e0 con 0x7f578c161340 2023-11-28T13:07:57.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.976+0000 7f5773fff700 1 -- 172.21.15.156:0/579579618 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578400ab80 con 0x7f578c161340 2023-11-28T13:07:57.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.977+0000 7f57919ff700 1 -- 172.21.15.156:0/579579618 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578c0b5130 con 0x7f578c161340 2023-11-28T13:07:57.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.977+0000 7f5773fff700 1 -- 172.21.15.156:0/579579618 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57840070d0 con 0x7f578c161340 2023-11-28T13:07:57.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.977+0000 7f57919ff700 1 -- 172.21.15.156:0/579579618 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578c0d15e0 con 0x7f578c161340 2023-11-28T13:07:57.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.978+0000 7f57919ff700 1 -- 172.21.15.156:0/579579618 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f578c0b52c0 con 0x7f578c161340 2023-11-28T13:07:57.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.979+0000 7f5773fff700 1 -- 172.21.15.156:0/579579618 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f5784010070 con 0x7f578c161340 2023-11-28T13:07:57.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.980+0000 7f5773fff700 1 --2- 172.21.15.156:0/579579618 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57780682a0 0x7f577806a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:57.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.980+0000 7f5773fff700 1 -- 172.21.15.156:0/579579618 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f57840851d0 con 0x7f578c161340 2023-11-28T13:07:57.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.980+0000 7f578affd700 1 --2- 172.21.15.156:0/579579618 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57780682a0 0x7f577806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:57.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.980+0000 7f578affd700 1 --2- 172.21.15.156:0/579579618 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57780682a0 0x7f577806a750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f577c006b20 tx=0x7f577c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:57.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:57.984+0000 7f5773fff700 1 -- 172.21.15.156:0/579579618 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5784051dd0 con 0x7f578c161340 2023-11-28T13:07:58.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.197+0000 7f57919ff700 1 -- 172.21.15.156:0/579579618 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f578c157330 con 0x7f578c161340 2023-11-28T13:07:58.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.198+0000 7f5773fff700 1 -- 172.21.15.156:0/579579618 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5784018020 con 0x7f578c161340 2023-11-28T13:07:58.199 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:07:58.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.200+0000 7f57919ff700 1 -- 172.21.15.156:0/579579618 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57780682a0 msgr2=0x7f577806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:58.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.200+0000 7f57919ff700 1 --2- 172.21.15.156:0/579579618 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57780682a0 0x7f577806a750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f577c006b20 tx=0x7f577c008040 comp rx=0 tx=0).stop 2023-11-28T13:07:58.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.200+0000 7f57919ff700 1 -- 172.21.15.156:0/579579618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f578c161340 msgr2=0x7f578c157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:58.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.200+0000 7f57919ff700 1 --2- 172.21.15.156:0/579579618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f578c161340 0x7f578c157610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f578400cb30 tx=0x7f578400ce40 comp rx=0 tx=0).stop 2023-11-28T13:07:58.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.201+0000 7f57919ff700 1 -- 172.21.15.156:0/579579618 shutdown_connections 2023-11-28T13:07:58.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.201+0000 7f57919ff700 1 --2- 172.21.15.156:0/579579618 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57780682a0 0x7f577806a750 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.201+0000 7f57919ff700 1 --2- 172.21.15.156:0/579579618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f578c161340 0x7f578c157610 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.201+0000 7f57919ff700 1 --2- 172.21.15.156:0/579579618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f578c0d2320 0x7f578c152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.201+0000 7f57919ff700 1 --2- 172.21.15.156:0/579579618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f578c15c9b0 0x7f578c1542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.201+0000 7f57919ff700 1 -- 172.21.15.156:0/579579618 >> 172.21.15.156:0/579579618 conn(0x7f578c072600 msgr2=0x7f578c15fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:58.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.201+0000 7f57919ff700 1 -- 172.21.15.156:0/579579618 shutdown_connections 2023-11-28T13:07:58.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.201+0000 7f57919ff700 1 -- 172.21.15.156:0/579579618 wait complete. 2023-11-28T13:07:58.219 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:07:58.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.429+0000 7fae8b5ff700 1 Processor -- start 2023-11-28T13:07:58.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.429+0000 7fae8b5ff700 1 -- start start 2023-11-28T13:07:58.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.430+0000 7fae8b5ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fae840d01b0 0x7fae840d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:58.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.430+0000 7fae8b5ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fae840b16d0 0x7fae840d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:58.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.430+0000 7fae8b5ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae840b0230 con 0x7fae840b20b0 2023-11-28T13:07:58.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.430+0000 7fae8b5ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae8415cec0 con 0x7fae840d01b0 2023-11-28T13:07:58.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.430+0000 7fae8b5ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae840b4570 con 0x7fae840b16d0 2023-11-28T13:07:58.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.430+0000 7fae8939b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fae840d01b0 0x7fae840d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:58.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.430+0000 7fae8939b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fae840d01b0 0x7fae840d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43446/0 (socket says 172.21.15.156:43446) 2023-11-28T13:07:58.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.430+0000 7fae8939b700 1 -- 172.21.15.156:0/1073170202 learned_addr learned my addr 172.21.15.156:0/1073170202 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:58.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.430+0000 7fae89b9c700 1 --2- 172.21.15.156:0/1073170202 >> v2:172.21.15.186:3300/0 conn(0x7fae840b16d0 0x7fae840d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:58.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.431+0000 7fae8939b700 1 -- 172.21.15.156:0/1073170202 >> v2:172.21.15.186:3300/0 conn(0x7fae840b16d0 msgr2=0x7fae840d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:58.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.431+0000 7fae8939b700 1 --2- 172.21.15.156:0/1073170202 >> v2:172.21.15.186:3300/0 conn(0x7fae840b16d0 0x7fae840d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.431+0000 7fae8939b700 1 -- 172.21.15.156:0/1073170202 >> v1:172.21.15.186:6789/0 conn(0x7fae840b20b0 legacy=0x7fae840b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:58.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.431+0000 7fae8939b700 1 -- 172.21.15.156:0/1073170202 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae840b4850 con 0x7fae840d01b0 2023-11-28T13:07:58.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.431+0000 7fae7bfff700 1 -- 172.21.15.156:0/1073170202 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4051556313 0 0) 0x7fae840b0230 con 0x7fae840b20b0 2023-11-28T13:07:58.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.431+0000 7fae8939b700 1 --2- 172.21.15.156:0/1073170202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae840d01b0 0x7fae840d0c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fae840cd630 tx=0x7fae80009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da304f8c4aaf5469 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:58.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.431+0000 7fae7bfff700 1 -- 172.21.15.156:0/1073170202 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80011830 con 0x7fae840d01b0 2023-11-28T13:07:58.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.431+0000 7fae7bfff700 1 -- 172.21.15.156:0/1073170202 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae80011990 con 0x7fae840d01b0 2023-11-28T13:07:58.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.431+0000 7fae7bfff700 1 -- 172.21.15.156:0/1073170202 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80011d20 con 0x7fae840d01b0 2023-11-28T13:07:58.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.432+0000 7fae8b5ff700 1 -- 172.21.15.156:0/1073170202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae840d01b0 msgr2=0x7fae840d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:58.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.432+0000 7fae8b5ff700 1 --2- 172.21.15.156:0/1073170202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae840d01b0 0x7fae840d0c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fae840cd630 tx=0x7fae80009bd0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.432+0000 7fae8b5ff700 1 -- 172.21.15.156:0/1073170202 shutdown_connections 2023-11-28T13:07:58.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.432+0000 7fae8b5ff700 1 --2- 172.21.15.156:0/1073170202 >> v2:172.21.15.186:3300/0 conn(0x7fae840b16d0 0x7fae840d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.432+0000 7fae8b5ff700 1 --2- 172.21.15.156:0/1073170202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae840d01b0 0x7fae840d0c20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.432+0000 7fae8b5ff700 1 -- 172.21.15.156:0/1073170202 >> 172.21.15.156:0/1073170202 conn(0x7fae8409c3f0 msgr2=0x7fae840d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:58.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.432+0000 7fae8b5ff700 1 -- 172.21.15.156:0/1073170202 shutdown_connections 2023-11-28T13:07:58.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.432+0000 7fae8b5ff700 1 -- 172.21.15.156:0/1073170202 wait complete. 2023-11-28T13:07:58.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.434+0000 7fae8b5ff700 1 Processor -- start 2023-11-28T13:07:58.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.434+0000 7fae8b5ff700 1 -- start start 2023-11-28T13:07:58.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.435+0000 7fae8b5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae840b16d0 0x7fae8415a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:58.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.435+0000 7fae8b5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae840b20b0 0x7fae840ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:58.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.435+0000 7fae8b5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae840d01b0 0x7fae841e4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:58.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.435+0000 7fae8b5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae840b5f20 con 0x7fae840d01b0 2023-11-28T13:07:58.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.435+0000 7fae8b5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae840b6200 con 0x7fae840b20b0 2023-11-28T13:07:58.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.435+0000 7fae8b5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae840b64e0 con 0x7fae840b16d0 2023-11-28T13:07:58.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.435+0000 7fae88b9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae840b20b0 0x7fae840ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:58.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.435+0000 7fae89b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae840d01b0 0x7fae841e4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:58.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.435+0000 7fae8939b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae840b16d0 0x7fae8415a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:58.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.435+0000 7fae89b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae840d01b0 0x7fae841e4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43460/0 (socket says 172.21.15.156:43460) 2023-11-28T13:07:58.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.435+0000 7fae89b9c700 1 -- 172.21.15.156:0/1018534928 learned_addr learned my addr 172.21.15.156:0/1018534928 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:58.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.435+0000 7fae89b9c700 1 -- 172.21.15.156:0/1018534928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae840b16d0 msgr2=0x7fae8415a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:58.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.435+0000 7fae89b9c700 1 --2- 172.21.15.156:0/1018534928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae840b16d0 0x7fae8415a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.435+0000 7fae89b9c700 1 -- 172.21.15.156:0/1018534928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae840b20b0 msgr2=0x7fae840ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:58.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.435+0000 7fae89b9c700 1 --2- 172.21.15.156:0/1018534928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae840b20b0 0x7fae840ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.435+0000 7fae89b9c700 1 -- 172.21.15.156:0/1018534928 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae80009550 con 0x7fae840d01b0 2023-11-28T13:07:58.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.435+0000 7fae88b9a700 1 --2- 172.21.15.156:0/1018534928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae840b20b0 0x7fae840ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:58.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.436+0000 7fae89b9c700 1 --2- 172.21.15.156:0/1018534928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae840d01b0 0x7fae841e4a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00efc0 tx=0x7fae7c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:58.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.436+0000 7fae79ffb700 1 -- 172.21.15.156:0/1018534928 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c009ea0 con 0x7fae840d01b0 2023-11-28T13:07:58.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.436+0000 7fae79ffb700 1 -- 172.21.15.156:0/1018534928 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7c00f040 con 0x7fae840d01b0 2023-11-28T13:07:58.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.436+0000 7fae8b5ff700 1 -- 172.21.15.156:0/1018534928 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae840afec0 con 0x7fae840d01b0 2023-11-28T13:07:58.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.436+0000 7fae79ffb700 1 -- 172.21.15.156:0/1018534928 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c00ced0 con 0x7fae840d01b0 2023-11-28T13:07:58.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.436+0000 7fae8b5ff700 1 -- 172.21.15.156:0/1018534928 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae840afbe0 con 0x7fae840d01b0 2023-11-28T13:07:58.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.437+0000 7fae8b5ff700 1 -- 172.21.15.156:0/1018534928 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae840b2600 con 0x7fae840d01b0 2023-11-28T13:07:58.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.439+0000 7fae79ffb700 1 -- 172.21.15.156:0/1018534928 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fae7c014480 con 0x7fae840d01b0 2023-11-28T13:07:58.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.439+0000 7fae79ffb700 1 --2- 172.21.15.156:0/1018534928 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fae680682a0 0x7fae6806a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:58.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.439+0000 7fae79ffb700 1 -- 172.21.15.156:0/1018534928 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fae7c086a40 con 0x7fae840d01b0 2023-11-28T13:07:58.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.440+0000 7fae8939b700 1 --2- 172.21.15.156:0/1018534928 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fae680682a0 0x7fae6806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:58.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.441+0000 7fae8939b700 1 --2- 172.21.15.156:0/1018534928 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fae680682a0 0x7fae6806a750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fae80014040 tx=0x7fae80008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:58.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.442+0000 7fae79ffb700 1 -- 172.21.15.156:0/1018534928 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fae7c0535e0 con 0x7fae840d01b0 2023-11-28T13:07:58.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.697+0000 7fae8b5ff700 1 -- 172.21.15.156:0/1018534928 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae840b6060 con 0x7fae840d01b0 2023-11-28T13:07:58.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.697+0000 7fae79ffb700 1 -- 172.21.15.156:0/1018534928 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3533 (secure 0 0 0) 0x7fae840b6060 con 0x7fae840d01b0 2023-11-28T13:07:58.698 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:58.698 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1283906891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1283906891},{"type":"v1","addr":"172.21.15.156:6835","nonce":1283906891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1738881422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1738881422},{"type":"v1","addr":"172.21.15.156:6833","nonce":1738881422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2235657629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2235657629},{"type":"v1","addr":"172.21.15.186:6836","nonce":2235657629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/402651964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":402651964},{"type":"v1","addr":"172.21.15.186:6837","nonce":402651964}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-28T13:07:58.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.699+0000 7fae8b5ff700 1 -- 172.21.15.156:0/1018534928 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fae680682a0 msgr2=0x7fae6806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:58.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.700+0000 7fae8b5ff700 1 --2- 172.21.15.156:0/1018534928 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fae680682a0 0x7fae6806a750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fae80014040 tx=0x7fae80008030 comp rx=0 tx=0).stop 2023-11-28T13:07:58.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.700+0000 7fae8b5ff700 1 -- 172.21.15.156:0/1018534928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae840d01b0 msgr2=0x7fae841e4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:58.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.700+0000 7fae8b5ff700 1 --2- 172.21.15.156:0/1018534928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae840d01b0 0x7fae841e4a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00efc0 tx=0x7fae7c00c3a0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.700+0000 7fae8b5ff700 1 -- 172.21.15.156:0/1018534928 shutdown_connections 2023-11-28T13:07:58.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.700+0000 7fae8b5ff700 1 --2- 172.21.15.156:0/1018534928 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fae680682a0 0x7fae6806a750 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.700+0000 7fae8b5ff700 1 --2- 172.21.15.156:0/1018534928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae840d01b0 0x7fae841e4a70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.700+0000 7fae8b5ff700 1 --2- 172.21.15.156:0/1018534928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fae840b16d0 0x7fae8415a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.700+0000 7fae8b5ff700 1 --2- 172.21.15.156:0/1018534928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae840b20b0 0x7fae840ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.700+0000 7fae8b5ff700 1 -- 172.21.15.156:0/1018534928 >> 172.21.15.156:0/1018534928 conn(0x7fae8409c3f0 msgr2=0x7fae840b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:58.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.700+0000 7fae8b5ff700 1 -- 172.21.15.156:0/1018534928 shutdown_connections 2023-11-28T13:07:58.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.700+0000 7fae8b5ff700 1 -- 172.21.15.156:0/1018534928 wait complete. 2023-11-28T13:07:58.702 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 3 2023-11-28T13:07:58.719 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:07:58.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.928+0000 7f88ef1ff700 1 Processor -- start 2023-11-28T13:07:58.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.928+0000 7f88ef1ff700 1 -- start start 2023-11-28T13:07:58.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.929+0000 7f88ef1ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f88e8161340 0x7f88e80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:58.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.930+0000 7f88ef1ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e80d1950 con 0x7f88e815c9b0 2023-11-28T13:07:58.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.930+0000 7f88ef1ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e80b5520 con 0x7f88e80d2320 2023-11-28T13:07:58.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.930+0000 7f88ef1ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e80b5800 con 0x7f88e8161340 2023-11-28T13:07:58.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.930+0000 7f88ecf9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f88e815c9b0 0x7f88e80b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:45604/0 (socket says 172.21.15.156:45604) 2023-11-28T13:07:58.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.930+0000 7f88ecf9b700 1 -- 172.21.15.156:0/3125308071 learned_addr learned my addr 172.21.15.156:0/3125308071 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:07:58.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.930+0000 7f88e7fff700 1 --2- 172.21.15.156:0/3125308071 >> v2:172.21.15.186:3300/0 conn(0x7f88e8161340 0x7f88e80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:58.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.930+0000 7f88e77fe700 1 -- 172.21.15.156:0/3125308071 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365532016 0 0) 0x7f88e80d1950 con 0x7f88e815c9b0 2023-11-28T13:07:58.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.930+0000 7f88e77fe700 1 -- 172.21.15.156:0/3125308071 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d80035b0 con 0x7f88e815c9b0 2023-11-28T13:07:58.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.930+0000 7f88e77fe700 1 -- 172.21.15.156:0/3125308071 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423009460 0 0) 0x7f88e80b5520 con 0x7f88e80d2320 2023-11-28T13:07:58.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.930+0000 7f88e77fe700 1 -- 172.21.15.156:0/3125308071 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88e80d1950 con 0x7f88e80d2320 2023-11-28T13:07:58.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.930+0000 7f88e7fff700 1 -- 172.21.15.156:0/3125308071 >> v1:172.21.15.186:6789/0 conn(0x7f88e80d2320 legacy=0x7f88e80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:58.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.930+0000 7f88e7fff700 1 -- 172.21.15.156:0/3125308071 >> v1:172.21.15.156:6789/0 conn(0x7f88e815c9b0 legacy=0x7f88e80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:58.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.930+0000 7f88e7fff700 1 -- 172.21.15.156:0/3125308071 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e80d1300 con 0x7f88e8161340 2023-11-28T13:07:58.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.931+0000 7f88e7fff700 1 --2- 172.21.15.156:0/3125308071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88e8161340 0x7f88e80d1de0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009840 tx=0x7f88d4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2efd9deadd1f9a6e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:58.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.931+0000 7f88e77fe700 1 -- 172.21.15.156:0/3125308071 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d4011830 con 0x7f88e8161340 2023-11-28T13:07:58.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.931+0000 7f88e77fe700 1 -- 172.21.15.156:0/3125308071 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d4011990 con 0x7f88e8161340 2023-11-28T13:07:58.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.931+0000 7f88e77fe700 1 -- 172.21.15.156:0/3125308071 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d4011d20 con 0x7f88e8161340 2023-11-28T13:07:58.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.931+0000 7f88ef1ff700 1 -- 172.21.15.156:0/3125308071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88e8161340 msgr2=0x7f88e80d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:58.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.931+0000 7f88ef1ff700 1 --2- 172.21.15.156:0/3125308071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88e8161340 0x7f88e80d1de0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009840 tx=0x7f88d4009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:58.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.932+0000 7f88ef1ff700 1 -- 172.21.15.156:0/3125308071 shutdown_connections 2023-11-28T13:07:58.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.932+0000 7f88ef1ff700 1 --2- 172.21.15.156:0/3125308071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88e8161340 0x7f88e80d1de0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.932+0000 7f88ef1ff700 1 -- 172.21.15.156:0/3125308071 >> 172.21.15.156:0/3125308071 conn(0x7f88e8072600 msgr2=0x7f88e80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:58.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.932+0000 7f88ef1ff700 1 -- 172.21.15.156:0/3125308071 shutdown_connections 2023-11-28T13:07:58.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.932+0000 7f88ef1ff700 1 -- 172.21.15.156:0/3125308071 wait complete. 2023-11-28T13:07:58.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.933+0000 7f88ef1ff700 1 Processor -- start 2023-11-28T13:07:58.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.934+0000 7f88ef1ff700 1 -- start start 2023-11-28T13:07:58.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.934+0000 7f88ef1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88e80d2320 0x7f88e815db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:58.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.934+0000 7f88ef1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88e815c9b0 0x7f88e80c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:58.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.934+0000 7f88ef1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88e8161340 0x7f88e80b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:58.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.934+0000 7f88ef1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e80c9390 con 0x7f88e815c9b0 2023-11-28T13:07:58.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.934+0000 7f88ef1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e815ee20 con 0x7f88e8161340 2023-11-28T13:07:58.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.934+0000 7f88ef1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e815f2b0 con 0x7f88e80d2320 2023-11-28T13:07:58.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.935+0000 7f88e7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88e815c9b0 0x7f88e80c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:58.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.935+0000 7f88e7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88e815c9b0 0x7f88e80c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43472/0 (socket says 172.21.15.156:43472) 2023-11-28T13:07:58.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.935+0000 7f88e7fff700 1 -- 172.21.15.156:0/3607507992 learned_addr learned my addr 172.21.15.156:0/3607507992 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:58.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.935+0000 7f88ecf9b700 1 --2- 172.21.15.156:0/3607507992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88e80d2320 0x7f88e815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:58.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.935+0000 7f88ed79c700 1 --2- 172.21.15.156:0/3607507992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88e8161340 0x7f88e80b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:58.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.935+0000 7f88e7fff700 1 -- 172.21.15.156:0/3607507992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88e80d2320 msgr2=0x7f88e815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:58.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.935+0000 7f88e7fff700 1 --2- 172.21.15.156:0/3607507992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88e80d2320 0x7f88e815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.935+0000 7f88e7fff700 1 -- 172.21.15.156:0/3607507992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88e8161340 msgr2=0x7f88e80b7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:58.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.935+0000 7f88e7fff700 1 --2- 172.21.15.156:0/3607507992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88e8161340 0x7f88e80b7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:58.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.935+0000 7f88e7fff700 1 -- 172.21.15.156:0/3607507992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d40094a0 con 0x7f88e815c9b0 2023-11-28T13:07:58.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.935+0000 7f88e7fff700 1 --2- 172.21.15.156:0/3607507992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88e815c9b0 0x7f88e80c8e50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f88d4014040 tx=0x7f88d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:58.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.936+0000 7f88e57fa700 1 -- 172.21.15.156:0/3607507992 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d401a6e0 con 0x7f88e815c9b0 2023-11-28T13:07:58.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.936+0000 7f88e57fa700 1 -- 172.21.15.156:0/3607507992 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d401a840 con 0x7f88e815c9b0 2023-11-28T13:07:58.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.936+0000 7f88e57fa700 1 -- 172.21.15.156:0/3607507992 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d401abb0 con 0x7f88e815c9b0 2023-11-28T13:07:58.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.936+0000 7f88ef1ff700 1 -- 172.21.15.156:0/3607507992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e815cd80 con 0x7f88e815c9b0 2023-11-28T13:07:58.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.936+0000 7f88ef1ff700 1 -- 172.21.15.156:0/3607507992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e80d15e0 con 0x7f88e815c9b0 2023-11-28T13:07:58.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.937+0000 7f88ef1ff700 1 -- 172.21.15.156:0/3607507992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e815cf10 con 0x7f88e815c9b0 2023-11-28T13:07:58.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.937+0000 7f88e57fa700 1 -- 172.21.15.156:0/3607507992 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f88d400f050 con 0x7f88e815c9b0 2023-11-28T13:07:58.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.939+0000 7f88e57fa700 1 --2- 172.21.15.156:0/3607507992 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f88cc0682f0 0x7f88cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:58.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.939+0000 7f88e57fa700 1 -- 172.21.15.156:0/3607507992 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f88d4086650 con 0x7f88e815c9b0 2023-11-28T13:07:58.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.939+0000 7f88ecf9b700 1 --2- 172.21.15.156:0/3607507992 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f88cc0682f0 0x7f88cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:58.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.939+0000 7f88ecf9b700 1 --2- 172.21.15.156:0/3607507992 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f88cc0682f0 0x7f88cc06a7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f88dc007fd0 tx=0x7f88dc007e00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:58.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:58.943+0000 7f88e57fa700 1 -- 172.21.15.156:0/3607507992 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f88d40531c0 con 0x7f88e815c9b0 2023-11-28T13:07:59.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.137+0000 7f88ef1ff700 1 -- 172.21.15.156:0/3607507992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88e80745c0 con 0x7f88e815c9b0 2023-11-28T13:07:59.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.138+0000 7f88e57fa700 1 -- 172.21.15.156:0/3607507992 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+14255 (secure 0 0 0) 0x7f88d4029090 con 0x7f88e815c9b0 2023-11-28T13:07:59.139 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:07:59.139 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":17,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:07:58.834897+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:07:59.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.140+0000 7f88ef1ff700 1 -- 172.21.15.156:0/3607507992 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f88cc0682f0 msgr2=0x7f88cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:59.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.140+0000 7f88ef1ff700 1 --2- 172.21.15.156:0/3607507992 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f88cc0682f0 0x7f88cc06a7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f88dc007fd0 tx=0x7f88dc007e00 comp rx=0 tx=0).stop 2023-11-28T13:07:59.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.140+0000 7f88ef1ff700 1 -- 172.21.15.156:0/3607507992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88e815c9b0 msgr2=0x7f88e80c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:59.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.140+0000 7f88ef1ff700 1 --2- 172.21.15.156:0/3607507992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88e815c9b0 0x7f88e80c8e50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f88d4014040 tx=0x7f88d4004750 comp rx=0 tx=0).stop 2023-11-28T13:07:59.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.140+0000 7f88ef1ff700 1 -- 172.21.15.156:0/3607507992 shutdown_connections 2023-11-28T13:07:59.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.141+0000 7f88ef1ff700 1 --2- 172.21.15.156:0/3607507992 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f88cc0682f0 0x7f88cc06a7a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:59.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.141+0000 7f88ef1ff700 1 --2- 172.21.15.156:0/3607507992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88e8161340 0x7f88e80b7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:59.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.141+0000 7f88ef1ff700 1 --2- 172.21.15.156:0/3607507992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88e80d2320 0x7f88e815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:59.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.141+0000 7f88ef1ff700 1 --2- 172.21.15.156:0/3607507992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88e815c9b0 0x7f88e80c8e50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:59.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.141+0000 7f88ef1ff700 1 -- 172.21.15.156:0/3607507992 >> 172.21.15.156:0/3607507992 conn(0x7f88e8072600 msgr2=0x7f88e80b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:59.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.141+0000 7f88ef1ff700 1 -- 172.21.15.156:0/3607507992 shutdown_connections 2023-11-28T13:07:59.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.141+0000 7f88ef1ff700 1 -- 172.21.15.156:0/3607507992 wait complete. 2023-11-28T13:07:59.159 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:07:59.160 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:07:59.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.357+0000 7f98fc7ff700 1 Processor -- start 2023-11-28T13:07:59.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.358+0000 7f98fc7ff700 1 -- start start 2023-11-28T13:07:59.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.359+0000 7f98fc7ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f98f4161340 0x7f98f40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:59.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.359+0000 7f98fc7ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f40d1950 con 0x7f98f40d2320 2023-11-28T13:07:59.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.359+0000 7f98fc7ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f40b5520 con 0x7f98f415c9b0 2023-11-28T13:07:59.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.359+0000 7f98fc7ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f40b5800 con 0x7f98f4161340 2023-11-28T13:07:59.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.359+0000 7f98f9d9a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f98f4161340 0x7f98f40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:59.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.359+0000 7f98f9d9a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f98f4161340 0x7f98f40d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43480/0 (socket says 172.21.15.156:43480) 2023-11-28T13:07:59.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.359+0000 7f98f9d9a700 1 -- 172.21.15.156:0/1374900161 learned_addr learned my addr 172.21.15.156:0/1374900161 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:59.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.359+0000 7f98fa59b700 1 --1- 172.21.15.156:0/1374900161 >> v1:172.21.15.186:6789/0 conn(0x7f98f415c9b0 0x7f98f40b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:47626/0 (socket says 172.21.15.156:47626) 2023-11-28T13:07:59.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.359+0000 7f98f9d9a700 1 -- 172.21.15.156:0/1374900161 >> v1:172.21.15.186:6789/0 conn(0x7f98f415c9b0 legacy=0x7f98f40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:59.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.359+0000 7f98f9d9a700 1 -- 172.21.15.156:0/1374900161 >> v1:172.21.15.156:6789/0 conn(0x7f98f40d2320 legacy=0x7f98f40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:59.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.359+0000 7f98f9d9a700 1 -- 172.21.15.156:0/1374900161 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f40d1300 con 0x7f98f4161340 2023-11-28T13:07:59.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.360+0000 7f98f9d9a700 1 --2- 172.21.15.156:0/1374900161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98f4161340 0x7f98f40d1de0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f98e40098f0 tx=0x7f98e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60de81b9dcb517ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:59.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.360+0000 7f98f9599700 1 -- 172.21.15.156:0/1374900161 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e400b8a0 con 0x7f98f4161340 2023-11-28T13:07:59.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.360+0000 7f98f9599700 1 -- 172.21.15.156:0/1374900161 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e400ba00 con 0x7f98f4161340 2023-11-28T13:07:59.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.360+0000 7f98f9599700 1 -- 172.21.15.156:0/1374900161 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e400bd90 con 0x7f98f4161340 2023-11-28T13:07:59.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.360+0000 7f98fc7ff700 1 -- 172.21.15.156:0/1374900161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98f4161340 msgr2=0x7f98f40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:59.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.360+0000 7f98fc7ff700 1 --2- 172.21.15.156:0/1374900161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98f4161340 0x7f98f40d1de0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f98e40098f0 tx=0x7f98e4009c00 comp rx=0 tx=0).stop 2023-11-28T13:07:59.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.361+0000 7f98fc7ff700 1 -- 172.21.15.156:0/1374900161 shutdown_connections 2023-11-28T13:07:59.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.361+0000 7f98fc7ff700 1 --2- 172.21.15.156:0/1374900161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98f4161340 0x7f98f40d1de0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:59.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.361+0000 7f98fc7ff700 1 -- 172.21.15.156:0/1374900161 >> 172.21.15.156:0/1374900161 conn(0x7f98f4072600 msgr2=0x7f98f40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:59.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.361+0000 7f98fc7ff700 1 -- 172.21.15.156:0/1374900161 shutdown_connections 2023-11-28T13:07:59.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.361+0000 7f98fc7ff700 1 -- 172.21.15.156:0/1374900161 wait complete. 2023-11-28T13:07:59.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.362+0000 7f98fc7ff700 1 Processor -- start 2023-11-28T13:07:59.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.362+0000 7f98fc7ff700 1 -- start start 2023-11-28T13:07:59.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.363+0000 7f98fc7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98f40d2320 0x7f98f4156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:59.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.363+0000 7f98fc7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98f415c9b0 0x7f98f41e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:59.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.363+0000 7f98fc7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98f4161340 0x7f98f4151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:59.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.363+0000 7f98fc7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f4152310 con 0x7f98f40d2320 2023-11-28T13:07:59.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.363+0000 7f98fc7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f41e78c0 con 0x7f98f4161340 2023-11-28T13:07:59.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.363+0000 7f98fc7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f41e7ba0 con 0x7f98f415c9b0 2023-11-28T13:07:59.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.364+0000 7f98fa59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98f40d2320 0x7f98f4156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:59.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.364+0000 7f98fa59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98f40d2320 0x7f98f4156860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43482/0 (socket says 172.21.15.156:43482) 2023-11-28T13:07:59.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.364+0000 7f98fa59b700 1 -- 172.21.15.156:0/1250597496 learned_addr learned my addr 172.21.15.156:0/1250597496 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:59.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.364+0000 7f98f9d9a700 1 --2- 172.21.15.156:0/1250597496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98f415c9b0 0x7f98f41e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:59.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.364+0000 7f98fad9c700 1 --2- 172.21.15.156:0/1250597496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98f4161340 0x7f98f4151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:59.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.364+0000 7f98fa59b700 1 -- 172.21.15.156:0/1250597496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98f415c9b0 msgr2=0x7f98f41e6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:59.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.364+0000 7f98fa59b700 1 --2- 172.21.15.156:0/1250597496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98f415c9b0 0x7f98f41e6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:59.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.364+0000 7f98fa59b700 1 -- 172.21.15.156:0/1250597496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98f4161340 msgr2=0x7f98f4151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:59.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.364+0000 7f98fa59b700 1 --2- 172.21.15.156:0/1250597496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98f4161340 0x7f98f4151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:59.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.364+0000 7f98fa59b700 1 -- 172.21.15.156:0/1250597496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e4009550 con 0x7f98f40d2320 2023-11-28T13:07:59.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.364+0000 7f98fa59b700 1 --2- 172.21.15.156:0/1250597496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98f40d2320 0x7f98f4156860 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f98f000c520 tx=0x7f98f000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:59.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.364+0000 7f98eaffd700 1 -- 172.21.15.156:0/1250597496 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0005cb0 con 0x7f98f40d2320 2023-11-28T13:07:59.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.364+0000 7f98eaffd700 1 -- 172.21.15.156:0/1250597496 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f0005e10 con 0x7f98f40d2320 2023-11-28T13:07:59.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.364+0000 7f98fc7ff700 1 -- 172.21.15.156:0/1250597496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f415cd80 con 0x7f98f40d2320 2023-11-28T13:07:59.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.365+0000 7f98eaffd700 1 -- 172.21.15.156:0/1250597496 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f00072d0 con 0x7f98f40d2320 2023-11-28T13:07:59.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.365+0000 7f98fc7ff700 1 -- 172.21.15.156:0/1250597496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f40d15e0 con 0x7f98f40d2320 2023-11-28T13:07:59.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.366+0000 7f98fc7ff700 1 -- 172.21.15.156:0/1250597496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f415cf10 con 0x7f98f40d2320 2023-11-28T13:07:59.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.366+0000 7f98eaffd700 1 -- 172.21.15.156:0/1250597496 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f98f0019030 con 0x7f98f40d2320 2023-11-28T13:07:59.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.367+0000 7f98eaffd700 1 --2- 172.21.15.156:0/1250597496 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f98d80682f0 0x7f98d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:59.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.367+0000 7f98eaffd700 1 -- 172.21.15.156:0/1250597496 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f98f0084b60 con 0x7f98f40d2320 2023-11-28T13:07:59.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.368+0000 7f98f9d9a700 1 --2- 172.21.15.156:0/1250597496 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f98d80682f0 0x7f98d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:59.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.368+0000 7f98f9d9a700 1 --2- 172.21.15.156:0/1250597496 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f98d80682f0 0x7f98d806a7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f98e4014040 tx=0x7f98e4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:59.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.371+0000 7f98eaffd700 1 -- 172.21.15.156:0/1250597496 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f98f004b5e0 con 0x7f98f40d2320 2023-11-28T13:07:59.574 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.574+0000 7f98fc7ff700 1 -- 172.21.15.156:0/1250597496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f98f40745c0 con 0x7f98f40d2320 2023-11-28T13:07:59.575 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.574+0000 7f98eaffd700 1 -- 172.21.15.156:0/1250597496 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f98f0017020 con 0x7f98f40d2320 2023-11-28T13:07:59.575 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:07:59.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.577+0000 7f98fc7ff700 1 -- 172.21.15.156:0/1250597496 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f98d80682f0 msgr2=0x7f98d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:59.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.577+0000 7f98fc7ff700 1 --2- 172.21.15.156:0/1250597496 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f98d80682f0 0x7f98d806a7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f98e4014040 tx=0x7f98e4008030 comp rx=0 tx=0).stop 2023-11-28T13:07:59.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.577+0000 7f98fc7ff700 1 -- 172.21.15.156:0/1250597496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98f40d2320 msgr2=0x7f98f4156860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:59.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.577+0000 7f98fc7ff700 1 --2- 172.21.15.156:0/1250597496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98f40d2320 0x7f98f4156860 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f98f000c520 tx=0x7f98f000c830 comp rx=0 tx=0).stop 2023-11-28T13:07:59.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.577+0000 7f98fc7ff700 1 -- 172.21.15.156:0/1250597496 shutdown_connections 2023-11-28T13:07:59.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.577+0000 7f98fc7ff700 1 --2- 172.21.15.156:0/1250597496 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f98d80682f0 0x7f98d806a7a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:59.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.577+0000 7f98fc7ff700 1 --2- 172.21.15.156:0/1250597496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98f4161340 0x7f98f4151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:59.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.577+0000 7f98fc7ff700 1 --2- 172.21.15.156:0/1250597496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98f40d2320 0x7f98f4156860 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:59.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.577+0000 7f98fc7ff700 1 --2- 172.21.15.156:0/1250597496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98f415c9b0 0x7f98f41e6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:59.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.577+0000 7f98fc7ff700 1 -- 172.21.15.156:0/1250597496 >> 172.21.15.156:0/1250597496 conn(0x7f98f4072600 msgr2=0x7f98f415ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:59.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.577+0000 7f98fc7ff700 1 -- 172.21.15.156:0/1250597496 shutdown_connections 2023-11-28T13:07:59.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.577+0000 7f98fc7ff700 1 -- 172.21.15.156:0/1250597496 wait complete. 2023-11-28T13:07:59.596 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:07:59.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.795+0000 7fbcef5ff700 1 Processor -- start 2023-11-28T13:07:59.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.795+0000 7fbcef5ff700 1 -- start start 2023-11-28T13:07:59.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.796+0000 7fbcef5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbce80d01b0 0x7fbce80d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:59.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.797+0000 7fbcef5ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce80b0230 con 0x7fbce80b20b0 2023-11-28T13:07:59.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.797+0000 7fbcef5ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce80b37d0 con 0x7fbce80b16d0 2023-11-28T13:07:59.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.797+0000 7fbcef5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce815cec0 con 0x7fbce80d01b0 2023-11-28T13:07:59.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.797+0000 7fbcecb9a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fbce80b20b0 0x7fbce80b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:45622/0 (socket says 172.21.15.156:45622) 2023-11-28T13:07:59.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.797+0000 7fbcecb9a700 1 -- 172.21.15.156:0/2880682253 learned_addr learned my addr 172.21.15.156:0/2880682253 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:07:59.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.797+0000 7fbcdffff700 1 -- 172.21.15.156:0/2880682253 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508638056 0 0) 0x7fbce80b0230 con 0x7fbce80b20b0 2023-11-28T13:07:59.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.797+0000 7fbcdffff700 1 -- 172.21.15.156:0/2880682253 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcd40035b0 con 0x7fbce80b20b0 2023-11-28T13:07:59.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.797+0000 7fbced39b700 1 --2- 172.21.15.156:0/2880682253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbce80d01b0 0x7fbce80d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:59.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.797+0000 7fbcdffff700 1 -- 172.21.15.156:0/2880682253 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928764702 0 0) 0x7fbce80b37d0 con 0x7fbce80b16d0 2023-11-28T13:07:59.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.797+0000 7fbcdffff700 1 -- 172.21.15.156:0/2880682253 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbce80b0230 con 0x7fbce80b16d0 2023-11-28T13:07:59.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.797+0000 7fbcdffff700 1 -- 172.21.15.156:0/2880682253 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 100388867 0 0) 0x7fbcd40035b0 con 0x7fbce80b20b0 2023-11-28T13:07:59.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcdffff700 1 -- 172.21.15.156:0/2880682253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbce80d01b0 msgr2=0x7fbce80d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:59.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcdffff700 1 --2- 172.21.15.156:0/2880682253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbce80d01b0 0x7fbce80d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:59.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcdffff700 1 -- 172.21.15.156:0/2880682253 >> v1:172.21.15.186:6789/0 conn(0x7fbce80b16d0 legacy=0x7fbce80b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:59.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcdffff700 1 -- 172.21.15.156:0/2880682253 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce80b4570 con 0x7fbce80b20b0 2023-11-28T13:07:59.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcdffff700 1 -- 172.21.15.156:0/2880682253 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7fbcd8001670 con 0x7fbce80b20b0 2023-11-28T13:07:59.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcdffff700 1 -- 172.21.15.156:0/2880682253 >> v1:172.21.15.156:6789/0 conn(0x7fbce80b20b0 legacy=0x7fbce80b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:59.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcdffff700 1 --2- 172.21.15.156:0/2880682253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbce80d01b0 0x7fbcd4003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:59.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcdffff700 1 --2- 172.21.15.156:0/2880682253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcd4004120 0x7fbcd4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:59.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcdffff700 1 --2- 172.21.15.156:0/2880682253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcd4004d90 0x7fbcd4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:59.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcdffff700 1 -- 172.21.15.156:0/2880682253 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce815cec0 con 0x7fbcd4004120 2023-11-28T13:07:59.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcdffff700 1 -- 172.21.15.156:0/2880682253 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd400bba0 con 0x7fbcd4004d90 2023-11-28T13:07:59.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcdffff700 1 -- 172.21.15.156:0/2880682253 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd400bd60 con 0x7fbce80d01b0 2023-11-28T13:07:59.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcecb9a700 1 --2- 172.21.15.156:0/2880682253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcd4004120 0x7fbcd4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:59.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbced39b700 1 --2- 172.21.15.156:0/2880682253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbce80d01b0 0x7fbcd4003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:59.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcedb9c700 1 --2- 172.21.15.156:0/2880682253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcd4004d90 0x7fbcd4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:59.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcecb9a700 1 -- 172.21.15.156:0/2880682253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbce80d01b0 msgr2=0x7fbcd4003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:59.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcecb9a700 1 --2- 172.21.15.156:0/2880682253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbce80d01b0 0x7fbcd4003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:59.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcecb9a700 1 -- 172.21.15.156:0/2880682253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcd4004d90 msgr2=0x7fbcd4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:59.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcecb9a700 1 --2- 172.21.15.156:0/2880682253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcd4004d90 0x7fbcd4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:59.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.798+0000 7fbcecb9a700 1 -- 172.21.15.156:0/2880682253 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd40035b0 con 0x7fbcd4004120 2023-11-28T13:07:59.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.799+0000 7fbcecb9a700 1 --2- 172.21.15.156:0/2880682253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcd4004120 0x7fbcd4004850 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800c210 tx=0x7fbcd4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a471bac03b72f15e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:59.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.799+0000 7fbcdffff700 1 -- 172.21.15.156:0/2880682253 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd800cd50 con 0x7fbcd4004120 2023-11-28T13:07:59.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.799+0000 7fbcdffff700 1 -- 172.21.15.156:0/2880682253 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd8005960 con 0x7fbcd4004120 2023-11-28T13:07:59.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.799+0000 7fbcdffff700 1 -- 172.21.15.156:0/2880682253 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8005cd0 con 0x7fbcd4004120 2023-11-28T13:07:59.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.799+0000 7fbcef5ff700 1 -- 172.21.15.156:0/2880682253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcd4004120 msgr2=0x7fbcd4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:59.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.799+0000 7fbcef5ff700 1 --2- 172.21.15.156:0/2880682253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcd4004120 0x7fbcd4004850 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800c210 tx=0x7fbcd4005880 comp rx=0 tx=0).stop 2023-11-28T13:07:59.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.799+0000 7fbcedb9c700 1 -- 172.21.15.156:0/2880682253 reap_dead start 2023-11-28T13:07:59.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.799+0000 7fbcef5ff700 1 -- 172.21.15.156:0/2880682253 shutdown_connections 2023-11-28T13:07:59.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.799+0000 7fbcef5ff700 1 -- 172.21.15.156:0/2880682253 >> 172.21.15.156:0/2880682253 conn(0x7fbce809c3f0 msgr2=0x7fbce80d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:59.807 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.799+0000 7fbcef5ff700 1 -- 172.21.15.156:0/2880682253 shutdown_connections 2023-11-28T13:07:59.807 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.799+0000 7fbcef5ff700 1 -- 172.21.15.156:0/2880682253 wait complete. 2023-11-28T13:07:59.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.801+0000 7fbcef5ff700 1 Processor -- start 2023-11-28T13:07:59.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.801+0000 7fbcef5ff700 1 -- start start 2023-11-28T13:07:59.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.802+0000 7fbcef5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbcd4004120 0x7fbce80ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:59.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.802+0000 7fbcef5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbce8156140 0x7fbce81565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:59.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.802+0000 7fbcef5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce81555c0 0x7fbce8155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:59.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.802+0000 7fbcef5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce815e390 con 0x7fbce81555c0 2023-11-28T13:07:59.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.802+0000 7fbcef5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce8155fa0 con 0x7fbce8156140 2023-11-28T13:07:59.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.802+0000 7fbcef5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce815f660 con 0x7fbcd4004120 2023-11-28T13:07:59.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.802+0000 7fbcedb9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce81555c0 0x7fbce8155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:59.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.803+0000 7fbcedb9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce81555c0 0x7fbce8155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43510/0 (socket says 172.21.15.156:43510) 2023-11-28T13:07:59.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.803+0000 7fbcedb9c700 1 -- 172.21.15.156:0/1305499717 learned_addr learned my addr 172.21.15.156:0/1305499717 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:07:59.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.803+0000 7fbcecb9a700 1 --2- 172.21.15.156:0/1305499717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbce8156140 0x7fbce81565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:59.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.803+0000 7fbced39b700 1 --2- 172.21.15.156:0/1305499717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbcd4004120 0x7fbce80ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:59.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.803+0000 7fbcedb9c700 1 -- 172.21.15.156:0/1305499717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbcd4004120 msgr2=0x7fbce80ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:59.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.803+0000 7fbcedb9c700 1 --2- 172.21.15.156:0/1305499717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbcd4004120 0x7fbce80ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:59.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.803+0000 7fbcedb9c700 1 -- 172.21.15.156:0/1305499717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbce8156140 msgr2=0x7fbce81565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:59.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.803+0000 7fbcedb9c700 1 --2- 172.21.15.156:0/1305499717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbce8156140 0x7fbce81565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:59.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.803+0000 7fbcedb9c700 1 -- 172.21.15.156:0/1305499717 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce80afbe0 con 0x7fbce81555c0 2023-11-28T13:07:59.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.803+0000 7fbcecb9a700 1 --2- 172.21.15.156:0/1305499717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbce8156140 0x7fbce81565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:59.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.803+0000 7fbcedb9c700 1 --2- 172.21.15.156:0/1305499717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce81555c0 0x7fbce8155a60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbce80b5c40 tx=0x7fbce000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:59.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.803+0000 7fbcddffb700 1 -- 172.21.15.156:0/1305499717 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0003c10 con 0x7fbce81555c0 2023-11-28T13:07:59.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.803+0000 7fbcddffb700 1 -- 172.21.15.156:0/1305499717 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce0003d70 con 0x7fbce81555c0 2023-11-28T13:07:59.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.803+0000 7fbcef5ff700 1 -- 172.21.15.156:0/1305499717 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce80b4850 con 0x7fbce81555c0 2023-11-28T13:07:59.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.803+0000 7fbcddffb700 1 -- 172.21.15.156:0/1305499717 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce000db20 con 0x7fbce81555c0 2023-11-28T13:07:59.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.804+0000 7fbcef5ff700 1 -- 172.21.15.156:0/1305499717 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce80afec0 con 0x7fbce81555c0 2023-11-28T13:07:59.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.805+0000 7fbcef5ff700 1 -- 172.21.15.156:0/1305499717 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce80b6b30 con 0x7fbce81555c0 2023-11-28T13:07:59.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.806+0000 7fbcddffb700 1 -- 172.21.15.156:0/1305499717 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbce000dc80 con 0x7fbce81555c0 2023-11-28T13:07:59.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.806+0000 7fbcddffb700 1 --2- 172.21.15.156:0/1305499717 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbccc0682f0 0x7fbccc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:59.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.806+0000 7fbcddffb700 1 -- 172.21.15.156:0/1305499717 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbce0085d90 con 0x7fbce81555c0 2023-11-28T13:07:59.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.807+0000 7fbced39b700 1 --2- 172.21.15.156:0/1305499717 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbccc0682f0 0x7fbccc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:59.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.807+0000 7fbced39b700 1 --2- 172.21.15.156:0/1305499717 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbccc0682f0 0x7fbccc06a7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbce80cf940 tx=0x7fbce4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:59.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:07:59.811+0000 7fbcddffb700 1 -- 172.21.15.156:0/1305499717 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbce80b6b30 con 0x7fbce81555c0 2023-11-28T13:08:00.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.005+0000 7fbcef5ff700 1 -- 172.21.15.156:0/1305499717 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbce80b6b30 con 0x7fbce81555c0 2023-11-28T13:08:00.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.006+0000 7fbcddffb700 1 -- 172.21.15.156:0/1305499717 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbce80b6b30 con 0x7fbce81555c0 2023-11-28T13:08:00.006 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:08:00.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.008+0000 7fbcef5ff700 1 -- 172.21.15.156:0/1305499717 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbccc0682f0 msgr2=0x7fbccc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:00.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.008+0000 7fbcef5ff700 1 --2- 172.21.15.156:0/1305499717 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbccc0682f0 0x7fbccc06a7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbce80cf940 tx=0x7fbce4008040 comp rx=0 tx=0).stop 2023-11-28T13:08:00.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.008+0000 7fbcef5ff700 1 -- 172.21.15.156:0/1305499717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce81555c0 msgr2=0x7fbce8155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:00.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.008+0000 7fbcef5ff700 1 --2- 172.21.15.156:0/1305499717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce81555c0 0x7fbce8155a60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbce80b5c40 tx=0x7fbce000d430 comp rx=0 tx=0).stop 2023-11-28T13:08:00.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.008+0000 7fbcef5ff700 1 -- 172.21.15.156:0/1305499717 shutdown_connections 2023-11-28T13:08:00.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.008+0000 7fbcef5ff700 1 --2- 172.21.15.156:0/1305499717 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbccc0682f0 0x7fbccc06a7a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:00.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.008+0000 7fbcef5ff700 1 --2- 172.21.15.156:0/1305499717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce81555c0 0x7fbce8155a60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:00.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.008+0000 7fbcef5ff700 1 --2- 172.21.15.156:0/1305499717 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbcd4004120 0x7fbce80ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:00.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.008+0000 7fbcef5ff700 1 --2- 172.21.15.156:0/1305499717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbce8156140 0x7fbce81565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:00.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.008+0000 7fbcef5ff700 1 -- 172.21.15.156:0/1305499717 >> 172.21.15.156:0/1305499717 conn(0x7fbce809c3f0 msgr2=0x7fbce80d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:00.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.008+0000 7fbcef5ff700 1 -- 172.21.15.156:0/1305499717 shutdown_connections 2023-11-28T13:08:00.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.008+0000 7fbcef5ff700 1 -- 172.21.15.156:0/1305499717 wait complete. 2023-11-28T13:08:00.027 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T13:08:00.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.223+0000 7f873a9ff700 1 Processor -- start 2023-11-28T13:08:00.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.223+0000 7f873a9ff700 1 -- start start 2023-11-28T13:08:00.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.224+0000 7f873a9ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f873415c9b0 0x7f87340b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:00.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.224+0000 7f873a9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8734161340 0x7f87340d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:00.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.225+0000 7f873a9ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87340d1950 con 0x7f87340d2320 2023-11-28T13:08:00.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.225+0000 7f873a9ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87340b5800 con 0x7f873415c9b0 2023-11-28T13:08:00.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.225+0000 7f873a9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87340b5520 con 0x7f8734161340 2023-11-28T13:08:00.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.225+0000 7f8733fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f873415c9b0 0x7f87340b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:00.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.225+0000 7f8733fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f873415c9b0 0x7f87340b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43520/0 (socket says 172.21.15.156:43520) 2023-11-28T13:08:00.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.225+0000 7f8733fff700 1 -- 172.21.15.156:0/770081664 learned_addr learned my addr 172.21.15.156:0/770081664 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:00.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.225+0000 7f872bfff700 1 --2- 172.21.15.156:0/770081664 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8734161340 0x7f87340d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:00.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.225+0000 7f8733fff700 1 -- 172.21.15.156:0/770081664 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8734161340 msgr2=0x7f87340d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:00.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.225+0000 7f8733fff700 1 --2- 172.21.15.156:0/770081664 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8734161340 0x7f87340d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:00.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.225+0000 7f8733fff700 1 -- 172.21.15.156:0/770081664 >> v1:172.21.15.186:6789/0 conn(0x7f87340d2320 legacy=0x7f87340cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:00.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.225+0000 7f8733fff700 1 -- 172.21.15.156:0/770081664 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87340d1300 con 0x7f873415c9b0 2023-11-28T13:08:00.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.225+0000 7f8733fff700 1 --2- 172.21.15.156:0/770081664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f873415c9b0 0x7f87340b4bf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f87240098f0 tx=0x7f8724009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9464045b932678b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:00.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.226+0000 7f87337fe700 1 -- 172.21.15.156:0/770081664 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872400b8a0 con 0x7f873415c9b0 2023-11-28T13:08:00.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.226+0000 7f87337fe700 1 -- 172.21.15.156:0/770081664 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872400ba00 con 0x7f873415c9b0 2023-11-28T13:08:00.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.226+0000 7f87337fe700 1 -- 172.21.15.156:0/770081664 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872400bd90 con 0x7f873415c9b0 2023-11-28T13:08:00.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.226+0000 7f873a9ff700 1 -- 172.21.15.156:0/770081664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f873415c9b0 msgr2=0x7f87340b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:00.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.226+0000 7f873a9ff700 1 --2- 172.21.15.156:0/770081664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f873415c9b0 0x7f87340b4bf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f87240098f0 tx=0x7f8724009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:00.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.226+0000 7f873a9ff700 1 -- 172.21.15.156:0/770081664 shutdown_connections 2023-11-28T13:08:00.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.226+0000 7f873a9ff700 1 --2- 172.21.15.156:0/770081664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f873415c9b0 0x7f87340b4bf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:00.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.226+0000 7f873a9ff700 1 --2- 172.21.15.156:0/770081664 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8734161340 0x7f87340d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:00.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.226+0000 7f873a9ff700 1 -- 172.21.15.156:0/770081664 >> 172.21.15.156:0/770081664 conn(0x7f8734072600 msgr2=0x7f87340cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:00.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.226+0000 7f873a9ff700 1 -- 172.21.15.156:0/770081664 shutdown_connections 2023-11-28T13:08:00.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.226+0000 7f873a9ff700 1 -- 172.21.15.156:0/770081664 wait complete. 2023-11-28T13:08:00.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.229+0000 7f873a9ff700 1 Processor -- start 2023-11-28T13:08:00.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.229+0000 7f873a9ff700 1 -- start start 2023-11-28T13:08:00.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.230+0000 7f873a9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87340d2320 0x7f8734150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:00.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.230+0000 7f873a9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f873415c9b0 0x7f873415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:00.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.230+0000 7f873a9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8734161340 0x7f873415f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:00.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.230+0000 7f873a9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87341e8ac0 con 0x7f8734161340 2023-11-28T13:08:00.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.230+0000 7f873a9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87341f6d80 con 0x7f87340d2320 2023-11-28T13:08:00.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.230+0000 7f873a9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87341f71a0 con 0x7f873415c9b0 2023-11-28T13:08:00.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.230+0000 7f8738f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8734161340 0x7f873415f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:00.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.230+0000 7f8738f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8734161340 0x7f873415f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43532/0 (socket says 172.21.15.156:43532) 2023-11-28T13:08:00.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.231+0000 7f8738f9c700 1 -- 172.21.15.156:0/2335607508 learned_addr learned my addr 172.21.15.156:0/2335607508 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:00.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.231+0000 7f872bfff700 1 --2- 172.21.15.156:0/2335607508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f873415c9b0 0x7f873415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:00.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.231+0000 7f8733fff700 1 --2- 172.21.15.156:0/2335607508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87340d2320 0x7f8734150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:00.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.231+0000 7f8738f9c700 1 -- 172.21.15.156:0/2335607508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f873415c9b0 msgr2=0x7f873415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:00.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.231+0000 7f8738f9c700 1 --2- 172.21.15.156:0/2335607508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f873415c9b0 0x7f873415f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:00.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.231+0000 7f8738f9c700 1 -- 172.21.15.156:0/2335607508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87340d2320 msgr2=0x7f8734150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:00.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.231+0000 7f8738f9c700 1 --2- 172.21.15.156:0/2335607508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87340d2320 0x7f8734150320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:00.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.231+0000 7f8738f9c700 1 -- 172.21.15.156:0/2335607508 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8724009550 con 0x7f8734161340 2023-11-28T13:08:00.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.231+0000 7f872bfff700 1 --2- 172.21.15.156:0/2335607508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f873415c9b0 0x7f873415f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:00.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.231+0000 7f8738f9c700 1 --2- 172.21.15.156:0/2335607508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8734161340 0x7f873415f820 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f872c00cb50 tx=0x7f872c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:00.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.231+0000 7f87317fa700 1 -- 172.21.15.156:0/2335607508 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c003c00 con 0x7f8734161340 2023-11-28T13:08:00.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.231+0000 7f87317fa700 1 -- 172.21.15.156:0/2335607508 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c003d60 con 0x7f8734161340 2023-11-28T13:08:00.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.231+0000 7f873a9ff700 1 -- 172.21.15.156:0/2335607508 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87340b5130 con 0x7f8734161340 2023-11-28T13:08:00.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.231+0000 7f87317fa700 1 -- 172.21.15.156:0/2335607508 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c00aac0 con 0x7f8734161340 2023-11-28T13:08:00.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.232+0000 7f873a9ff700 1 -- 172.21.15.156:0/2335607508 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87340d15e0 con 0x7f8734161340 2023-11-28T13:08:00.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.232+0000 7f873a9ff700 1 -- 172.21.15.156:0/2335607508 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87340b52c0 con 0x7f8734161340 2023-11-28T13:08:00.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.233+0000 7f87317fa700 1 -- 172.21.15.156:0/2335607508 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f872c010050 con 0x7f8734161340 2023-11-28T13:08:00.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.234+0000 7f87317fa700 1 --2- 172.21.15.156:0/2335607508 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f87140682f0 0x7f871406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:00.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.234+0000 7f87317fa700 1 -- 172.21.15.156:0/2335607508 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f872c085870 con 0x7f8734161340 2023-11-28T13:08:00.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.234+0000 7f8733fff700 1 --2- 172.21.15.156:0/2335607508 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f87140682f0 0x7f871406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:00.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.235+0000 7f8733fff700 1 --2- 172.21.15.156:0/2335607508 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f87140682f0 0x7f871406a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8724009c40 tx=0x7f8724008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:00.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.238+0000 7f87317fa700 1 -- 172.21.15.156:0/2335607508 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f872c052330 con 0x7f8734161340 2023-11-28T13:08:00.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.432+0000 7f873a9ff700 1 -- 172.21.15.156:0/2335607508 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8734150860 con 0x7f8734161340 2023-11-28T13:08:00.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.433+0000 7f87317fa700 1 -- 172.21.15.156:0/2335607508 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7f8734150860 con 0x7f8734161340 2023-11-28T13:08:00.434 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:00.434 INFO:teuthology.orchestra.run.smithi156.stdout:[ 2023-11-28T13:08:00.434 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:08:00.434 INFO:teuthology.orchestra.run.smithi156.stdout: "poolnum": 1, 2023-11-28T13:08:00.434 INFO:teuthology.orchestra.run.smithi156.stdout: "poolname": "rbd" 2023-11-28T13:08:00.434 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:08:00.434 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:08:00.434 INFO:teuthology.orchestra.run.smithi156.stdout: "poolnum": 2, 2023-11-28T13:08:00.435 INFO:teuthology.orchestra.run.smithi156.stdout: "poolname": ".mgr" 2023-11-28T13:08:00.435 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:08:00.435 INFO:teuthology.orchestra.run.smithi156.stdout:] 2023-11-28T13:08:00.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.435+0000 7f873a9ff700 1 -- 172.21.15.156:0/2335607508 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f87140682f0 msgr2=0x7f871406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:00.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.435+0000 7f873a9ff700 1 --2- 172.21.15.156:0/2335607508 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f87140682f0 0x7f871406a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8724009c40 tx=0x7f8724008030 comp rx=0 tx=0).stop 2023-11-28T13:08:00.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.435+0000 7f873a9ff700 1 -- 172.21.15.156:0/2335607508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8734161340 msgr2=0x7f873415f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:00.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.435+0000 7f873a9ff700 1 --2- 172.21.15.156:0/2335607508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8734161340 0x7f873415f820 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f872c00cb50 tx=0x7f872c00ce60 comp rx=0 tx=0).stop 2023-11-28T13:08:00.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.435+0000 7f873a9ff700 1 -- 172.21.15.156:0/2335607508 shutdown_connections 2023-11-28T13:08:00.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.435+0000 7f873a9ff700 1 --2- 172.21.15.156:0/2335607508 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f87140682f0 0x7f871406a7a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:00.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.435+0000 7f873a9ff700 1 --2- 172.21.15.156:0/2335607508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8734161340 0x7f873415f820 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:00.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.435+0000 7f873a9ff700 1 --2- 172.21.15.156:0/2335607508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87340d2320 0x7f8734150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:00.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.435+0000 7f873a9ff700 1 --2- 172.21.15.156:0/2335607508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f873415c9b0 0x7f873415f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:00.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.435+0000 7f873a9ff700 1 -- 172.21.15.156:0/2335607508 >> 172.21.15.156:0/2335607508 conn(0x7f8734072600 msgr2=0x7f873406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:00.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.435+0000 7f873a9ff700 1 -- 172.21.15.156:0/2335607508 shutdown_connections 2023-11-28T13:08:00.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.435+0000 7f873a9ff700 1 -- 172.21.15.156:0/2335607508 wait complete. 2023-11-28T13:08:00.454 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T13:08:00.454 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T13:08:00.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.656+0000 7ff95bfff700 1 Processor -- start 2023-11-28T13:08:00.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.656+0000 7ff95bfff700 1 -- start start 2023-11-28T13:08:00.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.657+0000 7ff95bfff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff9540cded0 0x7ff954161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:00.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.657+0000 7ff95bfff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff954160d70 con 0x7ff95415dc30 2023-11-28T13:08:00.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.657+0000 7ff95bfff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9540b5390 con 0x7ff95415a760 2023-11-28T13:08:00.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.657+0000 7ff95bfff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff95415e0d0 con 0x7ff9540cded0 2023-11-28T13:08:00.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.657+0000 7ff95a59c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff95415dc30 0x7ff9540d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:45626/0 (socket says 172.21.15.156:45626) 2023-11-28T13:08:00.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.657+0000 7ff95a59c700 1 -- 172.21.15.156:0/2074385441 learned_addr learned my addr 172.21.15.156:0/2074385441 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:00.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.657+0000 7ff95959a700 1 --2- 172.21.15.156:0/2074385441 >> v2:172.21.15.186:3300/0 conn(0x7ff9540cded0 0x7ff954161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:00.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.658+0000 7ff958d99700 1 -- 172.21.15.156:0/2074385441 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285023933 0 0) 0x7ff954160d70 con 0x7ff95415dc30 2023-11-28T13:08:00.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.658+0000 7ff958d99700 1 -- 172.21.15.156:0/2074385441 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9400035b0 con 0x7ff95415dc30 2023-11-28T13:08:00.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.658+0000 7ff958d99700 1 -- 172.21.15.156:0/2074385441 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371120699 0 0) 0x7ff9540b5390 con 0x7ff95415a760 2023-11-28T13:08:00.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.658+0000 7ff958d99700 1 -- 172.21.15.156:0/2074385441 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff954160d70 con 0x7ff95415a760 2023-11-28T13:08:00.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.658+0000 7ff95959a700 1 -- 172.21.15.156:0/2074385441 >> v1:172.21.15.186:6789/0 conn(0x7ff95415a760 legacy=0x7ff9540cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:00.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.658+0000 7ff95959a700 1 -- 172.21.15.156:0/2074385441 >> v1:172.21.15.156:6789/0 conn(0x7ff95415dc30 legacy=0x7ff9540d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:00.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.658+0000 7ff95959a700 1 -- 172.21.15.156:0/2074385441 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9540b5670 con 0x7ff9540cded0 2023-11-28T13:08:00.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.658+0000 7ff95959a700 1 --2- 172.21.15.156:0/2074385441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9540cded0 0x7ff954161200 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff944009840 tx=0x7ff944009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37a1f29e73f7f9ae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:00.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.658+0000 7ff958d99700 1 -- 172.21.15.156:0/2074385441 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944011830 con 0x7ff9540cded0 2023-11-28T13:08:00.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.658+0000 7ff958d99700 1 -- 172.21.15.156:0/2074385441 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff944011990 con 0x7ff9540cded0 2023-11-28T13:08:00.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.658+0000 7ff958d99700 1 -- 172.21.15.156:0/2074385441 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944011d20 con 0x7ff9540cded0 2023-11-28T13:08:00.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.659+0000 7ff95bfff700 1 -- 172.21.15.156:0/2074385441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9540cded0 msgr2=0x7ff954161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:00.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.659+0000 7ff95bfff700 1 --2- 172.21.15.156:0/2074385441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9540cded0 0x7ff954161200 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff944009840 tx=0x7ff944009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:00.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.659+0000 7ff95bfff700 1 -- 172.21.15.156:0/2074385441 shutdown_connections 2023-11-28T13:08:00.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.659+0000 7ff95bfff700 1 --2- 172.21.15.156:0/2074385441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9540cded0 0x7ff954161200 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:00.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.659+0000 7ff95bfff700 1 -- 172.21.15.156:0/2074385441 >> 172.21.15.156:0/2074385441 conn(0x7ff954072620 msgr2=0x7ff9540cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:00.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.659+0000 7ff95bfff700 1 -- 172.21.15.156:0/2074385441 shutdown_connections 2023-11-28T13:08:00.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.659+0000 7ff95bfff700 1 -- 172.21.15.156:0/2074385441 wait complete. 2023-11-28T13:08:00.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.661+0000 7ff95bfff700 1 Processor -- start 2023-11-28T13:08:00.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.661+0000 7ff95bfff700 1 -- start start 2023-11-28T13:08:00.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.662+0000 7ff95bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9540cded0 0x7ff9541f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:00.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.662+0000 7ff95bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff95415a760 0x7ff9541ef7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:00.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.662+0000 7ff95bfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff95415dc30 0x7ff954158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:00.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.662+0000 7ff95bfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9540c9b80 con 0x7ff95415a760 2023-11-28T13:08:00.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.662+0000 7ff95bfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9541f33f0 con 0x7ff9540cded0 2023-11-28T13:08:00.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.662+0000 7ff95bfff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9540c6db0 con 0x7ff95415dc30 2023-11-28T13:08:00.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.662+0000 7ff95959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff95415a760 0x7ff9541ef7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:00.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.662+0000 7ff95959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff95415a760 0x7ff9541ef7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43534/0 (socket says 172.21.15.156:43534) 2023-11-28T13:08:00.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.662+0000 7ff95959a700 1 -- 172.21.15.156:0/411957260 learned_addr learned my addr 172.21.15.156:0/411957260 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:00.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.662+0000 7ff95a59c700 1 --2- 172.21.15.156:0/411957260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff95415dc30 0x7ff954158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:00.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.662+0000 7ff959d9b700 1 --2- 172.21.15.156:0/411957260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9540cded0 0x7ff9541f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:00.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.662+0000 7ff95959a700 1 -- 172.21.15.156:0/411957260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff95415dc30 msgr2=0x7ff954158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:00.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.662+0000 7ff95959a700 1 --2- 172.21.15.156:0/411957260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff95415dc30 0x7ff954158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:00.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.662+0000 7ff95959a700 1 -- 172.21.15.156:0/411957260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9540cded0 msgr2=0x7ff9541f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:00.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.662+0000 7ff95959a700 1 --2- 172.21.15.156:0/411957260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9540cded0 0x7ff9541f2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:00.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.662+0000 7ff95959a700 1 -- 172.21.15.156:0/411957260 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9440094a0 con 0x7ff95415a760 2023-11-28T13:08:00.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.663+0000 7ff959d9b700 1 --2- 172.21.15.156:0/411957260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9540cded0 0x7ff9541f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:00.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.663+0000 7ff95a59c700 1 --2- 172.21.15.156:0/411957260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff95415dc30 0x7ff954158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:00.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.663+0000 7ff95959a700 1 --2- 172.21.15.156:0/411957260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff95415a760 0x7ff9541ef7e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff9440090b0 tx=0x7ff944004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:00.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.663+0000 7ff94a7fc700 1 -- 172.21.15.156:0/411957260 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94401a860 con 0x7ff95415a760 2023-11-28T13:08:00.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.663+0000 7ff94a7fc700 1 -- 172.21.15.156:0/411957260 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94401a9c0 con 0x7ff95415a760 2023-11-28T13:08:00.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.663+0000 7ff95bfff700 1 -- 172.21.15.156:0/411957260 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff954160a00 con 0x7ff95415a760 2023-11-28T13:08:00.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.663+0000 7ff94a7fc700 1 -- 172.21.15.156:0/411957260 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94401ad30 con 0x7ff95415a760 2023-11-28T13:08:00.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.663+0000 7ff95bfff700 1 -- 172.21.15.156:0/411957260 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff954160720 con 0x7ff95415a760 2023-11-28T13:08:00.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.664+0000 7ff95bfff700 1 -- 172.21.15.156:0/411957260 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9540b4bf0 con 0x7ff95415a760 2023-11-28T13:08:00.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.665+0000 7ff94a7fc700 1 -- 172.21.15.156:0/411957260 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff94402b030 con 0x7ff95415a760 2023-11-28T13:08:00.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.666+0000 7ff94a7fc700 1 --2- 172.21.15.156:0/411957260 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff9380682f0 0x7ff93806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:00.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.666+0000 7ff94a7fc700 1 -- 172.21.15.156:0/411957260 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff944086c30 con 0x7ff95415a760 2023-11-28T13:08:00.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.666+0000 7ff959d9b700 1 --2- 172.21.15.156:0/411957260 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff9380682f0 0x7ff93806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:00.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.667+0000 7ff959d9b700 1 --2- 172.21.15.156:0/411957260 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff9380682f0 0x7ff93806a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff9500011e0 tx=0x7ff950009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:00.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.670+0000 7ff94a7fc700 1 -- 172.21.15.156:0/411957260 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff94408d080 con 0x7ff95415a760 2023-11-28T13:08:00.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:00.863+0000 7ff95bfff700 1 -- 172.21.15.156:0/411957260 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff95404e670 con 0x7ff95415a760 2023-11-28T13:08:01.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.555+0000 7ff94a7fc700 1 -- 172.21.15.156:0/411957260 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff944029020 con 0x7ff95415a760 2023-11-28T13:08:01.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.560+0000 7ff95bfff700 1 -- 172.21.15.156:0/411957260 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff9380682f0 msgr2=0x7ff93806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:01.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.560+0000 7ff95bfff700 1 --2- 172.21.15.156:0/411957260 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff9380682f0 0x7ff93806a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff9500011e0 tx=0x7ff950009040 comp rx=0 tx=0).stop 2023-11-28T13:08:01.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.560+0000 7ff95bfff700 1 -- 172.21.15.156:0/411957260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff95415a760 msgr2=0x7ff9541ef7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:01.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.560+0000 7ff95bfff700 1 --2- 172.21.15.156:0/411957260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff95415a760 0x7ff9541ef7e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff9440090b0 tx=0x7ff944004750 comp rx=0 tx=0).stop 2023-11-28T13:08:01.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.561+0000 7ff95bfff700 1 -- 172.21.15.156:0/411957260 shutdown_connections 2023-11-28T13:08:01.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.561+0000 7ff95bfff700 1 --2- 172.21.15.156:0/411957260 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff9380682f0 0x7ff93806a7a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:01.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.561+0000 7ff95bfff700 1 --2- 172.21.15.156:0/411957260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff95415dc30 0x7ff954158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:01.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.561+0000 7ff95bfff700 1 --2- 172.21.15.156:0/411957260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9540cded0 0x7ff9541f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:01.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.561+0000 7ff95bfff700 1 --2- 172.21.15.156:0/411957260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff95415a760 0x7ff9541ef7e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:01.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.561+0000 7ff95bfff700 1 -- 172.21.15.156:0/411957260 >> 172.21.15.156:0/411957260 conn(0x7ff954072620 msgr2=0x7ff95406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:01.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.562+0000 7ff95bfff700 1 -- 172.21.15.156:0/411957260 shutdown_connections 2023-11-28T13:08:01.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.562+0000 7ff95bfff700 1 -- 172.21.15.156:0/411957260 wait complete. 2023-11-28T13:08:01.566 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_metadata' created 2023-11-28T13:08:01.585 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T13:08:01.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.789+0000 7f3f1b1ff700 1 Processor -- start 2023-11-28T13:08:01.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.789+0000 7f3f1b1ff700 1 -- start start 2023-11-28T13:08:01.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.790+0000 7f3f1b1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f140d01b0 0x7f3f140d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:01.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.790+0000 7f3f1b1ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3f140b20b0 0x7f3f140d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:01.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.790+0000 7f3f1b1ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f140b0230 con 0x7f3f140b29b0 2023-11-28T13:08:01.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.791+0000 7f3f1b1ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1415cec0 con 0x7f3f140b20b0 2023-11-28T13:08:01.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.791+0000 7f3f1b1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f140b4570 con 0x7f3f140d01b0 2023-11-28T13:08:01.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.791+0000 7f3f18f9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f140d01b0 0x7f3f140d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:01.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.791+0000 7f3f13fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3f140b20b0 0x7f3f140d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:01.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.791+0000 7f3f1979c700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f3f140b29b0 0x7f3f140b2e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:47662/0 (socket says 172.21.15.156:47662) 2023-11-28T13:08:01.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.791+0000 7f3f1979c700 1 -- 172.21.15.156:0/977088411 learned_addr learned my addr 172.21.15.156:0/977088411 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:01.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.792+0000 7f3f13fff700 1 -- 172.21.15.156:0/977088411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f140d01b0 msgr2=0x7f3f140d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:01.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.792+0000 7f3f13fff700 1 --2- 172.21.15.156:0/977088411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f140d01b0 0x7f3f140d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:01.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.792+0000 7f3f13fff700 1 -- 172.21.15.156:0/977088411 >> v1:172.21.15.186:6789/0 conn(0x7f3f140b29b0 legacy=0x7f3f140b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:01.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.792+0000 7f3f13fff700 1 -- 172.21.15.156:0/977088411 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f140b4850 con 0x7f3f140b20b0 2023-11-28T13:08:01.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.792+0000 7f3f18f9b700 1 --2- 172.21.15.156:0/977088411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f140d01b0 0x7f3f140d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:01.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.792+0000 7f3f13fff700 1 --2- 172.21.15.156:0/977088411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f140b20b0 0x7f3f140d1160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009840 tx=0x7f3f00009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7204e12774cd1e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:01.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.792+0000 7f3f137fe700 1 -- 172.21.15.156:0/977088411 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00011830 con 0x7f3f140b20b0 2023-11-28T13:08:01.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.792+0000 7f3f137fe700 1 -- 172.21.15.156:0/977088411 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f00011990 con 0x7f3f140b20b0 2023-11-28T13:08:01.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.792+0000 7f3f137fe700 1 -- 172.21.15.156:0/977088411 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00011d20 con 0x7f3f140b20b0 2023-11-28T13:08:01.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.793+0000 7f3f1b1ff700 1 -- 172.21.15.156:0/977088411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f140b20b0 msgr2=0x7f3f140d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:01.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.793+0000 7f3f1b1ff700 1 --2- 172.21.15.156:0/977088411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f140b20b0 0x7f3f140d1160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009840 tx=0x7f3f00009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:01.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.793+0000 7f3f1b1ff700 1 -- 172.21.15.156:0/977088411 shutdown_connections 2023-11-28T13:08:01.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.793+0000 7f3f1b1ff700 1 --2- 172.21.15.156:0/977088411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f140d01b0 0x7f3f140d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:01.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.793+0000 7f3f1b1ff700 1 --2- 172.21.15.156:0/977088411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f140b20b0 0x7f3f140d1160 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:01.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.793+0000 7f3f1b1ff700 1 -- 172.21.15.156:0/977088411 >> 172.21.15.156:0/977088411 conn(0x7f3f1409c3f0 msgr2=0x7f3f140d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:01.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.793+0000 7f3f1b1ff700 1 -- 172.21.15.156:0/977088411 shutdown_connections 2023-11-28T13:08:01.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.793+0000 7f3f1b1ff700 1 -- 172.21.15.156:0/977088411 wait complete. 2023-11-28T13:08:01.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.795+0000 7f3f1b1ff700 1 Processor -- start 2023-11-28T13:08:01.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.795+0000 7f3f1b1ff700 1 -- start start 2023-11-28T13:08:01.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.796+0000 7f3f1b1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f140b20b0 0x7f3f1415fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:01.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.796+0000 7f3f1b1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f140b29b0 0x7f3f141ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:01.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.796+0000 7f3f1b1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f140d01b0 0x7f3f141f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:01.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.796+0000 7f3f1b1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f140cd410 con 0x7f3f140d01b0 2023-11-28T13:08:01.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.796+0000 7f3f1b1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1415ff90 con 0x7f3f140b20b0 2023-11-28T13:08:01.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.796+0000 7f3f1b1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f140cc890 con 0x7f3f140b29b0 2023-11-28T13:08:01.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.796+0000 7f3f1979c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f140d01b0 0x7f3f141f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:01.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.796+0000 7f3f1979c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f140d01b0 0x7f3f141f5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43544/0 (socket says 172.21.15.156:43544) 2023-11-28T13:08:01.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.796+0000 7f3f1979c700 1 -- 172.21.15.156:0/2208071767 learned_addr learned my addr 172.21.15.156:0/2208071767 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:01.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.796+0000 7f3f18f9b700 1 --2- 172.21.15.156:0/2208071767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f140b20b0 0x7f3f1415fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:01.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.796+0000 7f3f13fff700 1 --2- 172.21.15.156:0/2208071767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f140b29b0 0x7f3f141ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:01.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.796+0000 7f3f1979c700 1 -- 172.21.15.156:0/2208071767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f140b29b0 msgr2=0x7f3f141ef8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:01.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.796+0000 7f3f1979c700 1 --2- 172.21.15.156:0/2208071767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f140b29b0 0x7f3f141ef8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:01.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.796+0000 7f3f1979c700 1 -- 172.21.15.156:0/2208071767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f140b20b0 msgr2=0x7f3f1415fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:01.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.796+0000 7f3f1979c700 1 --2- 172.21.15.156:0/2208071767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f140b20b0 0x7f3f1415fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:01.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.796+0000 7f3f1979c700 1 -- 172.21.15.156:0/2208071767 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f080094a0 con 0x7f3f140d01b0 2023-11-28T13:08:01.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.797+0000 7f3f1979c700 1 --2- 172.21.15.156:0/2208071767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f140d01b0 0x7f3f141f5ac0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00cbc0 tx=0x7f3f0c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:01.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.797+0000 7f3f117fa700 1 -- 172.21.15.156:0/2208071767 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c00a9e0 con 0x7f3f140d01b0 2023-11-28T13:08:01.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.797+0000 7f3f117fa700 1 -- 172.21.15.156:0/2208071767 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0c00ab40 con 0x7f3f140d01b0 2023-11-28T13:08:01.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.797+0000 7f3f117fa700 1 -- 172.21.15.156:0/2208071767 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c012400 con 0x7f3f140d01b0 2023-11-28T13:08:01.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.797+0000 7f3f1b1ff700 1 -- 172.21.15.156:0/2208071767 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f000094a0 con 0x7f3f140d01b0 2023-11-28T13:08:01.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.797+0000 7f3f1b1ff700 1 -- 172.21.15.156:0/2208071767 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f140afec0 con 0x7f3f140d01b0 2023-11-28T13:08:01.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.798+0000 7f3f1b1ff700 1 -- 172.21.15.156:0/2208071767 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f140afbe0 con 0x7f3f140d01b0 2023-11-28T13:08:01.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.799+0000 7f3f117fa700 1 -- 172.21.15.156:0/2208071767 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3f0c019030 con 0x7f3f140d01b0 2023-11-28T13:08:01.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.800+0000 7f3f117fa700 1 --2- 172.21.15.156:0/2208071767 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3ef80682f0 0x7f3ef806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:01.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.800+0000 7f3f117fa700 1 -- 172.21.15.156:0/2208071767 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f3f0c085420 con 0x7f3f140d01b0 2023-11-28T13:08:01.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.801+0000 7f3f18f9b700 1 --2- 172.21.15.156:0/2208071767 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3ef80682f0 0x7f3ef806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:01.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.802+0000 7f3f18f9b700 1 --2- 172.21.15.156:0/2208071767 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3ef80682f0 0x7f3ef806a7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3f140b7d20 tx=0x7f3f08009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:01.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.805+0000 7f3f117fa700 1 -- 172.21.15.156:0/2208071767 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3f0c04b5e0 con 0x7f3f140d01b0 2023-11-28T13:08:01.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:01.997+0000 7f3f1b1ff700 1 -- 172.21.15.156:0/2208071767 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3f140b2600 con 0x7f3f140d01b0 2023-11-28T13:08:02.558 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.557+0000 7f3f117fa700 1 -- 172.21.15.156:0/2208071767 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3f0c051f10 con 0x7f3f140d01b0 2023-11-28T13:08:02.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.567+0000 7f3f1b1ff700 1 -- 172.21.15.156:0/2208071767 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3ef80682f0 msgr2=0x7f3ef806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:02.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.567+0000 7f3f1b1ff700 1 --2- 172.21.15.156:0/2208071767 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3ef80682f0 0x7f3ef806a7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3f140b7d20 tx=0x7f3f08009210 comp rx=0 tx=0).stop 2023-11-28T13:08:02.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.567+0000 7f3f1b1ff700 1 -- 172.21.15.156:0/2208071767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f140d01b0 msgr2=0x7f3f141f5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:02.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.567+0000 7f3f1b1ff700 1 --2- 172.21.15.156:0/2208071767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f140d01b0 0x7f3f141f5ac0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00cbc0 tx=0x7f3f0c00ced0 comp rx=0 tx=0).stop 2023-11-28T13:08:02.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.571+0000 7f3f1b1ff700 1 -- 172.21.15.156:0/2208071767 shutdown_connections 2023-11-28T13:08:02.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.571+0000 7f3f1b1ff700 1 --2- 172.21.15.156:0/2208071767 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3ef80682f0 0x7f3ef806a7a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:02.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.571+0000 7f3f1b1ff700 1 --2- 172.21.15.156:0/2208071767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f140d01b0 0x7f3f141f5ac0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:02.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.571+0000 7f3f1b1ff700 1 --2- 172.21.15.156:0/2208071767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f140b20b0 0x7f3f1415fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:02.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.571+0000 7f3f1b1ff700 1 --2- 172.21.15.156:0/2208071767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f140b29b0 0x7f3f141ef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:02.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.571+0000 7f3f1b1ff700 1 -- 172.21.15.156:0/2208071767 >> 172.21.15.156:0/2208071767 conn(0x7f3f1409c3f0 msgr2=0x7f3f1406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:02.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.573+0000 7f3f1b1ff700 1 -- 172.21.15.156:0/2208071767 shutdown_connections 2023-11-28T13:08:02.574 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.573+0000 7f3f1b1ff700 1 -- 172.21.15.156:0/2208071767 wait complete. 2023-11-28T13:08:02.579 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_data' created 2023-11-28T13:08:02.600 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T13:08:02.646 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:08:02.651 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:08:02.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.804+0000 7fb5323ff700 1 Processor -- start 2023-11-28T13:08:02.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.804+0000 7fb5323ff700 1 -- start start 2023-11-28T13:08:02.807 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.806+0000 7fb5323ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb52c161340 0x7fb52c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:02.807 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.806+0000 7fb5323ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb52c0d2320 0x7fb52c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:02.807 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.806+0000 7fb5323ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52c0d1950 con 0x7fb52c15c9b0 2023-11-28T13:08:02.807 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.806+0000 7fb5323ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52c0b5800 con 0x7fb52c0d2320 2023-11-28T13:08:02.807 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.806+0000 7fb5323ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0b5300 con 0x7fb52c161340 2023-11-28T13:08:02.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.806+0000 7fb52bfff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb52c15c9b0 0x7fb52c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:45632/0 (socket says 172.21.15.156:45632) 2023-11-28T13:08:02.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.806+0000 7fb52bfff700 1 -- 172.21.15.156:0/1069246795 learned_addr learned my addr 172.21.15.156:0/1069246795 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:02.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.806+0000 7fb52b7fe700 1 --2- 172.21.15.156:0/1069246795 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb52c161340 0x7fb52c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:02.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.806+0000 7fb53099c700 1 --2- 172.21.15.156:0/1069246795 >> v2:172.21.15.186:3300/0 conn(0x7fb52c0d2320 0x7fb52c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:02.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.806+0000 7fb52affd700 1 -- 172.21.15.156:0/1069246795 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551340908 0 0) 0x7fb52c0d1950 con 0x7fb52c15c9b0 2023-11-28T13:08:02.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.807+0000 7fb52affd700 1 -- 172.21.15.156:0/1069246795 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5140035b0 con 0x7fb52c15c9b0 2023-11-28T13:08:02.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.807+0000 7fb52b7fe700 1 -- 172.21.15.156:0/1069246795 >> v2:172.21.15.186:3300/0 conn(0x7fb52c0d2320 msgr2=0x7fb52c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:02.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.807+0000 7fb52b7fe700 1 --2- 172.21.15.156:0/1069246795 >> v2:172.21.15.186:3300/0 conn(0x7fb52c0d2320 0x7fb52c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:02.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.807+0000 7fb52b7fe700 1 -- 172.21.15.156:0/1069246795 >> v1:172.21.15.156:6789/0 conn(0x7fb52c15c9b0 legacy=0x7fb52c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:02.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.807+0000 7fb52b7fe700 1 -- 172.21.15.156:0/1069246795 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c0b5520 con 0x7fb52c161340 2023-11-28T13:08:02.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.807+0000 7fb53099c700 1 --2- 172.21.15.156:0/1069246795 >> v2:172.21.15.186:3300/0 conn(0x7fb52c0d2320 0x7fb52c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:02.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.807+0000 7fb52b7fe700 1 --2- 172.21.15.156:0/1069246795 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb52c161340 0x7fb52c0d1de0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb520009840 tx=0x7fb520009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8dbe2ccd4a0ef02 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:02.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.807+0000 7fb52affd700 1 -- 172.21.15.156:0/1069246795 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520011830 con 0x7fb52c161340 2023-11-28T13:08:02.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.807+0000 7fb52affd700 1 -- 172.21.15.156:0/1069246795 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb520011990 con 0x7fb52c161340 2023-11-28T13:08:02.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.807+0000 7fb52affd700 1 -- 172.21.15.156:0/1069246795 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520011d00 con 0x7fb52c161340 2023-11-28T13:08:02.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.808+0000 7fb5323ff700 1 -- 172.21.15.156:0/1069246795 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb52c161340 msgr2=0x7fb52c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:02.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.808+0000 7fb5323ff700 1 --2- 172.21.15.156:0/1069246795 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb52c161340 0x7fb52c0d1de0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb520009840 tx=0x7fb520009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:02.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.808+0000 7fb5323ff700 1 -- 172.21.15.156:0/1069246795 shutdown_connections 2023-11-28T13:08:02.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.808+0000 7fb5323ff700 1 --2- 172.21.15.156:0/1069246795 >> v2:172.21.15.186:3300/0 conn(0x7fb52c0d2320 0x7fb52c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:02.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.808+0000 7fb5323ff700 1 --2- 172.21.15.156:0/1069246795 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb52c161340 0x7fb52c0d1de0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:02.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.808+0000 7fb5323ff700 1 -- 172.21.15.156:0/1069246795 >> 172.21.15.156:0/1069246795 conn(0x7fb52c072600 msgr2=0x7fb52c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:02.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.808+0000 7fb5323ff700 1 -- 172.21.15.156:0/1069246795 shutdown_connections 2023-11-28T13:08:02.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.808+0000 7fb5323ff700 1 -- 172.21.15.156:0/1069246795 wait complete. 2023-11-28T13:08:02.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.810+0000 7fb5323ff700 1 Processor -- start 2023-11-28T13:08:02.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.811+0000 7fb5323ff700 1 -- start start 2023-11-28T13:08:02.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.812+0000 7fb5323ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb52c0d2320 0x7fb52c0c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:02.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.812+0000 7fb5323ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb52c15c9b0 0x7fb52c1e6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:02.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.812+0000 7fb5323ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c161340 0x7fb52c0c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:02.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.812+0000 7fb5323ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c1e77f0 con 0x7fb52c0d2320 2023-11-28T13:08:02.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.812+0000 7fb5323ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c1e6980 con 0x7fb52c161340 2023-11-28T13:08:02.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.812+0000 7fb5323ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c1e6b70 con 0x7fb52c15c9b0 2023-11-28T13:08:02.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.812+0000 7fb52bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb52c0d2320 0x7fb52c0c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:02.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.812+0000 7fb52bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb52c0d2320 0x7fb52c0c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43548/0 (socket says 172.21.15.156:43548) 2023-11-28T13:08:02.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.812+0000 7fb52bfff700 1 -- 172.21.15.156:0/1091672694 learned_addr learned my addr 172.21.15.156:0/1091672694 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:02.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.812+0000 7fb52bfff700 1 -- 172.21.15.156:0/1091672694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb52c15c9b0 msgr2=0x7fb52c1e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:02.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.812+0000 7fb52bfff700 1 --2- 172.21.15.156:0/1091672694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb52c15c9b0 0x7fb52c1e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:02.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.812+0000 7fb52bfff700 1 -- 172.21.15.156:0/1091672694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c161340 msgr2=0x7fb52c0c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:02.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.812+0000 7fb52bfff700 1 --2- 172.21.15.156:0/1091672694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c161340 0x7fb52c0c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:02.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.812+0000 7fb52bfff700 1 -- 172.21.15.156:0/1091672694 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb524009a70 con 0x7fb52c0d2320 2023-11-28T13:08:02.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.813+0000 7fb52bfff700 1 --2- 172.21.15.156:0/1091672694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb52c0d2320 0x7fb52c0c78b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00c450 tx=0x7fb51c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:02.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.813+0000 7fb528ff9700 1 -- 172.21.15.156:0/1091672694 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0011b0 con 0x7fb52c0d2320 2023-11-28T13:08:02.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.813+0000 7fb528ff9700 1 -- 172.21.15.156:0/1091672694 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c005d70 con 0x7fb52c0d2320 2023-11-28T13:08:02.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.813+0000 7fb528ff9700 1 -- 172.21.15.156:0/1091672694 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0072d0 con 0x7fb52c0d2320 2023-11-28T13:08:02.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.813+0000 7fb5323ff700 1 -- 172.21.15.156:0/1091672694 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5200094a0 con 0x7fb52c0d2320 2023-11-28T13:08:02.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.813+0000 7fb5323ff700 1 -- 172.21.15.156:0/1091672694 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c0d15e0 con 0x7fb52c0d2320 2023-11-28T13:08:02.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.814+0000 7fb5323ff700 1 -- 172.21.15.156:0/1091672694 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c0d1300 con 0x7fb52c0d2320 2023-11-28T13:08:02.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.815+0000 7fb528ff9700 1 -- 172.21.15.156:0/1091672694 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb51c019030 con 0x7fb52c0d2320 2023-11-28T13:08:02.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.816+0000 7fb528ff9700 1 --2- 172.21.15.156:0/1091672694 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb5180682f0 0x7fb51806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:02.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.816+0000 7fb528ff9700 1 -- 172.21.15.156:0/1091672694 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fb51c084dd0 con 0x7fb52c0d2320 2023-11-28T13:08:02.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.816+0000 7fb52b7fe700 1 --2- 172.21.15.156:0/1091672694 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb5180682f0 0x7fb51806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:02.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.817+0000 7fb52b7fe700 1 --2- 172.21.15.156:0/1091672694 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb5180682f0 0x7fb51806a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0c80b0 tx=0x7fb520010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:02.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:02.820+0000 7fb528ff9700 1 -- 172.21.15.156:0/1091672694 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb51c04b5e0 con 0x7fb52c0d2320 2023-11-28T13:08:03.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.057+0000 7fb5323ff700 1 -- 172.21.15.156:0/1091672694 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb52c15cd80 con 0x7fb52c0d2320 2023-11-28T13:08:03.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.063+0000 7fb528ff9700 1 -- 172.21.15.156:0/1091672694 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb51c017020 con 0x7fb52c0d2320 2023-11-28T13:08:03.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.066+0000 7fb5323ff700 1 -- 172.21.15.156:0/1091672694 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb5180682f0 msgr2=0x7fb51806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:03.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.066+0000 7fb5323ff700 1 --2- 172.21.15.156:0/1091672694 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb5180682f0 0x7fb51806a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0c80b0 tx=0x7fb520010000 comp rx=0 tx=0).stop 2023-11-28T13:08:03.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.066+0000 7fb5323ff700 1 -- 172.21.15.156:0/1091672694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb52c0d2320 msgr2=0x7fb52c0c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:03.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.066+0000 7fb5323ff700 1 --2- 172.21.15.156:0/1091672694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb52c0d2320 0x7fb52c0c78b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00c450 tx=0x7fb51c00c810 comp rx=0 tx=0).stop 2023-11-28T13:08:03.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.068+0000 7fb5323ff700 1 -- 172.21.15.156:0/1091672694 shutdown_connections 2023-11-28T13:08:03.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.068+0000 7fb5323ff700 1 --2- 172.21.15.156:0/1091672694 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb5180682f0 0x7fb51806a7a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.068+0000 7fb5323ff700 1 --2- 172.21.15.156:0/1091672694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c161340 0x7fb52c0c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.068+0000 7fb5323ff700 1 --2- 172.21.15.156:0/1091672694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb52c0d2320 0x7fb52c0c78b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.068+0000 7fb5323ff700 1 --2- 172.21.15.156:0/1091672694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb52c15c9b0 0x7fb52c1e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.068+0000 7fb5323ff700 1 -- 172.21.15.156:0/1091672694 >> 172.21.15.156:0/1091672694 conn(0x7fb52c072600 msgr2=0x7fb52c06b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:03.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.069+0000 7fb5323ff700 1 -- 172.21.15.156:0/1091672694 shutdown_connections 2023-11-28T13:08:03.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.069+0000 7fb5323ff700 1 -- 172.21.15.156:0/1091672694 wait complete. 2023-11-28T13:08:03.075 INFO:teuthology.orchestra.run.smithi156.stderr:new fs with metadata pool 3 and data pool 4 2023-11-28T13:08:03.103 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:08:03.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.298+0000 7fc6e7dff700 1 Processor -- start 2023-11-28T13:08:03.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.298+0000 7fc6e7dff700 1 -- start start 2023-11-28T13:08:03.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.299+0000 7fc6e7dff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc6e0161340 0x7fc6e00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:03.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.299+0000 7fc6e7dff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e00d1950 con 0x7fc6e00d2320 2023-11-28T13:08:03.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.299+0000 7fc6e7dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e00b5520 con 0x7fc6e015c9b0 2023-11-28T13:08:03.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.299+0000 7fc6e7dff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e00b5800 con 0x7fc6e0161340 2023-11-28T13:08:03.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.299+0000 7fc6e539a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc6e0161340 0x7fc6e00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:03.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.299+0000 7fc6e539a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc6e0161340 0x7fc6e00d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43558/0 (socket says 172.21.15.156:43558) 2023-11-28T13:08:03.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.299+0000 7fc6e539a700 1 -- 172.21.15.156:0/847423363 learned_addr learned my addr 172.21.15.156:0/847423363 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:03.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.300+0000 7fc6e539a700 1 -- 172.21.15.156:0/847423363 >> v1:172.21.15.186:6789/0 conn(0x7fc6e015c9b0 legacy=0x7fc6e00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:03.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.300+0000 7fc6e539a700 1 -- 172.21.15.156:0/847423363 >> v1:172.21.15.156:6789/0 conn(0x7fc6e00d2320 legacy=0x7fc6e00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:03.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.300+0000 7fc6e4b99700 1 -- 172.21.15.156:0/847423363 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415579315 0 0) 0x7fc6e00b5520 con 0x7fc6e015c9b0 2023-11-28T13:08:03.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.300+0000 7fc6e539a700 1 -- 172.21.15.156:0/847423363 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e00d1300 con 0x7fc6e0161340 2023-11-28T13:08:03.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.300+0000 7fc6e539a700 1 --2- 172.21.15.156:0/847423363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6e0161340 0x7fc6e00d1de0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc6d00098f0 tx=0x7fc6d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8709e59399592d32 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:03.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.300+0000 7fc6e4b99700 1 -- 172.21.15.156:0/847423363 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0004230 con 0x7fc6e0161340 2023-11-28T13:08:03.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.300+0000 7fc6e4b99700 1 -- 172.21.15.156:0/847423363 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d0004390 con 0x7fc6e0161340 2023-11-28T13:08:03.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.300+0000 7fc6e4b99700 1 -- 172.21.15.156:0/847423363 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d00036a0 con 0x7fc6e0161340 2023-11-28T13:08:03.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.301+0000 7fc6e7dff700 1 -- 172.21.15.156:0/847423363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6e0161340 msgr2=0x7fc6e00d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:03.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.301+0000 7fc6e7dff700 1 --2- 172.21.15.156:0/847423363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6e0161340 0x7fc6e00d1de0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc6d00098f0 tx=0x7fc6d0009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:03.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.301+0000 7fc6e7dff700 1 -- 172.21.15.156:0/847423363 shutdown_connections 2023-11-28T13:08:03.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.301+0000 7fc6e7dff700 1 --2- 172.21.15.156:0/847423363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6e0161340 0x7fc6e00d1de0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.301+0000 7fc6e7dff700 1 -- 172.21.15.156:0/847423363 >> 172.21.15.156:0/847423363 conn(0x7fc6e0072600 msgr2=0x7fc6e00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:03.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.301+0000 7fc6e7dff700 1 -- 172.21.15.156:0/847423363 shutdown_connections 2023-11-28T13:08:03.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.301+0000 7fc6e7dff700 1 -- 172.21.15.156:0/847423363 wait complete. 2023-11-28T13:08:03.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.303+0000 7fc6e7dff700 1 Processor -- start 2023-11-28T13:08:03.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.303+0000 7fc6e7dff700 1 -- start start 2023-11-28T13:08:03.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.304+0000 7fc6e7dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6e00d2320 0x7fc6e0154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:03.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.304+0000 7fc6e7dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6e015c9b0 0x7fc6e01e7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:03.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.304+0000 7fc6e7dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6e0161340 0x7fc6e01e8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:03.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.304+0000 7fc6e7dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e01f26a0 con 0x7fc6e0161340 2023-11-28T13:08:03.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.304+0000 7fc6e7dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e01f5c70 con 0x7fc6e015c9b0 2023-11-28T13:08:03.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.304+0000 7fc6e7dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e01f6ad0 con 0x7fc6e00d2320 2023-11-28T13:08:03.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.304+0000 7fc6e639c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6e0161340 0x7fc6e01e8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:03.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.304+0000 7fc6e639c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6e0161340 0x7fc6e01e8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43562/0 (socket says 172.21.15.156:43562) 2023-11-28T13:08:03.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.304+0000 7fc6e639c700 1 -- 172.21.15.156:0/1410336455 learned_addr learned my addr 172.21.15.156:0/1410336455 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:03.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.304+0000 7fc6e5b9b700 1 --2- 172.21.15.156:0/1410336455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6e00d2320 0x7fc6e0154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.304+0000 7fc6e539a700 1 --2- 172.21.15.156:0/1410336455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6e015c9b0 0x7fc6e01e7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.304+0000 7fc6e639c700 1 -- 172.21.15.156:0/1410336455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6e00d2320 msgr2=0x7fc6e0154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.304+0000 7fc6e639c700 1 --2- 172.21.15.156:0/1410336455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6e00d2320 0x7fc6e0154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.304+0000 7fc6e639c700 1 -- 172.21.15.156:0/1410336455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6e015c9b0 msgr2=0x7fc6e01e7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.305+0000 7fc6e639c700 1 --2- 172.21.15.156:0/1410336455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6e015c9b0 0x7fc6e01e7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.305+0000 7fc6e639c700 1 -- 172.21.15.156:0/1410336455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d0009550 con 0x7fc6e0161340 2023-11-28T13:08:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.305+0000 7fc6e539a700 1 --2- 172.21.15.156:0/1410336455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6e015c9b0 0x7fc6e01e7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.305+0000 7fc6e639c700 1 --2- 172.21.15.156:0/1410336455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6e0161340 0x7fc6e01e8240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c880 tx=0x7fc6d800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.305+0000 7fc6d67fc700 1 -- 172.21.15.156:0/1410336455 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8003e30 con 0x7fc6e0161340 2023-11-28T13:08:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.305+0000 7fc6d67fc700 1 -- 172.21.15.156:0/1410336455 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d800ce80 con 0x7fc6e0161340 2023-11-28T13:08:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.305+0000 7fc6d67fc700 1 -- 172.21.15.156:0/1410336455 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d80052b0 con 0x7fc6e0161340 2023-11-28T13:08:03.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.305+0000 7fc6e7dff700 1 -- 172.21.15.156:0/1410336455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e015cd80 con 0x7fc6e0161340 2023-11-28T13:08:03.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.305+0000 7fc6e7dff700 1 -- 172.21.15.156:0/1410336455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e00d15e0 con 0x7fc6e0161340 2023-11-28T13:08:03.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.306+0000 7fc6e7dff700 1 -- 172.21.15.156:0/1410336455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e015cf10 con 0x7fc6e0161340 2023-11-28T13:08:03.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.307+0000 7fc6d67fc700 1 -- 172.21.15.156:0/1410336455 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc6d8010070 con 0x7fc6e0161340 2023-11-28T13:08:03.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.308+0000 7fc6d67fc700 1 --2- 172.21.15.156:0/1410336455 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6c40682f0 0x7fc6c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:03.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.308+0000 7fc6d67fc700 1 -- 172.21.15.156:0/1410336455 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc6d8084f00 con 0x7fc6e0161340 2023-11-28T13:08:03.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.308+0000 7fc6e5b9b700 1 --2- 172.21.15.156:0/1410336455 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6c40682f0 0x7fc6c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:03.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.309+0000 7fc6e5b9b700 1 --2- 172.21.15.156:0/1410336455 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6c40682f0 0x7fc6c406a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc000f60 tx=0x7fc6dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:03.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.312+0000 7fc6d67fc700 1 -- 172.21.15.156:0/1410336455 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc6d8051670 con 0x7fc6e0161340 2023-11-28T13:08:03.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.506+0000 7fc6e7dff700 1 -- 172.21.15.156:0/1410336455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6e00745c0 con 0x7fc6e0161340 2023-11-28T13:08:03.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.507+0000 7fc6d67fc700 1 -- 172.21.15.156:0/1410336455 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+17910 (secure 0 0 0) 0x7fc6d8017090 con 0x7fc6e0161340 2023-11-28T13:08:03.508 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:03.508 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":20,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:08:03.057729+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:08:00.903485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:08:02.037854+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:08:03.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.509+0000 7fc6e7dff700 1 -- 172.21.15.156:0/1410336455 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6c40682f0 msgr2=0x7fc6c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:03.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.509+0000 7fc6e7dff700 1 --2- 172.21.15.156:0/1410336455 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6c40682f0 0x7fc6c406a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc000f60 tx=0x7fc6dc009040 comp rx=0 tx=0).stop 2023-11-28T13:08:03.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.509+0000 7fc6e7dff700 1 -- 172.21.15.156:0/1410336455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6e0161340 msgr2=0x7fc6e01e8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:03.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.509+0000 7fc6e7dff700 1 --2- 172.21.15.156:0/1410336455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6e0161340 0x7fc6e01e8240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c880 tx=0x7fc6d800cc40 comp rx=0 tx=0).stop 2023-11-28T13:08:03.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.510+0000 7fc6e7dff700 1 -- 172.21.15.156:0/1410336455 shutdown_connections 2023-11-28T13:08:03.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.510+0000 7fc6e7dff700 1 --2- 172.21.15.156:0/1410336455 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6c40682f0 0x7fc6c406a7a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.510+0000 7fc6e7dff700 1 --2- 172.21.15.156:0/1410336455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6e0161340 0x7fc6e01e8240 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.510+0000 7fc6e7dff700 1 --2- 172.21.15.156:0/1410336455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6e00d2320 0x7fc6e0154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.510+0000 7fc6e7dff700 1 --2- 172.21.15.156:0/1410336455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6e015c9b0 0x7fc6e01e7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.510+0000 7fc6e7dff700 1 -- 172.21.15.156:0/1410336455 >> 172.21.15.156:0/1410336455 conn(0x7fc6e0072600 msgr2=0x7fc6e00b6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:03.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.510+0000 7fc6e7dff700 1 -- 172.21.15.156:0/1410336455 shutdown_connections 2023-11-28T13:08:03.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.510+0000 7fc6e7dff700 1 -- 172.21.15.156:0/1410336455 wait complete. 2023-11-28T13:08:03.529 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:08:03.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.725+0000 7f39fddff700 1 Processor -- start 2023-11-28T13:08:03.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.725+0000 7f39fddff700 1 -- start start 2023-11-28T13:08:03.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.726+0000 7f39fddff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f39f8161340 0x7f39f80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:03.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.726+0000 7f39fddff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f80d1950 con 0x7f39f80d2320 2023-11-28T13:08:03.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.726+0000 7f39fddff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f80b5520 con 0x7f39f815c9b0 2023-11-28T13:08:03.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.726+0000 7f39fddff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f80b5800 con 0x7f39f8161340 2023-11-28T13:08:03.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.727+0000 7f39f7fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f39f80d2320 0x7f39f80cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:45650/0 (socket says 172.21.15.156:45650) 2023-11-28T13:08:03.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.727+0000 7f39f7fff700 1 -- 172.21.15.156:0/2778656168 learned_addr learned my addr 172.21.15.156:0/2778656168 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:03.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.727+0000 7f39f6ffd700 1 --2- 172.21.15.156:0/2778656168 >> v2:172.21.15.186:3300/0 conn(0x7f39f8161340 0x7f39f80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:03.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.727+0000 7f39f67fc700 1 -- 172.21.15.156:0/2778656168 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341225049 0 0) 0x7f39f80d1950 con 0x7f39f80d2320 2023-11-28T13:08:03.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.727+0000 7f39f67fc700 1 -- 172.21.15.156:0/2778656168 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39e00035b0 con 0x7f39f80d2320 2023-11-28T13:08:03.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.727+0000 7f39f67fc700 1 -- 172.21.15.156:0/2778656168 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287351626 0 0) 0x7f39f80b5520 con 0x7f39f815c9b0 2023-11-28T13:08:03.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.727+0000 7f39f67fc700 1 -- 172.21.15.156:0/2778656168 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39f80d1950 con 0x7f39f815c9b0 2023-11-28T13:08:03.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.727+0000 7f39f67fc700 1 -- 172.21.15.156:0/2778656168 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3070885469 0 0) 0x7f39e00035b0 con 0x7f39f80d2320 2023-11-28T13:08:03.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.727+0000 7f39f67fc700 1 -- 172.21.15.156:0/2778656168 >> v2:172.21.15.186:3300/0 conn(0x7f39f8161340 msgr2=0x7f39f80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:03.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.727+0000 7f39f67fc700 1 --2- 172.21.15.156:0/2778656168 >> v2:172.21.15.186:3300/0 conn(0x7f39f8161340 0x7f39f80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.727+0000 7f39f67fc700 1 -- 172.21.15.156:0/2778656168 >> v1:172.21.15.186:6789/0 conn(0x7f39f815c9b0 legacy=0x7f39f80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:03.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.727+0000 7f39f67fc700 1 -- 172.21.15.156:0/2778656168 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f80d1300 con 0x7f39f80d2320 2023-11-28T13:08:03.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.727+0000 7f39f67fc700 1 -- 172.21.15.156:0/2778656168 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f39f00019f0 con 0x7f39f80d2320 2023-11-28T13:08:03.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.727+0000 7f39f67fc700 1 -- 172.21.15.156:0/2778656168 >> v1:172.21.15.156:6789/0 conn(0x7f39f80d2320 legacy=0x7f39f80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:03.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.727+0000 7f39f67fc700 1 --2- 172.21.15.156:0/2778656168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39e0003b90 0x7f39e0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:03.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.727+0000 7f39f67fc700 1 --2- 172.21.15.156:0/2778656168 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e0004850 0x7f39e0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:03.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.727+0000 7f39f67fc700 1 --2- 172.21.15.156:0/2778656168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39e0005200 0x7f39e000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:03.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.728+0000 7f39f67fc700 1 -- 172.21.15.156:0/2778656168 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f80b5800 con 0x7f39e0005200 2023-11-28T13:08:03.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.728+0000 7f39f67fc700 1 -- 172.21.15.156:0/2778656168 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e000c110 con 0x7f39e0003b90 2023-11-28T13:08:03.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.728+0000 7f39f67fc700 1 -- 172.21.15.156:0/2778656168 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e000c2d0 con 0x7f39e0004850 2023-11-28T13:08:03.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.728+0000 7f39f77fe700 1 --2- 172.21.15.156:0/2778656168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39e0003b90 0x7f39e0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:03.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.728+0000 7f39f7fff700 1 --2- 172.21.15.156:0/2778656168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39e0005200 0x7f39e000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:03.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.728+0000 7f39f6ffd700 1 --2- 172.21.15.156:0/2778656168 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e0004850 0x7f39e0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:03.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.728+0000 7f39f7fff700 1 -- 172.21.15.156:0/2778656168 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e0004850 msgr2=0x7f39e0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:03.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.728+0000 7f39f7fff700 1 --2- 172.21.15.156:0/2778656168 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e0004850 0x7f39e0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.728+0000 7f39f7fff700 1 -- 172.21.15.156:0/2778656168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39e0003b90 msgr2=0x7f39e0003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:03.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.728+0000 7f39f7fff700 1 --2- 172.21.15.156:0/2778656168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39e0003b90 0x7f39e0003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.728+0000 7f39f7fff700 1 -- 172.21.15.156:0/2778656168 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e00035b0 con 0x7f39e0005200 2023-11-28T13:08:03.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.728+0000 7f39f7fff700 1 -- 172.21.15.156:0/2778656168 reap_dead start 2023-11-28T13:08:03.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.728+0000 7f39f7fff700 1 --2- 172.21.15.156:0/2778656168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39e0005200 0x7f39e000ba10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f39f0002230 tx=0x7f39e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839c8599239e3bc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:03.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.729+0000 7f39f67fc700 1 -- 172.21.15.156:0/2778656168 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f00072d0 con 0x7f39e0005200 2023-11-28T13:08:03.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.729+0000 7f39f67fc700 1 -- 172.21.15.156:0/2778656168 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f0007430 con 0x7f39e0005200 2023-11-28T13:08:03.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.729+0000 7f39f67fc700 1 -- 172.21.15.156:0/2778656168 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f00077c0 con 0x7f39e0005200 2023-11-28T13:08:03.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.729+0000 7f39fddff700 1 -- 172.21.15.156:0/2778656168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39e0005200 msgr2=0x7f39e000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:03.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.729+0000 7f39fddff700 1 --2- 172.21.15.156:0/2778656168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39e0005200 0x7f39e000ba10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f39f0002230 tx=0x7f39e0005880 comp rx=0 tx=0).stop 2023-11-28T13:08:03.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.729+0000 7f39fddff700 1 -- 172.21.15.156:0/2778656168 shutdown_connections 2023-11-28T13:08:03.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.729+0000 7f39fddff700 1 --2- 172.21.15.156:0/2778656168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39e0005200 0x7f39e000ba10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.729+0000 7f39fddff700 1 -- 172.21.15.156:0/2778656168 >> 172.21.15.156:0/2778656168 conn(0x7f39f8072600 msgr2=0x7f39f80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:03.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.729+0000 7f39fddff700 1 -- 172.21.15.156:0/2778656168 shutdown_connections 2023-11-28T13:08:03.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.729+0000 7f39fddff700 1 -- 172.21.15.156:0/2778656168 wait complete. 2023-11-28T13:08:03.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.732+0000 7f39fddff700 1 Processor -- start 2023-11-28T13:08:03.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.732+0000 7f39fddff700 1 -- start start 2023-11-28T13:08:03.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.733+0000 7f39fddff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e0005200 0x7f39f8150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:03.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.733+0000 7f39fddff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39f815ef30 0x7f39f8156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:03.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.733+0000 7f39fddff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f815f3a0 0x7f39f8156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:03.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.733+0000 7f39fddff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8151200 con 0x7f39f815f3a0 2023-11-28T13:08:03.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.733+0000 7f39fddff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f80c6e40 con 0x7f39f815ef30 2023-11-28T13:08:03.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.733+0000 7f39fddff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f80c7120 con 0x7f39e0005200 2023-11-28T13:08:03.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.733+0000 7f39f7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f815f3a0 0x7f39f8156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:03.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.733+0000 7f39f7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f815f3a0 0x7f39f8156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43590/0 (socket says 172.21.15.156:43590) 2023-11-28T13:08:03.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.733+0000 7f39f7fff700 1 -- 172.21.15.156:0/1319142739 learned_addr learned my addr 172.21.15.156:0/1319142739 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:03.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.733+0000 7f39f77fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e0005200 0x7f39f8150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:03.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.733+0000 7f39f6ffd700 1 --2- 172.21.15.156:0/1319142739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39f815ef30 0x7f39f8156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:03.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.734+0000 7f39f7fff700 1 -- 172.21.15.156:0/1319142739 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e0005200 msgr2=0x7f39f8150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:03.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.734+0000 7f39f7fff700 1 --2- 172.21.15.156:0/1319142739 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e0005200 0x7f39f8150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.734+0000 7f39f7fff700 1 -- 172.21.15.156:0/1319142739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39f815ef30 msgr2=0x7f39f8156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:03.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.734+0000 7f39f7fff700 1 --2- 172.21.15.156:0/1319142739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39f815ef30 0x7f39f8156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.734+0000 7f39f7fff700 1 -- 172.21.15.156:0/1319142739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f815cd80 con 0x7f39f815f3a0 2023-11-28T13:08:03.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.734+0000 7f39f7fff700 1 --2- 172.21.15.156:0/1319142739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f815f3a0 0x7f39f8156f40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f39f0016040 tx=0x7f39f0001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:03.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.734+0000 7f39dffff700 1 -- 172.21.15.156:0/1319142739 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0024030 con 0x7f39f815f3a0 2023-11-28T13:08:03.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.734+0000 7f39dffff700 1 -- 172.21.15.156:0/1319142739 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f0012620 con 0x7f39f815f3a0 2023-11-28T13:08:03.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.734+0000 7f39dffff700 1 -- 172.21.15.156:0/1319142739 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f00129d0 con 0x7f39f815f3a0 2023-11-28T13:08:03.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.734+0000 7f39fddff700 1 -- 172.21.15.156:0/1319142739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f80d15e0 con 0x7f39f815f3a0 2023-11-28T13:08:03.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.734+0000 7f39fddff700 1 -- 172.21.15.156:0/1319142739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f815cf10 con 0x7f39f815f3a0 2023-11-28T13:08:03.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.736+0000 7f39fddff700 1 -- 172.21.15.156:0/1319142739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f80c6bc0 con 0x7f39f815f3a0 2023-11-28T13:08:03.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.736+0000 7f39dffff700 1 -- 172.21.15.156:0/1319142739 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f39f0010070 con 0x7f39f815f3a0 2023-11-28T13:08:03.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.737+0000 7f39dffff700 1 --2- 172.21.15.156:0/1319142739 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39e40682f0 0x7f39e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:03.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.737+0000 7f39dffff700 1 -- 172.21.15.156:0/1319142739 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39f0025070 con 0x7f39f815f3a0 2023-11-28T13:08:03.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.737+0000 7f39f77fe700 1 --2- 172.21.15.156:0/1319142739 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39e40682f0 0x7f39e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:03.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.738+0000 7f39f77fe700 1 --2- 172.21.15.156:0/1319142739 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39e40682f0 0x7f39e406a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f39e8007fd0 tx=0x7f39e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:03.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.741+0000 7f39dffff700 1 -- 172.21.15.156:0/1319142739 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f39f0053c00 con 0x7f39f815f3a0 2023-11-28T13:08:03.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.932+0000 7f39fddff700 1 -- 172.21.15.156:0/1319142739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39f80745c0 con 0x7f39f815f3a0 2023-11-28T13:08:03.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.933+0000 7f39dffff700 1 -- 172.21.15.156:0/1319142739 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+17910 (secure 0 0 0) 0x7f39f0018090 con 0x7f39f815f3a0 2023-11-28T13:08:03.934 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:03.934 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":20,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:08:03.057729+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:08:00.903485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:08:02.037854+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:08:03.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.935+0000 7f39fddff700 1 -- 172.21.15.156:0/1319142739 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39e40682f0 msgr2=0x7f39e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:03.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.935+0000 7f39fddff700 1 --2- 172.21.15.156:0/1319142739 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39e40682f0 0x7f39e406a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f39e8007fd0 tx=0x7f39e8009040 comp rx=0 tx=0).stop 2023-11-28T13:08:03.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.935+0000 7f39fddff700 1 -- 172.21.15.156:0/1319142739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f815f3a0 msgr2=0x7f39f8156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:03.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.935+0000 7f39fddff700 1 --2- 172.21.15.156:0/1319142739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f815f3a0 0x7f39f8156f40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f39f0016040 tx=0x7f39f0001d00 comp rx=0 tx=0).stop 2023-11-28T13:08:03.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.935+0000 7f39fddff700 1 -- 172.21.15.156:0/1319142739 shutdown_connections 2023-11-28T13:08:03.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.935+0000 7f39fddff700 1 --2- 172.21.15.156:0/1319142739 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39e40682f0 0x7f39e406a7a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.935+0000 7f39fddff700 1 --2- 172.21.15.156:0/1319142739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f815f3a0 0x7f39f8156f40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.935+0000 7f39fddff700 1 --2- 172.21.15.156:0/1319142739 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e0005200 0x7f39f8150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.935+0000 7f39fddff700 1 --2- 172.21.15.156:0/1319142739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39f815ef30 0x7f39f8156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:03.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.935+0000 7f39fddff700 1 -- 172.21.15.156:0/1319142739 >> 172.21.15.156:0/1319142739 conn(0x7f39f8072600 msgr2=0x7f39f815a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:03.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.935+0000 7f39fddff700 1 -- 172.21.15.156:0/1319142739 shutdown_connections 2023-11-28T13:08:03.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:03.935+0000 7f39fddff700 1 -- 172.21.15.156:0/1319142739 wait complete. 2023-11-28T13:08:03.955 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T13:08:04.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.160+0000 7f550b3ff700 1 Processor -- start 2023-11-28T13:08:04.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.160+0000 7f550b3ff700 1 -- start start 2023-11-28T13:08:04.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.161+0000 7f550b3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5504161340 0x7f55040d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:04.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.161+0000 7f550b3ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f55040d2320 0x7f55040cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:04.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.161+0000 7f550b3ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55040d1950 con 0x7f550415c9b0 2023-11-28T13:08:04.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.161+0000 7f550b3ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55040b5800 con 0x7f55040d2320 2023-11-28T13:08:04.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.161+0000 7f550b3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55040b5300 con 0x7f5504161340 2023-11-28T13:08:04.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.161+0000 7f550919b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f550415c9b0 0x7f55040b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:45666/0 (socket says 172.21.15.156:45666) 2023-11-28T13:08:04.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.161+0000 7f550919b700 1 -- 172.21.15.156:0/4183997886 learned_addr learned my addr 172.21.15.156:0/4183997886 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:04.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f550999c700 1 --2- 172.21.15.156:0/4183997886 >> v2:172.21.15.186:3300/0 conn(0x7f55040d2320 0x7f55040cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f550899a700 1 --2- 172.21.15.156:0/4183997886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5504161340 0x7f55040d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f54fbfff700 1 -- 172.21.15.156:0/4183997886 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468371284 0 0) 0x7f55040d1950 con 0x7f550415c9b0 2023-11-28T13:08:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f54fbfff700 1 -- 172.21.15.156:0/4183997886 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54f00035b0 con 0x7f550415c9b0 2023-11-28T13:08:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f54fbfff700 1 -- 172.21.15.156:0/4183997886 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 982788834 0 0) 0x7f54f00035b0 con 0x7f550415c9b0 2023-11-28T13:08:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f54fbfff700 1 -- 172.21.15.156:0/4183997886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5504161340 msgr2=0x7f55040d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f54fbfff700 1 --2- 172.21.15.156:0/4183997886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5504161340 0x7f55040d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f54fbfff700 1 -- 172.21.15.156:0/4183997886 >> v2:172.21.15.186:3300/0 conn(0x7f55040d2320 msgr2=0x7f55040cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f54fbfff700 1 --2- 172.21.15.156:0/4183997886 >> v2:172.21.15.186:3300/0 conn(0x7f55040d2320 0x7f55040cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f54fbfff700 1 -- 172.21.15.156:0/4183997886 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55040b5520 con 0x7f550415c9b0 2023-11-28T13:08:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f54fbfff700 1 -- 172.21.15.156:0/4183997886 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f5500001670 con 0x7f550415c9b0 2023-11-28T13:08:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f54fbfff700 1 -- 172.21.15.156:0/4183997886 >> v1:172.21.15.156:6789/0 conn(0x7f550415c9b0 legacy=0x7f55040b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f54fbfff700 1 --2- 172.21.15.156:0/4183997886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5504161340 0x7f54f0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f54fbfff700 1 --2- 172.21.15.156:0/4183997886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54f0003fb0 0x7f54f0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f54fbfff700 1 --2- 172.21.15.156:0/4183997886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54f0004d90 0x7f54f0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f54fbfff700 1 -- 172.21.15.156:0/4183997886 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55040b5800 con 0x7f54f0004d90 2023-11-28T13:08:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f54fbfff700 1 -- 172.21.15.156:0/4183997886 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55040b5300 con 0x7f54f0003fb0 2023-11-28T13:08:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f54fbfff700 1 -- 172.21.15.156:0/4183997886 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f000ba60 con 0x7f5504161340 2023-11-28T13:08:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.162+0000 7f550999c700 1 --2- 172.21.15.156:0/4183997886 >> v2:172.21.15.186:3300/0 conn(0x7f55040d2320 0x7f55040cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.163+0000 7f550919b700 1 --2- 172.21.15.156:0/4183997886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5504161340 0x7f54f0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.163+0000 7f550899a700 1 --2- 172.21.15.156:0/4183997886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54f0003fb0 0x7f54f0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.163+0000 7f550999c700 1 --2- 172.21.15.156:0/4183997886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54f0004d90 0x7f54f0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.163+0000 7f550999c700 1 -- 172.21.15.156:0/4183997886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5504161340 msgr2=0x7f54f0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.163+0000 7f550999c700 1 --2- 172.21.15.156:0/4183997886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5504161340 0x7f54f0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:04.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.163+0000 7f550999c700 1 -- 172.21.15.156:0/4183997886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54f0003fb0 msgr2=0x7f54f0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:04.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.163+0000 7f550999c700 1 --2- 172.21.15.156:0/4183997886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54f0003fb0 0x7f54f0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:04.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.163+0000 7f550999c700 1 -- 172.21.15.156:0/4183997886 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f00035b0 con 0x7f54f0004d90 2023-11-28T13:08:04.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.163+0000 7f550919b700 1 --2- 172.21.15.156:0/4183997886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5504161340 0x7f54f0003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:04.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.163+0000 7f550999c700 1 --2- 172.21.15.156:0/4183997886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54f0004d90 0x7f54f0005200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00e350 tx=0x7f54f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d7a86f2d8ad56f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:04.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.163+0000 7f54fbfff700 1 -- 172.21.15.156:0/4183997886 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc00ee10 con 0x7f54f0004d90 2023-11-28T13:08:04.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.164+0000 7f54fbfff700 1 -- 172.21.15.156:0/4183997886 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc00f040 con 0x7f54f0004d90 2023-11-28T13:08:04.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.164+0000 7f54fbfff700 1 -- 172.21.15.156:0/4183997886 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc003c40 con 0x7f54f0004d90 2023-11-28T13:08:04.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.164+0000 7f550b3ff700 1 -- 172.21.15.156:0/4183997886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54f0004d90 msgr2=0x7f54f0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:04.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.164+0000 7f550b3ff700 1 --2- 172.21.15.156:0/4183997886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54f0004d90 0x7f54f0005200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00e350 tx=0x7f54f0005880 comp rx=0 tx=0).stop 2023-11-28T13:08:04.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.164+0000 7f550999c700 1 -- 172.21.15.156:0/4183997886 reap_dead start 2023-11-28T13:08:04.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.164+0000 7f550b3ff700 1 -- 172.21.15.156:0/4183997886 shutdown_connections 2023-11-28T13:08:04.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.164+0000 7f550b3ff700 1 -- 172.21.15.156:0/4183997886 >> 172.21.15.156:0/4183997886 conn(0x7f5504072600 msgr2=0x7f55040cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:04.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.164+0000 7f550b3ff700 1 -- 172.21.15.156:0/4183997886 shutdown_connections 2023-11-28T13:08:04.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.164+0000 7f550b3ff700 1 -- 172.21.15.156:0/4183997886 wait complete. 2023-11-28T13:08:04.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.166+0000 7f550b3ff700 1 Processor -- start 2023-11-28T13:08:04.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.166+0000 7f550b3ff700 1 -- start start 2023-11-28T13:08:04.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.167+0000 7f550b3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54f0004d90 0x7f5504151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:04.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.167+0000 7f550b3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55041595d0 0x7f5504154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:04.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.167+0000 7f550b3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5504159a40 0x7f550414fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:04.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.167+0000 7f550b3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5504154c90 con 0x7f5504159a40 2023-11-28T13:08:04.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.167+0000 7f550b3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5504156b10 con 0x7f54f0004d90 2023-11-28T13:08:04.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.167+0000 7f550b3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55041519b0 con 0x7f55041595d0 2023-11-28T13:08:04.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.167+0000 7f550999c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5504159a40 0x7f550414fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:04.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.167+0000 7f550999c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5504159a40 0x7f550414fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43610/0 (socket says 172.21.15.156:43610) 2023-11-28T13:08:04.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.167+0000 7f550999c700 1 -- 172.21.15.156:0/1485944038 learned_addr learned my addr 172.21.15.156:0/1485944038 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:04.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.167+0000 7f550919b700 1 --2- 172.21.15.156:0/1485944038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54f0004d90 0x7f5504151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:04.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.167+0000 7f550899a700 1 --2- 172.21.15.156:0/1485944038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55041595d0 0x7f5504154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:04.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.168+0000 7f550999c700 1 -- 172.21.15.156:0/1485944038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55041595d0 msgr2=0x7f5504154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:04.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.168+0000 7f550999c700 1 --2- 172.21.15.156:0/1485944038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55041595d0 0x7f5504154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:04.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.168+0000 7f550999c700 1 -- 172.21.15.156:0/1485944038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54f0004d90 msgr2=0x7f5504151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:04.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.168+0000 7f550999c700 1 --2- 172.21.15.156:0/1485944038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54f0004d90 0x7f5504151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:04.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.168+0000 7f550999c700 1 -- 172.21.15.156:0/1485944038 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55040d15e0 con 0x7f5504159a40 2023-11-28T13:08:04.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.168+0000 7f550999c700 1 --2- 172.21.15.156:0/1485944038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5504159a40 0x7f550414fcd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f54f00047a0 tx=0x7f54fc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:04.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.168+0000 7f54f9ffb700 1 -- 172.21.15.156:0/1485944038 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc004440 con 0x7f5504159a40 2023-11-28T13:08:04.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.168+0000 7f54f9ffb700 1 -- 172.21.15.156:0/1485944038 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc0045e0 con 0x7f5504159a40 2023-11-28T13:08:04.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.168+0000 7f550b3ff700 1 -- 172.21.15.156:0/1485944038 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55040d1300 con 0x7f5504159a40 2023-11-28T13:08:04.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.168+0000 7f54f9ffb700 1 -- 172.21.15.156:0/1485944038 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc014480 con 0x7f5504159a40 2023-11-28T13:08:04.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.168+0000 7f550b3ff700 1 -- 172.21.15.156:0/1485944038 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550415cd80 con 0x7f5504159a40 2023-11-28T13:08:04.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.169+0000 7f550b3ff700 1 -- 172.21.15.156:0/1485944038 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55041549f0 con 0x7f5504159a40 2023-11-28T13:08:04.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.170+0000 7f54f9ffb700 1 -- 172.21.15.156:0/1485944038 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f54fc01a020 con 0x7f5504159a40 2023-11-28T13:08:04.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.171+0000 7f54f9ffb700 1 --2- 172.21.15.156:0/1485944038 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f54e80682f0 0x7f54e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:04.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.171+0000 7f54f9ffb700 1 -- 172.21.15.156:0/1485944038 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f54fc087130 con 0x7f5504159a40 2023-11-28T13:08:04.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.171+0000 7f550919b700 1 --2- 172.21.15.156:0/1485944038 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f54e80682f0 0x7f54e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:04.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.172+0000 7f550919b700 1 --2- 172.21.15.156:0/1485944038 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f54e80682f0 0x7f54e806a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5500001c90 tx=0x7f5500001b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:04.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.175+0000 7f54f9ffb700 1 -- 172.21.15.156:0/1485944038 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f55041549f0 con 0x7f5504159a40 2023-11-28T13:08:04.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:04.413+0000 7f550b3ff700 1 -- 172.21.15.156:0/1485944038 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f55041549f0 con 0x7f5504159a40 2023-11-28T13:08:05.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.078+0000 7f54f9ffb700 1 -- 172.21.15.156:0/1485944038 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f54fc0538a0 con 0x7f5504159a40 2023-11-28T13:08:05.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.081+0000 7f550b3ff700 1 -- 172.21.15.156:0/1485944038 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f54e80682f0 msgr2=0x7f54e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:05.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.081+0000 7f550b3ff700 1 --2- 172.21.15.156:0/1485944038 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f54e80682f0 0x7f54e806a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5500001c90 tx=0x7f5500001b80 comp rx=0 tx=0).stop 2023-11-28T13:08:05.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.081+0000 7f550b3ff700 1 -- 172.21.15.156:0/1485944038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5504159a40 msgr2=0x7f550414fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:05.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.081+0000 7f550b3ff700 1 --2- 172.21.15.156:0/1485944038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5504159a40 0x7f550414fcd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f54f00047a0 tx=0x7f54fc009750 comp rx=0 tx=0).stop 2023-11-28T13:08:05.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.081+0000 7f550b3ff700 1 -- 172.21.15.156:0/1485944038 shutdown_connections 2023-11-28T13:08:05.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.081+0000 7f550b3ff700 1 --2- 172.21.15.156:0/1485944038 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f54e80682f0 0x7f54e806a7a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:05.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.081+0000 7f550b3ff700 1 --2- 172.21.15.156:0/1485944038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5504159a40 0x7f550414fcd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:05.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.081+0000 7f550b3ff700 1 --2- 172.21.15.156:0/1485944038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54f0004d90 0x7f5504151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:05.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.081+0000 7f550b3ff700 1 --2- 172.21.15.156:0/1485944038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55041595d0 0x7f5504154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:05.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.081+0000 7f550b3ff700 1 -- 172.21.15.156:0/1485944038 >> 172.21.15.156:0/1485944038 conn(0x7f5504072600 msgr2=0x7f55040b5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:05.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.081+0000 7f550b3ff700 1 -- 172.21.15.156:0/1485944038 shutdown_connections 2023-11-28T13:08:05.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.082+0000 7f550b3ff700 1 -- 172.21.15.156:0/1485944038 wait complete. 2023-11-28T13:08:05.103 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T13:08:05.103 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T13:08:05.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.303+0000 7f99e03ff700 1 Processor -- start 2023-11-28T13:08:05.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.303+0000 7f99e03ff700 1 -- start start 2023-11-28T13:08:05.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.304+0000 7f99e03ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f99d815c9b0 0x7f99d80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:05.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.305+0000 7f99e03ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d80d1950 con 0x7f99d80cd7c0 2023-11-28T13:08:05.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.305+0000 7f99e03ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d80d24f0 con 0x7f99d8161340 2023-11-28T13:08:05.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.305+0000 7f99e03ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d80b5800 con 0x7f99d815c9b0 2023-11-28T13:08:05.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.305+0000 7f99de99c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f99d80cd7c0 0x7f99d80b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:45668/0 (socket says 172.21.15.156:45668) 2023-11-28T13:08:05.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.305+0000 7f99de99c700 1 -- 172.21.15.156:0/2731516819 learned_addr learned my addr 172.21.15.156:0/2731516819 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:05.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.305+0000 7f99de19b700 1 --2- 172.21.15.156:0/2731516819 >> v2:172.21.15.186:3300/0 conn(0x7f99d815c9b0 0x7f99d80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:05.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.305+0000 7f99dd199700 1 -- 172.21.15.156:0/2731516819 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1699109166 0 0) 0x7f99d80d1950 con 0x7f99d80cd7c0 2023-11-28T13:08:05.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.305+0000 7f99dd199700 1 -- 172.21.15.156:0/2731516819 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99c40035b0 con 0x7f99d80cd7c0 2023-11-28T13:08:05.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.306+0000 7f99de19b700 1 -- 172.21.15.156:0/2731516819 >> v1:172.21.15.186:6789/0 conn(0x7f99d8161340 legacy=0x7f99d80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:05.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.306+0000 7f99dd199700 1 -- 172.21.15.156:0/2731516819 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528369920 0 0) 0x7f99d80d24f0 con 0x7f99d8161340 2023-11-28T13:08:05.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.306+0000 7f99de19b700 1 -- 172.21.15.156:0/2731516819 >> v1:172.21.15.156:6789/0 conn(0x7f99d80cd7c0 legacy=0x7f99d80b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:05.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.306+0000 7f99de19b700 1 -- 172.21.15.156:0/2731516819 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d80b5520 con 0x7f99d815c9b0 2023-11-28T13:08:05.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.306+0000 7f99de19b700 1 --2- 172.21.15.156:0/2731516819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99d815c9b0 0x7f99d80b4bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009840 tx=0x7f99d4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b52e2d7d00c854e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:05.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.306+0000 7f99dd199700 1 -- 172.21.15.156:0/2731516819 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d00019f0 con 0x7f99d815c9b0 2023-11-28T13:08:05.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.306+0000 7f99dd199700 1 -- 172.21.15.156:0/2731516819 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d4004030 con 0x7f99d815c9b0 2023-11-28T13:08:05.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.306+0000 7f99dd199700 1 -- 172.21.15.156:0/2731516819 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d40043c0 con 0x7f99d815c9b0 2023-11-28T13:08:05.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.307+0000 7f99e03ff700 1 -- 172.21.15.156:0/2731516819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99d815c9b0 msgr2=0x7f99d80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:05.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.307+0000 7f99e03ff700 1 --2- 172.21.15.156:0/2731516819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99d815c9b0 0x7f99d80b4bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009840 tx=0x7f99d4009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:05.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.307+0000 7f99e03ff700 1 -- 172.21.15.156:0/2731516819 shutdown_connections 2023-11-28T13:08:05.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.307+0000 7f99e03ff700 1 --2- 172.21.15.156:0/2731516819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99d815c9b0 0x7f99d80b4bf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:05.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.307+0000 7f99e03ff700 1 -- 172.21.15.156:0/2731516819 >> 172.21.15.156:0/2731516819 conn(0x7f99d8072600 msgr2=0x7f99d80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:05.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.307+0000 7f99e03ff700 1 -- 172.21.15.156:0/2731516819 shutdown_connections 2023-11-28T13:08:05.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.307+0000 7f99e03ff700 1 -- 172.21.15.156:0/2731516819 wait complete. 2023-11-28T13:08:05.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.309+0000 7f99e03ff700 1 Processor -- start 2023-11-28T13:08:05.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.309+0000 7f99e03ff700 1 -- start start 2023-11-28T13:08:05.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.310+0000 7f99e03ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99d80cd7c0 0x7f99d80cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:05.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.310+0000 7f99e03ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99d815c9b0 0x7f99d8150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:05.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.310+0000 7f99e03ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99d8161340 0x7f99d81e6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:05.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.310+0000 7f99e03ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d815ddd0 con 0x7f99d815c9b0 2023-11-28T13:08:05.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.310+0000 7f99e03ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d8150dd0 con 0x7f99d8161340 2023-11-28T13:08:05.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.310+0000 7f99e03ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d80ccbb0 con 0x7f99d80cd7c0 2023-11-28T13:08:05.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.310+0000 7f99dd99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99d815c9b0 0x7f99d8150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:05.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.310+0000 7f99dd99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99d815c9b0 0x7f99d8150890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43616/0 (socket says 172.21.15.156:43616) 2023-11-28T13:08:05.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.310+0000 7f99dd99a700 1 -- 172.21.15.156:0/1744637199 learned_addr learned my addr 172.21.15.156:0/1744637199 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:05.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.310+0000 7f99de19b700 1 --2- 172.21.15.156:0/1744637199 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99d80cd7c0 0x7f99d80cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:05.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.310+0000 7f99de99c700 1 --2- 172.21.15.156:0/1744637199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99d8161340 0x7f99d81e6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:05.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.310+0000 7f99dd99a700 1 -- 172.21.15.156:0/1744637199 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99d80cd7c0 msgr2=0x7f99d80cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:05.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.310+0000 7f99dd99a700 1 --2- 172.21.15.156:0/1744637199 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99d80cd7c0 0x7f99d80cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:05.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.310+0000 7f99dd99a700 1 -- 172.21.15.156:0/1744637199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99d8161340 msgr2=0x7f99d81e6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:05.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.310+0000 7f99dd99a700 1 --2- 172.21.15.156:0/1744637199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99d8161340 0x7f99d81e6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:05.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.311+0000 7f99dd99a700 1 -- 172.21.15.156:0/1744637199 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d40094a0 con 0x7f99d815c9b0 2023-11-28T13:08:05.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.311+0000 7f99de19b700 1 --2- 172.21.15.156:0/1744637199 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99d80cd7c0 0x7f99d80cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:05.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.311+0000 7f99dd99a700 1 --2- 172.21.15.156:0/1744637199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99d815c9b0 0x7f99d8150890 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f99c80011e0 tx=0x7f99c800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:05.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.311+0000 7f99ceffd700 1 -- 172.21.15.156:0/1744637199 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c80054e0 con 0x7f99d815c9b0 2023-11-28T13:08:05.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.311+0000 7f99ceffd700 1 -- 172.21.15.156:0/1744637199 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c8003680 con 0x7f99d815c9b0 2023-11-28T13:08:05.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.311+0000 7f99e03ff700 1 -- 172.21.15.156:0/1744637199 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d80d15e0 con 0x7f99d815c9b0 2023-11-28T13:08:05.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.311+0000 7f99ceffd700 1 -- 172.21.15.156:0/1744637199 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c8003a30 con 0x7f99d815c9b0 2023-11-28T13:08:05.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.311+0000 7f99e03ff700 1 -- 172.21.15.156:0/1744637199 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d80d1300 con 0x7f99d815c9b0 2023-11-28T13:08:05.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.312+0000 7f99e03ff700 1 -- 172.21.15.156:0/1744637199 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99d80b5130 con 0x7f99d815c9b0 2023-11-28T13:08:05.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.313+0000 7f99ceffd700 1 -- 172.21.15.156:0/1744637199 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f99c8019030 con 0x7f99d815c9b0 2023-11-28T13:08:05.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.314+0000 7f99ceffd700 1 --2- 172.21.15.156:0/1744637199 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f99bc0682f0 0x7f99bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:05.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.314+0000 7f99ceffd700 1 -- 172.21.15.156:0/1744637199 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f99c80850a0 con 0x7f99d815c9b0 2023-11-28T13:08:05.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.314+0000 7f99de19b700 1 --2- 172.21.15.156:0/1744637199 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f99bc0682f0 0x7f99bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:05.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.315+0000 7f99de19b700 1 --2- 172.21.15.156:0/1744637199 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f99bc0682f0 0x7f99bc06a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99d400a010 tx=0x7f99d4008000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:05.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.318+0000 7f99ceffd700 1 -- 172.21.15.156:0/1744637199 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f99c8051900 con 0x7f99d815c9b0 2023-11-28T13:08:05.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:05.556+0000 7f99e03ff700 1 -- 172.21.15.156:0/1744637199 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f99d80745c0 con 0x7f99d815c9b0 2023-11-28T13:08:06.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.080+0000 7f99ceffd700 1 -- 172.21.15.156:0/1744637199 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f99c80170c0 con 0x7f99d815c9b0 2023-11-28T13:08:06.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.083+0000 7f99e03ff700 1 -- 172.21.15.156:0/1744637199 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f99bc0682f0 msgr2=0x7f99bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:06.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.083+0000 7f99e03ff700 1 --2- 172.21.15.156:0/1744637199 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f99bc0682f0 0x7f99bc06a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99d400a010 tx=0x7f99d4008000 comp rx=0 tx=0).stop 2023-11-28T13:08:06.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.083+0000 7f99e03ff700 1 -- 172.21.15.156:0/1744637199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99d815c9b0 msgr2=0x7f99d8150890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:06.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.083+0000 7f99e03ff700 1 --2- 172.21.15.156:0/1744637199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99d815c9b0 0x7f99d8150890 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f99c80011e0 tx=0x7f99c800cae0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.083+0000 7f99e03ff700 1 -- 172.21.15.156:0/1744637199 shutdown_connections 2023-11-28T13:08:06.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.083+0000 7f99e03ff700 1 --2- 172.21.15.156:0/1744637199 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f99bc0682f0 0x7f99bc06a7a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.083+0000 7f99e03ff700 1 --2- 172.21.15.156:0/1744637199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99d8161340 0x7f99d81e6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.083+0000 7f99e03ff700 1 --2- 172.21.15.156:0/1744637199 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99d80cd7c0 0x7f99d80cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.083+0000 7f99e03ff700 1 --2- 172.21.15.156:0/1744637199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99d815c9b0 0x7f99d8150890 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.083+0000 7f99e03ff700 1 -- 172.21.15.156:0/1744637199 >> 172.21.15.156:0/1744637199 conn(0x7f99d8072600 msgr2=0x7f99d815ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:06.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.083+0000 7f99e03ff700 1 -- 172.21.15.156:0/1744637199 shutdown_connections 2023-11-28T13:08:06.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.083+0000 7f99e03ff700 1 -- 172.21.15.156:0/1744637199 wait complete. 2023-11-28T13:08:06.102 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T13:08:06.103 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:08:06.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.297+0000 7ff2aa3ff700 1 Processor -- start 2023-11-28T13:08:06.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.297+0000 7ff2aa3ff700 1 -- start start 2023-11-28T13:08:06.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.299+0000 7ff2aa3ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff2a40d01b0 0x7ff2a40d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:06.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.299+0000 7ff2aa3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a40b16d0 0x7ff2a40d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:06.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.299+0000 7ff2aa3ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a40b0230 con 0x7ff2a40b20b0 2023-11-28T13:08:06.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.299+0000 7ff2aa3ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a415cec0 con 0x7ff2a40d01b0 2023-11-28T13:08:06.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.299+0000 7ff2aa3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40b4570 con 0x7ff2a40b16d0 2023-11-28T13:08:06.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.299+0000 7ff2a37fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff2a40b20b0 0x7ff2a40b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:34578/0 (socket says 172.21.15.156:34578) 2023-11-28T13:08:06.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.299+0000 7ff2a37fe700 1 -- 172.21.15.156:0/3908951665 learned_addr learned my addr 172.21.15.156:0/3908951665 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:06.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.299+0000 7ff2a3fff700 1 --2- 172.21.15.156:0/3908951665 >> v2:172.21.15.186:3300/0 conn(0x7ff2a40d01b0 0x7ff2a40d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:06.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.299+0000 7ff2a899c700 1 --2- 172.21.15.156:0/3908951665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a40b16d0 0x7ff2a40d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:06.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.299+0000 7ff2a2ffd700 1 -- 172.21.15.156:0/3908951665 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076299793 0 0) 0x7ff2a40b0230 con 0x7ff2a40b20b0 2023-11-28T13:08:06.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.299+0000 7ff2a2ffd700 1 -- 172.21.15.156:0/3908951665 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff28c0035b0 con 0x7ff2a40b20b0 2023-11-28T13:08:06.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.300+0000 7ff2a3fff700 1 -- 172.21.15.156:0/3908951665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a40b16d0 msgr2=0x7ff2a40d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:06.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.300+0000 7ff2a3fff700 1 --2- 172.21.15.156:0/3908951665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a40b16d0 0x7ff2a40d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.300+0000 7ff2a3fff700 1 -- 172.21.15.156:0/3908951665 >> v1:172.21.15.156:6789/0 conn(0x7ff2a40b20b0 legacy=0x7ff2a40b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:06.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.300+0000 7ff2a3fff700 1 -- 172.21.15.156:0/3908951665 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a40b4850 con 0x7ff2a40d01b0 2023-11-28T13:08:06.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.300+0000 7ff2a899c700 1 --2- 172.21.15.156:0/3908951665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a40b16d0 0x7ff2a40d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:06.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.300+0000 7ff2a3fff700 1 --2- 172.21.15.156:0/3908951665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a40d01b0 0x7ff2a40d0c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff294009840 tx=0x7ff294009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c4dfee737fc9484 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:06.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.300+0000 7ff2a2ffd700 1 -- 172.21.15.156:0/3908951665 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294011830 con 0x7ff2a40d01b0 2023-11-28T13:08:06.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.300+0000 7ff2a2ffd700 1 -- 172.21.15.156:0/3908951665 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff294011990 con 0x7ff2a40d01b0 2023-11-28T13:08:06.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.300+0000 7ff2a2ffd700 1 -- 172.21.15.156:0/3908951665 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294011d20 con 0x7ff2a40d01b0 2023-11-28T13:08:06.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.301+0000 7ff2aa3ff700 1 -- 172.21.15.156:0/3908951665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a40d01b0 msgr2=0x7ff2a40d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:06.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.301+0000 7ff2aa3ff700 1 --2- 172.21.15.156:0/3908951665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a40d01b0 0x7ff2a40d0c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff294009840 tx=0x7ff294009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:06.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.301+0000 7ff2aa3ff700 1 -- 172.21.15.156:0/3908951665 shutdown_connections 2023-11-28T13:08:06.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.301+0000 7ff2aa3ff700 1 --2- 172.21.15.156:0/3908951665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a40b16d0 0x7ff2a40d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.301+0000 7ff2aa3ff700 1 --2- 172.21.15.156:0/3908951665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a40d01b0 0x7ff2a40d0c20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.301+0000 7ff2aa3ff700 1 -- 172.21.15.156:0/3908951665 >> 172.21.15.156:0/3908951665 conn(0x7ff2a409c3f0 msgr2=0x7ff2a40d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:06.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.301+0000 7ff2aa3ff700 1 -- 172.21.15.156:0/3908951665 shutdown_connections 2023-11-28T13:08:06.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.301+0000 7ff2aa3ff700 1 -- 172.21.15.156:0/3908951665 wait complete. 2023-11-28T13:08:06.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.303+0000 7ff2aa3ff700 1 Processor -- start 2023-11-28T13:08:06.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.303+0000 7ff2aa3ff700 1 -- start start 2023-11-28T13:08:06.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.304+0000 7ff2aa3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a40b16d0 0x7ff2a40cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:06.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.304+0000 7ff2aa3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a40b20b0 0x7ff2a41584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:06.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.304+0000 7ff2aa3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a40d01b0 0x7ff2a40cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:06.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.304+0000 7ff2aa3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a409ce70 con 0x7ff2a40b16d0 2023-11-28T13:08:06.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.304+0000 7ff2aa3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40b5e80 con 0x7ff2a40d01b0 2023-11-28T13:08:06.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.304+0000 7ff2aa3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40cce50 con 0x7ff2a40b20b0 2023-11-28T13:08:06.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.304+0000 7ff2a3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a40b16d0 0x7ff2a40cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:06.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.304+0000 7ff2a3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a40b16d0 0x7ff2a40cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58662/0 (socket says 172.21.15.156:58662) 2023-11-28T13:08:06.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.304+0000 7ff2a3fff700 1 -- 172.21.15.156:0/2121497673 learned_addr learned my addr 172.21.15.156:0/2121497673 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:06.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.304+0000 7ff2a37fe700 1 --2- 172.21.15.156:0/2121497673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a40b20b0 0x7ff2a41584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:06.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.304+0000 7ff2a899c700 1 --2- 172.21.15.156:0/2121497673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a40d01b0 0x7ff2a40cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:06.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.304+0000 7ff2a3fff700 1 -- 172.21.15.156:0/2121497673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a40b20b0 msgr2=0x7ff2a41584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:06.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.304+0000 7ff2a3fff700 1 --2- 172.21.15.156:0/2121497673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a40b20b0 0x7ff2a41584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.304+0000 7ff2a3fff700 1 -- 172.21.15.156:0/2121497673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a40d01b0 msgr2=0x7ff2a40cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:06.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.304+0000 7ff2a3fff700 1 --2- 172.21.15.156:0/2121497673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a40d01b0 0x7ff2a40cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.304+0000 7ff2a3fff700 1 -- 172.21.15.156:0/2121497673 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c009a70 con 0x7ff2a40b16d0 2023-11-28T13:08:06.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.305+0000 7ff2a3fff700 1 --2- 172.21.15.156:0/2121497673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a40b16d0 0x7ff2a40cc870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff294009fd0 tx=0x7ff294004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:06.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.305+0000 7ff2a0ff9700 1 -- 172.21.15.156:0/2121497673 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2940036a0 con 0x7ff2a40b16d0 2023-11-28T13:08:06.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.305+0000 7ff2a0ff9700 1 -- 172.21.15.156:0/2121497673 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29401e070 con 0x7ff2a40b16d0 2023-11-28T13:08:06.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.305+0000 7ff2aa3ff700 1 -- 172.21.15.156:0/2121497673 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2940094a0 con 0x7ff2a40b16d0 2023-11-28T13:08:06.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.305+0000 7ff2a0ff9700 1 -- 172.21.15.156:0/2121497673 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294003a60 con 0x7ff2a40b16d0 2023-11-28T13:08:06.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.305+0000 7ff2aa3ff700 1 -- 172.21.15.156:0/2121497673 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a40afec0 con 0x7ff2a40b16d0 2023-11-28T13:08:06.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.306+0000 7ff2aa3ff700 1 -- 172.21.15.156:0/2121497673 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a40afbe0 con 0x7ff2a40b16d0 2023-11-28T13:08:06.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.307+0000 7ff2a0ff9700 1 -- 172.21.15.156:0/2121497673 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff29400f050 con 0x7ff2a40b16d0 2023-11-28T13:08:06.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.308+0000 7ff2a0ff9700 1 --2- 172.21.15.156:0/2121497673 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2900682f0 0x7ff29006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:06.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.308+0000 7ff2a0ff9700 1 -- 172.21.15.156:0/2121497673 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff294086be0 con 0x7ff2a40b16d0 2023-11-28T13:08:06.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.308+0000 7ff2a37fe700 1 --2- 172.21.15.156:0/2121497673 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2900682f0 0x7ff29006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:06.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.309+0000 7ff2a37fe700 1 --2- 172.21.15.156:0/2121497673 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2900682f0 0x7ff29006a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff2a415c8a0 tx=0x7ff298009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:06.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.312+0000 7ff2a0ff9700 1 -- 172.21.15.156:0/2121497673 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff294053440 con 0x7ff2a40b16d0 2023-11-28T13:08:06.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.546+0000 7ff2aa3ff700 1 -- 172.21.15.156:0/2121497673 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2a40b2600 con 0x7ff2a40b16d0 2023-11-28T13:08:06.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.546+0000 7ff2a0ff9700 1 -- 172.21.15.156:0/2121497673 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4651 (secure 0 0 0) 0x7ff294015090 con 0x7ff2a40b16d0 2023-11-28T13:08:06.547 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:06.548 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1283906891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1283906891},{"type":"v1","addr":"172.21.15.156:6835","nonce":1283906891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1738881422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1738881422},{"type":"v1","addr":"172.21.15.156:6833","nonce":1738881422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2235657629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2235657629},{"type":"v1","addr":"172.21.15.186:6836","nonce":2235657629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:08:03.058116+0000","modified":"2023-11-28T13:08:06.077600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/402651964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":402651964},{"type":"v1","addr":"172.21.15.186:6837","nonce":402651964}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:08:06.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.548+0000 7ff2aa3ff700 1 -- 172.21.15.156:0/2121497673 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2900682f0 msgr2=0x7ff29006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:06.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.548+0000 7ff2aa3ff700 1 --2- 172.21.15.156:0/2121497673 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2900682f0 0x7ff29006a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff2a415c8a0 tx=0x7ff298009040 comp rx=0 tx=0).stop 2023-11-28T13:08:06.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.549+0000 7ff2aa3ff700 1 -- 172.21.15.156:0/2121497673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a40b16d0 msgr2=0x7ff2a40cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:06.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.549+0000 7ff2aa3ff700 1 --2- 172.21.15.156:0/2121497673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a40b16d0 0x7ff2a40cc870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff294009fd0 tx=0x7ff294004750 comp rx=0 tx=0).stop 2023-11-28T13:08:06.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.549+0000 7ff2aa3ff700 1 -- 172.21.15.156:0/2121497673 shutdown_connections 2023-11-28T13:08:06.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.549+0000 7ff2aa3ff700 1 --2- 172.21.15.156:0/2121497673 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2900682f0 0x7ff29006a7a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.549+0000 7ff2aa3ff700 1 --2- 172.21.15.156:0/2121497673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a40d01b0 0x7ff2a40cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.549+0000 7ff2aa3ff700 1 --2- 172.21.15.156:0/2121497673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a40b16d0 0x7ff2a40cc870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.549+0000 7ff2aa3ff700 1 --2- 172.21.15.156:0/2121497673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a40b20b0 0x7ff2a41584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.549+0000 7ff2aa3ff700 1 -- 172.21.15.156:0/2121497673 >> 172.21.15.156:0/2121497673 conn(0x7ff2a409c3f0 msgr2=0x7ff2a40b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:06.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.549+0000 7ff2aa3ff700 1 -- 172.21.15.156:0/2121497673 shutdown_connections 2023-11-28T13:08:06.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.549+0000 7ff2aa3ff700 1 -- 172.21.15.156:0/2121497673 wait complete. 2023-11-28T13:08:06.552 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 8 2023-11-28T13:08:06.568 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:08:06.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.764+0000 7fe71c3ff700 1 Processor -- start 2023-11-28T13:08:06.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.764+0000 7fe71c3ff700 1 -- start start 2023-11-28T13:08:06.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.765+0000 7fe71c3ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe7140d01b0 0x7fe7140d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:06.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.766+0000 7fe71c3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7140b16d0 0x7fe7140d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:06.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.766+0000 7fe71c3ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7140b0230 con 0x7fe7140b20b0 2023-11-28T13:08:06.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.766+0000 7fe71c3ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe71415cec0 con 0x7fe7140d01b0 2023-11-28T13:08:06.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.766+0000 7fe71c3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7140b4570 con 0x7fe7140b16d0 2023-11-28T13:08:06.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.766+0000 7fe71a19b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe7140d01b0 0x7fe7140d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:06.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.766+0000 7fe71a19b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe7140d01b0 0x7fe7140d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58674/0 (socket says 172.21.15.156:58674) 2023-11-28T13:08:06.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.766+0000 7fe71a19b700 1 -- 172.21.15.156:0/201282960 learned_addr learned my addr 172.21.15.156:0/201282960 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:06.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.766+0000 7fe71999a700 1 --1- 172.21.15.156:0/201282960 >> v1:172.21.15.156:6789/0 conn(0x7fe7140b20b0 0x7fe7140b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:34592/0 (socket says 172.21.15.156:34592) 2023-11-28T13:08:06.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.766+0000 7fe71a99c700 1 --2- 172.21.15.156:0/201282960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7140b16d0 0x7fe7140d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:06.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.766+0000 7fe71a19b700 1 -- 172.21.15.156:0/201282960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7140b16d0 msgr2=0x7fe7140d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:06.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.766+0000 7fe71a19b700 1 --2- 172.21.15.156:0/201282960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7140b16d0 0x7fe7140d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.766+0000 7fe71a19b700 1 -- 172.21.15.156:0/201282960 >> v1:172.21.15.156:6789/0 conn(0x7fe7140b20b0 legacy=0x7fe7140b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:06.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.766+0000 7fe71a19b700 1 -- 172.21.15.156:0/201282960 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7140b4850 con 0x7fe7140d01b0 2023-11-28T13:08:06.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.766+0000 7fe71a99c700 1 --2- 172.21.15.156:0/201282960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7140b16d0 0x7fe7140d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:06.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.766+0000 7fe719199700 1 -- 172.21.15.156:0/201282960 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3105600863 0 0) 0x7fe7140b0230 con 0x7fe7140b20b0 2023-11-28T13:08:06.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.767+0000 7fe71a19b700 1 --2- 172.21.15.156:0/201282960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7140d01b0 0x7fe7140d0c20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe7100098f0 tx=0x7fe710009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82602974e504ea73 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:06.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.767+0000 7fe719199700 1 -- 172.21.15.156:0/201282960 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71000b8a0 con 0x7fe7140d01b0 2023-11-28T13:08:06.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.767+0000 7fe719199700 1 -- 172.21.15.156:0/201282960 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71000ba00 con 0x7fe7140d01b0 2023-11-28T13:08:06.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.767+0000 7fe719199700 1 -- 172.21.15.156:0/201282960 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71000bd90 con 0x7fe7140d01b0 2023-11-28T13:08:06.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.767+0000 7fe71c3ff700 1 -- 172.21.15.156:0/201282960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7140d01b0 msgr2=0x7fe7140d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:06.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.767+0000 7fe71c3ff700 1 --2- 172.21.15.156:0/201282960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7140d01b0 0x7fe7140d0c20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe7100098f0 tx=0x7fe710009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:06.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.767+0000 7fe71c3ff700 1 -- 172.21.15.156:0/201282960 shutdown_connections 2023-11-28T13:08:06.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.767+0000 7fe71c3ff700 1 --2- 172.21.15.156:0/201282960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7140b16d0 0x7fe7140d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.767+0000 7fe71c3ff700 1 --2- 172.21.15.156:0/201282960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7140d01b0 0x7fe7140d0c20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.767+0000 7fe71c3ff700 1 -- 172.21.15.156:0/201282960 >> 172.21.15.156:0/201282960 conn(0x7fe71409c3f0 msgr2=0x7fe7140d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:06.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.768+0000 7fe71c3ff700 1 -- 172.21.15.156:0/201282960 shutdown_connections 2023-11-28T13:08:06.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.768+0000 7fe71c3ff700 1 -- 172.21.15.156:0/201282960 wait complete. 2023-11-28T13:08:06.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.769+0000 7fe71c3ff700 1 Processor -- start 2023-11-28T13:08:06.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.769+0000 7fe71c3ff700 1 -- start start 2023-11-28T13:08:06.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.770+0000 7fe71c3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7140b16d0 0x7fe7141e63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:06.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.770+0000 7fe71c3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7140b20b0 0x7fe7140b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:06.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.770+0000 7fe71c3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7140d01b0 0x7fe7140ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:06.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.770+0000 7fe71c3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7141e50b0 con 0x7fe7140d01b0 2023-11-28T13:08:06.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.770+0000 7fe71c3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7141e5f10 con 0x7fe7140b20b0 2023-11-28T13:08:06.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.770+0000 7fe71c3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7141e5300 con 0x7fe7140b16d0 2023-11-28T13:08:06.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.770+0000 7fe71a99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7140d01b0 0x7fe7140ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:06.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.771+0000 7fe71a99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7140d01b0 0x7fe7140ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58690/0 (socket says 172.21.15.156:58690) 2023-11-28T13:08:06.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.771+0000 7fe71a99c700 1 -- 172.21.15.156:0/3741859325 learned_addr learned my addr 172.21.15.156:0/3741859325 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:06.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.771+0000 7fe71a19b700 1 --2- 172.21.15.156:0/3741859325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7140b16d0 0x7fe7141e63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:06.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.771+0000 7fe71999a700 1 --2- 172.21.15.156:0/3741859325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7140b20b0 0x7fe7140b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:06.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.771+0000 7fe71a99c700 1 -- 172.21.15.156:0/3741859325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7140b16d0 msgr2=0x7fe7141e63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:06.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.771+0000 7fe71a99c700 1 --2- 172.21.15.156:0/3741859325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7140b16d0 0x7fe7141e63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.771+0000 7fe71a99c700 1 -- 172.21.15.156:0/3741859325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7140b20b0 msgr2=0x7fe7140b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:06.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.771+0000 7fe71a99c700 1 --2- 172.21.15.156:0/3741859325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7140b20b0 0x7fe7140b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.771+0000 7fe71a99c700 1 -- 172.21.15.156:0/3741859325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe710009550 con 0x7fe7140d01b0 2023-11-28T13:08:06.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.771+0000 7fe71a99c700 1 --2- 172.21.15.156:0/3741859325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7140d01b0 0x7fe7140ca120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe70c007f20 tx=0x7fe70c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:06.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.771+0000 7fe70affd700 1 -- 172.21.15.156:0/3741859325 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00dde0 con 0x7fe7140d01b0 2023-11-28T13:08:06.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.771+0000 7fe70affd700 1 -- 172.21.15.156:0/3741859325 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c00f040 con 0x7fe7140d01b0 2023-11-28T13:08:06.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.771+0000 7fe71c3ff700 1 -- 172.21.15.156:0/3741859325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7140afec0 con 0x7fe7140d01b0 2023-11-28T13:08:06.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.771+0000 7fe70affd700 1 -- 172.21.15.156:0/3741859325 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c003c60 con 0x7fe7140d01b0 2023-11-28T13:08:06.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.772+0000 7fe71c3ff700 1 -- 172.21.15.156:0/3741859325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7140afbe0 con 0x7fe7140d01b0 2023-11-28T13:08:06.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.773+0000 7fe71c3ff700 1 -- 172.21.15.156:0/3741859325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7140b2600 con 0x7fe7140d01b0 2023-11-28T13:08:06.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.773+0000 7fe70affd700 1 -- 172.21.15.156:0/3741859325 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe70c01c030 con 0x7fe7140d01b0 2023-11-28T13:08:06.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.775+0000 7fe70affd700 1 --2- 172.21.15.156:0/3741859325 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6f80682f0 0x7fe6f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:06.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.775+0000 7fe70affd700 1 -- 172.21.15.156:0/3741859325 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe70c085d30 con 0x7fe7140d01b0 2023-11-28T13:08:06.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.775+0000 7fe71a19b700 1 --2- 172.21.15.156:0/3741859325 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6f80682f0 0x7fe6f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:06.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.776+0000 7fe71a19b700 1 --2- 172.21.15.156:0/3741859325 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6f80682f0 0x7fe6f806a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe710014040 tx=0x7fe710008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:06.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.779+0000 7fe70affd700 1 -- 172.21.15.156:0/3741859325 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe70c0524a0 con 0x7fe7140d01b0 2023-11-28T13:08:06.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.974+0000 7fe71c3ff700 1 -- 172.21.15.156:0/3741859325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe71409c7c0 con 0x7fe7140d01b0 2023-11-28T13:08:06.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.975+0000 7fe70affd700 1 -- 172.21.15.156:0/3741859325 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17897 (secure 0 0 0) 0x7fe70c01a090 con 0x7fe7140d01b0 2023-11-28T13:08:06.976 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:06.976 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":21,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:08:04.060053+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:08:00.903485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:08:02.037854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:08:06.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.978+0000 7fe71c3ff700 1 -- 172.21.15.156:0/3741859325 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6f80682f0 msgr2=0x7fe6f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:06.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.978+0000 7fe71c3ff700 1 --2- 172.21.15.156:0/3741859325 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6f80682f0 0x7fe6f806a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe710014040 tx=0x7fe710008030 comp rx=0 tx=0).stop 2023-11-28T13:08:06.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.978+0000 7fe71c3ff700 1 -- 172.21.15.156:0/3741859325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7140d01b0 msgr2=0x7fe7140ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:06.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.978+0000 7fe71c3ff700 1 --2- 172.21.15.156:0/3741859325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7140d01b0 0x7fe7140ca120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe70c007f20 tx=0x7fe70c00d3b0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.978+0000 7fe71c3ff700 1 -- 172.21.15.156:0/3741859325 shutdown_connections 2023-11-28T13:08:06.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.978+0000 7fe71c3ff700 1 --2- 172.21.15.156:0/3741859325 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6f80682f0 0x7fe6f806a7a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.978+0000 7fe71c3ff700 1 --2- 172.21.15.156:0/3741859325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7140d01b0 0x7fe7140ca120 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.978+0000 7fe71c3ff700 1 --2- 172.21.15.156:0/3741859325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7140b16d0 0x7fe7141e63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.978+0000 7fe71c3ff700 1 --2- 172.21.15.156:0/3741859325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7140b20b0 0x7fe7140b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:06.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.978+0000 7fe71c3ff700 1 -- 172.21.15.156:0/3741859325 >> 172.21.15.156:0/3741859325 conn(0x7fe71409c3f0 msgr2=0x7fe71406b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:06.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.978+0000 7fe71c3ff700 1 -- 172.21.15.156:0/3741859325 shutdown_connections 2023-11-28T13:08:06.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:06.978+0000 7fe71c3ff700 1 -- 172.21.15.156:0/3741859325 wait complete. 2023-11-28T13:08:06.997 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T13:08:06.998 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:08:07.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.194+0000 7fac08bff700 1 Processor -- start 2023-11-28T13:08:07.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.194+0000 7fac08bff700 1 -- start start 2023-11-28T13:08:07.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.195+0000 7fac08bff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fac0415c9b0 0x7fac040b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:07.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.195+0000 7fac08bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fac04161340 0x7fac040d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:07.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.195+0000 7fac08bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac040d2320 0x7fac040cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:07.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.195+0000 7fac08bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac040b5800 con 0x7fac04161340 2023-11-28T13:08:07.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.195+0000 7fac08bff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac040d1950 con 0x7fac0415c9b0 2023-11-28T13:08:07.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.195+0000 7fac08bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac040b5520 con 0x7fac040d2320 2023-11-28T13:08:07.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.195+0000 7fabfbfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fac04161340 0x7fac040d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:07.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.195+0000 7fabfbfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fac04161340 0x7fac040d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58694/0 (socket says 172.21.15.156:58694) 2023-11-28T13:08:07.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.195+0000 7fabfbfff700 1 -- 172.21.15.156:0/3051746450 learned_addr learned my addr 172.21.15.156:0/3051746450 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:07.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.195+0000 7fac0259c700 1 --2- 172.21.15.156:0/3051746450 >> v2:172.21.15.186:3300/0 conn(0x7fac0415c9b0 0x7fac040b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:07.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.195+0000 7fac02d9d700 1 --2- 172.21.15.156:0/3051746450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac040d2320 0x7fac040cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:07.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.196+0000 7fabfbfff700 1 -- 172.21.15.156:0/3051746450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac040d2320 msgr2=0x7fac040cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:07.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.196+0000 7fabfbfff700 1 --2- 172.21.15.156:0/3051746450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac040d2320 0x7fac040cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.196+0000 7fabfbfff700 1 -- 172.21.15.156:0/3051746450 >> v2:172.21.15.186:3300/0 conn(0x7fac0415c9b0 msgr2=0x7fac040b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:07.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.196+0000 7fabfbfff700 1 --2- 172.21.15.156:0/3051746450 >> v2:172.21.15.186:3300/0 conn(0x7fac0415c9b0 0x7fac040b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.196+0000 7fabfbfff700 1 -- 172.21.15.156:0/3051746450 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac040d1300 con 0x7fac04161340 2023-11-28T13:08:07.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.196+0000 7fabfbfff700 1 --2- 172.21.15.156:0/3051746450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac04161340 0x7fac040d1de0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fac04067460 tx=0x7fabec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b205e1f17854a81b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:07.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.196+0000 7fac01d9b700 1 -- 172.21.15.156:0/3051746450 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec00b870 con 0x7fac04161340 2023-11-28T13:08:07.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.196+0000 7fac01d9b700 1 -- 172.21.15.156:0/3051746450 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabec00b9d0 con 0x7fac04161340 2023-11-28T13:08:07.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.196+0000 7fac01d9b700 1 -- 172.21.15.156:0/3051746450 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec00bd60 con 0x7fac04161340 2023-11-28T13:08:07.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.197+0000 7fac08bff700 1 -- 172.21.15.156:0/3051746450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac04161340 msgr2=0x7fac040d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:07.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.197+0000 7fac08bff700 1 --2- 172.21.15.156:0/3051746450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac04161340 0x7fac040d1de0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fac04067460 tx=0x7fabec009bd0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.197+0000 7fac08bff700 1 -- 172.21.15.156:0/3051746450 shutdown_connections 2023-11-28T13:08:07.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.197+0000 7fac08bff700 1 --2- 172.21.15.156:0/3051746450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac040d2320 0x7fac040cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.197+0000 7fac08bff700 1 --2- 172.21.15.156:0/3051746450 >> v2:172.21.15.186:3300/0 conn(0x7fac0415c9b0 0x7fac040b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.197+0000 7fac08bff700 1 --2- 172.21.15.156:0/3051746450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac04161340 0x7fac040d1de0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.197+0000 7fac08bff700 1 -- 172.21.15.156:0/3051746450 >> 172.21.15.156:0/3051746450 conn(0x7fac04072600 msgr2=0x7fac040cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:07.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.197+0000 7fac08bff700 1 -- 172.21.15.156:0/3051746450 shutdown_connections 2023-11-28T13:08:07.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.197+0000 7fac08bff700 1 -- 172.21.15.156:0/3051746450 wait complete. 2023-11-28T13:08:07.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.199+0000 7fac08bff700 1 Processor -- start 2023-11-28T13:08:07.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.199+0000 7fac08bff700 1 -- start start 2023-11-28T13:08:07.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.200+0000 7fac08bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac040d2320 0x7fac041f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:07.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.200+0000 7fac08bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac0415c9b0 0x7fac041f19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:07.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.200+0000 7fac08bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac04161340 0x7fac0415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:07.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.200+0000 7fac08bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac041437d0 con 0x7fac040d2320 2023-11-28T13:08:07.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.200+0000 7fac08bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac041f6720 con 0x7fac04161340 2023-11-28T13:08:07.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.200+0000 7fac08bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac04151990 con 0x7fac0415c9b0 2023-11-28T13:08:07.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.200+0000 7fac0259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac040d2320 0x7fac041f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:07.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.200+0000 7fac0259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac040d2320 0x7fac041f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58710/0 (socket says 172.21.15.156:58710) 2023-11-28T13:08:07.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.200+0000 7fac0259c700 1 -- 172.21.15.156:0/592102100 learned_addr learned my addr 172.21.15.156:0/592102100 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:07.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.200+0000 7fac02d9d700 1 --2- 172.21.15.156:0/592102100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac04161340 0x7fac0415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:07.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.200+0000 7fabfbfff700 1 --2- 172.21.15.156:0/592102100 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac0415c9b0 0x7fac041f19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:07.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.200+0000 7fac0259c700 1 -- 172.21.15.156:0/592102100 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac0415c9b0 msgr2=0x7fac041f19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:07.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.200+0000 7fac0259c700 1 --2- 172.21.15.156:0/592102100 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac0415c9b0 0x7fac041f19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.200+0000 7fac0259c700 1 -- 172.21.15.156:0/592102100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac04161340 msgr2=0x7fac0415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:07.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.200+0000 7fac0259c700 1 --2- 172.21.15.156:0/592102100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac04161340 0x7fac0415ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.200+0000 7fac0259c700 1 -- 172.21.15.156:0/592102100 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec009550 con 0x7fac040d2320 2023-11-28T13:08:07.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.200+0000 7fabfbfff700 1 --2- 172.21.15.156:0/592102100 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac0415c9b0 0x7fac041f19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:07.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.201+0000 7fac0259c700 1 --2- 172.21.15.156:0/592102100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac040d2320 0x7fac041f61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fabf400e810 tx=0x7fabf400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:07.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.201+0000 7fabfaffd700 1 -- 172.21.15.156:0/592102100 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf4013610 con 0x7fac040d2320 2023-11-28T13:08:07.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.201+0000 7fabfaffd700 1 -- 172.21.15.156:0/592102100 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf4013770 con 0x7fac040d2320 2023-11-28T13:08:07.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.201+0000 7fac08bff700 1 -- 172.21.15.156:0/592102100 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac040b5130 con 0x7fac040d2320 2023-11-28T13:08:07.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.201+0000 7fabfaffd700 1 -- 172.21.15.156:0/592102100 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf4013b20 con 0x7fac040d2320 2023-11-28T13:08:07.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.201+0000 7fac08bff700 1 -- 172.21.15.156:0/592102100 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac040d15e0 con 0x7fac040d2320 2023-11-28T13:08:07.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.202+0000 7fac08bff700 1 -- 172.21.15.156:0/592102100 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac040b52c0 con 0x7fac040d2320 2023-11-28T13:08:07.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.203+0000 7fabfaffd700 1 -- 172.21.15.156:0/592102100 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fabf4023070 con 0x7fac040d2320 2023-11-28T13:08:07.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.203+0000 7fabfaffd700 1 --2- 172.21.15.156:0/592102100 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fabe40682f0 0x7fabe406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:07.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.204+0000 7fabfaffd700 1 -- 172.21.15.156:0/592102100 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fabf4085a80 con 0x7fac040d2320 2023-11-28T13:08:07.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.204+0000 7fabfbfff700 1 --2- 172.21.15.156:0/592102100 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fabe40682f0 0x7fabe406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:07.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.205+0000 7fabfbfff700 1 --2- 172.21.15.156:0/592102100 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fabe40682f0 0x7fabe406a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fabec000c00 tx=0x7fabec008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:07.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.207+0000 7fabfaffd700 1 -- 172.21.15.156:0/592102100 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fabf4052170 con 0x7fac040d2320 2023-11-28T13:08:07.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.391+0000 7fac08bff700 1 -- 172.21.15.156:0/592102100 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fac040745c0 con 0x7fabe40682f0 2023-11-28T13:08:07.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.404+0000 7fabfaffd700 1 -- 172.21.15.156:0/592102100 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287003 (secure 0 0 0) 0x7fac040745c0 con 0x7fabe40682f0 2023-11-28T13:08:07.409 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:07.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.409+0000 7fac08bff700 1 -- 172.21.15.156:0/592102100 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fabe40682f0 msgr2=0x7fabe406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:07.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.409+0000 7fac08bff700 1 --2- 172.21.15.156:0/592102100 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fabe40682f0 0x7fabe406a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fabec000c00 tx=0x7fabec008030 comp rx=0 tx=0).stop 2023-11-28T13:08:07.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.409+0000 7fac08bff700 1 -- 172.21.15.156:0/592102100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac040d2320 msgr2=0x7fac041f61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:07.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.409+0000 7fac08bff700 1 --2- 172.21.15.156:0/592102100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac040d2320 0x7fac041f61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fabf400e810 tx=0x7fabf400ebd0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.409+0000 7fac08bff700 1 -- 172.21.15.156:0/592102100 shutdown_connections 2023-11-28T13:08:07.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.409+0000 7fac08bff700 1 --2- 172.21.15.156:0/592102100 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fabe40682f0 0x7fabe406a7a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.409+0000 7fac08bff700 1 --2- 172.21.15.156:0/592102100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac04161340 0x7fac0415ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.409+0000 7fac08bff700 1 --2- 172.21.15.156:0/592102100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac040d2320 0x7fac041f61e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.409+0000 7fac08bff700 1 --2- 172.21.15.156:0/592102100 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac0415c9b0 0x7fac041f19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.409+0000 7fac08bff700 1 -- 172.21.15.156:0/592102100 >> 172.21.15.156:0/592102100 conn(0x7fac04072600 msgr2=0x7fac0415d020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:07.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.409+0000 7fac08bff700 1 -- 172.21.15.156:0/592102100 shutdown_connections 2023-11-28T13:08:07.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.409+0000 7fac08bff700 1 -- 172.21.15.156:0/592102100 wait complete. 2023-11-28T13:08:07.414 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T13:08:07.428 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-11-28T13:08:06.014466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":82456,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":79488,"kb_avail":754892264,"statfs":{"total":773094113280,"available":773009678336,"internally_reserved":0,"allocated":2973696,"data_stored":1813437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81395712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.460715"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371323+0000","last_change":"2023-11-28T13:08:03.085544+0000","last_active":"2023-11-28T13:08:04.371323+0000","last_peered":"2023-11-28T13:08:04.371323+0000","last_clean":"2023-11-28T13:08:04.371323+0000","last_became_active":"2023-11-28T13:08:03.085265+0000","last_became_peered":"2023-11-28T13:08:03.085265+0000","last_unstale":"2023-11-28T13:08:04.371323+0000","last_undegraded":"2023-11-28T13:08:04.371323+0000","last_fullsized":"2023-11-28T13:08:04.371323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:37.423803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069498+0000","last_change":"2023-11-28T13:08:03.084729+0000","last_active":"2023-11-28T13:08:04.069498+0000","last_peered":"2023-11-28T13:08:04.069498+0000","last_clean":"2023-11-28T13:08:04.069498+0000","last_became_active":"2023-11-28T13:08:03.084473+0000","last_became_peered":"2023-11-28T13:08:03.084473+0000","last_unstale":"2023-11-28T13:08:04.069498+0000","last_undegraded":"2023-11-28T13:08:04.069498+0000","last_fullsized":"2023-11-28T13:08:04.069498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:53.049281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371761+0000","last_change":"2023-11-28T13:08:03.086292+0000","last_active":"2023-11-28T13:08:04.371761+0000","last_peered":"2023-11-28T13:08:04.371761+0000","last_clean":"2023-11-28T13:08:04.371761+0000","last_became_active":"2023-11-28T13:08:03.085984+0000","last_became_peered":"2023-11-28T13:08:03.085984+0000","last_unstale":"2023-11-28T13:08:04.371761+0000","last_undegraded":"2023-11-28T13:08:04.371761+0000","last_fullsized":"2023-11-28T13:08:04.371761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:58.749445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372036+0000","last_change":"2023-11-28T13:08:03.080558+0000","last_active":"2023-11-28T13:08:04.372036+0000","last_peered":"2023-11-28T13:08:04.372036+0000","last_clean":"2023-11-28T13:08:04.372036+0000","last_became_active":"2023-11-28T13:08:03.080267+0000","last_became_peered":"2023-11-28T13:08:03.080267+0000","last_unstale":"2023-11-28T13:08:04.372036+0000","last_undegraded":"2023-11-28T13:08:04.372036+0000","last_fullsized":"2023-11-28T13:08:04.372036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:38.726877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372532+0000","last_change":"2023-11-28T13:08:03.081890+0000","last_active":"2023-11-28T13:08:04.372532+0000","last_peered":"2023-11-28T13:08:04.372532+0000","last_clean":"2023-11-28T13:08:04.372532+0000","last_became_active":"2023-11-28T13:08:03.081638+0000","last_became_peered":"2023-11-28T13:08:03.081638+0000","last_unstale":"2023-11-28T13:08:04.372532+0000","last_undegraded":"2023-11-28T13:08:04.372532+0000","last_fullsized":"2023-11-28T13:08:04.372532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:53.049281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371404+0000","last_change":"2023-11-28T13:08:03.081127+0000","last_active":"2023-11-28T13:08:04.371404+0000","last_peered":"2023-11-28T13:08:04.371404+0000","last_clean":"2023-11-28T13:08:04.371404+0000","last_became_active":"2023-11-28T13:08:03.077682+0000","last_became_peered":"2023-11-28T13:08:03.077682+0000","last_unstale":"2023-11-28T13:08:04.371404+0000","last_undegraded":"2023-11-28T13:08:04.371404+0000","last_fullsized":"2023-11-28T13:08:04.371404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:58.947317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070150+0000","last_change":"2023-11-28T13:08:03.099999+0000","last_active":"2023-11-28T13:08:04.070150+0000","last_peered":"2023-11-28T13:08:04.070150+0000","last_clean":"2023-11-28T13:08:04.070150+0000","last_became_active":"2023-11-28T13:08:03.099733+0000","last_became_peered":"2023-11-28T13:08:03.099733+0000","last_unstale":"2023-11-28T13:08:04.070150+0000","last_undegraded":"2023-11-28T13:08:04.070150+0000","last_fullsized":"2023-11-28T13:08:04.070150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:46.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069502+0000","last_change":"2023-11-28T13:08:03.094943+0000","last_active":"2023-11-28T13:08:04.069502+0000","last_peered":"2023-11-28T13:08:04.069502+0000","last_clean":"2023-11-28T13:08:04.069502+0000","last_became_active":"2023-11-28T13:08:03.094688+0000","last_became_peered":"2023-11-28T13:08:03.094688+0000","last_unstale":"2023-11-28T13:08:04.069502+0000","last_undegraded":"2023-11-28T13:08:04.069502+0000","last_fullsized":"2023-11-28T13:08:04.069502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:14.015315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372476+0000","last_change":"2023-11-28T13:08:03.083815+0000","last_active":"2023-11-28T13:08:04.372476+0000","last_peered":"2023-11-28T13:08:04.372476+0000","last_clean":"2023-11-28T13:08:04.372476+0000","last_became_active":"2023-11-28T13:08:03.083292+0000","last_became_peered":"2023-11-28T13:08:03.083292+0000","last_unstale":"2023-11-28T13:08:04.372476+0000","last_undegraded":"2023-11-28T13:08:04.372476+0000","last_fullsized":"2023-11-28T13:08:04.372476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:14.015315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070743+0000","last_change":"2023-11-28T13:08:03.083685+0000","last_active":"2023-11-28T13:08:04.070743+0000","last_peered":"2023-11-28T13:08:04.070743+0000","last_clean":"2023-11-28T13:08:04.070743+0000","last_became_active":"2023-11-28T13:08:03.077038+0000","last_became_peered":"2023-11-28T13:08:03.077038+0000","last_unstale":"2023-11-28T13:08:04.070743+0000","last_undegraded":"2023-11-28T13:08:04.070743+0000","last_fullsized":"2023-11-28T13:08:04.070743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:34.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371966+0000","last_change":"2023-11-28T13:08:03.085905+0000","last_active":"2023-11-28T13:08:04.371966+0000","last_peered":"2023-11-28T13:08:04.371966+0000","last_clean":"2023-11-28T13:08:04.371966+0000","last_became_active":"2023-11-28T13:08:03.085524+0000","last_became_peered":"2023-11-28T13:08:03.085524+0000","last_unstale":"2023-11-28T13:08:04.371966+0000","last_undegraded":"2023-11-28T13:08:04.371966+0000","last_fullsized":"2023-11-28T13:08:04.371966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:34.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371160+0000","last_change":"2023-11-28T13:08:03.091440+0000","last_active":"2023-11-28T13:08:04.371160+0000","last_peered":"2023-11-28T13:08:04.371160+0000","last_clean":"2023-11-28T13:08:04.371160+0000","last_became_active":"2023-11-28T13:08:03.091117+0000","last_became_peered":"2023-11-28T13:08:03.091117+0000","last_unstale":"2023-11-28T13:08:04.371160+0000","last_undegraded":"2023-11-28T13:08:04.371160+0000","last_fullsized":"2023-11-28T13:08:04.371160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:41.849774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371255+0000","last_change":"2023-11-28T13:08:03.080912+0000","last_active":"2023-11-28T13:08:04.371255+0000","last_peered":"2023-11-28T13:08:04.371255+0000","last_clean":"2023-11-28T13:08:04.371255+0000","last_became_active":"2023-11-28T13:08:03.080578+0000","last_became_peered":"2023-11-28T13:08:03.080578+0000","last_unstale":"2023-11-28T13:08:04.371255+0000","last_undegraded":"2023-11-28T13:08:04.371255+0000","last_fullsized":"2023-11-28T13:08:04.371255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:38.657150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069701+0000","last_change":"2023-11-28T13:08:03.083208+0000","last_active":"2023-11-28T13:08:04.069701+0000","last_peered":"2023-11-28T13:08:04.069701+0000","last_clean":"2023-11-28T13:08:04.069701+0000","last_became_active":"2023-11-28T13:08:03.082944+0000","last_became_peered":"2023-11-28T13:08:03.082944+0000","last_unstale":"2023-11-28T13:08:04.069701+0000","last_undegraded":"2023-11-28T13:08:04.069701+0000","last_fullsized":"2023-11-28T13:08:04.069701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:36.892676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070334+0000","last_change":"2023-11-28T13:08:03.098345+0000","last_active":"2023-11-28T13:08:04.070334+0000","last_peered":"2023-11-28T13:08:04.070334+0000","last_clean":"2023-11-28T13:08:04.070334+0000","last_became_active":"2023-11-28T13:08:03.098105+0000","last_became_peered":"2023-11-28T13:08:03.098105+0000","last_unstale":"2023-11-28T13:08:04.070334+0000","last_undegraded":"2023-11-28T13:08:04.070334+0000","last_fullsized":"2023-11-28T13:08:04.070334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:43.340474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371254+0000","last_change":"2023-11-28T13:08:03.076945+0000","last_active":"2023-11-28T13:08:04.371254+0000","last_peered":"2023-11-28T13:08:04.371254+0000","last_clean":"2023-11-28T13:08:04.371254+0000","last_became_active":"2023-11-28T13:08:03.076639+0000","last_became_peered":"2023-11-28T13:08:03.076639+0000","last_unstale":"2023-11-28T13:08:04.371254+0000","last_undegraded":"2023-11-28T13:08:04.371254+0000","last_fullsized":"2023-11-28T13:08:04.371254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:41.076578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372578+0000","last_change":"2023-11-28T13:08:03.088044+0000","last_active":"2023-11-28T13:08:04.372578+0000","last_peered":"2023-11-28T13:08:04.372578+0000","last_clean":"2023-11-28T13:08:04.372578+0000","last_became_active":"2023-11-28T13:08:03.087703+0000","last_became_peered":"2023-11-28T13:08:03.087703+0000","last_unstale":"2023-11-28T13:08:04.372578+0000","last_undegraded":"2023-11-28T13:08:04.372578+0000","last_fullsized":"2023-11-28T13:08:04.372578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:43.340474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372432+0000","last_change":"2023-11-28T13:08:03.077748+0000","last_active":"2023-11-28T13:08:04.372432+0000","last_peered":"2023-11-28T13:08:04.372432+0000","last_clean":"2023-11-28T13:08:04.372432+0000","last_became_active":"2023-11-28T13:08:03.077488+0000","last_became_peered":"2023-11-28T13:08:03.077488+0000","last_unstale":"2023-11-28T13:08:04.372432+0000","last_undegraded":"2023-11-28T13:08:04.372432+0000","last_fullsized":"2023-11-28T13:08:04.372432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:14.015315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371120+0000","last_change":"2023-11-28T13:08:03.079221+0000","last_active":"2023-11-28T13:08:04.371120+0000","last_peered":"2023-11-28T13:08:04.371120+0000","last_clean":"2023-11-28T13:08:04.371120+0000","last_became_active":"2023-11-28T13:08:03.077084+0000","last_became_peered":"2023-11-28T13:08:03.077084+0000","last_unstale":"2023-11-28T13:08:04.371120+0000","last_undegraded":"2023-11-28T13:08:04.371120+0000","last_fullsized":"2023-11-28T13:08:04.371120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:02.499466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371692+0000","last_change":"2023-11-28T13:08:03.083322+0000","last_active":"2023-11-28T13:08:04.371692+0000","last_peered":"2023-11-28T13:08:04.371692+0000","last_clean":"2023-11-28T13:08:04.371692+0000","last_became_active":"2023-11-28T13:08:03.083064+0000","last_became_peered":"2023-11-28T13:08:03.083064+0000","last_unstale":"2023-11-28T13:08:04.371692+0000","last_undegraded":"2023-11-28T13:08:04.371692+0000","last_fullsized":"2023-11-28T13:08:04.371692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:13.351182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070670+0000","last_change":"2023-11-28T13:08:03.095031+0000","last_active":"2023-11-28T13:08:04.070670+0000","last_peered":"2023-11-28T13:08:04.070670+0000","last_clean":"2023-11-28T13:08:04.070670+0000","last_became_active":"2023-11-28T13:08:03.094746+0000","last_became_peered":"2023-11-28T13:08:03.094746+0000","last_unstale":"2023-11-28T13:08:04.070670+0000","last_undegraded":"2023-11-28T13:08:04.070670+0000","last_fullsized":"2023-11-28T13:08:04.070670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:49.707036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372182+0000","last_change":"2023-11-28T13:08:03.082127+0000","last_active":"2023-11-28T13:08:04.372182+0000","last_peered":"2023-11-28T13:08:04.372182+0000","last_clean":"2023-11-28T13:08:04.372182+0000","last_became_active":"2023-11-28T13:08:03.081861+0000","last_became_peered":"2023-11-28T13:08:03.081861+0000","last_unstale":"2023-11-28T13:08:04.372182+0000","last_undegraded":"2023-11-28T13:08:04.372182+0000","last_fullsized":"2023-11-28T13:08:04.372182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:05:35.135974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064206+0000","last_change":"2023-11-28T13:08:03.083718+0000","last_active":"2023-11-28T13:08:04.064206+0000","last_peered":"2023-11-28T13:08:04.064206+0000","last_clean":"2023-11-28T13:08:04.064206+0000","last_became_active":"2023-11-28T13:08:03.082420+0000","last_became_peered":"2023-11-28T13:08:03.082420+0000","last_unstale":"2023-11-28T13:08:04.064206+0000","last_undegraded":"2023-11-28T13:08:04.064206+0000","last_fullsized":"2023-11-28T13:08:04.064206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:58.181120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070265+0000","last_change":"2023-11-28T13:08:03.095780+0000","last_active":"2023-11-28T13:08:04.070265+0000","last_peered":"2023-11-28T13:08:04.070265+0000","last_clean":"2023-11-28T13:08:04.070265+0000","last_became_active":"2023-11-28T13:08:03.095522+0000","last_became_peered":"2023-11-28T13:08:03.095522+0000","last_unstale":"2023-11-28T13:08:04.070265+0000","last_undegraded":"2023-11-28T13:08:04.070265+0000","last_fullsized":"2023-11-28T13:08:04.070265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:37:44.657148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371784+0000","last_change":"2023-11-28T13:08:03.081168+0000","last_active":"2023-11-28T13:08:04.371784+0000","last_peered":"2023-11-28T13:08:04.371784+0000","last_clean":"2023-11-28T13:08:04.371784+0000","last_became_active":"2023-11-28T13:08:03.080867+0000","last_became_peered":"2023-11-28T13:08:03.080867+0000","last_unstale":"2023-11-28T13:08:04.371784+0000","last_undegraded":"2023-11-28T13:08:04.371784+0000","last_fullsized":"2023-11-28T13:08:04.371784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:44.188797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371597+0000","last_change":"2023-11-28T13:08:03.074696+0000","last_active":"2023-11-28T13:08:04.371597+0000","last_peered":"2023-11-28T13:08:04.371597+0000","last_clean":"2023-11-28T13:08:04.371597+0000","last_became_active":"2023-11-28T13:08:03.074392+0000","last_became_peered":"2023-11-28T13:08:03.074392+0000","last_unstale":"2023-11-28T13:08:04.371597+0000","last_undegraded":"2023-11-28T13:08:04.371597+0000","last_fullsized":"2023-11-28T13:08:04.371597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:50.062766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070127+0000","last_change":"2023-11-28T13:08:03.092477+0000","last_active":"2023-11-28T13:08:04.070127+0000","last_peered":"2023-11-28T13:08:04.070127+0000","last_clean":"2023-11-28T13:08:04.070127+0000","last_became_active":"2023-11-28T13:08:03.092243+0000","last_became_peered":"2023-11-28T13:08:03.092243+0000","last_unstale":"2023-11-28T13:08:04.070127+0000","last_undegraded":"2023-11-28T13:08:04.070127+0000","last_fullsized":"2023-11-28T13:08:04.070127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:46.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371521+0000","last_change":"2023-11-28T13:08:02.582255+0000","last_active":"2023-11-28T13:08:04.371521+0000","last_peered":"2023-11-28T13:08:04.371521+0000","last_clean":"2023-11-28T13:08:04.371521+0000","last_became_active":"2023-11-28T13:08:02.581960+0000","last_became_peered":"2023-11-28T13:08:02.581960+0000","last_unstale":"2023-11-28T13:08:04.371521+0000","last_undegraded":"2023-11-28T13:08:04.371521+0000","last_fullsized":"2023-11-28T13:08:04.371521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:49.593563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064820+0000","last_change":"2023-11-28T13:08:03.080261+0000","last_active":"2023-11-28T13:08:04.064820+0000","last_peered":"2023-11-28T13:08:04.064820+0000","last_clean":"2023-11-28T13:08:04.064820+0000","last_became_active":"2023-11-28T13:08:03.079964+0000","last_became_peered":"2023-11-28T13:08:03.079964+0000","last_unstale":"2023-11-28T13:08:04.064820+0000","last_undegraded":"2023-11-28T13:08:04.064820+0000","last_fullsized":"2023-11-28T13:08:04.064820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:30.263143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371820+0000","last_change":"2023-11-28T13:08:02.575220+0000","last_active":"2023-11-28T13:08:04.371820+0000","last_peered":"2023-11-28T13:08:04.371820+0000","last_clean":"2023-11-28T13:08:04.371820+0000","last_became_active":"2023-11-28T13:08:02.574933+0000","last_became_peered":"2023-11-28T13:08:02.574933+0000","last_unstale":"2023-11-28T13:08:04.371820+0000","last_undegraded":"2023-11-28T13:08:04.371820+0000","last_fullsized":"2023-11-28T13:08:04.371820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:56.910035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372498+0000","last_change":"2023-11-28T13:08:03.077902+0000","last_active":"2023-11-28T13:08:04.372498+0000","last_peered":"2023-11-28T13:08:04.372498+0000","last_clean":"2023-11-28T13:08:04.372498+0000","last_became_active":"2023-11-28T13:08:03.077379+0000","last_became_peered":"2023-11-28T13:08:03.077379+0000","last_unstale":"2023-11-28T13:08:04.372498+0000","last_undegraded":"2023-11-28T13:08:04.372498+0000","last_fullsized":"2023-11-28T13:08:04.372498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:34.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064779+0000","last_change":"2023-11-28T13:08:02.574513+0000","last_active":"2023-11-28T13:08:04.064779+0000","last_peered":"2023-11-28T13:08:04.064779+0000","last_clean":"2023-11-28T13:08:04.064779+0000","last_became_active":"2023-11-28T13:08:02.573509+0000","last_became_peered":"2023-11-28T13:08:02.573509+0000","last_unstale":"2023-11-28T13:08:04.064779+0000","last_undegraded":"2023-11-28T13:08:04.064779+0000","last_fullsized":"2023-11-28T13:08:04.064779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:01.496804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372381+0000","last_change":"2023-11-28T13:08:03.091021+0000","last_active":"2023-11-28T13:08:04.372381+0000","last_peered":"2023-11-28T13:08:04.372381+0000","last_clean":"2023-11-28T13:08:04.372381+0000","last_became_active":"2023-11-28T13:08:03.090717+0000","last_became_peered":"2023-11-28T13:08:03.090717+0000","last_unstale":"2023-11-28T13:08:04.372381+0000","last_undegraded":"2023-11-28T13:08:04.372381+0000","last_fullsized":"2023-11-28T13:08:04.372381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:50.062766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372655+0000","last_change":"2023-11-28T13:08:02.580525+0000","last_active":"2023-11-28T13:08:04.372655+0000","last_peered":"2023-11-28T13:08:04.372655+0000","last_clean":"2023-11-28T13:08:04.372655+0000","last_became_active":"2023-11-28T13:08:02.580220+0000","last_became_peered":"2023-11-28T13:08:02.580220+0000","last_unstale":"2023-11-28T13:08:04.372655+0000","last_undegraded":"2023-11-28T13:08:04.372655+0000","last_fullsized":"2023-11-28T13:08:04.372655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:01.496804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064707+0000","last_change":"2023-11-28T13:08:03.080244+0000","last_active":"2023-11-28T13:08:04.064707+0000","last_peered":"2023-11-28T13:08:04.064707+0000","last_clean":"2023-11-28T13:08:04.064707+0000","last_became_active":"2023-11-28T13:08:03.079988+0000","last_became_peered":"2023-11-28T13:08:03.079988+0000","last_unstale":"2023-11-28T13:08:04.064707+0000","last_undegraded":"2023-11-28T13:08:04.064707+0000","last_fullsized":"2023-11-28T13:08:04.064707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:49.707036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070421+0000","last_change":"2023-11-28T13:08:02.573388+0000","last_active":"2023-11-28T13:08:04.070421+0000","last_peered":"2023-11-28T13:08:04.070421+0000","last_clean":"2023-11-28T13:08:04.070421+0000","last_became_active":"2023-11-28T13:08:02.573059+0000","last_became_peered":"2023-11-28T13:08:02.573059+0000","last_unstale":"2023-11-28T13:08:04.070421+0000","last_undegraded":"2023-11-28T13:08:04.070421+0000","last_fullsized":"2023-11-28T13:08:04.070421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:46:29.622650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064417+0000","last_change":"2023-11-28T13:08:03.088704+0000","last_active":"2023-11-28T13:08:04.064417+0000","last_peered":"2023-11-28T13:08:04.064417+0000","last_clean":"2023-11-28T13:08:04.064417+0000","last_became_active":"2023-11-28T13:08:03.088417+0000","last_became_peered":"2023-11-28T13:08:03.088417+0000","last_unstale":"2023-11-28T13:08:04.064417+0000","last_undegraded":"2023-11-28T13:08:04.064417+0000","last_fullsized":"2023-11-28T13:08:04.064417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:41.196768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070940+0000","last_change":"2023-11-28T13:08:02.576140+0000","last_active":"2023-11-28T13:08:04.070940+0000","last_peered":"2023-11-28T13:08:04.070940+0000","last_clean":"2023-11-28T13:08:04.070940+0000","last_became_active":"2023-11-28T13:08:02.575721+0000","last_became_peered":"2023-11-28T13:08:02.575721+0000","last_unstale":"2023-11-28T13:08:04.070940+0000","last_undegraded":"2023-11-28T13:08:04.070940+0000","last_fullsized":"2023-11-28T13:08:04.070940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:12.214372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371693+0000","last_change":"2023-11-28T13:08:03.079421+0000","last_active":"2023-11-28T13:08:04.371693+0000","last_peered":"2023-11-28T13:08:04.371693+0000","last_clean":"2023-11-28T13:08:04.371693+0000","last_became_active":"2023-11-28T13:08:03.078471+0000","last_became_peered":"2023-11-28T13:08:03.078471+0000","last_unstale":"2023-11-28T13:08:04.371693+0000","last_undegraded":"2023-11-28T13:08:04.371693+0000","last_fullsized":"2023-11-28T13:08:04.371693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:05:35.135974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371902+0000","last_change":"2023-11-28T13:08:02.581355+0000","last_active":"2023-11-28T13:08:04.371902+0000","last_peered":"2023-11-28T13:08:04.371902+0000","last_clean":"2023-11-28T13:08:04.371902+0000","last_became_active":"2023-11-28T13:08:02.581051+0000","last_became_peered":"2023-11-28T13:08:02.581051+0000","last_unstale":"2023-11-28T13:08:04.371902+0000","last_undegraded":"2023-11-28T13:08:04.371902+0000","last_fullsized":"2023-11-28T13:08:04.371902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:43.117989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371569+0000","last_change":"2023-11-28T13:08:03.085302+0000","last_active":"2023-11-28T13:08:04.371569+0000","last_peered":"2023-11-28T13:08:04.371569+0000","last_clean":"2023-11-28T13:08:04.371569+0000","last_became_active":"2023-11-28T13:08:03.085061+0000","last_became_peered":"2023-11-28T13:08:03.085061+0000","last_unstale":"2023-11-28T13:08:04.371569+0000","last_undegraded":"2023-11-28T13:08:04.371569+0000","last_fullsized":"2023-11-28T13:08:04.371569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:13.625233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070589+0000","last_change":"2023-11-28T13:08:02.584763+0000","last_active":"2023-11-28T13:08:04.070589+0000","last_peered":"2023-11-28T13:08:04.070589+0000","last_clean":"2023-11-28T13:08:04.070589+0000","last_became_active":"2023-11-28T13:08:02.583655+0000","last_became_peered":"2023-11-28T13:08:02.583655+0000","last_unstale":"2023-11-28T13:08:04.070589+0000","last_undegraded":"2023-11-28T13:08:04.070589+0000","last_fullsized":"2023-11-28T13:08:04.070589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:53:59.398155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064403+0000","last_change":"2023-11-28T13:08:03.095515+0000","last_active":"2023-11-28T13:08:04.064403+0000","last_peered":"2023-11-28T13:08:04.064403+0000","last_clean":"2023-11-28T13:08:04.064403+0000","last_became_active":"2023-11-28T13:08:03.095004+0000","last_became_peered":"2023-11-28T13:08:03.095004+0000","last_unstale":"2023-11-28T13:08:04.064403+0000","last_undegraded":"2023-11-28T13:08:04.064403+0000","last_fullsized":"2023-11-28T13:08:04.064403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:34.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372572+0000","last_change":"2023-11-28T13:08:02.580082+0000","last_active":"2023-11-28T13:08:04.372572+0000","last_peered":"2023-11-28T13:08:04.372572+0000","last_clean":"2023-11-28T13:08:04.372572+0000","last_became_active":"2023-11-28T13:08:02.579745+0000","last_became_peered":"2023-11-28T13:08:02.579745+0000","last_unstale":"2023-11-28T13:08:04.372572+0000","last_undegraded":"2023-11-28T13:08:04.372572+0000","last_fullsized":"2023-11-28T13:08:04.372572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:12.363239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070163+0000","last_change":"2023-11-28T13:08:03.082966+0000","last_active":"2023-11-28T13:08:04.070163+0000","last_peered":"2023-11-28T13:08:04.070163+0000","last_clean":"2023-11-28T13:08:04.070163+0000","last_became_active":"2023-11-28T13:08:03.082710+0000","last_became_peered":"2023-11-28T13:08:03.082710+0000","last_unstale":"2023-11-28T13:08:04.070163+0000","last_undegraded":"2023-11-28T13:08:04.070163+0000","last_fullsized":"2023-11-28T13:08:04.070163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:49.707036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070236+0000","last_change":"2023-11-28T13:08:02.585538+0000","last_active":"2023-11-28T13:08:04.070236+0000","last_peered":"2023-11-28T13:08:04.070236+0000","last_clean":"2023-11-28T13:08:04.070236+0000","last_became_active":"2023-11-28T13:08:02.584407+0000","last_became_peered":"2023-11-28T13:08:02.584407+0000","last_unstale":"2023-11-28T13:08:04.070236+0000","last_undegraded":"2023-11-28T13:08:04.070236+0000","last_fullsized":"2023-11-28T13:08:04.070236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:01.496804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372389+0000","last_change":"2023-11-28T13:08:03.077754+0000","last_active":"2023-11-28T13:08:04.372389+0000","last_peered":"2023-11-28T13:08:04.372389+0000","last_clean":"2023-11-28T13:08:04.372389+0000","last_became_active":"2023-11-28T13:08:03.077073+0000","last_became_peered":"2023-11-28T13:08:03.077073+0000","last_unstale":"2023-11-28T13:08:04.372389+0000","last_undegraded":"2023-11-28T13:08:04.372389+0000","last_fullsized":"2023-11-28T13:08:04.372389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:46.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371255+0000","last_change":"2023-11-28T13:08:02.574654+0000","last_active":"2023-11-28T13:08:04.371255+0000","last_peered":"2023-11-28T13:08:04.371255+0000","last_clean":"2023-11-28T13:08:04.371255+0000","last_became_active":"2023-11-28T13:08:02.574354+0000","last_became_peered":"2023-11-28T13:08:02.574354+0000","last_unstale":"2023-11-28T13:08:04.371255+0000","last_undegraded":"2023-11-28T13:08:04.371255+0000","last_fullsized":"2023-11-28T13:08:04.371255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:43.117989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064965+0000","last_change":"2023-11-28T13:08:03.084191+0000","last_active":"2023-11-28T13:08:04.064965+0000","last_peered":"2023-11-28T13:08:04.064965+0000","last_clean":"2023-11-28T13:08:04.064965+0000","last_became_active":"2023-11-28T13:08:03.083879+0000","last_became_peered":"2023-11-28T13:08:03.083879+0000","last_unstale":"2023-11-28T13:08:04.064965+0000","last_undegraded":"2023-11-28T13:08:04.064965+0000","last_fullsized":"2023-11-28T13:08:04.064965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:14.015315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.071066+0000","last_change":"2023-11-28T13:08:02.584107+0000","last_active":"2023-11-28T13:08:04.071066+0000","last_peered":"2023-11-28T13:08:04.071066+0000","last_clean":"2023-11-28T13:08:04.071066+0000","last_became_active":"2023-11-28T13:08:02.583702+0000","last_became_peered":"2023-11-28T13:08:02.583702+0000","last_unstale":"2023-11-28T13:08:04.071066+0000","last_undegraded":"2023-11-28T13:08:04.071066+0000","last_fullsized":"2023-11-28T13:08:04.071066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:37.724216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069638+0000","last_change":"2023-11-28T13:08:03.077169+0000","last_active":"2023-11-28T13:08:04.069638+0000","last_peered":"2023-11-28T13:08:04.069638+0000","last_clean":"2023-11-28T13:08:04.069638+0000","last_became_active":"2023-11-28T13:08:03.076875+0000","last_became_peered":"2023-11-28T13:08:03.076875+0000","last_unstale":"2023-11-28T13:08:04.069638+0000","last_undegraded":"2023-11-28T13:08:04.069638+0000","last_fullsized":"2023-11-28T13:08:04.069638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:37:44.657148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.071007+0000","last_change":"2023-11-28T13:08:02.584070+0000","last_active":"2023-11-28T13:08:04.071007+0000","last_peered":"2023-11-28T13:08:04.071007+0000","last_clean":"2023-11-28T13:08:04.071007+0000","last_became_active":"2023-11-28T13:08:02.583527+0000","last_became_peered":"2023-11-28T13:08:02.583527+0000","last_unstale":"2023-11-28T13:08:04.071007+0000","last_undegraded":"2023-11-28T13:08:04.071007+0000","last_fullsized":"2023-11-28T13:08:04.071007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:49.060104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371361+0000","last_change":"2023-11-28T13:08:03.082338+0000","last_active":"2023-11-28T13:08:04.371361+0000","last_peered":"2023-11-28T13:08:04.371361+0000","last_clean":"2023-11-28T13:08:04.371361+0000","last_became_active":"2023-11-28T13:08:03.082006+0000","last_became_peered":"2023-11-28T13:08:03.082006+0000","last_unstale":"2023-11-28T13:08:04.371361+0000","last_undegraded":"2023-11-28T13:08:04.371361+0000","last_fullsized":"2023-11-28T13:08:04.371361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:56:48.282770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070812+0000","last_change":"2023-11-28T13:08:02.587071+0000","last_active":"2023-11-28T13:08:04.070812+0000","last_peered":"2023-11-28T13:08:04.070812+0000","last_clean":"2023-11-28T13:08:04.070812+0000","last_became_active":"2023-11-28T13:08:02.586541+0000","last_became_peered":"2023-11-28T13:08:02.586541+0000","last_unstale":"2023-11-28T13:08:04.070812+0000","last_undegraded":"2023-11-28T13:08:04.070812+0000","last_fullsized":"2023-11-28T13:08:04.070812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:56.535608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372224+0000","last_change":"2023-11-28T13:08:03.081008+0000","last_active":"2023-11-28T13:08:04.372224+0000","last_peered":"2023-11-28T13:08:04.372224+0000","last_clean":"2023-11-28T13:08:04.372224+0000","last_became_active":"2023-11-28T13:08:03.080718+0000","last_became_peered":"2023-11-28T13:08:03.080718+0000","last_unstale":"2023-11-28T13:08:04.372224+0000","last_undegraded":"2023-11-28T13:08:04.372224+0000","last_fullsized":"2023-11-28T13:08:04.372224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:00.400817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069777+0000","last_change":"2023-11-28T13:08:02.577784+0000","last_active":"2023-11-28T13:08:04.069777+0000","last_peered":"2023-11-28T13:08:04.069777+0000","last_clean":"2023-11-28T13:08:04.069777+0000","last_became_active":"2023-11-28T13:08:02.577364+0000","last_became_peered":"2023-11-28T13:08:02.577364+0000","last_unstale":"2023-11-28T13:08:04.069777+0000","last_undegraded":"2023-11-28T13:08:04.069777+0000","last_fullsized":"2023-11-28T13:08:04.069777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:37.654488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372256+0000","last_change":"2023-11-28T13:08:03.075105+0000","last_active":"2023-11-28T13:08:04.372256+0000","last_peered":"2023-11-28T13:08:04.372256+0000","last_clean":"2023-11-28T13:08:04.372256+0000","last_became_active":"2023-11-28T13:08:03.074768+0000","last_became_peered":"2023-11-28T13:08:03.074768+0000","last_unstale":"2023-11-28T13:08:04.372256+0000","last_undegraded":"2023-11-28T13:08:04.372256+0000","last_fullsized":"2023-11-28T13:08:04.372256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:37:44.657148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064578+0000","last_change":"2023-11-28T13:08:02.582099+0000","last_active":"2023-11-28T13:08:04.064578+0000","last_peered":"2023-11-28T13:08:04.064578+0000","last_clean":"2023-11-28T13:08:04.064578+0000","last_became_active":"2023-11-28T13:08:02.581651+0000","last_became_peered":"2023-11-28T13:08:02.581651+0000","last_unstale":"2023-11-28T13:08:04.064578+0000","last_undegraded":"2023-11-28T13:08:04.064578+0000","last_fullsized":"2023-11-28T13:08:04.064578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:12.214372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371794+0000","last_change":"2023-11-28T13:08:03.088535+0000","last_active":"2023-11-28T13:08:04.371794+0000","last_peered":"2023-11-28T13:08:04.371794+0000","last_clean":"2023-11-28T13:08:04.371794+0000","last_became_active":"2023-11-28T13:08:03.088263+0000","last_became_peered":"2023-11-28T13:08:03.088263+0000","last_unstale":"2023-11-28T13:08:04.371794+0000","last_undegraded":"2023-11-28T13:08:04.371794+0000","last_fullsized":"2023-11-28T13:08:04.371794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:22.644742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372159+0000","last_change":"2023-11-28T13:08:02.584910+0000","last_active":"2023-11-28T13:08:04.372159+0000","last_peered":"2023-11-28T13:08:04.372159+0000","last_clean":"2023-11-28T13:08:04.372159+0000","last_became_active":"2023-11-28T13:08:02.584573+0000","last_became_peered":"2023-11-28T13:08:02.584573+0000","last_unstale":"2023-11-28T13:08:04.372159+0000","last_undegraded":"2023-11-28T13:08:04.372159+0000","last_fullsized":"2023-11-28T13:08:04.372159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:49.593563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372100+0000","last_change":"2023-11-28T13:08:03.083587+0000","last_active":"2023-11-28T13:08:04.372100+0000","last_peered":"2023-11-28T13:08:04.372100+0000","last_clean":"2023-11-28T13:08:04.372100+0000","last_became_active":"2023-11-28T13:08:03.081658+0000","last_became_peered":"2023-11-28T13:08:03.081658+0000","last_unstale":"2023-11-28T13:08:04.372100+0000","last_undegraded":"2023-11-28T13:08:04.372100+0000","last_fullsized":"2023-11-28T13:08:04.372100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:53.049281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371468+0000","last_change":"2023-11-28T13:08:02.582045+0000","last_active":"2023-11-28T13:08:04.371468+0000","last_peered":"2023-11-28T13:08:04.371468+0000","last_clean":"2023-11-28T13:08:04.371468+0000","last_became_active":"2023-11-28T13:08:02.581687+0000","last_became_peered":"2023-11-28T13:08:02.581687+0000","last_unstale":"2023-11-28T13:08:04.371468+0000","last_undegraded":"2023-11-28T13:08:04.371468+0000","last_fullsized":"2023-11-28T13:08:04.371468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:56.910035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372483+0000","last_change":"2023-11-28T13:08:03.090341+0000","last_active":"2023-11-28T13:08:04.372483+0000","last_peered":"2023-11-28T13:08:04.372483+0000","last_clean":"2023-11-28T13:08:04.372483+0000","last_became_active":"2023-11-28T13:08:03.086320+0000","last_became_peered":"2023-11-28T13:08:03.086320+0000","last_unstale":"2023-11-28T13:08:04.372483+0000","last_undegraded":"2023-11-28T13:08:04.372483+0000","last_fullsized":"2023-11-28T13:08:04.372483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:30.263143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064693+0000","last_change":"2023-11-28T13:08:02.577006+0000","last_active":"2023-11-28T13:08:04.064693+0000","last_peered":"2023-11-28T13:08:04.064693+0000","last_clean":"2023-11-28T13:08:04.064693+0000","last_became_active":"2023-11-28T13:08:02.576734+0000","last_became_peered":"2023-11-28T13:08:02.576734+0000","last_unstale":"2023-11-28T13:08:04.064693+0000","last_undegraded":"2023-11-28T13:08:04.064693+0000","last_fullsized":"2023-11-28T13:08:04.064693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:40.847112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372143+0000","last_change":"2023-11-28T13:08:03.080548+0000","last_active":"2023-11-28T13:08:04.372143+0000","last_peered":"2023-11-28T13:08:04.372143+0000","last_clean":"2023-11-28T13:08:04.372143+0000","last_became_active":"2023-11-28T13:08:03.077609+0000","last_became_peered":"2023-11-28T13:08:03.077609+0000","last_unstale":"2023-11-28T13:08:04.372143+0000","last_undegraded":"2023-11-28T13:08:04.372143+0000","last_fullsized":"2023-11-28T13:08:04.372143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:57.732415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064801+0000","last_change":"2023-11-28T13:08:02.579378+0000","last_active":"2023-11-28T13:08:04.064801+0000","last_peered":"2023-11-28T13:08:04.064801+0000","last_clean":"2023-11-28T13:08:04.064801+0000","last_became_active":"2023-11-28T13:08:02.579080+0000","last_became_peered":"2023-11-28T13:08:02.579080+0000","last_unstale":"2023-11-28T13:08:04.064801+0000","last_undegraded":"2023-11-28T13:08:04.064801+0000","last_fullsized":"2023-11-28T13:08:04.064801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:36.421141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.075126+0000","last_change":"2023-11-28T13:07:56.890378+0000","last_active":"2023-11-28T13:08:03.075126+0000","last_peered":"2023-11-28T13:08:03.075126+0000","last_clean":"2023-11-28T13:08:03.075126+0000","last_became_active":"2023-11-28T13:07:56.890058+0000","last_became_peered":"2023-11-28T13:07:56.890058+0000","last_unstale":"2023-11-28T13:08:03.075126+0000","last_undegraded":"2023-11-28T13:08:03.075126+0000","last_fullsized":"2023-11-28T13:08:03.075126+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:05.344602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064197+0000","last_change":"2023-11-28T13:08:03.084094+0000","last_active":"2023-11-28T13:08:04.064197+0000","last_peered":"2023-11-28T13:08:04.064197+0000","last_clean":"2023-11-28T13:08:04.064197+0000","last_became_active":"2023-11-28T13:08:03.083811+0000","last_became_peered":"2023-11-28T13:08:03.083811+0000","last_unstale":"2023-11-28T13:08:04.064197+0000","last_undegraded":"2023-11-28T13:08:04.064197+0000","last_fullsized":"2023-11-28T13:08:04.064197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:53.049281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.072799+0000","last_change":"2023-11-28T13:07:56.844509+0000","last_active":"2023-11-28T13:08:03.072799+0000","last_peered":"2023-11-28T13:08:03.072799+0000","last_clean":"2023-11-28T13:08:03.072799+0000","last_became_active":"2023-11-28T13:07:56.844081+0000","last_became_peered":"2023-11-28T13:07:56.844081+0000","last_unstale":"2023-11-28T13:08:03.072799+0000","last_undegraded":"2023-11-28T13:08:03.072799+0000","last_fullsized":"2023-11-28T13:08:03.072799+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372520+0000","last_change":"2023-11-28T13:08:02.576708+0000","last_active":"2023-11-28T13:08:04.372520+0000","last_peered":"2023-11-28T13:08:04.372520+0000","last_clean":"2023-11-28T13:08:04.372520+0000","last_became_active":"2023-11-28T13:08:02.572927+0000","last_became_peered":"2023-11-28T13:08:02.572927+0000","last_unstale":"2023-11-28T13:08:04.372520+0000","last_undegraded":"2023-11-28T13:08:04.372520+0000","last_fullsized":"2023-11-28T13:08:04.372520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:43.117989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372278+0000","last_change":"2023-11-28T13:08:03.093540+0000","last_active":"2023-11-28T13:08:04.372278+0000","last_peered":"2023-11-28T13:08:04.372278+0000","last_clean":"2023-11-28T13:08:04.372278+0000","last_became_active":"2023-11-28T13:08:03.089698+0000","last_became_peered":"2023-11-28T13:08:03.089698+0000","last_unstale":"2023-11-28T13:08:04.372278+0000","last_undegraded":"2023-11-28T13:08:04.372278+0000","last_fullsized":"2023-11-28T13:08:04.372278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:29:46.171857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.062393+0000","last_change":"2023-11-28T13:07:56.844699+0000","last_active":"2023-11-28T13:08:03.062393+0000","last_peered":"2023-11-28T13:08:03.062393+0000","last_clean":"2023-11-28T13:08:03.062393+0000","last_became_active":"2023-11-28T13:07:56.844391+0000","last_became_peered":"2023-11-28T13:07:56.844391+0000","last_unstale":"2023-11-28T13:08:03.062393+0000","last_undegraded":"2023-11-28T13:08:03.062393+0000","last_fullsized":"2023-11-28T13:08:03.062393+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.074953+0000","last_change":"2023-11-28T13:07:57.844020+0000","last_active":"2023-11-28T13:08:03.074953+0000","last_peered":"2023-11-28T13:08:03.074953+0000","last_clean":"2023-11-28T13:08:03.074953+0000","last_became_active":"2023-11-28T13:07:57.842981+0000","last_became_peered":"2023-11-28T13:07:57.842981+0000","last_unstale":"2023-11-28T13:08:03.074953+0000","last_undegraded":"2023-11-28T13:08:03.074953+0000","last_fullsized":"2023-11-28T13:08:03.074953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_clean_scrub_stamp":"2023-11-28T13:07:56.825711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:44.973218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371955+0000","last_change":"2023-11-28T13:08:02.583813+0000","last_active":"2023-11-28T13:08:04.371955+0000","last_peered":"2023-11-28T13:08:04.371955+0000","last_clean":"2023-11-28T13:08:04.371955+0000","last_became_active":"2023-11-28T13:08:02.583528+0000","last_became_peered":"2023-11-28T13:08:02.583528+0000","last_unstale":"2023-11-28T13:08:04.371955+0000","last_undegraded":"2023-11-28T13:08:04.371955+0000","last_fullsized":"2023-11-28T13:08:04.371955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:49.593563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372411+0000","last_change":"2023-11-28T13:08:03.077611+0000","last_active":"2023-11-28T13:08:04.372411+0000","last_peered":"2023-11-28T13:08:04.372411+0000","last_clean":"2023-11-28T13:08:04.372411+0000","last_became_active":"2023-11-28T13:08:03.076746+0000","last_became_peered":"2023-11-28T13:08:03.076746+0000","last_unstale":"2023-11-28T13:08:04.372411+0000","last_undegraded":"2023-11-28T13:08:04.372411+0000","last_fullsized":"2023-11-28T13:08:04.372411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:49.707036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.075246+0000","last_change":"2023-11-28T13:07:57.306318+0000","last_active":"2023-11-28T13:08:03.075246+0000","last_peered":"2023-11-28T13:08:03.075246+0000","last_clean":"2023-11-28T13:08:03.075246+0000","last_became_active":"2023-11-28T13:07:57.305937+0000","last_became_peered":"2023-11-28T13:07:57.305937+0000","last_unstale":"2023-11-28T13:08:03.075246+0000","last_undegraded":"2023-11-28T13:08:03.075246+0000","last_fullsized":"2023-11-28T13:08:03.075246+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371445+0000","last_change":"2023-11-28T13:08:02.569783+0000","last_active":"2023-11-28T13:08:04.371445+0000","last_peered":"2023-11-28T13:08:04.371445+0000","last_clean":"2023-11-28T13:08:04.371445+0000","last_became_active":"2023-11-28T13:08:02.569505+0000","last_became_peered":"2023-11-28T13:08:02.569505+0000","last_unstale":"2023-11-28T13:08:04.371445+0000","last_undegraded":"2023-11-28T13:08:04.371445+0000","last_fullsized":"2023-11-28T13:08:04.371445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:12.363239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372117+0000","last_change":"2023-11-28T13:08:03.094969+0000","last_active":"2023-11-28T13:08:04.372117+0000","last_peered":"2023-11-28T13:08:04.372117+0000","last_clean":"2023-11-28T13:08:04.372117+0000","last_became_active":"2023-11-28T13:08:03.094723+0000","last_became_peered":"2023-11-28T13:08:03.094723+0000","last_unstale":"2023-11-28T13:08:04.372117+0000","last_undegraded":"2023-11-28T13:08:04.372117+0000","last_fullsized":"2023-11-28T13:08:04.372117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:37:44.657148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.074562+0000","last_change":"2023-11-28T13:07:57.306277+0000","last_active":"2023-11-28T13:08:03.074562+0000","last_peered":"2023-11-28T13:08:03.074562+0000","last_clean":"2023-11-28T13:08:03.074562+0000","last_became_active":"2023-11-28T13:07:57.305983+0000","last_became_peered":"2023-11-28T13:07:57.305983+0000","last_unstale":"2023-11-28T13:08:03.074562+0000","last_undegraded":"2023-11-28T13:08:03.074562+0000","last_fullsized":"2023-11-28T13:08:03.074562+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064565+0000","last_change":"2023-11-28T13:08:02.578428+0000","last_active":"2023-11-28T13:08:04.064565+0000","last_peered":"2023-11-28T13:08:04.064565+0000","last_clean":"2023-11-28T13:08:04.064565+0000","last_became_active":"2023-11-28T13:08:02.578147+0000","last_became_peered":"2023-11-28T13:08:02.578147+0000","last_unstale":"2023-11-28T13:08:04.064565+0000","last_undegraded":"2023-11-28T13:08:04.064565+0000","last_fullsized":"2023-11-28T13:08:04.064565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:43.117989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372191+0000","last_change":"2023-11-28T13:08:03.094387+0000","last_active":"2023-11-28T13:08:04.372191+0000","last_peered":"2023-11-28T13:08:04.372191+0000","last_clean":"2023-11-28T13:08:04.372191+0000","last_became_active":"2023-11-28T13:08:03.094117+0000","last_became_peered":"2023-11-28T13:08:03.094117+0000","last_unstale":"2023-11-28T13:08:04.372191+0000","last_undegraded":"2023-11-28T13:08:04.372191+0000","last_fullsized":"2023-11-28T13:08:04.372191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:38.387013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.068929+0000","last_change":"2023-11-28T13:07:57.306177+0000","last_active":"2023-11-28T13:08:03.068929+0000","last_peered":"2023-11-28T13:08:03.068929+0000","last_clean":"2023-11-28T13:08:03.068929+0000","last_became_active":"2023-11-28T13:07:57.305853+0000","last_became_peered":"2023-11-28T13:07:57.305853+0000","last_unstale":"2023-11-28T13:08:03.068929+0000","last_undegraded":"2023-11-28T13:08:03.068929+0000","last_fullsized":"2023-11-28T13:08:03.068929+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371895+0000","last_change":"2023-11-28T13:08:02.584788+0000","last_active":"2023-11-28T13:08:04.371895+0000","last_peered":"2023-11-28T13:08:04.371895+0000","last_clean":"2023-11-28T13:08:04.371895+0000","last_became_active":"2023-11-28T13:08:02.584281+0000","last_became_peered":"2023-11-28T13:08:02.584281+0000","last_unstale":"2023-11-28T13:08:04.371895+0000","last_undegraded":"2023-11-28T13:08:04.371895+0000","last_fullsized":"2023-11-28T13:08:04.371895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:40.847112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371995+0000","last_change":"2023-11-28T13:08:03.074701+0000","last_active":"2023-11-28T13:08:04.371995+0000","last_peered":"2023-11-28T13:08:04.371995+0000","last_clean":"2023-11-28T13:08:04.371995+0000","last_became_active":"2023-11-28T13:08:03.074336+0000","last_became_peered":"2023-11-28T13:08:03.074336+0000","last_unstale":"2023-11-28T13:08:04.371995+0000","last_undegraded":"2023-11-28T13:08:04.371995+0000","last_fullsized":"2023-11-28T13:08:04.371995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:38.387013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.074084+0000","last_change":"2023-11-28T13:07:57.306321+0000","last_active":"2023-11-28T13:08:03.074084+0000","last_peered":"2023-11-28T13:08:03.074084+0000","last_clean":"2023-11-28T13:08:03.074084+0000","last_became_active":"2023-11-28T13:07:57.305923+0000","last_became_peered":"2023-11-28T13:07:57.305923+0000","last_unstale":"2023-11-28T13:08:03.074084+0000","last_undegraded":"2023-11-28T13:08:03.074084+0000","last_fullsized":"2023-11-28T13:08:03.074084+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:41.104679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371927+0000","last_change":"2023-11-28T13:08:02.570790+0000","last_active":"2023-11-28T13:08:04.371927+0000","last_peered":"2023-11-28T13:08:04.371927+0000","last_clean":"2023-11-28T13:08:04.371927+0000","last_became_active":"2023-11-28T13:08:02.569907+0000","last_became_peered":"2023-11-28T13:08:02.569907+0000","last_unstale":"2023-11-28T13:08:04.371927+0000","last_undegraded":"2023-11-28T13:08:04.371927+0000","last_fullsized":"2023-11-28T13:08:04.371927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:46.379743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372610+0000","last_change":"2023-11-28T13:08:03.087530+0000","last_active":"2023-11-28T13:08:04.372610+0000","last_peered":"2023-11-28T13:08:04.372610+0000","last_clean":"2023-11-28T13:08:04.372610+0000","last_became_active":"2023-11-28T13:08:03.087191+0000","last_became_peered":"2023-11-28T13:08:03.087191+0000","last_unstale":"2023-11-28T13:08:04.372610+0000","last_undegraded":"2023-11-28T13:08:04.372610+0000","last_fullsized":"2023-11-28T13:08:04.372610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:38.726877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.074497+0000","last_change":"2023-11-28T13:07:56.844305+0000","last_active":"2023-11-28T13:08:03.074497+0000","last_peered":"2023-11-28T13:08:03.074497+0000","last_clean":"2023-11-28T13:08:03.074497+0000","last_became_active":"2023-11-28T13:07:56.843980+0000","last_became_peered":"2023-11-28T13:07:56.843980+0000","last_unstale":"2023-11-28T13:08:03.074497+0000","last_undegraded":"2023-11-28T13:08:03.074497+0000","last_fullsized":"2023-11-28T13:08:03.074497+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070218+0000","last_change":"2023-11-28T13:08:02.577243+0000","last_active":"2023-11-28T13:08:04.070218+0000","last_peered":"2023-11-28T13:08:04.070218+0000","last_clean":"2023-11-28T13:08:04.070218+0000","last_became_active":"2023-11-28T13:08:02.576748+0000","last_became_peered":"2023-11-28T13:08:02.576748+0000","last_unstale":"2023-11-28T13:08:04.070218+0000","last_undegraded":"2023-11-28T13:08:04.070218+0000","last_fullsized":"2023-11-28T13:08:04.070218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:01.496804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371797+0000","last_change":"2023-11-28T13:08:03.095221+0000","last_active":"2023-11-28T13:08:04.371797+0000","last_peered":"2023-11-28T13:08:04.371797+0000","last_clean":"2023-11-28T13:08:04.371797+0000","last_became_active":"2023-11-28T13:08:03.091841+0000","last_became_peered":"2023-11-28T13:08:03.091841+0000","last_unstale":"2023-11-28T13:08:04.371797+0000","last_undegraded":"2023-11-28T13:08:04.371797+0000","last_fullsized":"2023-11-28T13:08:04.371797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:43.340474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372325+0000","last_change":"2023-11-28T13:08:02.583653+0000","last_active":"2023-11-28T13:08:04.372325+0000","last_peered":"2023-11-28T13:08:04.372325+0000","last_clean":"2023-11-28T13:08:04.372325+0000","last_became_active":"2023-11-28T13:08:02.583397+0000","last_became_peered":"2023-11-28T13:08:02.583397+0000","last_unstale":"2023-11-28T13:08:04.372325+0000","last_undegraded":"2023-11-28T13:08:04.372325+0000","last_fullsized":"2023-11-28T13:08:04.372325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:40.847112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371135+0000","last_change":"2023-11-28T13:08:03.096716+0000","last_active":"2023-11-28T13:08:04.371135+0000","last_peered":"2023-11-28T13:08:04.371135+0000","last_clean":"2023-11-28T13:08:04.371135+0000","last_became_active":"2023-11-28T13:08:03.096239+0000","last_became_peered":"2023-11-28T13:08:03.096239+0000","last_unstale":"2023-11-28T13:08:04.371135+0000","last_undegraded":"2023-11-28T13:08:04.371135+0000","last_fullsized":"2023-11-28T13:08:04.371135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:13.217034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069581+0000","last_change":"2023-11-28T13:08:02.581525+0000","last_active":"2023-11-28T13:08:04.069581+0000","last_peered":"2023-11-28T13:08:04.069581+0000","last_clean":"2023-11-28T13:08:04.069581+0000","last_became_active":"2023-11-28T13:08:02.581253+0000","last_became_peered":"2023-11-28T13:08:02.581253+0000","last_unstale":"2023-11-28T13:08:04.069581+0000","last_undegraded":"2023-11-28T13:08:04.069581+0000","last_fullsized":"2023-11-28T13:08:04.069581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:36.421141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372111+0000","last_change":"2023-11-28T13:08:03.084538+0000","last_active":"2023-11-28T13:08:04.372111+0000","last_peered":"2023-11-28T13:08:04.372111+0000","last_clean":"2023-11-28T13:08:04.372111+0000","last_became_active":"2023-11-28T13:08:03.084256+0000","last_became_peered":"2023-11-28T13:08:03.084256+0000","last_unstale":"2023-11-28T13:08:04.372111+0000","last_undegraded":"2023-11-28T13:08:04.372111+0000","last_fullsized":"2023-11-28T13:08:04.372111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:46:46.224857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070254+0000","last_change":"2023-11-28T13:08:02.583812+0000","last_active":"2023-11-28T13:08:04.070254+0000","last_peered":"2023-11-28T13:08:04.070254+0000","last_clean":"2023-11-28T13:08:04.070254+0000","last_became_active":"2023-11-28T13:08:02.583517+0000","last_became_peered":"2023-11-28T13:08:02.583517+0000","last_unstale":"2023-11-28T13:08:04.070254+0000","last_undegraded":"2023-11-28T13:08:04.070254+0000","last_fullsized":"2023-11-28T13:08:04.070254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:12.214372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371726+0000","last_change":"2023-11-28T13:08:03.088006+0000","last_active":"2023-11-28T13:08:04.371726+0000","last_peered":"2023-11-28T13:08:04.371726+0000","last_clean":"2023-11-28T13:08:04.371726+0000","last_became_active":"2023-11-28T13:08:03.086226+0000","last_became_peered":"2023-11-28T13:08:03.086226+0000","last_unstale":"2023-11-28T13:08:04.371726+0000","last_undegraded":"2023-11-28T13:08:04.371726+0000","last_fullsized":"2023-11-28T13:08:04.371726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:08.985463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064555+0000","last_change":"2023-11-28T13:08:02.574322+0000","last_active":"2023-11-28T13:08:04.064555+0000","last_peered":"2023-11-28T13:08:04.064555+0000","last_clean":"2023-11-28T13:08:04.064555+0000","last_became_active":"2023-11-28T13:08:02.573611+0000","last_became_peered":"2023-11-28T13:08:02.573611+0000","last_unstale":"2023-11-28T13:08:04.064555+0000","last_undegraded":"2023-11-28T13:08:04.064555+0000","last_fullsized":"2023-11-28T13:08:04.064555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:37.654488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064305+0000","last_change":"2023-11-28T13:08:03.093751+0000","last_active":"2023-11-28T13:08:04.064305+0000","last_peered":"2023-11-28T13:08:04.064305+0000","last_clean":"2023-11-28T13:08:04.064305+0000","last_became_active":"2023-11-28T13:08:03.093493+0000","last_became_peered":"2023-11-28T13:08:03.093493+0000","last_unstale":"2023-11-28T13:08:04.064305+0000","last_undegraded":"2023-11-28T13:08:04.064305+0000","last_fullsized":"2023-11-28T13:08:04.064305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:46.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372733+0000","last_change":"2023-11-28T13:08:02.580561+0000","last_active":"2023-11-28T13:08:04.372733+0000","last_peered":"2023-11-28T13:08:04.372733+0000","last_clean":"2023-11-28T13:08:04.372733+0000","last_became_active":"2023-11-28T13:08:02.580273+0000","last_became_peered":"2023-11-28T13:08:02.580273+0000","last_unstale":"2023-11-28T13:08:04.372733+0000","last_undegraded":"2023-11-28T13:08:04.372733+0000","last_fullsized":"2023-11-28T13:08:04.372733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:37.654488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371182+0000","last_change":"2023-11-28T13:08:03.089735+0000","last_active":"2023-11-28T13:08:04.371182+0000","last_peered":"2023-11-28T13:08:04.371182+0000","last_clean":"2023-11-28T13:08:04.371182+0000","last_became_active":"2023-11-28T13:08:03.089492+0000","last_became_peered":"2023-11-28T13:08:03.089492+0000","last_unstale":"2023-11-28T13:08:04.371182+0000","last_undegraded":"2023-11-28T13:08:04.371182+0000","last_fullsized":"2023-11-28T13:08:04.371182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:41.196768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371414+0000","last_change":"2023-11-28T13:08:03.082551+0000","last_active":"2023-11-28T13:08:04.371414+0000","last_peered":"2023-11-28T13:08:04.371414+0000","last_clean":"2023-11-28T13:08:04.371414+0000","last_became_active":"2023-11-28T13:08:03.082220+0000","last_became_peered":"2023-11-28T13:08:03.082220+0000","last_unstale":"2023-11-28T13:08:04.371414+0000","last_undegraded":"2023-11-28T13:08:04.371414+0000","last_fullsized":"2023-11-28T13:08:04.371414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:57.538270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064240+0000","last_change":"2023-11-28T13:08:03.095511+0000","last_active":"2023-11-28T13:08:04.064240+0000","last_peered":"2023-11-28T13:08:04.064240+0000","last_clean":"2023-11-28T13:08:04.064240+0000","last_became_active":"2023-11-28T13:08:03.095214+0000","last_became_peered":"2023-11-28T13:08:03.095214+0000","last_unstale":"2023-11-28T13:08:04.064240+0000","last_undegraded":"2023-11-28T13:08:04.064240+0000","last_fullsized":"2023-11-28T13:08:04.064240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:13.351182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371631+0000","last_change":"2023-11-28T13:08:03.089514+0000","last_active":"2023-11-28T13:08:04.371631+0000","last_peered":"2023-11-28T13:08:04.371631+0000","last_clean":"2023-11-28T13:08:04.371631+0000","last_became_active":"2023-11-28T13:08:03.089266+0000","last_became_peered":"2023-11-28T13:08:03.089266+0000","last_unstale":"2023-11-28T13:08:04.371631+0000","last_undegraded":"2023-11-28T13:08:04.371631+0000","last_fullsized":"2023-11-28T13:08:04.371631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:58.181120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064673+0000","last_change":"2023-11-28T13:08:03.084369+0000","last_active":"2023-11-28T13:08:04.064673+0000","last_peered":"2023-11-28T13:08:04.064673+0000","last_clean":"2023-11-28T13:08:04.064673+0000","last_became_active":"2023-11-28T13:08:03.084081+0000","last_became_peered":"2023-11-28T13:08:03.084081+0000","last_unstale":"2023-11-28T13:08:04.064673+0000","last_undegraded":"2023-11-28T13:08:04.064673+0000","last_fullsized":"2023-11-28T13:08:04.064673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:13.625233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371927+0000","last_change":"2023-11-28T13:08:03.093572+0000","last_active":"2023-11-28T13:08:04.371927+0000","last_peered":"2023-11-28T13:08:04.371927+0000","last_clean":"2023-11-28T13:08:04.371927+0000","last_became_active":"2023-11-28T13:08:03.093306+0000","last_became_peered":"2023-11-28T13:08:03.093306+0000","last_unstale":"2023-11-28T13:08:04.371927+0000","last_undegraded":"2023-11-28T13:08:04.371927+0000","last_fullsized":"2023-11-28T13:08:04.371927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:36.892676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9084,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362756,"statfs":{"total":96636764160,"available":96627462144,"internally_reserved":0,"allocated":839680,"data_stored":671592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7952,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363888,"statfs":{"total":96636764160,"available":96628621312,"internally_reserved":0,"allocated":204800,"data_stored":72807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13152,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358688,"statfs":{"total":96636764160,"available":96623296512,"internally_reserved":0,"allocated":811008,"data_stored":669828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12124,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359716,"statfs":{"total":96636764160,"available":96624349184,"internally_reserved":0,"allocated":217088,"data_stored":79040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11996,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":11776,"kb_avail":94359844,"statfs":{"total":96636764160,"available":96624480256,"internally_reserved":0,"allocated":217088,"data_stored":79460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12058624},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12128,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359712,"statfs":{"total":96636764160,"available":96624345088,"internally_reserved":0,"allocated":221184,"data_stored":79550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8048,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363792,"statfs":{"total":96636764160,"available":96628523008,"internally_reserved":0,"allocated":237568,"data_stored":81064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7972,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363868,"statfs":{"total":96636764160,"available":96628600832,"internally_reserved":0,"allocated":225280,"data_stored":80096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:08:07.434 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:08:07.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.665+0000 7f4158bff700 1 Processor -- start 2023-11-28T13:08:07.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.665+0000 7f4158bff700 1 -- start start 2023-11-28T13:08:07.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.666+0000 7f4158bff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f41540d01b0 0x7f41540d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:07.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.666+0000 7f4158bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41540b16d0 0x7f41540d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:07.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.666+0000 7f4158bff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41540b0230 con 0x7f41540b20b0 2023-11-28T13:08:07.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.666+0000 7f4158bff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f415415cec0 con 0x7f41540d01b0 2023-11-28T13:08:07.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.666+0000 7f4158bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41540b4570 con 0x7f41540b16d0 2023-11-28T13:08:07.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.667+0000 7f4152d9d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41540b16d0 0x7f41540d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:07.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.667+0000 7f415259c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f41540d01b0 0x7f41540d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:07.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.667+0000 7f4152d9d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41540b16d0 0x7f41540d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36734/0 (socket says 172.21.15.156:36734) 2023-11-28T13:08:07.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.667+0000 7f4151d9b700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f41540b20b0 0x7f41540b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:38468/0 (socket says 172.21.15.156:38468) 2023-11-28T13:08:07.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.667+0000 7f4152d9d700 1 -- 172.21.15.156:0/3908585186 learned_addr learned my addr 172.21.15.156:0/3908585186 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:07.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.667+0000 7f415259c700 1 -- 172.21.15.156:0/3908585186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41540b16d0 msgr2=0x7f41540d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:07.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.667+0000 7f415259c700 1 --2- 172.21.15.156:0/3908585186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41540b16d0 0x7f41540d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.667+0000 7f415259c700 1 -- 172.21.15.156:0/3908585186 >> v1:172.21.15.186:6789/0 conn(0x7f41540b20b0 legacy=0x7f41540b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:07.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.667+0000 7f415259c700 1 -- 172.21.15.156:0/3908585186 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41540b4850 con 0x7f41540d01b0 2023-11-28T13:08:07.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.667+0000 7f4152d9d700 1 --2- 172.21.15.156:0/3908585186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41540b16d0 0x7f41540d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:07.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.667+0000 7f415159a700 1 -- 172.21.15.156:0/3908585186 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1627390144 0 0) 0x7f41540b0230 con 0x7f41540b20b0 2023-11-28T13:08:07.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.668+0000 7f415259c700 1 --2- 172.21.15.156:0/3908585186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41540d01b0 0x7f41540d0c20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4144009840 tx=0x7f4144009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a08d9fe2e1cabd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:07.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.668+0000 7f415159a700 1 -- 172.21.15.156:0/3908585186 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144011830 con 0x7f41540d01b0 2023-11-28T13:08:07.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.668+0000 7f415159a700 1 -- 172.21.15.156:0/3908585186 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4144011990 con 0x7f41540d01b0 2023-11-28T13:08:07.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.668+0000 7f415159a700 1 -- 172.21.15.156:0/3908585186 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144011d20 con 0x7f41540d01b0 2023-11-28T13:08:07.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.668+0000 7f4158bff700 1 -- 172.21.15.156:0/3908585186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41540d01b0 msgr2=0x7f41540d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:07.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.668+0000 7f4158bff700 1 --2- 172.21.15.156:0/3908585186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41540d01b0 0x7f41540d0c20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4144009840 tx=0x7f4144009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:07.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.669+0000 7f4158bff700 1 -- 172.21.15.156:0/3908585186 shutdown_connections 2023-11-28T13:08:07.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.669+0000 7f4158bff700 1 --2- 172.21.15.156:0/3908585186 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41540b16d0 0x7f41540d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.669+0000 7f4158bff700 1 --2- 172.21.15.156:0/3908585186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41540d01b0 0x7f41540d0c20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.669+0000 7f4158bff700 1 -- 172.21.15.156:0/3908585186 >> 172.21.15.156:0/3908585186 conn(0x7f415409c3f0 msgr2=0x7f41540d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:07.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.669+0000 7f4158bff700 1 -- 172.21.15.156:0/3908585186 shutdown_connections 2023-11-28T13:08:07.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.669+0000 7f4158bff700 1 -- 172.21.15.156:0/3908585186 wait complete. 2023-11-28T13:08:07.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.670+0000 7f4158bff700 1 Processor -- start 2023-11-28T13:08:07.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.671+0000 7f4158bff700 1 -- start start 2023-11-28T13:08:07.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.671+0000 7f4158bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41540b16d0 0x7f41541e63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:07.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.671+0000 7f4158bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41540b20b0 0x7f41540b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:07.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.672+0000 7f4158bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41540d01b0 0x7f41540ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:07.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.672+0000 7f4158bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41541e50b0 con 0x7f41540b16d0 2023-11-28T13:08:07.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.672+0000 7f4158bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41541e5f10 con 0x7f41540d01b0 2023-11-28T13:08:07.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.672+0000 7f4158bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41541e5300 con 0x7f41540b20b0 2023-11-28T13:08:07.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.672+0000 7f415259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41540b16d0 0x7f41541e63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:07.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.672+0000 7f415259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41540b16d0 0x7f41541e63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58718/0 (socket says 172.21.15.156:58718) 2023-11-28T13:08:07.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.672+0000 7f415259c700 1 -- 172.21.15.156:0/1513071284 learned_addr learned my addr 172.21.15.156:0/1513071284 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:07.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.672+0000 7f4151d9b700 1 --2- 172.21.15.156:0/1513071284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41540b20b0 0x7f41540b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:07.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.672+0000 7f4152d9d700 1 --2- 172.21.15.156:0/1513071284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41540d01b0 0x7f41540ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:07.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.672+0000 7f415259c700 1 -- 172.21.15.156:0/1513071284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41540b20b0 msgr2=0x7f41540b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:07.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.672+0000 7f415259c700 1 --2- 172.21.15.156:0/1513071284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41540b20b0 0x7f41540b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.672+0000 7f415259c700 1 -- 172.21.15.156:0/1513071284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41540d01b0 msgr2=0x7f41540ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:07.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.672+0000 7f415259c700 1 --2- 172.21.15.156:0/1513071284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41540d01b0 0x7f41540ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.672+0000 7f415259c700 1 -- 172.21.15.156:0/1513071284 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414c009b30 con 0x7f41540b16d0 2023-11-28T13:08:07.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.673+0000 7f415259c700 1 --2- 172.21.15.156:0/1513071284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41540b16d0 0x7f41541e63a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4144009fd0 tx=0x7f4144004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:07.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.673+0000 7f4142ffd700 1 -- 172.21.15.156:0/1513071284 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41440036a0 con 0x7f41540b16d0 2023-11-28T13:08:07.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.673+0000 7f4142ffd700 1 -- 172.21.15.156:0/1513071284 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414401e070 con 0x7f41540b16d0 2023-11-28T13:08:07.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.673+0000 7f4142ffd700 1 -- 172.21.15.156:0/1513071284 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144003a60 con 0x7f41540b16d0 2023-11-28T13:08:07.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.673+0000 7f4158bff700 1 -- 172.21.15.156:0/1513071284 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41440094a0 con 0x7f41540b16d0 2023-11-28T13:08:07.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.673+0000 7f4158bff700 1 -- 172.21.15.156:0/1513071284 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41540afec0 con 0x7f41540b16d0 2023-11-28T13:08:07.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.674+0000 7f4158bff700 1 -- 172.21.15.156:0/1513071284 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41540afbe0 con 0x7f41540b16d0 2023-11-28T13:08:07.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.675+0000 7f4142ffd700 1 -- 172.21.15.156:0/1513071284 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f414400f050 con 0x7f41540b16d0 2023-11-28T13:08:07.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.675+0000 7f4142ffd700 1 --2- 172.21.15.156:0/1513071284 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f41340682f0 0x7f413406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:07.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.676+0000 7f4142ffd700 1 -- 172.21.15.156:0/1513071284 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4144086cb0 con 0x7f41540b16d0 2023-11-28T13:08:07.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.676+0000 7f4151d9b700 1 --2- 172.21.15.156:0/1513071284 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f41340682f0 0x7f413406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:07.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.677+0000 7f4151d9b700 1 --2- 172.21.15.156:0/1513071284 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f41340682f0 0x7f413406a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f41540b8ea0 tx=0x7f4148007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:07.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.680+0000 7f4142ffd700 1 -- 172.21.15.156:0/1513071284 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f41440533a0 con 0x7f41540b16d0 2023-11-28T13:08:07.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.869+0000 7f4158bff700 1 -- 172.21.15.156:0/1513071284 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41541e6050 con 0x7f41340682f0 2023-11-28T13:08:07.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.878+0000 7f4142ffd700 1 -- 172.21.15.156:0/1513071284 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287003 (secure 0 0 0) 0x7f4148001370 con 0x7f41340682f0 2023-11-28T13:08:07.882 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:07.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.882+0000 7f4158bff700 1 -- 172.21.15.156:0/1513071284 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f41340682f0 msgr2=0x7f413406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:07.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.882+0000 7f4158bff700 1 --2- 172.21.15.156:0/1513071284 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f41340682f0 0x7f413406a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f41540b8ea0 tx=0x7f4148007de0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.882+0000 7f4158bff700 1 -- 172.21.15.156:0/1513071284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41540b16d0 msgr2=0x7f41541e63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:07.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.882+0000 7f4158bff700 1 --2- 172.21.15.156:0/1513071284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41540b16d0 0x7f41541e63a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4144009fd0 tx=0x7f4144004750 comp rx=0 tx=0).stop 2023-11-28T13:08:07.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.882+0000 7f4158bff700 1 -- 172.21.15.156:0/1513071284 shutdown_connections 2023-11-28T13:08:07.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.882+0000 7f4158bff700 1 --2- 172.21.15.156:0/1513071284 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f41340682f0 0x7f413406a7a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.883+0000 7f4158bff700 1 --2- 172.21.15.156:0/1513071284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41540d01b0 0x7f41540ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.883+0000 7f4158bff700 1 --2- 172.21.15.156:0/1513071284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41540b16d0 0x7f41541e63a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.883+0000 7f4158bff700 1 --2- 172.21.15.156:0/1513071284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41540b20b0 0x7f41540b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:07.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.883+0000 7f4158bff700 1 -- 172.21.15.156:0/1513071284 >> 172.21.15.156:0/1513071284 conn(0x7f415409c3f0 msgr2=0x7f415406b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:07.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.883+0000 7f4158bff700 1 -- 172.21.15.156:0/1513071284 shutdown_connections 2023-11-28T13:08:07.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:07.883+0000 7f4158bff700 1 -- 172.21.15.156:0/1513071284 wait complete. 2023-11-28T13:08:07.886 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T13:08:07.902 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-11-28T13:08:06.014466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":82456,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":79488,"kb_avail":754892264,"statfs":{"total":773094113280,"available":773009678336,"internally_reserved":0,"allocated":2973696,"data_stored":1813437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81395712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.460715"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371323+0000","last_change":"2023-11-28T13:08:03.085544+0000","last_active":"2023-11-28T13:08:04.371323+0000","last_peered":"2023-11-28T13:08:04.371323+0000","last_clean":"2023-11-28T13:08:04.371323+0000","last_became_active":"2023-11-28T13:08:03.085265+0000","last_became_peered":"2023-11-28T13:08:03.085265+0000","last_unstale":"2023-11-28T13:08:04.371323+0000","last_undegraded":"2023-11-28T13:08:04.371323+0000","last_fullsized":"2023-11-28T13:08:04.371323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:37.423803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069498+0000","last_change":"2023-11-28T13:08:03.084729+0000","last_active":"2023-11-28T13:08:04.069498+0000","last_peered":"2023-11-28T13:08:04.069498+0000","last_clean":"2023-11-28T13:08:04.069498+0000","last_became_active":"2023-11-28T13:08:03.084473+0000","last_became_peered":"2023-11-28T13:08:03.084473+0000","last_unstale":"2023-11-28T13:08:04.069498+0000","last_undegraded":"2023-11-28T13:08:04.069498+0000","last_fullsized":"2023-11-28T13:08:04.069498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:53.049281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371761+0000","last_change":"2023-11-28T13:08:03.086292+0000","last_active":"2023-11-28T13:08:04.371761+0000","last_peered":"2023-11-28T13:08:04.371761+0000","last_clean":"2023-11-28T13:08:04.371761+0000","last_became_active":"2023-11-28T13:08:03.085984+0000","last_became_peered":"2023-11-28T13:08:03.085984+0000","last_unstale":"2023-11-28T13:08:04.371761+0000","last_undegraded":"2023-11-28T13:08:04.371761+0000","last_fullsized":"2023-11-28T13:08:04.371761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:58.749445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372036+0000","last_change":"2023-11-28T13:08:03.080558+0000","last_active":"2023-11-28T13:08:04.372036+0000","last_peered":"2023-11-28T13:08:04.372036+0000","last_clean":"2023-11-28T13:08:04.372036+0000","last_became_active":"2023-11-28T13:08:03.080267+0000","last_became_peered":"2023-11-28T13:08:03.080267+0000","last_unstale":"2023-11-28T13:08:04.372036+0000","last_undegraded":"2023-11-28T13:08:04.372036+0000","last_fullsized":"2023-11-28T13:08:04.372036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:38.726877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372532+0000","last_change":"2023-11-28T13:08:03.081890+0000","last_active":"2023-11-28T13:08:04.372532+0000","last_peered":"2023-11-28T13:08:04.372532+0000","last_clean":"2023-11-28T13:08:04.372532+0000","last_became_active":"2023-11-28T13:08:03.081638+0000","last_became_peered":"2023-11-28T13:08:03.081638+0000","last_unstale":"2023-11-28T13:08:04.372532+0000","last_undegraded":"2023-11-28T13:08:04.372532+0000","last_fullsized":"2023-11-28T13:08:04.372532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:53.049281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371404+0000","last_change":"2023-11-28T13:08:03.081127+0000","last_active":"2023-11-28T13:08:04.371404+0000","last_peered":"2023-11-28T13:08:04.371404+0000","last_clean":"2023-11-28T13:08:04.371404+0000","last_became_active":"2023-11-28T13:08:03.077682+0000","last_became_peered":"2023-11-28T13:08:03.077682+0000","last_unstale":"2023-11-28T13:08:04.371404+0000","last_undegraded":"2023-11-28T13:08:04.371404+0000","last_fullsized":"2023-11-28T13:08:04.371404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:58.947317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070150+0000","last_change":"2023-11-28T13:08:03.099999+0000","last_active":"2023-11-28T13:08:04.070150+0000","last_peered":"2023-11-28T13:08:04.070150+0000","last_clean":"2023-11-28T13:08:04.070150+0000","last_became_active":"2023-11-28T13:08:03.099733+0000","last_became_peered":"2023-11-28T13:08:03.099733+0000","last_unstale":"2023-11-28T13:08:04.070150+0000","last_undegraded":"2023-11-28T13:08:04.070150+0000","last_fullsized":"2023-11-28T13:08:04.070150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:46.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069502+0000","last_change":"2023-11-28T13:08:03.094943+0000","last_active":"2023-11-28T13:08:04.069502+0000","last_peered":"2023-11-28T13:08:04.069502+0000","last_clean":"2023-11-28T13:08:04.069502+0000","last_became_active":"2023-11-28T13:08:03.094688+0000","last_became_peered":"2023-11-28T13:08:03.094688+0000","last_unstale":"2023-11-28T13:08:04.069502+0000","last_undegraded":"2023-11-28T13:08:04.069502+0000","last_fullsized":"2023-11-28T13:08:04.069502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:14.015315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372476+0000","last_change":"2023-11-28T13:08:03.083815+0000","last_active":"2023-11-28T13:08:04.372476+0000","last_peered":"2023-11-28T13:08:04.372476+0000","last_clean":"2023-11-28T13:08:04.372476+0000","last_became_active":"2023-11-28T13:08:03.083292+0000","last_became_peered":"2023-11-28T13:08:03.083292+0000","last_unstale":"2023-11-28T13:08:04.372476+0000","last_undegraded":"2023-11-28T13:08:04.372476+0000","last_fullsized":"2023-11-28T13:08:04.372476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:14.015315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070743+0000","last_change":"2023-11-28T13:08:03.083685+0000","last_active":"2023-11-28T13:08:04.070743+0000","last_peered":"2023-11-28T13:08:04.070743+0000","last_clean":"2023-11-28T13:08:04.070743+0000","last_became_active":"2023-11-28T13:08:03.077038+0000","last_became_peered":"2023-11-28T13:08:03.077038+0000","last_unstale":"2023-11-28T13:08:04.070743+0000","last_undegraded":"2023-11-28T13:08:04.070743+0000","last_fullsized":"2023-11-28T13:08:04.070743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:34.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371966+0000","last_change":"2023-11-28T13:08:03.085905+0000","last_active":"2023-11-28T13:08:04.371966+0000","last_peered":"2023-11-28T13:08:04.371966+0000","last_clean":"2023-11-28T13:08:04.371966+0000","last_became_active":"2023-11-28T13:08:03.085524+0000","last_became_peered":"2023-11-28T13:08:03.085524+0000","last_unstale":"2023-11-28T13:08:04.371966+0000","last_undegraded":"2023-11-28T13:08:04.371966+0000","last_fullsized":"2023-11-28T13:08:04.371966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:34.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371160+0000","last_change":"2023-11-28T13:08:03.091440+0000","last_active":"2023-11-28T13:08:04.371160+0000","last_peered":"2023-11-28T13:08:04.371160+0000","last_clean":"2023-11-28T13:08:04.371160+0000","last_became_active":"2023-11-28T13:08:03.091117+0000","last_became_peered":"2023-11-28T13:08:03.091117+0000","last_unstale":"2023-11-28T13:08:04.371160+0000","last_undegraded":"2023-11-28T13:08:04.371160+0000","last_fullsized":"2023-11-28T13:08:04.371160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:41.849774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371255+0000","last_change":"2023-11-28T13:08:03.080912+0000","last_active":"2023-11-28T13:08:04.371255+0000","last_peered":"2023-11-28T13:08:04.371255+0000","last_clean":"2023-11-28T13:08:04.371255+0000","last_became_active":"2023-11-28T13:08:03.080578+0000","last_became_peered":"2023-11-28T13:08:03.080578+0000","last_unstale":"2023-11-28T13:08:04.371255+0000","last_undegraded":"2023-11-28T13:08:04.371255+0000","last_fullsized":"2023-11-28T13:08:04.371255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:38.657150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069701+0000","last_change":"2023-11-28T13:08:03.083208+0000","last_active":"2023-11-28T13:08:04.069701+0000","last_peered":"2023-11-28T13:08:04.069701+0000","last_clean":"2023-11-28T13:08:04.069701+0000","last_became_active":"2023-11-28T13:08:03.082944+0000","last_became_peered":"2023-11-28T13:08:03.082944+0000","last_unstale":"2023-11-28T13:08:04.069701+0000","last_undegraded":"2023-11-28T13:08:04.069701+0000","last_fullsized":"2023-11-28T13:08:04.069701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:36.892676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070334+0000","last_change":"2023-11-28T13:08:03.098345+0000","last_active":"2023-11-28T13:08:04.070334+0000","last_peered":"2023-11-28T13:08:04.070334+0000","last_clean":"2023-11-28T13:08:04.070334+0000","last_became_active":"2023-11-28T13:08:03.098105+0000","last_became_peered":"2023-11-28T13:08:03.098105+0000","last_unstale":"2023-11-28T13:08:04.070334+0000","last_undegraded":"2023-11-28T13:08:04.070334+0000","last_fullsized":"2023-11-28T13:08:04.070334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:43.340474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371254+0000","last_change":"2023-11-28T13:08:03.076945+0000","last_active":"2023-11-28T13:08:04.371254+0000","last_peered":"2023-11-28T13:08:04.371254+0000","last_clean":"2023-11-28T13:08:04.371254+0000","last_became_active":"2023-11-28T13:08:03.076639+0000","last_became_peered":"2023-11-28T13:08:03.076639+0000","last_unstale":"2023-11-28T13:08:04.371254+0000","last_undegraded":"2023-11-28T13:08:04.371254+0000","last_fullsized":"2023-11-28T13:08:04.371254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:41.076578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372578+0000","last_change":"2023-11-28T13:08:03.088044+0000","last_active":"2023-11-28T13:08:04.372578+0000","last_peered":"2023-11-28T13:08:04.372578+0000","last_clean":"2023-11-28T13:08:04.372578+0000","last_became_active":"2023-11-28T13:08:03.087703+0000","last_became_peered":"2023-11-28T13:08:03.087703+0000","last_unstale":"2023-11-28T13:08:04.372578+0000","last_undegraded":"2023-11-28T13:08:04.372578+0000","last_fullsized":"2023-11-28T13:08:04.372578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:43.340474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372432+0000","last_change":"2023-11-28T13:08:03.077748+0000","last_active":"2023-11-28T13:08:04.372432+0000","last_peered":"2023-11-28T13:08:04.372432+0000","last_clean":"2023-11-28T13:08:04.372432+0000","last_became_active":"2023-11-28T13:08:03.077488+0000","last_became_peered":"2023-11-28T13:08:03.077488+0000","last_unstale":"2023-11-28T13:08:04.372432+0000","last_undegraded":"2023-11-28T13:08:04.372432+0000","last_fullsized":"2023-11-28T13:08:04.372432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:14.015315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371120+0000","last_change":"2023-11-28T13:08:03.079221+0000","last_active":"2023-11-28T13:08:04.371120+0000","last_peered":"2023-11-28T13:08:04.371120+0000","last_clean":"2023-11-28T13:08:04.371120+0000","last_became_active":"2023-11-28T13:08:03.077084+0000","last_became_peered":"2023-11-28T13:08:03.077084+0000","last_unstale":"2023-11-28T13:08:04.371120+0000","last_undegraded":"2023-11-28T13:08:04.371120+0000","last_fullsized":"2023-11-28T13:08:04.371120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:02.499466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371692+0000","last_change":"2023-11-28T13:08:03.083322+0000","last_active":"2023-11-28T13:08:04.371692+0000","last_peered":"2023-11-28T13:08:04.371692+0000","last_clean":"2023-11-28T13:08:04.371692+0000","last_became_active":"2023-11-28T13:08:03.083064+0000","last_became_peered":"2023-11-28T13:08:03.083064+0000","last_unstale":"2023-11-28T13:08:04.371692+0000","last_undegraded":"2023-11-28T13:08:04.371692+0000","last_fullsized":"2023-11-28T13:08:04.371692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:13.351182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070670+0000","last_change":"2023-11-28T13:08:03.095031+0000","last_active":"2023-11-28T13:08:04.070670+0000","last_peered":"2023-11-28T13:08:04.070670+0000","last_clean":"2023-11-28T13:08:04.070670+0000","last_became_active":"2023-11-28T13:08:03.094746+0000","last_became_peered":"2023-11-28T13:08:03.094746+0000","last_unstale":"2023-11-28T13:08:04.070670+0000","last_undegraded":"2023-11-28T13:08:04.070670+0000","last_fullsized":"2023-11-28T13:08:04.070670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:49.707036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372182+0000","last_change":"2023-11-28T13:08:03.082127+0000","last_active":"2023-11-28T13:08:04.372182+0000","last_peered":"2023-11-28T13:08:04.372182+0000","last_clean":"2023-11-28T13:08:04.372182+0000","last_became_active":"2023-11-28T13:08:03.081861+0000","last_became_peered":"2023-11-28T13:08:03.081861+0000","last_unstale":"2023-11-28T13:08:04.372182+0000","last_undegraded":"2023-11-28T13:08:04.372182+0000","last_fullsized":"2023-11-28T13:08:04.372182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:05:35.135974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064206+0000","last_change":"2023-11-28T13:08:03.083718+0000","last_active":"2023-11-28T13:08:04.064206+0000","last_peered":"2023-11-28T13:08:04.064206+0000","last_clean":"2023-11-28T13:08:04.064206+0000","last_became_active":"2023-11-28T13:08:03.082420+0000","last_became_peered":"2023-11-28T13:08:03.082420+0000","last_unstale":"2023-11-28T13:08:04.064206+0000","last_undegraded":"2023-11-28T13:08:04.064206+0000","last_fullsized":"2023-11-28T13:08:04.064206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:58.181120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070265+0000","last_change":"2023-11-28T13:08:03.095780+0000","last_active":"2023-11-28T13:08:04.070265+0000","last_peered":"2023-11-28T13:08:04.070265+0000","last_clean":"2023-11-28T13:08:04.070265+0000","last_became_active":"2023-11-28T13:08:03.095522+0000","last_became_peered":"2023-11-28T13:08:03.095522+0000","last_unstale":"2023-11-28T13:08:04.070265+0000","last_undegraded":"2023-11-28T13:08:04.070265+0000","last_fullsized":"2023-11-28T13:08:04.070265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:37:44.657148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371784+0000","last_change":"2023-11-28T13:08:03.081168+0000","last_active":"2023-11-28T13:08:04.371784+0000","last_peered":"2023-11-28T13:08:04.371784+0000","last_clean":"2023-11-28T13:08:04.371784+0000","last_became_active":"2023-11-28T13:08:03.080867+0000","last_became_peered":"2023-11-28T13:08:03.080867+0000","last_unstale":"2023-11-28T13:08:04.371784+0000","last_undegraded":"2023-11-28T13:08:04.371784+0000","last_fullsized":"2023-11-28T13:08:04.371784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:44.188797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371597+0000","last_change":"2023-11-28T13:08:03.074696+0000","last_active":"2023-11-28T13:08:04.371597+0000","last_peered":"2023-11-28T13:08:04.371597+0000","last_clean":"2023-11-28T13:08:04.371597+0000","last_became_active":"2023-11-28T13:08:03.074392+0000","last_became_peered":"2023-11-28T13:08:03.074392+0000","last_unstale":"2023-11-28T13:08:04.371597+0000","last_undegraded":"2023-11-28T13:08:04.371597+0000","last_fullsized":"2023-11-28T13:08:04.371597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:50.062766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070127+0000","last_change":"2023-11-28T13:08:03.092477+0000","last_active":"2023-11-28T13:08:04.070127+0000","last_peered":"2023-11-28T13:08:04.070127+0000","last_clean":"2023-11-28T13:08:04.070127+0000","last_became_active":"2023-11-28T13:08:03.092243+0000","last_became_peered":"2023-11-28T13:08:03.092243+0000","last_unstale":"2023-11-28T13:08:04.070127+0000","last_undegraded":"2023-11-28T13:08:04.070127+0000","last_fullsized":"2023-11-28T13:08:04.070127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:46.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371521+0000","last_change":"2023-11-28T13:08:02.582255+0000","last_active":"2023-11-28T13:08:04.371521+0000","last_peered":"2023-11-28T13:08:04.371521+0000","last_clean":"2023-11-28T13:08:04.371521+0000","last_became_active":"2023-11-28T13:08:02.581960+0000","last_became_peered":"2023-11-28T13:08:02.581960+0000","last_unstale":"2023-11-28T13:08:04.371521+0000","last_undegraded":"2023-11-28T13:08:04.371521+0000","last_fullsized":"2023-11-28T13:08:04.371521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:49.593563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064820+0000","last_change":"2023-11-28T13:08:03.080261+0000","last_active":"2023-11-28T13:08:04.064820+0000","last_peered":"2023-11-28T13:08:04.064820+0000","last_clean":"2023-11-28T13:08:04.064820+0000","last_became_active":"2023-11-28T13:08:03.079964+0000","last_became_peered":"2023-11-28T13:08:03.079964+0000","last_unstale":"2023-11-28T13:08:04.064820+0000","last_undegraded":"2023-11-28T13:08:04.064820+0000","last_fullsized":"2023-11-28T13:08:04.064820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:30.263143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371820+0000","last_change":"2023-11-28T13:08:02.575220+0000","last_active":"2023-11-28T13:08:04.371820+0000","last_peered":"2023-11-28T13:08:04.371820+0000","last_clean":"2023-11-28T13:08:04.371820+0000","last_became_active":"2023-11-28T13:08:02.574933+0000","last_became_peered":"2023-11-28T13:08:02.574933+0000","last_unstale":"2023-11-28T13:08:04.371820+0000","last_undegraded":"2023-11-28T13:08:04.371820+0000","last_fullsized":"2023-11-28T13:08:04.371820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:56.910035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372498+0000","last_change":"2023-11-28T13:08:03.077902+0000","last_active":"2023-11-28T13:08:04.372498+0000","last_peered":"2023-11-28T13:08:04.372498+0000","last_clean":"2023-11-28T13:08:04.372498+0000","last_became_active":"2023-11-28T13:08:03.077379+0000","last_became_peered":"2023-11-28T13:08:03.077379+0000","last_unstale":"2023-11-28T13:08:04.372498+0000","last_undegraded":"2023-11-28T13:08:04.372498+0000","last_fullsized":"2023-11-28T13:08:04.372498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:34.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064779+0000","last_change":"2023-11-28T13:08:02.574513+0000","last_active":"2023-11-28T13:08:04.064779+0000","last_peered":"2023-11-28T13:08:04.064779+0000","last_clean":"2023-11-28T13:08:04.064779+0000","last_became_active":"2023-11-28T13:08:02.573509+0000","last_became_peered":"2023-11-28T13:08:02.573509+0000","last_unstale":"2023-11-28T13:08:04.064779+0000","last_undegraded":"2023-11-28T13:08:04.064779+0000","last_fullsized":"2023-11-28T13:08:04.064779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:01.496804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372381+0000","last_change":"2023-11-28T13:08:03.091021+0000","last_active":"2023-11-28T13:08:04.372381+0000","last_peered":"2023-11-28T13:08:04.372381+0000","last_clean":"2023-11-28T13:08:04.372381+0000","last_became_active":"2023-11-28T13:08:03.090717+0000","last_became_peered":"2023-11-28T13:08:03.090717+0000","last_unstale":"2023-11-28T13:08:04.372381+0000","last_undegraded":"2023-11-28T13:08:04.372381+0000","last_fullsized":"2023-11-28T13:08:04.372381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:50.062766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372655+0000","last_change":"2023-11-28T13:08:02.580525+0000","last_active":"2023-11-28T13:08:04.372655+0000","last_peered":"2023-11-28T13:08:04.372655+0000","last_clean":"2023-11-28T13:08:04.372655+0000","last_became_active":"2023-11-28T13:08:02.580220+0000","last_became_peered":"2023-11-28T13:08:02.580220+0000","last_unstale":"2023-11-28T13:08:04.372655+0000","last_undegraded":"2023-11-28T13:08:04.372655+0000","last_fullsized":"2023-11-28T13:08:04.372655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:01.496804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064707+0000","last_change":"2023-11-28T13:08:03.080244+0000","last_active":"2023-11-28T13:08:04.064707+0000","last_peered":"2023-11-28T13:08:04.064707+0000","last_clean":"2023-11-28T13:08:04.064707+0000","last_became_active":"2023-11-28T13:08:03.079988+0000","last_became_peered":"2023-11-28T13:08:03.079988+0000","last_unstale":"2023-11-28T13:08:04.064707+0000","last_undegraded":"2023-11-28T13:08:04.064707+0000","last_fullsized":"2023-11-28T13:08:04.064707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:49.707036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070421+0000","last_change":"2023-11-28T13:08:02.573388+0000","last_active":"2023-11-28T13:08:04.070421+0000","last_peered":"2023-11-28T13:08:04.070421+0000","last_clean":"2023-11-28T13:08:04.070421+0000","last_became_active":"2023-11-28T13:08:02.573059+0000","last_became_peered":"2023-11-28T13:08:02.573059+0000","last_unstale":"2023-11-28T13:08:04.070421+0000","last_undegraded":"2023-11-28T13:08:04.070421+0000","last_fullsized":"2023-11-28T13:08:04.070421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:46:29.622650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064417+0000","last_change":"2023-11-28T13:08:03.088704+0000","last_active":"2023-11-28T13:08:04.064417+0000","last_peered":"2023-11-28T13:08:04.064417+0000","last_clean":"2023-11-28T13:08:04.064417+0000","last_became_active":"2023-11-28T13:08:03.088417+0000","last_became_peered":"2023-11-28T13:08:03.088417+0000","last_unstale":"2023-11-28T13:08:04.064417+0000","last_undegraded":"2023-11-28T13:08:04.064417+0000","last_fullsized":"2023-11-28T13:08:04.064417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:41.196768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070940+0000","last_change":"2023-11-28T13:08:02.576140+0000","last_active":"2023-11-28T13:08:04.070940+0000","last_peered":"2023-11-28T13:08:04.070940+0000","last_clean":"2023-11-28T13:08:04.070940+0000","last_became_active":"2023-11-28T13:08:02.575721+0000","last_became_peered":"2023-11-28T13:08:02.575721+0000","last_unstale":"2023-11-28T13:08:04.070940+0000","last_undegraded":"2023-11-28T13:08:04.070940+0000","last_fullsized":"2023-11-28T13:08:04.070940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:12.214372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371693+0000","last_change":"2023-11-28T13:08:03.079421+0000","last_active":"2023-11-28T13:08:04.371693+0000","last_peered":"2023-11-28T13:08:04.371693+0000","last_clean":"2023-11-28T13:08:04.371693+0000","last_became_active":"2023-11-28T13:08:03.078471+0000","last_became_peered":"2023-11-28T13:08:03.078471+0000","last_unstale":"2023-11-28T13:08:04.371693+0000","last_undegraded":"2023-11-28T13:08:04.371693+0000","last_fullsized":"2023-11-28T13:08:04.371693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:05:35.135974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371902+0000","last_change":"2023-11-28T13:08:02.581355+0000","last_active":"2023-11-28T13:08:04.371902+0000","last_peered":"2023-11-28T13:08:04.371902+0000","last_clean":"2023-11-28T13:08:04.371902+0000","last_became_active":"2023-11-28T13:08:02.581051+0000","last_became_peered":"2023-11-28T13:08:02.581051+0000","last_unstale":"2023-11-28T13:08:04.371902+0000","last_undegraded":"2023-11-28T13:08:04.371902+0000","last_fullsized":"2023-11-28T13:08:04.371902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:43.117989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371569+0000","last_change":"2023-11-28T13:08:03.085302+0000","last_active":"2023-11-28T13:08:04.371569+0000","last_peered":"2023-11-28T13:08:04.371569+0000","last_clean":"2023-11-28T13:08:04.371569+0000","last_became_active":"2023-11-28T13:08:03.085061+0000","last_became_peered":"2023-11-28T13:08:03.085061+0000","last_unstale":"2023-11-28T13:08:04.371569+0000","last_undegraded":"2023-11-28T13:08:04.371569+0000","last_fullsized":"2023-11-28T13:08:04.371569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:13.625233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070589+0000","last_change":"2023-11-28T13:08:02.584763+0000","last_active":"2023-11-28T13:08:04.070589+0000","last_peered":"2023-11-28T13:08:04.070589+0000","last_clean":"2023-11-28T13:08:04.070589+0000","last_became_active":"2023-11-28T13:08:02.583655+0000","last_became_peered":"2023-11-28T13:08:02.583655+0000","last_unstale":"2023-11-28T13:08:04.070589+0000","last_undegraded":"2023-11-28T13:08:04.070589+0000","last_fullsized":"2023-11-28T13:08:04.070589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:53:59.398155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064403+0000","last_change":"2023-11-28T13:08:03.095515+0000","last_active":"2023-11-28T13:08:04.064403+0000","last_peered":"2023-11-28T13:08:04.064403+0000","last_clean":"2023-11-28T13:08:04.064403+0000","last_became_active":"2023-11-28T13:08:03.095004+0000","last_became_peered":"2023-11-28T13:08:03.095004+0000","last_unstale":"2023-11-28T13:08:04.064403+0000","last_undegraded":"2023-11-28T13:08:04.064403+0000","last_fullsized":"2023-11-28T13:08:04.064403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:34.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372572+0000","last_change":"2023-11-28T13:08:02.580082+0000","last_active":"2023-11-28T13:08:04.372572+0000","last_peered":"2023-11-28T13:08:04.372572+0000","last_clean":"2023-11-28T13:08:04.372572+0000","last_became_active":"2023-11-28T13:08:02.579745+0000","last_became_peered":"2023-11-28T13:08:02.579745+0000","last_unstale":"2023-11-28T13:08:04.372572+0000","last_undegraded":"2023-11-28T13:08:04.372572+0000","last_fullsized":"2023-11-28T13:08:04.372572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:12.363239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070163+0000","last_change":"2023-11-28T13:08:03.082966+0000","last_active":"2023-11-28T13:08:04.070163+0000","last_peered":"2023-11-28T13:08:04.070163+0000","last_clean":"2023-11-28T13:08:04.070163+0000","last_became_active":"2023-11-28T13:08:03.082710+0000","last_became_peered":"2023-11-28T13:08:03.082710+0000","last_unstale":"2023-11-28T13:08:04.070163+0000","last_undegraded":"2023-11-28T13:08:04.070163+0000","last_fullsized":"2023-11-28T13:08:04.070163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:49.707036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070236+0000","last_change":"2023-11-28T13:08:02.585538+0000","last_active":"2023-11-28T13:08:04.070236+0000","last_peered":"2023-11-28T13:08:04.070236+0000","last_clean":"2023-11-28T13:08:04.070236+0000","last_became_active":"2023-11-28T13:08:02.584407+0000","last_became_peered":"2023-11-28T13:08:02.584407+0000","last_unstale":"2023-11-28T13:08:04.070236+0000","last_undegraded":"2023-11-28T13:08:04.070236+0000","last_fullsized":"2023-11-28T13:08:04.070236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:01.496804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372389+0000","last_change":"2023-11-28T13:08:03.077754+0000","last_active":"2023-11-28T13:08:04.372389+0000","last_peered":"2023-11-28T13:08:04.372389+0000","last_clean":"2023-11-28T13:08:04.372389+0000","last_became_active":"2023-11-28T13:08:03.077073+0000","last_became_peered":"2023-11-28T13:08:03.077073+0000","last_unstale":"2023-11-28T13:08:04.372389+0000","last_undegraded":"2023-11-28T13:08:04.372389+0000","last_fullsized":"2023-11-28T13:08:04.372389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:46.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371255+0000","last_change":"2023-11-28T13:08:02.574654+0000","last_active":"2023-11-28T13:08:04.371255+0000","last_peered":"2023-11-28T13:08:04.371255+0000","last_clean":"2023-11-28T13:08:04.371255+0000","last_became_active":"2023-11-28T13:08:02.574354+0000","last_became_peered":"2023-11-28T13:08:02.574354+0000","last_unstale":"2023-11-28T13:08:04.371255+0000","last_undegraded":"2023-11-28T13:08:04.371255+0000","last_fullsized":"2023-11-28T13:08:04.371255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:43.117989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064965+0000","last_change":"2023-11-28T13:08:03.084191+0000","last_active":"2023-11-28T13:08:04.064965+0000","last_peered":"2023-11-28T13:08:04.064965+0000","last_clean":"2023-11-28T13:08:04.064965+0000","last_became_active":"2023-11-28T13:08:03.083879+0000","last_became_peered":"2023-11-28T13:08:03.083879+0000","last_unstale":"2023-11-28T13:08:04.064965+0000","last_undegraded":"2023-11-28T13:08:04.064965+0000","last_fullsized":"2023-11-28T13:08:04.064965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:14.015315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.071066+0000","last_change":"2023-11-28T13:08:02.584107+0000","last_active":"2023-11-28T13:08:04.071066+0000","last_peered":"2023-11-28T13:08:04.071066+0000","last_clean":"2023-11-28T13:08:04.071066+0000","last_became_active":"2023-11-28T13:08:02.583702+0000","last_became_peered":"2023-11-28T13:08:02.583702+0000","last_unstale":"2023-11-28T13:08:04.071066+0000","last_undegraded":"2023-11-28T13:08:04.071066+0000","last_fullsized":"2023-11-28T13:08:04.071066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:37.724216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069638+0000","last_change":"2023-11-28T13:08:03.077169+0000","last_active":"2023-11-28T13:08:04.069638+0000","last_peered":"2023-11-28T13:08:04.069638+0000","last_clean":"2023-11-28T13:08:04.069638+0000","last_became_active":"2023-11-28T13:08:03.076875+0000","last_became_peered":"2023-11-28T13:08:03.076875+0000","last_unstale":"2023-11-28T13:08:04.069638+0000","last_undegraded":"2023-11-28T13:08:04.069638+0000","last_fullsized":"2023-11-28T13:08:04.069638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:37:44.657148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.071007+0000","last_change":"2023-11-28T13:08:02.584070+0000","last_active":"2023-11-28T13:08:04.071007+0000","last_peered":"2023-11-28T13:08:04.071007+0000","last_clean":"2023-11-28T13:08:04.071007+0000","last_became_active":"2023-11-28T13:08:02.583527+0000","last_became_peered":"2023-11-28T13:08:02.583527+0000","last_unstale":"2023-11-28T13:08:04.071007+0000","last_undegraded":"2023-11-28T13:08:04.071007+0000","last_fullsized":"2023-11-28T13:08:04.071007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:49.060104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371361+0000","last_change":"2023-11-28T13:08:03.082338+0000","last_active":"2023-11-28T13:08:04.371361+0000","last_peered":"2023-11-28T13:08:04.371361+0000","last_clean":"2023-11-28T13:08:04.371361+0000","last_became_active":"2023-11-28T13:08:03.082006+0000","last_became_peered":"2023-11-28T13:08:03.082006+0000","last_unstale":"2023-11-28T13:08:04.371361+0000","last_undegraded":"2023-11-28T13:08:04.371361+0000","last_fullsized":"2023-11-28T13:08:04.371361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:56:48.282770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070812+0000","last_change":"2023-11-28T13:08:02.587071+0000","last_active":"2023-11-28T13:08:04.070812+0000","last_peered":"2023-11-28T13:08:04.070812+0000","last_clean":"2023-11-28T13:08:04.070812+0000","last_became_active":"2023-11-28T13:08:02.586541+0000","last_became_peered":"2023-11-28T13:08:02.586541+0000","last_unstale":"2023-11-28T13:08:04.070812+0000","last_undegraded":"2023-11-28T13:08:04.070812+0000","last_fullsized":"2023-11-28T13:08:04.070812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:56.535608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372224+0000","last_change":"2023-11-28T13:08:03.081008+0000","last_active":"2023-11-28T13:08:04.372224+0000","last_peered":"2023-11-28T13:08:04.372224+0000","last_clean":"2023-11-28T13:08:04.372224+0000","last_became_active":"2023-11-28T13:08:03.080718+0000","last_became_peered":"2023-11-28T13:08:03.080718+0000","last_unstale":"2023-11-28T13:08:04.372224+0000","last_undegraded":"2023-11-28T13:08:04.372224+0000","last_fullsized":"2023-11-28T13:08:04.372224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:00.400817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069777+0000","last_change":"2023-11-28T13:08:02.577784+0000","last_active":"2023-11-28T13:08:04.069777+0000","last_peered":"2023-11-28T13:08:04.069777+0000","last_clean":"2023-11-28T13:08:04.069777+0000","last_became_active":"2023-11-28T13:08:02.577364+0000","last_became_peered":"2023-11-28T13:08:02.577364+0000","last_unstale":"2023-11-28T13:08:04.069777+0000","last_undegraded":"2023-11-28T13:08:04.069777+0000","last_fullsized":"2023-11-28T13:08:04.069777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:37.654488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372256+0000","last_change":"2023-11-28T13:08:03.075105+0000","last_active":"2023-11-28T13:08:04.372256+0000","last_peered":"2023-11-28T13:08:04.372256+0000","last_clean":"2023-11-28T13:08:04.372256+0000","last_became_active":"2023-11-28T13:08:03.074768+0000","last_became_peered":"2023-11-28T13:08:03.074768+0000","last_unstale":"2023-11-28T13:08:04.372256+0000","last_undegraded":"2023-11-28T13:08:04.372256+0000","last_fullsized":"2023-11-28T13:08:04.372256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:37:44.657148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064578+0000","last_change":"2023-11-28T13:08:02.582099+0000","last_active":"2023-11-28T13:08:04.064578+0000","last_peered":"2023-11-28T13:08:04.064578+0000","last_clean":"2023-11-28T13:08:04.064578+0000","last_became_active":"2023-11-28T13:08:02.581651+0000","last_became_peered":"2023-11-28T13:08:02.581651+0000","last_unstale":"2023-11-28T13:08:04.064578+0000","last_undegraded":"2023-11-28T13:08:04.064578+0000","last_fullsized":"2023-11-28T13:08:04.064578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:12.214372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371794+0000","last_change":"2023-11-28T13:08:03.088535+0000","last_active":"2023-11-28T13:08:04.371794+0000","last_peered":"2023-11-28T13:08:04.371794+0000","last_clean":"2023-11-28T13:08:04.371794+0000","last_became_active":"2023-11-28T13:08:03.088263+0000","last_became_peered":"2023-11-28T13:08:03.088263+0000","last_unstale":"2023-11-28T13:08:04.371794+0000","last_undegraded":"2023-11-28T13:08:04.371794+0000","last_fullsized":"2023-11-28T13:08:04.371794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:22.644742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372159+0000","last_change":"2023-11-28T13:08:02.584910+0000","last_active":"2023-11-28T13:08:04.372159+0000","last_peered":"2023-11-28T13:08:04.372159+0000","last_clean":"2023-11-28T13:08:04.372159+0000","last_became_active":"2023-11-28T13:08:02.584573+0000","last_became_peered":"2023-11-28T13:08:02.584573+0000","last_unstale":"2023-11-28T13:08:04.372159+0000","last_undegraded":"2023-11-28T13:08:04.372159+0000","last_fullsized":"2023-11-28T13:08:04.372159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:49.593563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372100+0000","last_change":"2023-11-28T13:08:03.083587+0000","last_active":"2023-11-28T13:08:04.372100+0000","last_peered":"2023-11-28T13:08:04.372100+0000","last_clean":"2023-11-28T13:08:04.372100+0000","last_became_active":"2023-11-28T13:08:03.081658+0000","last_became_peered":"2023-11-28T13:08:03.081658+0000","last_unstale":"2023-11-28T13:08:04.372100+0000","last_undegraded":"2023-11-28T13:08:04.372100+0000","last_fullsized":"2023-11-28T13:08:04.372100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:53.049281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371468+0000","last_change":"2023-11-28T13:08:02.582045+0000","last_active":"2023-11-28T13:08:04.371468+0000","last_peered":"2023-11-28T13:08:04.371468+0000","last_clean":"2023-11-28T13:08:04.371468+0000","last_became_active":"2023-11-28T13:08:02.581687+0000","last_became_peered":"2023-11-28T13:08:02.581687+0000","last_unstale":"2023-11-28T13:08:04.371468+0000","last_undegraded":"2023-11-28T13:08:04.371468+0000","last_fullsized":"2023-11-28T13:08:04.371468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:56.910035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372483+0000","last_change":"2023-11-28T13:08:03.090341+0000","last_active":"2023-11-28T13:08:04.372483+0000","last_peered":"2023-11-28T13:08:04.372483+0000","last_clean":"2023-11-28T13:08:04.372483+0000","last_became_active":"2023-11-28T13:08:03.086320+0000","last_became_peered":"2023-11-28T13:08:03.086320+0000","last_unstale":"2023-11-28T13:08:04.372483+0000","last_undegraded":"2023-11-28T13:08:04.372483+0000","last_fullsized":"2023-11-28T13:08:04.372483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:30.263143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064693+0000","last_change":"2023-11-28T13:08:02.577006+0000","last_active":"2023-11-28T13:08:04.064693+0000","last_peered":"2023-11-28T13:08:04.064693+0000","last_clean":"2023-11-28T13:08:04.064693+0000","last_became_active":"2023-11-28T13:08:02.576734+0000","last_became_peered":"2023-11-28T13:08:02.576734+0000","last_unstale":"2023-11-28T13:08:04.064693+0000","last_undegraded":"2023-11-28T13:08:04.064693+0000","last_fullsized":"2023-11-28T13:08:04.064693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:40.847112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372143+0000","last_change":"2023-11-28T13:08:03.080548+0000","last_active":"2023-11-28T13:08:04.372143+0000","last_peered":"2023-11-28T13:08:04.372143+0000","last_clean":"2023-11-28T13:08:04.372143+0000","last_became_active":"2023-11-28T13:08:03.077609+0000","last_became_peered":"2023-11-28T13:08:03.077609+0000","last_unstale":"2023-11-28T13:08:04.372143+0000","last_undegraded":"2023-11-28T13:08:04.372143+0000","last_fullsized":"2023-11-28T13:08:04.372143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:57.732415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064801+0000","last_change":"2023-11-28T13:08:02.579378+0000","last_active":"2023-11-28T13:08:04.064801+0000","last_peered":"2023-11-28T13:08:04.064801+0000","last_clean":"2023-11-28T13:08:04.064801+0000","last_became_active":"2023-11-28T13:08:02.579080+0000","last_became_peered":"2023-11-28T13:08:02.579080+0000","last_unstale":"2023-11-28T13:08:04.064801+0000","last_undegraded":"2023-11-28T13:08:04.064801+0000","last_fullsized":"2023-11-28T13:08:04.064801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:36.421141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.075126+0000","last_change":"2023-11-28T13:07:56.890378+0000","last_active":"2023-11-28T13:08:03.075126+0000","last_peered":"2023-11-28T13:08:03.075126+0000","last_clean":"2023-11-28T13:08:03.075126+0000","last_became_active":"2023-11-28T13:07:56.890058+0000","last_became_peered":"2023-11-28T13:07:56.890058+0000","last_unstale":"2023-11-28T13:08:03.075126+0000","last_undegraded":"2023-11-28T13:08:03.075126+0000","last_fullsized":"2023-11-28T13:08:03.075126+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:05.344602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064197+0000","last_change":"2023-11-28T13:08:03.084094+0000","last_active":"2023-11-28T13:08:04.064197+0000","last_peered":"2023-11-28T13:08:04.064197+0000","last_clean":"2023-11-28T13:08:04.064197+0000","last_became_active":"2023-11-28T13:08:03.083811+0000","last_became_peered":"2023-11-28T13:08:03.083811+0000","last_unstale":"2023-11-28T13:08:04.064197+0000","last_undegraded":"2023-11-28T13:08:04.064197+0000","last_fullsized":"2023-11-28T13:08:04.064197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:53.049281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.072799+0000","last_change":"2023-11-28T13:07:56.844509+0000","last_active":"2023-11-28T13:08:03.072799+0000","last_peered":"2023-11-28T13:08:03.072799+0000","last_clean":"2023-11-28T13:08:03.072799+0000","last_became_active":"2023-11-28T13:07:56.844081+0000","last_became_peered":"2023-11-28T13:07:56.844081+0000","last_unstale":"2023-11-28T13:08:03.072799+0000","last_undegraded":"2023-11-28T13:08:03.072799+0000","last_fullsized":"2023-11-28T13:08:03.072799+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372520+0000","last_change":"2023-11-28T13:08:02.576708+0000","last_active":"2023-11-28T13:08:04.372520+0000","last_peered":"2023-11-28T13:08:04.372520+0000","last_clean":"2023-11-28T13:08:04.372520+0000","last_became_active":"2023-11-28T13:08:02.572927+0000","last_became_peered":"2023-11-28T13:08:02.572927+0000","last_unstale":"2023-11-28T13:08:04.372520+0000","last_undegraded":"2023-11-28T13:08:04.372520+0000","last_fullsized":"2023-11-28T13:08:04.372520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:43.117989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372278+0000","last_change":"2023-11-28T13:08:03.093540+0000","last_active":"2023-11-28T13:08:04.372278+0000","last_peered":"2023-11-28T13:08:04.372278+0000","last_clean":"2023-11-28T13:08:04.372278+0000","last_became_active":"2023-11-28T13:08:03.089698+0000","last_became_peered":"2023-11-28T13:08:03.089698+0000","last_unstale":"2023-11-28T13:08:04.372278+0000","last_undegraded":"2023-11-28T13:08:04.372278+0000","last_fullsized":"2023-11-28T13:08:04.372278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:29:46.171857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.062393+0000","last_change":"2023-11-28T13:07:56.844699+0000","last_active":"2023-11-28T13:08:03.062393+0000","last_peered":"2023-11-28T13:08:03.062393+0000","last_clean":"2023-11-28T13:08:03.062393+0000","last_became_active":"2023-11-28T13:07:56.844391+0000","last_became_peered":"2023-11-28T13:07:56.844391+0000","last_unstale":"2023-11-28T13:08:03.062393+0000","last_undegraded":"2023-11-28T13:08:03.062393+0000","last_fullsized":"2023-11-28T13:08:03.062393+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.074953+0000","last_change":"2023-11-28T13:07:57.844020+0000","last_active":"2023-11-28T13:08:03.074953+0000","last_peered":"2023-11-28T13:08:03.074953+0000","last_clean":"2023-11-28T13:08:03.074953+0000","last_became_active":"2023-11-28T13:07:57.842981+0000","last_became_peered":"2023-11-28T13:07:57.842981+0000","last_unstale":"2023-11-28T13:08:03.074953+0000","last_undegraded":"2023-11-28T13:08:03.074953+0000","last_fullsized":"2023-11-28T13:08:03.074953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_clean_scrub_stamp":"2023-11-28T13:07:56.825711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:44.973218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371955+0000","last_change":"2023-11-28T13:08:02.583813+0000","last_active":"2023-11-28T13:08:04.371955+0000","last_peered":"2023-11-28T13:08:04.371955+0000","last_clean":"2023-11-28T13:08:04.371955+0000","last_became_active":"2023-11-28T13:08:02.583528+0000","last_became_peered":"2023-11-28T13:08:02.583528+0000","last_unstale":"2023-11-28T13:08:04.371955+0000","last_undegraded":"2023-11-28T13:08:04.371955+0000","last_fullsized":"2023-11-28T13:08:04.371955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:49.593563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372411+0000","last_change":"2023-11-28T13:08:03.077611+0000","last_active":"2023-11-28T13:08:04.372411+0000","last_peered":"2023-11-28T13:08:04.372411+0000","last_clean":"2023-11-28T13:08:04.372411+0000","last_became_active":"2023-11-28T13:08:03.076746+0000","last_became_peered":"2023-11-28T13:08:03.076746+0000","last_unstale":"2023-11-28T13:08:04.372411+0000","last_undegraded":"2023-11-28T13:08:04.372411+0000","last_fullsized":"2023-11-28T13:08:04.372411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:49.707036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.075246+0000","last_change":"2023-11-28T13:07:57.306318+0000","last_active":"2023-11-28T13:08:03.075246+0000","last_peered":"2023-11-28T13:08:03.075246+0000","last_clean":"2023-11-28T13:08:03.075246+0000","last_became_active":"2023-11-28T13:07:57.305937+0000","last_became_peered":"2023-11-28T13:07:57.305937+0000","last_unstale":"2023-11-28T13:08:03.075246+0000","last_undegraded":"2023-11-28T13:08:03.075246+0000","last_fullsized":"2023-11-28T13:08:03.075246+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371445+0000","last_change":"2023-11-28T13:08:02.569783+0000","last_active":"2023-11-28T13:08:04.371445+0000","last_peered":"2023-11-28T13:08:04.371445+0000","last_clean":"2023-11-28T13:08:04.371445+0000","last_became_active":"2023-11-28T13:08:02.569505+0000","last_became_peered":"2023-11-28T13:08:02.569505+0000","last_unstale":"2023-11-28T13:08:04.371445+0000","last_undegraded":"2023-11-28T13:08:04.371445+0000","last_fullsized":"2023-11-28T13:08:04.371445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:12.363239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372117+0000","last_change":"2023-11-28T13:08:03.094969+0000","last_active":"2023-11-28T13:08:04.372117+0000","last_peered":"2023-11-28T13:08:04.372117+0000","last_clean":"2023-11-28T13:08:04.372117+0000","last_became_active":"2023-11-28T13:08:03.094723+0000","last_became_peered":"2023-11-28T13:08:03.094723+0000","last_unstale":"2023-11-28T13:08:04.372117+0000","last_undegraded":"2023-11-28T13:08:04.372117+0000","last_fullsized":"2023-11-28T13:08:04.372117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:37:44.657148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.074562+0000","last_change":"2023-11-28T13:07:57.306277+0000","last_active":"2023-11-28T13:08:03.074562+0000","last_peered":"2023-11-28T13:08:03.074562+0000","last_clean":"2023-11-28T13:08:03.074562+0000","last_became_active":"2023-11-28T13:07:57.305983+0000","last_became_peered":"2023-11-28T13:07:57.305983+0000","last_unstale":"2023-11-28T13:08:03.074562+0000","last_undegraded":"2023-11-28T13:08:03.074562+0000","last_fullsized":"2023-11-28T13:08:03.074562+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064565+0000","last_change":"2023-11-28T13:08:02.578428+0000","last_active":"2023-11-28T13:08:04.064565+0000","last_peered":"2023-11-28T13:08:04.064565+0000","last_clean":"2023-11-28T13:08:04.064565+0000","last_became_active":"2023-11-28T13:08:02.578147+0000","last_became_peered":"2023-11-28T13:08:02.578147+0000","last_unstale":"2023-11-28T13:08:04.064565+0000","last_undegraded":"2023-11-28T13:08:04.064565+0000","last_fullsized":"2023-11-28T13:08:04.064565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:43.117989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372191+0000","last_change":"2023-11-28T13:08:03.094387+0000","last_active":"2023-11-28T13:08:04.372191+0000","last_peered":"2023-11-28T13:08:04.372191+0000","last_clean":"2023-11-28T13:08:04.372191+0000","last_became_active":"2023-11-28T13:08:03.094117+0000","last_became_peered":"2023-11-28T13:08:03.094117+0000","last_unstale":"2023-11-28T13:08:04.372191+0000","last_undegraded":"2023-11-28T13:08:04.372191+0000","last_fullsized":"2023-11-28T13:08:04.372191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:38.387013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.068929+0000","last_change":"2023-11-28T13:07:57.306177+0000","last_active":"2023-11-28T13:08:03.068929+0000","last_peered":"2023-11-28T13:08:03.068929+0000","last_clean":"2023-11-28T13:08:03.068929+0000","last_became_active":"2023-11-28T13:07:57.305853+0000","last_became_peered":"2023-11-28T13:07:57.305853+0000","last_unstale":"2023-11-28T13:08:03.068929+0000","last_undegraded":"2023-11-28T13:08:03.068929+0000","last_fullsized":"2023-11-28T13:08:03.068929+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371895+0000","last_change":"2023-11-28T13:08:02.584788+0000","last_active":"2023-11-28T13:08:04.371895+0000","last_peered":"2023-11-28T13:08:04.371895+0000","last_clean":"2023-11-28T13:08:04.371895+0000","last_became_active":"2023-11-28T13:08:02.584281+0000","last_became_peered":"2023-11-28T13:08:02.584281+0000","last_unstale":"2023-11-28T13:08:04.371895+0000","last_undegraded":"2023-11-28T13:08:04.371895+0000","last_fullsized":"2023-11-28T13:08:04.371895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:40.847112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371995+0000","last_change":"2023-11-28T13:08:03.074701+0000","last_active":"2023-11-28T13:08:04.371995+0000","last_peered":"2023-11-28T13:08:04.371995+0000","last_clean":"2023-11-28T13:08:04.371995+0000","last_became_active":"2023-11-28T13:08:03.074336+0000","last_became_peered":"2023-11-28T13:08:03.074336+0000","last_unstale":"2023-11-28T13:08:04.371995+0000","last_undegraded":"2023-11-28T13:08:04.371995+0000","last_fullsized":"2023-11-28T13:08:04.371995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:38.387013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.074084+0000","last_change":"2023-11-28T13:07:57.306321+0000","last_active":"2023-11-28T13:08:03.074084+0000","last_peered":"2023-11-28T13:08:03.074084+0000","last_clean":"2023-11-28T13:08:03.074084+0000","last_became_active":"2023-11-28T13:07:57.305923+0000","last_became_peered":"2023-11-28T13:07:57.305923+0000","last_unstale":"2023-11-28T13:08:03.074084+0000","last_undegraded":"2023-11-28T13:08:03.074084+0000","last_fullsized":"2023-11-28T13:08:03.074084+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:41.104679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371927+0000","last_change":"2023-11-28T13:08:02.570790+0000","last_active":"2023-11-28T13:08:04.371927+0000","last_peered":"2023-11-28T13:08:04.371927+0000","last_clean":"2023-11-28T13:08:04.371927+0000","last_became_active":"2023-11-28T13:08:02.569907+0000","last_became_peered":"2023-11-28T13:08:02.569907+0000","last_unstale":"2023-11-28T13:08:04.371927+0000","last_undegraded":"2023-11-28T13:08:04.371927+0000","last_fullsized":"2023-11-28T13:08:04.371927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:46.379743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372610+0000","last_change":"2023-11-28T13:08:03.087530+0000","last_active":"2023-11-28T13:08:04.372610+0000","last_peered":"2023-11-28T13:08:04.372610+0000","last_clean":"2023-11-28T13:08:04.372610+0000","last_became_active":"2023-11-28T13:08:03.087191+0000","last_became_peered":"2023-11-28T13:08:03.087191+0000","last_unstale":"2023-11-28T13:08:04.372610+0000","last_undegraded":"2023-11-28T13:08:04.372610+0000","last_fullsized":"2023-11-28T13:08:04.372610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:38.726877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.074497+0000","last_change":"2023-11-28T13:07:56.844305+0000","last_active":"2023-11-28T13:08:03.074497+0000","last_peered":"2023-11-28T13:08:03.074497+0000","last_clean":"2023-11-28T13:08:03.074497+0000","last_became_active":"2023-11-28T13:07:56.843980+0000","last_became_peered":"2023-11-28T13:07:56.843980+0000","last_unstale":"2023-11-28T13:08:03.074497+0000","last_undegraded":"2023-11-28T13:08:03.074497+0000","last_fullsized":"2023-11-28T13:08:03.074497+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070218+0000","last_change":"2023-11-28T13:08:02.577243+0000","last_active":"2023-11-28T13:08:04.070218+0000","last_peered":"2023-11-28T13:08:04.070218+0000","last_clean":"2023-11-28T13:08:04.070218+0000","last_became_active":"2023-11-28T13:08:02.576748+0000","last_became_peered":"2023-11-28T13:08:02.576748+0000","last_unstale":"2023-11-28T13:08:04.070218+0000","last_undegraded":"2023-11-28T13:08:04.070218+0000","last_fullsized":"2023-11-28T13:08:04.070218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:01.496804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371797+0000","last_change":"2023-11-28T13:08:03.095221+0000","last_active":"2023-11-28T13:08:04.371797+0000","last_peered":"2023-11-28T13:08:04.371797+0000","last_clean":"2023-11-28T13:08:04.371797+0000","last_became_active":"2023-11-28T13:08:03.091841+0000","last_became_peered":"2023-11-28T13:08:03.091841+0000","last_unstale":"2023-11-28T13:08:04.371797+0000","last_undegraded":"2023-11-28T13:08:04.371797+0000","last_fullsized":"2023-11-28T13:08:04.371797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:43.340474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372325+0000","last_change":"2023-11-28T13:08:02.583653+0000","last_active":"2023-11-28T13:08:04.372325+0000","last_peered":"2023-11-28T13:08:04.372325+0000","last_clean":"2023-11-28T13:08:04.372325+0000","last_became_active":"2023-11-28T13:08:02.583397+0000","last_became_peered":"2023-11-28T13:08:02.583397+0000","last_unstale":"2023-11-28T13:08:04.372325+0000","last_undegraded":"2023-11-28T13:08:04.372325+0000","last_fullsized":"2023-11-28T13:08:04.372325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:40.847112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371135+0000","last_change":"2023-11-28T13:08:03.096716+0000","last_active":"2023-11-28T13:08:04.371135+0000","last_peered":"2023-11-28T13:08:04.371135+0000","last_clean":"2023-11-28T13:08:04.371135+0000","last_became_active":"2023-11-28T13:08:03.096239+0000","last_became_peered":"2023-11-28T13:08:03.096239+0000","last_unstale":"2023-11-28T13:08:04.371135+0000","last_undegraded":"2023-11-28T13:08:04.371135+0000","last_fullsized":"2023-11-28T13:08:04.371135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:13.217034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069581+0000","last_change":"2023-11-28T13:08:02.581525+0000","last_active":"2023-11-28T13:08:04.069581+0000","last_peered":"2023-11-28T13:08:04.069581+0000","last_clean":"2023-11-28T13:08:04.069581+0000","last_became_active":"2023-11-28T13:08:02.581253+0000","last_became_peered":"2023-11-28T13:08:02.581253+0000","last_unstale":"2023-11-28T13:08:04.069581+0000","last_undegraded":"2023-11-28T13:08:04.069581+0000","last_fullsized":"2023-11-28T13:08:04.069581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:36.421141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372111+0000","last_change":"2023-11-28T13:08:03.084538+0000","last_active":"2023-11-28T13:08:04.372111+0000","last_peered":"2023-11-28T13:08:04.372111+0000","last_clean":"2023-11-28T13:08:04.372111+0000","last_became_active":"2023-11-28T13:08:03.084256+0000","last_became_peered":"2023-11-28T13:08:03.084256+0000","last_unstale":"2023-11-28T13:08:04.372111+0000","last_undegraded":"2023-11-28T13:08:04.372111+0000","last_fullsized":"2023-11-28T13:08:04.372111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:46:46.224857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070254+0000","last_change":"2023-11-28T13:08:02.583812+0000","last_active":"2023-11-28T13:08:04.070254+0000","last_peered":"2023-11-28T13:08:04.070254+0000","last_clean":"2023-11-28T13:08:04.070254+0000","last_became_active":"2023-11-28T13:08:02.583517+0000","last_became_peered":"2023-11-28T13:08:02.583517+0000","last_unstale":"2023-11-28T13:08:04.070254+0000","last_undegraded":"2023-11-28T13:08:04.070254+0000","last_fullsized":"2023-11-28T13:08:04.070254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:12.214372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371726+0000","last_change":"2023-11-28T13:08:03.088006+0000","last_active":"2023-11-28T13:08:04.371726+0000","last_peered":"2023-11-28T13:08:04.371726+0000","last_clean":"2023-11-28T13:08:04.371726+0000","last_became_active":"2023-11-28T13:08:03.086226+0000","last_became_peered":"2023-11-28T13:08:03.086226+0000","last_unstale":"2023-11-28T13:08:04.371726+0000","last_undegraded":"2023-11-28T13:08:04.371726+0000","last_fullsized":"2023-11-28T13:08:04.371726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:08.985463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064555+0000","last_change":"2023-11-28T13:08:02.574322+0000","last_active":"2023-11-28T13:08:04.064555+0000","last_peered":"2023-11-28T13:08:04.064555+0000","last_clean":"2023-11-28T13:08:04.064555+0000","last_became_active":"2023-11-28T13:08:02.573611+0000","last_became_peered":"2023-11-28T13:08:02.573611+0000","last_unstale":"2023-11-28T13:08:04.064555+0000","last_undegraded":"2023-11-28T13:08:04.064555+0000","last_fullsized":"2023-11-28T13:08:04.064555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:37.654488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064305+0000","last_change":"2023-11-28T13:08:03.093751+0000","last_active":"2023-11-28T13:08:04.064305+0000","last_peered":"2023-11-28T13:08:04.064305+0000","last_clean":"2023-11-28T13:08:04.064305+0000","last_became_active":"2023-11-28T13:08:03.093493+0000","last_became_peered":"2023-11-28T13:08:03.093493+0000","last_unstale":"2023-11-28T13:08:04.064305+0000","last_undegraded":"2023-11-28T13:08:04.064305+0000","last_fullsized":"2023-11-28T13:08:04.064305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:46.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372733+0000","last_change":"2023-11-28T13:08:02.580561+0000","last_active":"2023-11-28T13:08:04.372733+0000","last_peered":"2023-11-28T13:08:04.372733+0000","last_clean":"2023-11-28T13:08:04.372733+0000","last_became_active":"2023-11-28T13:08:02.580273+0000","last_became_peered":"2023-11-28T13:08:02.580273+0000","last_unstale":"2023-11-28T13:08:04.372733+0000","last_undegraded":"2023-11-28T13:08:04.372733+0000","last_fullsized":"2023-11-28T13:08:04.372733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:37.654488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371182+0000","last_change":"2023-11-28T13:08:03.089735+0000","last_active":"2023-11-28T13:08:04.371182+0000","last_peered":"2023-11-28T13:08:04.371182+0000","last_clean":"2023-11-28T13:08:04.371182+0000","last_became_active":"2023-11-28T13:08:03.089492+0000","last_became_peered":"2023-11-28T13:08:03.089492+0000","last_unstale":"2023-11-28T13:08:04.371182+0000","last_undegraded":"2023-11-28T13:08:04.371182+0000","last_fullsized":"2023-11-28T13:08:04.371182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:41.196768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371414+0000","last_change":"2023-11-28T13:08:03.082551+0000","last_active":"2023-11-28T13:08:04.371414+0000","last_peered":"2023-11-28T13:08:04.371414+0000","last_clean":"2023-11-28T13:08:04.371414+0000","last_became_active":"2023-11-28T13:08:03.082220+0000","last_became_peered":"2023-11-28T13:08:03.082220+0000","last_unstale":"2023-11-28T13:08:04.371414+0000","last_undegraded":"2023-11-28T13:08:04.371414+0000","last_fullsized":"2023-11-28T13:08:04.371414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:57.538270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064240+0000","last_change":"2023-11-28T13:08:03.095511+0000","last_active":"2023-11-28T13:08:04.064240+0000","last_peered":"2023-11-28T13:08:04.064240+0000","last_clean":"2023-11-28T13:08:04.064240+0000","last_became_active":"2023-11-28T13:08:03.095214+0000","last_became_peered":"2023-11-28T13:08:03.095214+0000","last_unstale":"2023-11-28T13:08:04.064240+0000","last_undegraded":"2023-11-28T13:08:04.064240+0000","last_fullsized":"2023-11-28T13:08:04.064240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:13.351182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371631+0000","last_change":"2023-11-28T13:08:03.089514+0000","last_active":"2023-11-28T13:08:04.371631+0000","last_peered":"2023-11-28T13:08:04.371631+0000","last_clean":"2023-11-28T13:08:04.371631+0000","last_became_active":"2023-11-28T13:08:03.089266+0000","last_became_peered":"2023-11-28T13:08:03.089266+0000","last_unstale":"2023-11-28T13:08:04.371631+0000","last_undegraded":"2023-11-28T13:08:04.371631+0000","last_fullsized":"2023-11-28T13:08:04.371631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:58.181120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064673+0000","last_change":"2023-11-28T13:08:03.084369+0000","last_active":"2023-11-28T13:08:04.064673+0000","last_peered":"2023-11-28T13:08:04.064673+0000","last_clean":"2023-11-28T13:08:04.064673+0000","last_became_active":"2023-11-28T13:08:03.084081+0000","last_became_peered":"2023-11-28T13:08:03.084081+0000","last_unstale":"2023-11-28T13:08:04.064673+0000","last_undegraded":"2023-11-28T13:08:04.064673+0000","last_fullsized":"2023-11-28T13:08:04.064673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:13.625233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371927+0000","last_change":"2023-11-28T13:08:03.093572+0000","last_active":"2023-11-28T13:08:04.371927+0000","last_peered":"2023-11-28T13:08:04.371927+0000","last_clean":"2023-11-28T13:08:04.371927+0000","last_became_active":"2023-11-28T13:08:03.093306+0000","last_became_peered":"2023-11-28T13:08:03.093306+0000","last_unstale":"2023-11-28T13:08:04.371927+0000","last_undegraded":"2023-11-28T13:08:04.371927+0000","last_fullsized":"2023-11-28T13:08:04.371927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:36.892676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9084,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362756,"statfs":{"total":96636764160,"available":96627462144,"internally_reserved":0,"allocated":839680,"data_stored":671592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7952,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363888,"statfs":{"total":96636764160,"available":96628621312,"internally_reserved":0,"allocated":204800,"data_stored":72807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13152,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358688,"statfs":{"total":96636764160,"available":96623296512,"internally_reserved":0,"allocated":811008,"data_stored":669828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12124,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359716,"statfs":{"total":96636764160,"available":96624349184,"internally_reserved":0,"allocated":217088,"data_stored":79040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11996,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":11776,"kb_avail":94359844,"statfs":{"total":96636764160,"available":96624480256,"internally_reserved":0,"allocated":217088,"data_stored":79460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12058624},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12128,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359712,"statfs":{"total":96636764160,"available":96624345088,"internally_reserved":0,"allocated":221184,"data_stored":79550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8048,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363792,"statfs":{"total":96636764160,"available":96628523008,"internally_reserved":0,"allocated":237568,"data_stored":81064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7972,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363868,"statfs":{"total":96636764160,"available":96628600832,"internally_reserved":0,"allocated":225280,"data_stored":80096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:08:07.906 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T13:08:07.910 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-28T13:08:07.910 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-28T13:08:07.910 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-28T13:08:07.910 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-28T13:08:08.037 INFO:tasks.ceph.mds.d.smithi186.stderr:2023-11-28T13:08:08.036+0000 7f5f6a5f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:08:08.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.139+0000 7fbbb79ff700 1 Processor -- start 2023-11-28T13:08:08.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.139+0000 7fbbb79ff700 1 -- start start 2023-11-28T13:08:08.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.140+0000 7fbbb79ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbb015c9b0 0x7fbbb00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:08.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.140+0000 7fbbb79ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbbb00cd7c0 0x7fbbb00cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:08.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.140+0000 7fbbb79ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb00ce160 con 0x7fbbb0161340 2023-11-28T13:08:08.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.140+0000 7fbbb79ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb00b5800 con 0x7fbbb00cd7c0 2023-11-28T13:08:08.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.140+0000 7fbbb79ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00d1950 con 0x7fbbb015c9b0 2023-11-28T13:08:08.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.141+0000 7fbbb579b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbb015c9b0 0x7fbbb00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:08.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.141+0000 7fbbb579b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbb015c9b0 0x7fbbb00b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36758/0 (socket says 172.21.15.156:36758) 2023-11-28T13:08:08.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.141+0000 7fbbb579b700 1 -- 172.21.15.156:0/723624925 learned_addr learned my addr 172.21.15.156:0/723624925 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:08.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.141+0000 7fbbb4f9a700 1 --1- 172.21.15.156:0/723624925 >> v1:172.21.15.186:6789/0 conn(0x7fbbb0161340 0x7fbbb00d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:38470/0 (socket says 172.21.15.156:38470) 2023-11-28T13:08:08.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.141+0000 7fbbb5f9c700 1 --2- 172.21.15.156:0/723624925 >> v2:172.21.15.186:3300/0 conn(0x7fbbb00cd7c0 0x7fbbb00cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:08.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.141+0000 7fbbb579b700 1 -- 172.21.15.156:0/723624925 >> v2:172.21.15.186:3300/0 conn(0x7fbbb00cd7c0 msgr2=0x7fbbb00cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:08.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.141+0000 7fbbb579b700 1 --2- 172.21.15.156:0/723624925 >> v2:172.21.15.186:3300/0 conn(0x7fbbb00cd7c0 0x7fbbb00cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.141+0000 7fbbb579b700 1 -- 172.21.15.156:0/723624925 >> v1:172.21.15.186:6789/0 conn(0x7fbbb0161340 legacy=0x7fbbb00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:08.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.141+0000 7fbbb579b700 1 -- 172.21.15.156:0/723624925 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb00d24f0 con 0x7fbbb015c9b0 2023-11-28T13:08:08.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.141+0000 7fbba7fff700 1 -- 172.21.15.156:0/723624925 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245518423 0 0) 0x7fbbb00ce160 con 0x7fbbb0161340 2023-11-28T13:08:08.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.141+0000 7fbbb579b700 1 --2- 172.21.15.156:0/723624925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbb015c9b0 0x7fbbb00b4bf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0098f0 tx=0x7fbbac009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18fbe0769cac22ae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:08.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.142+0000 7fbba7fff700 1 -- 172.21.15.156:0/723624925 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac011830 con 0x7fbbb015c9b0 2023-11-28T13:08:08.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.142+0000 7fbba7fff700 1 -- 172.21.15.156:0/723624925 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac011990 con 0x7fbbb015c9b0 2023-11-28T13:08:08.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.142+0000 7fbba7fff700 1 -- 172.21.15.156:0/723624925 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac011d20 con 0x7fbbb015c9b0 2023-11-28T13:08:08.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.142+0000 7fbbb79ff700 1 -- 172.21.15.156:0/723624925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbb015c9b0 msgr2=0x7fbbb00b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:08.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.142+0000 7fbbb79ff700 1 --2- 172.21.15.156:0/723624925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbb015c9b0 0x7fbbb00b4bf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0098f0 tx=0x7fbbac009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:08.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.142+0000 7fbbb79ff700 1 -- 172.21.15.156:0/723624925 shutdown_connections 2023-11-28T13:08:08.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.142+0000 7fbbb79ff700 1 --2- 172.21.15.156:0/723624925 >> v2:172.21.15.186:3300/0 conn(0x7fbbb00cd7c0 0x7fbbb00cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.142+0000 7fbbb79ff700 1 --2- 172.21.15.156:0/723624925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbb015c9b0 0x7fbbb00b4bf0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.142+0000 7fbbb79ff700 1 -- 172.21.15.156:0/723624925 >> 172.21.15.156:0/723624925 conn(0x7fbbb0072600 msgr2=0x7fbbb00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:08.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.142+0000 7fbbb79ff700 1 -- 172.21.15.156:0/723624925 shutdown_connections 2023-11-28T13:08:08.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.142+0000 7fbbb79ff700 1 -- 172.21.15.156:0/723624925 wait complete. 2023-11-28T13:08:08.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.144+0000 7fbbb79ff700 1 Processor -- start 2023-11-28T13:08:08.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.144+0000 7fbbb79ff700 1 -- start start 2023-11-28T13:08:08.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.145+0000 7fbbb79ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbb00cd7c0 0x7fbbb01e8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:08.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.145+0000 7fbbb79ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbb015c9b0 0x7fbbb01e6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:08.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.145+0000 7fbbb79ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbb0161340 0x7fbbb0159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:08.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.145+0000 7fbbb79ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00c9770 con 0x7fbbb015c9b0 2023-11-28T13:08:08.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.145+0000 7fbbb79ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01e6b40 con 0x7fbbb0161340 2023-11-28T13:08:08.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.145+0000 7fbbb79ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01e6e20 con 0x7fbbb00cd7c0 2023-11-28T13:08:08.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.145+0000 7fbbb4f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbb015c9b0 0x7fbbb01e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:08.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.145+0000 7fbbb579b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbb00cd7c0 0x7fbbb01e8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:08.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.145+0000 7fbbb4f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbb015c9b0 0x7fbbb01e6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58722/0 (socket says 172.21.15.156:58722) 2023-11-28T13:08:08.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.145+0000 7fbbb4f9a700 1 -- 172.21.15.156:0/1687432695 learned_addr learned my addr 172.21.15.156:0/1687432695 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:08.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.145+0000 7fbbb579b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbb00cd7c0 0x7fbbb01e8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36764/0 (socket says 172.21.15.156:36764) 2023-11-28T13:08:08.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.145+0000 7fbbb5f9c700 1 --2- 172.21.15.156:0/1687432695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbb0161340 0x7fbbb0159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:08.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.146+0000 7fbbb579b700 1 -- 172.21.15.156:0/1687432695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbb0161340 msgr2=0x7fbbb0159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:08.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.146+0000 7fbbb579b700 1 --2- 172.21.15.156:0/1687432695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbb0161340 0x7fbbb0159800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.146+0000 7fbbb579b700 1 -- 172.21.15.156:0/1687432695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbb015c9b0 msgr2=0x7fbbb01e6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:08.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.146+0000 7fbbb579b700 1 --2- 172.21.15.156:0/1687432695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbb015c9b0 0x7fbbb01e6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.146+0000 7fbbb579b700 1 -- 172.21.15.156:0/1687432695 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac009550 con 0x7fbbb00cd7c0 2023-11-28T13:08:08.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.146+0000 7fbbb4f9a700 1 --2- 172.21.15.156:0/1687432695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbb015c9b0 0x7fbbb01e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:08.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.146+0000 7fbbb579b700 1 --2- 172.21.15.156:0/1687432695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbb00cd7c0 0x7fbbb01e8c30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbbac016000 tx=0x7fbbac004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:08.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.146+0000 7fbba5ffb700 1 -- 172.21.15.156:0/1687432695 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac004510 con 0x7fbbb00cd7c0 2023-11-28T13:08:08.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.146+0000 7fbba5ffb700 1 -- 172.21.15.156:0/1687432695 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac01e070 con 0x7fbbb00cd7c0 2023-11-28T13:08:08.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.146+0000 7fbbb79ff700 1 -- 172.21.15.156:0/1687432695 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb00d1300 con 0x7fbbb00cd7c0 2023-11-28T13:08:08.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.147+0000 7fbba5ffb700 1 -- 172.21.15.156:0/1687432695 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0038e0 con 0x7fbbb00cd7c0 2023-11-28T13:08:08.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.147+0000 7fbbb79ff700 1 -- 172.21.15.156:0/1687432695 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb00b5520 con 0x7fbbb00cd7c0 2023-11-28T13:08:08.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.147+0000 7fbbb79ff700 1 -- 172.21.15.156:0/1687432695 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb00d15e0 con 0x7fbbb00cd7c0 2023-11-28T13:08:08.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.149+0000 7fbba5ffb700 1 -- 172.21.15.156:0/1687432695 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbbac00f050 con 0x7fbbb00cd7c0 2023-11-28T13:08:08.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.150+0000 7fbba5ffb700 1 --2- 172.21.15.156:0/1687432695 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbb940682f0 0x7fbb9406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:08.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.150+0000 7fbba5ffb700 1 -- 172.21.15.156:0/1687432695 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbbac0874b0 con 0x7fbbb00cd7c0 2023-11-28T13:08:08.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.150+0000 7fbbb4f9a700 1 --2- 172.21.15.156:0/1687432695 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbb940682f0 0x7fbb9406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:08.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.151+0000 7fbbb4f9a700 1 --2- 172.21.15.156:0/1687432695 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbb940682f0 0x7fbb9406a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbbb015b3b0 tx=0x7fbba0007f30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:08.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.154+0000 7fbba5ffb700 1 -- 172.21.15.156:0/1687432695 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbbac053c20 con 0x7fbbb00cd7c0 2023-11-28T13:08:08.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.396+0000 7fbbb79ff700 1 -- 172.21.15.156:0/1687432695 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fbbb00745c0 con 0x7fbbb00cd7c0 2023-11-28T13:08:08.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.409+0000 7fbba5ffb700 1 -- 172.21.15.156:0/1687432695 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+166386 (secure 0 0 0) 0x7fbbac057240 con 0x7fbbb00cd7c0 2023-11-28T13:08:08.412 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:08.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.412+0000 7fbbb79ff700 1 -- 172.21.15.156:0/1687432695 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbb940682f0 msgr2=0x7fbb9406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:08.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.412+0000 7fbbb79ff700 1 --2- 172.21.15.156:0/1687432695 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbb940682f0 0x7fbb9406a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbbb015b3b0 tx=0x7fbba0007f30 comp rx=0 tx=0).stop 2023-11-28T13:08:08.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.412+0000 7fbbb79ff700 1 -- 172.21.15.156:0/1687432695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbb00cd7c0 msgr2=0x7fbbb01e8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:08.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.412+0000 7fbbb79ff700 1 --2- 172.21.15.156:0/1687432695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbb00cd7c0 0x7fbbb01e8c30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbbac016000 tx=0x7fbbac004750 comp rx=0 tx=0).stop 2023-11-28T13:08:08.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.413+0000 7fbbb79ff700 1 -- 172.21.15.156:0/1687432695 shutdown_connections 2023-11-28T13:08:08.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.413+0000 7fbbb79ff700 1 --2- 172.21.15.156:0/1687432695 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbb940682f0 0x7fbb9406a7a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.413+0000 7fbbb79ff700 1 --2- 172.21.15.156:0/1687432695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbb0161340 0x7fbbb0159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.413+0000 7fbbb79ff700 1 --2- 172.21.15.156:0/1687432695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbb00cd7c0 0x7fbbb01e8c30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.413+0000 7fbbb79ff700 1 --2- 172.21.15.156:0/1687432695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbb015c9b0 0x7fbbb01e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.413+0000 7fbbb79ff700 1 -- 172.21.15.156:0/1687432695 >> 172.21.15.156:0/1687432695 conn(0x7fbbb0072600 msgr2=0x7fbbb0160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:08.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.413+0000 7fbbb79ff700 1 -- 172.21.15.156:0/1687432695 shutdown_connections 2023-11-28T13:08:08.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.413+0000 7fbbb79ff700 1 -- 172.21.15.156:0/1687432695 wait complete. 2023-11-28T13:08:08.432 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":5,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":130711},{"type":"v1","addr":"172.21.15.186:6833","nonce":130711}]},"active_addr":"172.21.15.186:6833/130711","active_change":"2023-11-28T13:07:49.996436+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.186:0","nonce":1421370351}]},{"addrvec":[{"type":"v2","addr":"172.21.15.186:0","nonce":3329864026}]},{"addrvec":[{"type":"v2","addr":"172.21.15.186:0","nonce":3764493247}]},{"addrvec":[{"type":"v2","addr":"172.21.15.186:0","nonce":3796743670}]}]} 2023-11-28T13:08:08.435 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-28T13:08:08.435 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-28T13:08:08.435 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:08:08.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.634+0000 7f034adff700 1 Processor -- start 2023-11-28T13:08:08.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.634+0000 7f034adff700 1 -- start start 2023-11-28T13:08:08.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.636+0000 7f034adff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03440d1330 0x7f03440b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:08.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.636+0000 7f034adff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f03440b20b0 0x7f03440b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:08.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.636+0000 7f034adff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03440b0230 con 0x7f03440d01b0 2023-11-28T13:08:08.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.636+0000 7f034adff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034415cec0 con 0x7f03440b20b0 2023-11-28T13:08:08.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.636+0000 7f034adff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03440b4570 con 0x7f03440d1330 2023-11-28T13:08:08.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.636+0000 7f034939c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f03440b20b0 0x7f03440b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:08.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.636+0000 7f034939c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f03440b20b0 0x7f03440b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58726/0 (socket says 172.21.15.156:58726) 2023-11-28T13:08:08.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.636+0000 7f034939c700 1 -- 172.21.15.156:0/76888479 learned_addr learned my addr 172.21.15.156:0/76888479 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:08.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.636+0000 7f0343fff700 1 --2- 172.21.15.156:0/76888479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03440d1330 0x7f03440b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:08.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.636+0000 7f034939c700 1 -- 172.21.15.156:0/76888479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03440d1330 msgr2=0x7f03440b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:08.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.636+0000 7f034939c700 1 --2- 172.21.15.156:0/76888479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03440d1330 0x7f03440b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.636+0000 7f034939c700 1 -- 172.21.15.156:0/76888479 >> v1:172.21.15.186:6789/0 conn(0x7f03440d01b0 legacy=0x7f03440d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:08.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.636+0000 7f034939c700 1 -- 172.21.15.156:0/76888479 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03440b4850 con 0x7f03440b20b0 2023-11-28T13:08:08.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.636+0000 7f0343fff700 1 --2- 172.21.15.156:0/76888479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03440d1330 0x7f03440b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:08.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.637+0000 7f034939c700 1 --2- 172.21.15.156:0/76888479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03440b20b0 0x7f03440b0c00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f033c009e50 tx=0x7f033c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b341e02e874f97 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:08.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.637+0000 7f03437fe700 1 -- 172.21.15.156:0/76888479 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c00e070 con 0x7f03440b20b0 2023-11-28T13:08:08.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.637+0000 7f03437fe700 1 -- 172.21.15.156:0/76888479 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c00b9c0 con 0x7f03440b20b0 2023-11-28T13:08:08.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.637+0000 7f03437fe700 1 -- 172.21.15.156:0/76888479 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c00bd70 con 0x7f03440b20b0 2023-11-28T13:08:08.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.637+0000 7f034adff700 1 -- 172.21.15.156:0/76888479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03440b20b0 msgr2=0x7f03440b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:08.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.637+0000 7f034adff700 1 --2- 172.21.15.156:0/76888479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03440b20b0 0x7f03440b0c00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f033c009e50 tx=0x7f033c00b330 comp rx=0 tx=0).stop 2023-11-28T13:08:08.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.637+0000 7f034adff700 1 -- 172.21.15.156:0/76888479 shutdown_connections 2023-11-28T13:08:08.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.637+0000 7f034adff700 1 --2- 172.21.15.156:0/76888479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03440b20b0 0x7f03440b0c00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.637+0000 7f034adff700 1 --2- 172.21.15.156:0/76888479 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03440d1330 0x7f03440b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.637+0000 7f034adff700 1 -- 172.21.15.156:0/76888479 >> 172.21.15.156:0/76888479 conn(0x7f034409c3f0 msgr2=0x7f03440d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:08.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.638+0000 7f034adff700 1 -- 172.21.15.156:0/76888479 shutdown_connections 2023-11-28T13:08:08.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.638+0000 7f034adff700 1 -- 172.21.15.156:0/76888479 wait complete. 2023-11-28T13:08:08.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.639+0000 7f034adff700 1 Processor -- start 2023-11-28T13:08:08.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.639+0000 7f034adff700 1 -- start start 2023-11-28T13:08:08.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.640+0000 7f034adff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03440b20b0 0x7f03441f5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:08.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.640+0000 7f034adff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03440d01b0 0x7f034415f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:08.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.640+0000 7f034adff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03440d1330 0x7f034415f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:08.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.640+0000 7f034adff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0344148020 con 0x7f03440b20b0 2023-11-28T13:08:08.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.640+0000 7f034adff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03441e5fa0 con 0x7f03440d01b0 2023-11-28T13:08:08.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.640+0000 7f034adff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03441482c0 con 0x7f03440d1330 2023-11-28T13:08:08.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.640+0000 7f0348b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03440b20b0 0x7f03441f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:08.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.641+0000 7f0348b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03440b20b0 0x7f03441f5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58734/0 (socket says 172.21.15.156:58734) 2023-11-28T13:08:08.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.641+0000 7f0348b9b700 1 -- 172.21.15.156:0/268460925 learned_addr learned my addr 172.21.15.156:0/268460925 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:08.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.641+0000 7f0343fff700 1 --2- 172.21.15.156:0/268460925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03440d01b0 0x7f034415f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:08.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.641+0000 7f034939c700 1 --2- 172.21.15.156:0/268460925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03440d1330 0x7f034415f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:08.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.641+0000 7f0348b9b700 1 -- 172.21.15.156:0/268460925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03440d1330 msgr2=0x7f034415f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:08.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.641+0000 7f0348b9b700 1 --2- 172.21.15.156:0/268460925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03440d1330 0x7f034415f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.641+0000 7f0348b9b700 1 -- 172.21.15.156:0/268460925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03440d01b0 msgr2=0x7f034415f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:08.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.641+0000 7f0348b9b700 1 --2- 172.21.15.156:0/268460925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03440d01b0 0x7f034415f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.641+0000 7f0348b9b700 1 -- 172.21.15.156:0/268460925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c009b30 con 0x7f03440b20b0 2023-11-28T13:08:08.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.641+0000 7f0348b9b700 1 --2- 172.21.15.156:0/268460925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03440b20b0 0x7f03441f5450 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f033800c4e0 tx=0x7f033800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:08.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.641+0000 7f03417fa700 1 -- 172.21.15.156:0/268460925 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03380052d0 con 0x7f03440b20b0 2023-11-28T13:08:08.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.641+0000 7f03417fa700 1 -- 172.21.15.156:0/268460925 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0338005430 con 0x7f03440b20b0 2023-11-28T13:08:08.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.641+0000 7f034adff700 1 -- 172.21.15.156:0/268460925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03440afec0 con 0x7f03440b20b0 2023-11-28T13:08:08.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.641+0000 7f03417fa700 1 -- 172.21.15.156:0/268460925 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03380038c0 con 0x7f03440b20b0 2023-11-28T13:08:08.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.642+0000 7f034adff700 1 -- 172.21.15.156:0/268460925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03440afbe0 con 0x7f03440b20b0 2023-11-28T13:08:08.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.642+0000 7f034adff700 1 -- 172.21.15.156:0/268460925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03440b2600 con 0x7f03440b20b0 2023-11-28T13:08:08.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.643+0000 7f03417fa700 1 -- 172.21.15.156:0/268460925 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0338019030 con 0x7f03440b20b0 2023-11-28T13:08:08.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.644+0000 7f03417fa700 1 --2- 172.21.15.156:0/268460925 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f03280682f0 0x7f032806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:08.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.645+0000 7f03417fa700 1 -- 172.21.15.156:0/268460925 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0338085040 con 0x7f03440b20b0 2023-11-28T13:08:08.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.645+0000 7f0343fff700 1 --2- 172.21.15.156:0/268460925 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f03280682f0 0x7f032806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:08.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.646+0000 7f0343fff700 1 --2- 172.21.15.156:0/268460925 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f03280682f0 0x7f032806a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f034414af10 tx=0x7f0330008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:08.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.648+0000 7f03417fa700 1 -- 172.21.15.156:0/268460925 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f03380518a0 con 0x7f03440b20b0 2023-11-28T13:08:08.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.842+0000 7f034adff700 1 -- 172.21.15.156:0/268460925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03440d3e70 con 0x7f03440b20b0 2023-11-28T13:08:08.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.843+0000 7f03417fa700 1 -- 172.21.15.156:0/268460925 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17897 (secure 0 0 0) 0x7f0338054ec0 con 0x7f03440b20b0 2023-11-28T13:08:08.844 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:08.844 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":21,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:08:04.060053+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:08:00.903485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:08:02.037854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:08:08.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.846+0000 7f034adff700 1 -- 172.21.15.156:0/268460925 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f03280682f0 msgr2=0x7f032806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.846+0000 7f034adff700 1 --2- 172.21.15.156:0/268460925 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f03280682f0 0x7f032806a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f034414af10 tx=0x7f0330008040 comp rx=0 tx=0).stop 2023-11-28T13:08:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.846+0000 7f034adff700 1 -- 172.21.15.156:0/268460925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03440b20b0 msgr2=0x7f03441f5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.846+0000 7f034adff700 1 --2- 172.21.15.156:0/268460925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03440b20b0 0x7f03441f5450 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f033800c4e0 tx=0x7f033800c8a0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.846+0000 7f034adff700 1 -- 172.21.15.156:0/268460925 shutdown_connections 2023-11-28T13:08:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.846+0000 7f034adff700 1 --2- 172.21.15.156:0/268460925 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f03280682f0 0x7f032806a7a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.846+0000 7f034adff700 1 --2- 172.21.15.156:0/268460925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03440d1330 0x7f034415f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.846+0000 7f034adff700 1 --2- 172.21.15.156:0/268460925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03440b20b0 0x7f03441f5450 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.846+0000 7f034adff700 1 --2- 172.21.15.156:0/268460925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03440d01b0 0x7f034415f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.846+0000 7f034adff700 1 -- 172.21.15.156:0/268460925 >> 172.21.15.156:0/268460925 conn(0x7f034409c3f0 msgr2=0x7f034415c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.846+0000 7f034adff700 1 -- 172.21.15.156:0/268460925 shutdown_connections 2023-11-28T13:08:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:08.846+0000 7f034adff700 1 -- 172.21.15.156:0/268460925 wait complete. 2023-11-28T13:08:08.865 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-28T13:08:08.865 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:08:09.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.060+0000 7f61bc5ff700 1 Processor -- start 2023-11-28T13:08:09.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.060+0000 7f61bc5ff700 1 -- start start 2023-11-28T13:08:09.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.061+0000 7f61bc5ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f61b4161340 0x7f61b40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.061+0000 7f61bc5ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f61b40d2320 0x7f61b40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.061+0000 7f61bc5ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b40d1950 con 0x7f61b415c9b0 2023-11-28T13:08:09.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.061+0000 7f61bc5ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b40b5800 con 0x7f61b40d2320 2023-11-28T13:08:09.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.061+0000 7f61bc5ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b40b5300 con 0x7f61b4161340 2023-11-28T13:08:09.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.061+0000 7f61bab9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f61b40d2320 0x7f61b40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.061+0000 7f61ba39b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f61b415c9b0 0x7f61b40b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:34608/0 (socket says 172.21.15.156:34608) 2023-11-28T13:08:09.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.061+0000 7f61ba39b700 1 -- 172.21.15.156:0/2812316011 learned_addr learned my addr 172.21.15.156:0/2812316011 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:09.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.061+0000 7f61b9b9a700 1 --2- 172.21.15.156:0/2812316011 >> v2:172.21.15.186:3300/0 conn(0x7f61b4161340 0x7f61b40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.062+0000 7f61b9399700 1 -- 172.21.15.156:0/2812316011 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1873817195 0 0) 0x7f61b40d1950 con 0x7f61b415c9b0 2023-11-28T13:08:09.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.062+0000 7f61b9399700 1 -- 172.21.15.156:0/2812316011 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61a00035b0 con 0x7f61b415c9b0 2023-11-28T13:08:09.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.062+0000 7f61bab9c700 1 -- 172.21.15.156:0/2812316011 >> v2:172.21.15.186:3300/0 conn(0x7f61b4161340 msgr2=0x7f61b40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:09.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.062+0000 7f61bab9c700 1 --2- 172.21.15.156:0/2812316011 >> v2:172.21.15.186:3300/0 conn(0x7f61b4161340 0x7f61b40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.062+0000 7f61bab9c700 1 -- 172.21.15.156:0/2812316011 >> v1:172.21.15.156:6789/0 conn(0x7f61b415c9b0 legacy=0x7f61b40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.062+0000 7f61bab9c700 1 -- 172.21.15.156:0/2812316011 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b40b5520 con 0x7f61b40d2320 2023-11-28T13:08:09.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.062+0000 7f61bab9c700 1 --2- 172.21.15.156:0/2812316011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61b40d2320 0x7f61b40cd7c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f61b40685c0 tx=0x7f61ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05059a9551f0daa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.062+0000 7f61b9399700 1 -- 172.21.15.156:0/2812316011 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac00e070 con 0x7f61b40d2320 2023-11-28T13:08:09.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.062+0000 7f61b9399700 1 -- 172.21.15.156:0/2812316011 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac00bae0 con 0x7f61b40d2320 2023-11-28T13:08:09.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.062+0000 7f61b9399700 1 -- 172.21.15.156:0/2812316011 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac00be90 con 0x7f61b40d2320 2023-11-28T13:08:09.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.063+0000 7f61bc5ff700 1 -- 172.21.15.156:0/2812316011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61b40d2320 msgr2=0x7f61b40cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.063+0000 7f61bc5ff700 1 --2- 172.21.15.156:0/2812316011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61b40d2320 0x7f61b40cd7c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f61b40685c0 tx=0x7f61ac00b330 comp rx=0 tx=0).stop 2023-11-28T13:08:09.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.063+0000 7f61bc5ff700 1 -- 172.21.15.156:0/2812316011 shutdown_connections 2023-11-28T13:08:09.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.063+0000 7f61bc5ff700 1 --2- 172.21.15.156:0/2812316011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61b40d2320 0x7f61b40cd7c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.063+0000 7f61bc5ff700 1 --2- 172.21.15.156:0/2812316011 >> v2:172.21.15.186:3300/0 conn(0x7f61b4161340 0x7f61b40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.063+0000 7f61bc5ff700 1 -- 172.21.15.156:0/2812316011 >> 172.21.15.156:0/2812316011 conn(0x7f61b4072600 msgr2=0x7f61b40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.063+0000 7f61bc5ff700 1 -- 172.21.15.156:0/2812316011 shutdown_connections 2023-11-28T13:08:09.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.063+0000 7f61bc5ff700 1 -- 172.21.15.156:0/2812316011 wait complete. 2023-11-28T13:08:09.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.065+0000 7f61bc5ff700 1 Processor -- start 2023-11-28T13:08:09.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.065+0000 7f61bc5ff700 1 -- start start 2023-11-28T13:08:09.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.066+0000 7f61bc5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61b40d2320 0x7f61b41577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.066+0000 7f61bc5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61b415c9b0 0x7f61b4152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.066+0000 7f61bc5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61b4161340 0x7f61b41f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.066+0000 7f61bc5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b41f2650 con 0x7f61b40d2320 2023-11-28T13:08:09.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.066+0000 7f61bc5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b4156010 con 0x7f61b4161340 2023-11-28T13:08:09.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.066+0000 7f61bc5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b4154e20 con 0x7f61b415c9b0 2023-11-28T13:08:09.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.066+0000 7f61ba39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61b40d2320 0x7f61b41577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.066+0000 7f61ba39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61b40d2320 0x7f61b41577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58756/0 (socket says 172.21.15.156:58756) 2023-11-28T13:08:09.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.066+0000 7f61ba39b700 1 -- 172.21.15.156:0/3352357349 learned_addr learned my addr 172.21.15.156:0/3352357349 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:09.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.066+0000 7f61b9b9a700 1 --2- 172.21.15.156:0/3352357349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61b415c9b0 0x7f61b4152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.066+0000 7f61bab9c700 1 --2- 172.21.15.156:0/3352357349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61b4161340 0x7f61b41f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.066+0000 7f61ba39b700 1 -- 172.21.15.156:0/3352357349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61b415c9b0 msgr2=0x7f61b4152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.066+0000 7f61ba39b700 1 --2- 172.21.15.156:0/3352357349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61b415c9b0 0x7f61b4152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.066+0000 7f61ba39b700 1 -- 172.21.15.156:0/3352357349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61b4161340 msgr2=0x7f61b41f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.066+0000 7f61ba39b700 1 --2- 172.21.15.156:0/3352357349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61b4161340 0x7f61b41f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.066+0000 7f61ba39b700 1 -- 172.21.15.156:0/3352357349 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac009a70 con 0x7f61b40d2320 2023-11-28T13:08:09.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.066+0000 7f61bab9c700 1 --2- 172.21.15.156:0/3352357349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61b4161340 0x7f61b41f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:09.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.067+0000 7f61ba39b700 1 --2- 172.21.15.156:0/3352357349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61b40d2320 0x7f61b41577c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f61b000c4d0 tx=0x7f61b000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.067+0000 7f61aaffd700 1 -- 172.21.15.156:0/3352357349 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0005d70 con 0x7f61b40d2320 2023-11-28T13:08:09.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.067+0000 7f61aaffd700 1 -- 172.21.15.156:0/3352357349 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b00070d0 con 0x7f61b40d2320 2023-11-28T13:08:09.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.067+0000 7f61bc5ff700 1 -- 172.21.15.156:0/3352357349 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b40d15e0 con 0x7f61b40d2320 2023-11-28T13:08:09.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.067+0000 7f61aaffd700 1 -- 172.21.15.156:0/3352357349 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0007460 con 0x7f61b40d2320 2023-11-28T13:08:09.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.067+0000 7f61bc5ff700 1 -- 172.21.15.156:0/3352357349 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b40d1300 con 0x7f61b40d2320 2023-11-28T13:08:09.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.068+0000 7f61bc5ff700 1 -- 172.21.15.156:0/3352357349 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b415cd80 con 0x7f61b40d2320 2023-11-28T13:08:09.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.069+0000 7f61aaffd700 1 -- 172.21.15.156:0/3352357349 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f61b0019030 con 0x7f61b40d2320 2023-11-28T13:08:09.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.070+0000 7f61aaffd700 1 --2- 172.21.15.156:0/3352357349 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f61980682f0 0x7f619806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.070+0000 7f61aaffd700 1 -- 172.21.15.156:0/3352357349 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61b0085080 con 0x7f61b40d2320 2023-11-28T13:08:09.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.070+0000 7f61b9b9a700 1 --2- 172.21.15.156:0/3352357349 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f61980682f0 0x7f619806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.071+0000 7f61b9b9a700 1 --2- 172.21.15.156:0/3352357349 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f61980682f0 0x7f619806a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f61b40b5230 tx=0x7f61a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.074+0000 7f61aaffd700 1 -- 172.21.15.156:0/3352357349 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f61b00518e0 con 0x7f61b40d2320 2023-11-28T13:08:09.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.263+0000 7f61bc5ff700 1 -- 172.21.15.156:0/3352357349 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61b40745c0 con 0x7f61b40d2320 2023-11-28T13:08:09.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.265+0000 7f61aaffd700 1 -- 172.21.15.156:0/3352357349 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17897 (secure 0 0 0) 0x7f61b0054f00 con 0x7f61b40d2320 2023-11-28T13:08:09.266 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:09.266 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":21,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:08:04.060053+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:08:00.903485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:08:02.037854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:08:09.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.267+0000 7f61bc5ff700 1 -- 172.21.15.156:0/3352357349 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f61980682f0 msgr2=0x7f619806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.267+0000 7f61bc5ff700 1 --2- 172.21.15.156:0/3352357349 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f61980682f0 0x7f619806a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f61b40b5230 tx=0x7f61a4008040 comp rx=0 tx=0).stop 2023-11-28T13:08:09.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.267+0000 7f61bc5ff700 1 -- 172.21.15.156:0/3352357349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61b40d2320 msgr2=0x7f61b41577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.267+0000 7f61bc5ff700 1 --2- 172.21.15.156:0/3352357349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61b40d2320 0x7f61b41577c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f61b000c4d0 tx=0x7f61b000c890 comp rx=0 tx=0).stop 2023-11-28T13:08:09.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.267+0000 7f61bc5ff700 1 -- 172.21.15.156:0/3352357349 shutdown_connections 2023-11-28T13:08:09.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.267+0000 7f61bc5ff700 1 --2- 172.21.15.156:0/3352357349 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f61980682f0 0x7f619806a7a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.267+0000 7f61bc5ff700 1 --2- 172.21.15.156:0/3352357349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61b4161340 0x7f61b41f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.267+0000 7f61bc5ff700 1 --2- 172.21.15.156:0/3352357349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61b40d2320 0x7f61b41577c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.267+0000 7f61bc5ff700 1 --2- 172.21.15.156:0/3352357349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61b415c9b0 0x7f61b4152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.267+0000 7f61bc5ff700 1 -- 172.21.15.156:0/3352357349 >> 172.21.15.156:0/3352357349 conn(0x7f61b4072600 msgr2=0x7f61b4160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.267+0000 7f61bc5ff700 1 -- 172.21.15.156:0/3352357349 shutdown_connections 2023-11-28T13:08:09.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.267+0000 7f61bc5ff700 1 -- 172.21.15.156:0/3352357349 wait complete. 2023-11-28T13:08:09.286 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-28T13:08:09.287 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-28T13:08:09.287 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-28T13:08:09.288 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-28T13:08:09.288 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-28T13:08:09.288 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-28T13:08:09.289 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-28T13:08:09.289 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-28T13:08:09.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.640+0000 7f5da43ff700 1 Processor -- start 2023-11-28T13:08:09.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.641+0000 7f5da43ff700 1 -- start start 2023-11-28T13:08:09.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.643+0000 7f5da43ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5d9c06d500 0x7f5d9c0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.643+0000 7f5da43ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c06c000 con 0x7f5d9c160b80 2023-11-28T13:08:09.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.643+0000 7f5da43ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c06c2e0 con 0x7f5d9c0ade50 2023-11-28T13:08:09.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.643+0000 7f5da43ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9c0ae350 con 0x7f5d9c06d500 2023-11-28T13:08:09.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.644+0000 7f5da299c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f5d9c160b80 0x7f5d9c15ca00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:34610/0 (socket says 172.21.15.156:34610) 2023-11-28T13:08:09.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.644+0000 7f5da299c700 1 -- 172.21.15.156:0/2660458605 learned_addr learned my addr 172.21.15.156:0/2660458605 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:09.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.645+0000 7f5da1199700 1 -- 172.21.15.156:0/2660458605 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658358380 0 0) 0x7f5d9c06c000 con 0x7f5d9c160b80 2023-11-28T13:08:09.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.645+0000 7f5da1199700 1 -- 172.21.15.156:0/2660458605 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d880035b0 con 0x7f5d9c160b80 2023-11-28T13:08:09.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.646+0000 7f5da1199700 1 -- 172.21.15.156:0/2660458605 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 268240533 0 0) 0x7f5d880035b0 con 0x7f5d9c160b80 2023-11-28T13:08:09.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.646+0000 7f5da1199700 1 -- 172.21.15.156:0/2660458605 >> v2:172.21.15.186:3300/0 conn(0x7f5d9c06d500 msgr2=0x7f5d9c0cc6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T13:08:09.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.646+0000 7f5da1199700 1 --2- 172.21.15.156:0/2660458605 >> v2:172.21.15.186:3300/0 conn(0x7f5d9c06d500 0x7f5d9c0cc6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.646+0000 7f5da1199700 1 -- 172.21.15.156:0/2660458605 >> v1:172.21.15.186:6789/0 conn(0x7f5d9c0ade50 legacy=0x7f5d9c06d8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T13:08:09.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.646+0000 7f5da1199700 1 -- 172.21.15.156:0/2660458605 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c160380 con 0x7f5d9c160b80 2023-11-28T13:08:09.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.646+0000 7f5da1199700 1 -- 172.21.15.156:0/2660458605 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f5d98001670 con 0x7f5d9c160b80 2023-11-28T13:08:09.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.646+0000 7f5da1199700 1 -- 172.21.15.156:0/2660458605 >> v1:172.21.15.156:6789/0 conn(0x7f5d9c160b80 legacy=0x7f5d9c15ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.646+0000 7f5da1199700 1 --2- 172.21.15.156:0/2660458605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d88003a00 0x7f5d88003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.646+0000 7f5da1199700 1 --2- 172.21.15.156:0/2660458605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d88004850 0x7f5d88004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.646+0000 7f5da1199700 1 --2- 172.21.15.156:0/2660458605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d88005320 0x7f5d88014220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.646+0000 7f5da1199700 1 -- 172.21.15.156:0/2660458605 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0ae350 con 0x7f5d88005320 2023-11-28T13:08:09.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.646+0000 7f5da1199700 1 -- 172.21.15.156:0/2660458605 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d880149f0 con 0x7f5d88003a00 2023-11-28T13:08:09.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.646+0000 7f5da1199700 1 -- 172.21.15.156:0/2660458605 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d88014bb0 con 0x7f5d88004850 2023-11-28T13:08:09.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.647+0000 7f5da299c700 1 --2- 172.21.15.156:0/2660458605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d88005320 0x7f5d88014220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.647+0000 7f5da199a700 1 --2- 172.21.15.156:0/2660458605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d88004850 0x7f5d88004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.647+0000 7f5da219b700 1 --2- 172.21.15.156:0/2660458605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d88003a00 0x7f5d88003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.647+0000 7f5da299c700 1 -- 172.21.15.156:0/2660458605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d88004850 msgr2=0x7f5d88004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:09.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.647+0000 7f5da299c700 1 --2- 172.21.15.156:0/2660458605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d88004850 0x7f5d88004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.647+0000 7f5da299c700 1 -- 172.21.15.156:0/2660458605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d88003a00 msgr2=0x7f5d88003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:09.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.647+0000 7f5da299c700 1 --2- 172.21.15.156:0/2660458605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d88003a00 0x7f5d88003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.647+0000 7f5da299c700 1 -- 172.21.15.156:0/2660458605 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c160380 con 0x7f5d88005320 2023-11-28T13:08:09.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.647+0000 7f5da199a700 1 --2- 172.21.15.156:0/2660458605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d88004850 0x7f5d88004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:09.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.648+0000 7f5da219b700 1 --2- 172.21.15.156:0/2660458605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d88003a00 0x7f5d88003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:09.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.648+0000 7f5da299c700 1 --2- 172.21.15.156:0/2660458605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d88005320 0x7f5d88014220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800c370 tx=0x7f5d88005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e119fe616020d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.648+0000 7f5da299c700 1 -- 172.21.15.156:0/2660458605 reap_dead start 2023-11-28T13:08:09.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.651+0000 7f5da1199700 1 -- 172.21.15.156:0/2660458605 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98005960 con 0x7f5d88005320 2023-11-28T13:08:09.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.651+0000 7f5da1199700 1 -- 172.21.15.156:0/2660458605 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d98005ac0 con 0x7f5d88005320 2023-11-28T13:08:09.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.651+0000 7f5da1199700 1 -- 172.21.15.156:0/2660458605 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98005e30 con 0x7f5d88005320 2023-11-28T13:08:09.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.651+0000 7fb1b0fff700 1 Processor -- start 2023-11-28T13:08:09.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.652+0000 7f5da43ff700 1 -- 172.21.15.156:0/2660458605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d88005320 msgr2=0x7f5d88014220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.652+0000 7f5da43ff700 1 --2- 172.21.15.156:0/2660458605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d88005320 0x7f5d88014220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800c370 tx=0x7f5d88005880 comp rx=0 tx=0).stop 2023-11-28T13:08:09.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.652+0000 7efe7cd9d700 1 Processor -- start 2023-11-28T13:08:09.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.652+0000 7f5da43ff700 1 -- 172.21.15.156:0/2660458605 shutdown_connections 2023-11-28T13:08:09.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.652+0000 7f5da43ff700 1 --2- 172.21.15.156:0/2660458605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d88005320 0x7f5d88014220 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.652+0000 7f5da43ff700 1 -- 172.21.15.156:0/2660458605 >> 172.21.15.156:0/2660458605 conn(0x7f5d9c0b06f0 msgr2=0x7f5d9c0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.652+0000 7fb1b0fff700 1 -- start start 2023-11-28T13:08:09.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.653+0000 7efe7cd9d700 1 -- start start 2023-11-28T13:08:09.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.653+0000 7f5da43ff700 1 -- 172.21.15.156:0/2660458605 shutdown_connections 2023-11-28T13:08:09.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.654+0000 7fb1b0fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb1ac0ce650 0x7fb1ac0cb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.655+0000 7fb1b0fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1ac15cf90 con 0x7fb1ac0712b0 2023-11-28T13:08:09.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.655+0000 7fb1b0fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1ac071680 con 0x7fb1ac0cd270 2023-11-28T13:08:09.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.655+0000 7fb1b0fff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ac15c380 con 0x7fb1ac0ce650 2023-11-28T13:08:09.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.655+0000 7efe7cd9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efe78158950 0x7efe7806ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.653+0000 7f5da43ff700 1 -- 172.21.15.156:0/2660458605 wait complete. 2023-11-28T13:08:09.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.656+0000 7f5da43ff700 1 Processor -- start 2023-11-28T13:08:09.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.657+0000 7f5da43ff700 1 -- start start 2023-11-28T13:08:09.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.657+0000 7efe777fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efe78158950 0x7efe7806ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.657+0000 7efe777fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efe78158950 0x7efe7806ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58788/0 (socket says 172.21.15.156:58788) 2023-11-28T13:08:09.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.656+0000 7f79e75ff700 1 Processor -- start 2023-11-28T13:08:09.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.657+0000 7f79e75ff700 1 -- start start 2023-11-28T13:08:09.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.658+0000 7efe7cd9d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7efe7806c810 0x7efe7815c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.658+0000 7efe7cd9d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe7815c8e0 0x7efe780ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.658+0000 7efe7cd9d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe780cc790 con 0x7efe78158950 2023-11-28T13:08:09.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.658+0000 7efe7cd9d700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe78158d20 con 0x7efe7806c810 2023-11-28T13:08:09.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.658+0000 7efe7cd9d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe780caa10 con 0x7efe7815c8e0 2023-11-28T13:08:09.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.655+0000 7fb1a9d9b700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fb1ac0cd270 0x7fb1ac0cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:38502/0 (socket says 172.21.15.156:38502) 2023-11-28T13:08:09.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.655+0000 7fb1a9d9b700 1 -- 172.21.15.156:0/2234900083 learned_addr learned my addr 172.21.15.156:0/2234900083 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:09.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.656+0000 7fb1a959a700 1 -- 172.21.15.156:0/2234900083 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2705333129 0 0) 0x7fb1ac071680 con 0x7fb1ac0cd270 2023-11-28T13:08:09.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.656+0000 7fb1a959a700 1 -- 172.21.15.156:0/2234900083 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1940057c0 con 0x7fb1ac0cd270 2023-11-28T13:08:09.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.656+0000 7fb1aad9d700 1 --2- 172.21.15.156:0/2234900083 >> v2:172.21.15.186:3300/0 conn(0x7fb1ac0ce650 0x7fb1ac0cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.656+0000 7fb1a959a700 1 -- 172.21.15.156:0/2234900083 <== mon.1 v1:172.21.15.186:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3674804675 0 0) 0x7fb1940057c0 con 0x7fb1ac0cd270 2023-11-28T13:08:09.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.656+0000 7fb1a959a700 1 -- 172.21.15.156:0/2234900083 >> v2:172.21.15.186:3300/0 conn(0x7fb1ac0ce650 msgr2=0x7fb1ac0cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:09.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.656+0000 7fb1a959a700 1 --2- 172.21.15.156:0/2234900083 >> v2:172.21.15.186:3300/0 conn(0x7fb1ac0ce650 0x7fb1ac0cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.656+0000 7fb1a959a700 1 -- 172.21.15.156:0/2234900083 >> v1:172.21.15.156:6789/0 conn(0x7fb1ac0712b0 legacy=0x7fb1ac06c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.656+0000 7fb1a959a700 1 -- 172.21.15.156:0/2234900083 --> v1:172.21.15.186:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac15c660 con 0x7fb1ac0cd270 2023-11-28T13:08:09.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.656+0000 7fb1a959a700 1 -- 172.21.15.156:0/2234900083 <== mon.1 v1:172.21.15.186:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7fb1a0001650 con 0x7fb1ac0cd270 2023-11-28T13:08:09.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.656+0000 7fb1a959a700 1 -- 172.21.15.156:0/2234900083 >> v1:172.21.15.186:6789/0 conn(0x7fb1ac0cd270 legacy=0x7fb1ac0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.656+0000 7fb1a959a700 1 --2- 172.21.15.156:0/2234900083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb194005c10 0x7fb194006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.657+0000 7fb1a959a700 1 --2- 172.21.15.156:0/2234900083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb194006560 0x7fb194006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.657+0000 7fb1a959a700 1 --2- 172.21.15.156:0/2234900083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb194007300 0x7fb19400bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.657+0000 7fb1a959a700 1 -- 172.21.15.156:0/2234900083 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac15c380 con 0x7fb194005c10 2023-11-28T13:08:09.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.657+0000 7fb1a959a700 1 -- 172.21.15.156:0/2234900083 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19400c280 con 0x7fb194006560 2023-11-28T13:08:09.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.657+0000 7fb1a959a700 1 -- 172.21.15.156:0/2234900083 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb19400c440 con 0x7fb194007300 2023-11-28T13:08:09.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.657+0000 7fb1aa59c700 1 --2- 172.21.15.156:0/2234900083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb194005c10 0x7fb194006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.657+0000 7fb1a9d9b700 1 --2- 172.21.15.156:0/2234900083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb194006560 0x7fb194006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.658+0000 7efe777fe700 1 -- 172.21.15.156:0/547252887 learned_addr learned my addr 172.21.15.156:0/547252887 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:09.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.659+0000 7f5da43ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d88005320 0x7f5d9c0b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.658+0000 7efe76ffd700 1 --2- 172.21.15.156:0/547252887 >> v2:172.21.15.186:3300/0 conn(0x7efe7806c810 0x7efe7815c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.658+0000 7efe77fff700 1 --2- 172.21.15.156:0/547252887 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe7815c8e0 0x7efe780ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.658+0000 7fb1aa59c700 1 -- 172.21.15.156:0/2234900083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb194007300 msgr2=0x7fb19400bc10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T13:08:09.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.658+0000 7fb1aa59c700 1 --2- 172.21.15.156:0/2234900083 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb194007300 0x7fb19400bc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.658+0000 7fb1aa59c700 1 -- 172.21.15.156:0/2234900083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb194006560 msgr2=0x7fb194006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:09.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.658+0000 7fb1aa59c700 1 --2- 172.21.15.156:0/2234900083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb194006560 0x7fb194006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.658+0000 7fb1aa59c700 1 -- 172.21.15.156:0/2234900083 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac15c660 con 0x7fb194005c10 2023-11-28T13:08:09.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.658+0000 7fb1a9d9b700 1 --2- 172.21.15.156:0/2234900083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb194006560 0x7fb194006a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:09.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.658+0000 7f79e75ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79d8004b40 0x7f79d80e2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.658+0000 7f79e75ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f79d804f980 0x7f79d8054530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.659+0000 7f79e75ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d80e32d0 con 0x7f79d8007ac0 2023-11-28T13:08:09.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.659+0000 7f79e75ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d8053dc0 con 0x7f79d804f980 2023-11-28T13:08:09.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.659+0000 7f79e75ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8054010 con 0x7f79d8004b40 2023-11-28T13:08:09.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.659+0000 7fe355b9d700 1 Processor -- start 2023-11-28T13:08:09.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.659+0000 7fe355b9d700 1 -- start start 2023-11-28T13:08:09.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.659+0000 7f5da43ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d9c0af480 0x7f5d9c0af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.659+0000 7efe76ffd700 1 -- 172.21.15.156:0/547252887 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe7815c8e0 msgr2=0x7efe780ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:09.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.659+0000 7efe76ffd700 1 --2- 172.21.15.156:0/547252887 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe7815c8e0 0x7efe780ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.659+0000 7efe76ffd700 1 -- 172.21.15.156:0/547252887 >> v2:172.21.15.156:3300/0 conn(0x7efe78158950 msgr2=0x7efe7806ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:09.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.659+0000 7efe76ffd700 1 --2- 172.21.15.156:0/547252887 >> v2:172.21.15.156:3300/0 conn(0x7efe78158950 0x7efe7806ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.659+0000 7efe76ffd700 1 -- 172.21.15.156:0/547252887 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe780cacf0 con 0x7efe7806c810 2023-11-28T13:08:09.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.659+0000 7efe76ffd700 1 --2- 172.21.15.156:0/547252887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7806c810 0x7efe7815c3a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efe7815c300 tx=0x7efe6c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37833a2bb2b410f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.660+0000 7fb1aad9d700 1 -- 172.21.15.156:0/2234900083 reap_dead start 2023-11-28T13:08:09.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.660+0000 7efe767fc700 1 -- 172.21.15.156:0/547252887 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c010a30 con 0x7efe7806c810 2023-11-28T13:08:09.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.660+0000 7efe767fc700 1 -- 172.21.15.156:0/547252887 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6c010b90 con 0x7efe7806c810 2023-11-28T13:08:09.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.660+0000 7efe767fc700 1 -- 172.21.15.156:0/547252887 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c018440 con 0x7efe7806c810 2023-11-28T13:08:09.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.659+0000 7f79e5b9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f79d804f980 0x7f79d8054530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.660+0000 7f79e5b9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f79d804f980 0x7f79d8054530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58816/0 (socket says 172.21.15.156:58816) 2023-11-28T13:08:09.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.660+0000 7f79e5b9c700 1 -- 172.21.15.156:0/383360152 learned_addr learned my addr 172.21.15.156:0/383360152 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:09.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.660+0000 7f79e539b700 1 --2- 172.21.15.156:0/383360152 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79d8004b40 0x7f79d80e2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.661+0000 7f79e5b9c700 1 -- 172.21.15.156:0/383360152 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79d8004b40 msgr2=0x7f79d80e2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:09.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.661+0000 7f79e5b9c700 1 --2- 172.21.15.156:0/383360152 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79d8004b40 0x7f79d80e2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.661+0000 7f79e5b9c700 1 -- 172.21.15.156:0/383360152 >> v1:172.21.15.156:6789/0 conn(0x7f79d8007ac0 legacy=0x7f79d8052b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.661+0000 7fb1aa59c700 1 --2- 172.21.15.156:0/2234900083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb194005c10 0x7fb194006020 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00c290 tx=0x7fb194007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce97cc96e595cd0d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.661+0000 7f79e5b9c700 1 -- 172.21.15.156:0/383360152 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d80512a0 con 0x7f79d804f980 2023-11-28T13:08:09.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.659+0000 7f5da43ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d9c0c8450 0x7f5d9c0c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.661+0000 7f5da43ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0afe30 con 0x7f5d88005320 2023-11-28T13:08:09.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.661+0000 7f5da43ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0c5f90 con 0x7f5d9c0af480 2023-11-28T13:08:09.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.661+0000 7f5da43ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0c6270 con 0x7f5d9c0c8450 2023-11-28T13:08:09.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.660+0000 7f5da199a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d9c0af480 0x7f5d9c0af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.661+0000 7f5da199a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d9c0af480 0x7f5d9c0af8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:34984/0 (socket says 172.21.15.156:34984) 2023-11-28T13:08:09.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.661+0000 7f5da199a700 1 -- 172.21.15.156:0/2374934589 learned_addr learned my addr 172.21.15.156:0/2374934589 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:09.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.660+0000 7f5da219b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d88005320 0x7f5d9c0b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.661+0000 7fe355b9d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe350158b80 0x7fe3500c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.661+0000 7fe355b9d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3500b1080 0x7fe35006c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.661+0000 7fe355b9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe3500cc660 0x7fe3500cca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.661+0000 7fe355b9d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3500c9620 con 0x7fe3500cc660 2023-11-28T13:08:09.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.661+0000 7fe355b9d700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3500ccfc0 con 0x7fe350158b80 2023-11-28T13:08:09.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.661+0000 7fe355b9d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35006ce20 con 0x7fe3500b1080 2023-11-28T13:08:09.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.662+0000 7fe347fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3500b1080 0x7fe35006c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.662+0000 7fe347fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3500b1080 0x7fe35006c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36822/0 (socket says 172.21.15.156:36822) 2023-11-28T13:08:09.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.662+0000 7fe347fff700 1 -- 172.21.15.156:0/517149737 learned_addr learned my addr 172.21.15.156:0/517149737 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:09.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.662+0000 7f5da199a700 1 -- 172.21.15.156:0/2374934589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d9c0c8450 msgr2=0x7f5d9c0c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.662+0000 7f5da199a700 1 --2- 172.21.15.156:0/2374934589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d9c0c8450 0x7f5d9c0c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.662+0000 7f5da199a700 1 -- 172.21.15.156:0/2374934589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d88005320 msgr2=0x7f5d9c0b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.662+0000 7f5da199a700 1 --2- 172.21.15.156:0/2374934589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d88005320 0x7f5d9c0b1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.662+0000 7f5da199a700 1 -- 172.21.15.156:0/2374934589 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c06c970 con 0x7f5d9c0af480 2023-11-28T13:08:09.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.662+0000 7f79e5b9c700 1 --2- 172.21.15.156:0/383360152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d804f980 0x7f79d8054530 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f79e002d030 tx=0x7f79e0069a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c42ef6a6c069e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.662+0000 7f5da199a700 1 --2- 172.21.15.156:0/2374934589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d9c0af480 0x7f5d9c0af8f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00d620 tx=0x7f5d8c00d9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.662+0000 7f79d7fff700 1 -- 172.21.15.156:0/383360152 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e00673d0 con 0x7f79d804f980 2023-11-28T13:08:09.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.663+0000 7fb1a959a700 1 -- 172.21.15.156:0/2234900083 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c001e70 con 0x7fb194005c10 2023-11-28T13:08:09.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.663+0000 7f79d7fff700 1 -- 172.21.15.156:0/383360152 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e0067530 con 0x7f79d804f980 2023-11-28T13:08:09.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.663+0000 7f79d7fff700 1 -- 172.21.15.156:0/383360152 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0072040 con 0x7f79d804f980 2023-11-28T13:08:09.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.663+0000 7fe347fff700 1 -- 172.21.15.156:0/517149737 >> v2:172.21.15.186:3300/0 conn(0x7fe350158b80 msgr2=0x7fe3500c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:09.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.663+0000 7fe347fff700 1 --2- 172.21.15.156:0/517149737 >> v2:172.21.15.186:3300/0 conn(0x7fe350158b80 0x7fe3500c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.663+0000 7fe347fff700 1 -- 172.21.15.156:0/517149737 >> v2:172.21.15.156:3300/0 conn(0x7fe3500cc660 msgr2=0x7fe3500cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:09.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.663+0000 7fe347fff700 1 --2- 172.21.15.156:0/517149737 >> v2:172.21.15.156:3300/0 conn(0x7fe3500cc660 0x7fe3500cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.663+0000 7fe347fff700 1 -- 172.21.15.156:0/517149737 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3500cad50 con 0x7fe3500b1080 2023-11-28T13:08:09.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.663+0000 7f5d92ffd700 1 -- 172.21.15.156:0/2374934589 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0125f0 con 0x7f5d9c0af480 2023-11-28T13:08:09.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.663+0000 7f5d92ffd700 1 -- 172.21.15.156:0/2374934589 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c012750 con 0x7f5d9c0af480 2023-11-28T13:08:09.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.663+0000 7f5d92ffd700 1 -- 172.21.15.156:0/2374934589 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c012b00 con 0x7f5d9c0af480 2023-11-28T13:08:09.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.663+0000 7f5da43ff700 1 -- 172.21.15.156:0/2374934589 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c160660 con 0x7f5d9c0af480 2023-11-28T13:08:09.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.663+0000 7f5da43ff700 1 -- 172.21.15.156:0/2374934589 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c160fe0 con 0x7f5d9c0af480 2023-11-28T13:08:09.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.664+0000 7fe347fff700 1 --2- 172.21.15.156:0/517149737 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3500b1080 0x7fe35006c8e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe3500cc180 tx=0x7fe33800be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a71f637b5efd6b2b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.663+0000 7fb1a959a700 1 -- 172.21.15.156:0/2234900083 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19c00ce30 con 0x7fb194005c10 2023-11-28T13:08:09.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.663+0000 7fb1a959a700 1 -- 172.21.15.156:0/2234900083 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c003c40 con 0x7fb194005c10 2023-11-28T13:08:09.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.664+0000 7fb1b0fff700 1 -- 172.21.15.156:0/2234900083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb194005c10 msgr2=0x7fb194006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.664+0000 7fb1b0fff700 1 --2- 172.21.15.156:0/2234900083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb194005c10 0x7fb194006020 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00c290 tx=0x7fb194007a90 comp rx=0 tx=0).stop 2023-11-28T13:08:09.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.664+0000 7f79e75ff700 1 -- 172.21.15.156:0/383360152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d804f980 msgr2=0x7f79d8054530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.664+0000 7f79e75ff700 1 --2- 172.21.15.156:0/383360152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d804f980 0x7f79d8054530 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f79e002d030 tx=0x7f79e0069a90 comp rx=0 tx=0).stop 2023-11-28T13:08:09.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.664+0000 7f79e75ff700 1 -- 172.21.15.156:0/383360152 shutdown_connections 2023-11-28T13:08:09.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.664+0000 7f79e75ff700 1 --2- 172.21.15.156:0/383360152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d804f980 0x7f79d8054530 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.664+0000 7f79e75ff700 1 --2- 172.21.15.156:0/383360152 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79d8004b40 0x7f79d80e2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.664+0000 7f79e75ff700 1 -- 172.21.15.156:0/383360152 >> 172.21.15.156:0/383360152 conn(0x7f79d8051a60 msgr2=0x7f79d8051e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.665+0000 7fe34ffff700 1 -- 172.21.15.156:0/517149737 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33800dd70 con 0x7fe3500b1080 2023-11-28T13:08:09.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.665+0000 7fe34ffff700 1 -- 172.21.15.156:0/517149737 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe338012450 con 0x7fe3500b1080 2023-11-28T13:08:09.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.665+0000 7fe34ffff700 1 -- 172.21.15.156:0/517149737 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338012800 con 0x7fe3500b1080 2023-11-28T13:08:09.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.666+0000 7fe355b9d700 1 -- 172.21.15.156:0/517149737 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3500b1080 msgr2=0x7fe35006c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.666+0000 7fe355b9d700 1 --2- 172.21.15.156:0/517149737 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3500b1080 0x7fe35006c8e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe3500cc180 tx=0x7fe33800be70 comp rx=0 tx=0).stop 2023-11-28T13:08:09.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.666+0000 7fe355b9d700 1 -- 172.21.15.156:0/517149737 shutdown_connections 2023-11-28T13:08:09.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.666+0000 7fe355b9d700 1 --2- 172.21.15.156:0/517149737 >> v2:172.21.15.156:3300/0 conn(0x7fe3500cc660 0x7fe3500cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.666+0000 7fe355b9d700 1 --2- 172.21.15.156:0/517149737 >> v2:172.21.15.186:3300/0 conn(0x7fe350158b80 0x7fe3500c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.666+0000 7fe355b9d700 1 --2- 172.21.15.156:0/517149737 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3500b1080 0x7fe35006c8e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.666+0000 7fe355b9d700 1 -- 172.21.15.156:0/517149737 >> 172.21.15.156:0/517149737 conn(0x7fe3500cb510 msgr2=0x7fe3500cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.665+0000 7f5da43ff700 1 -- 172.21.15.156:0/2374934589 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5d84000fc0 con 0x7f5d9c0af480 2023-11-28T13:08:09.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.663+0000 7efe7cd9d700 1 -- 172.21.15.156:0/547252887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7806c810 msgr2=0x7efe7815c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.663+0000 7efe7cd9d700 1 --2- 172.21.15.156:0/547252887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7806c810 0x7efe7815c3a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efe7815c300 tx=0x7efe6c009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:09.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.665+0000 7efe7cd9d700 1 -- 172.21.15.156:0/547252887 shutdown_connections 2023-11-28T13:08:09.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.665+0000 7efe7cd9d700 1 --2- 172.21.15.156:0/547252887 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe7815c8e0 0x7efe780ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.665+0000 7efe7cd9d700 1 --2- 172.21.15.156:0/547252887 >> v2:172.21.15.156:3300/0 conn(0x7efe78158950 0x7efe7806ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.665+0000 7efe7cd9d700 1 --2- 172.21.15.156:0/547252887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7806c810 0x7efe7815c3a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.665+0000 7efe7cd9d700 1 -- 172.21.15.156:0/547252887 >> 172.21.15.156:0/547252887 conn(0x7efe780cb510 msgr2=0x7efe780cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.665+0000 7efe7cd9d700 1 -- 172.21.15.156:0/547252887 shutdown_connections 2023-11-28T13:08:09.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.665+0000 7efe7cd9d700 1 -- 172.21.15.156:0/547252887 wait complete. 2023-11-28T13:08:09.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.666+0000 7fb1b0fff700 1 -- 172.21.15.156:0/2234900083 shutdown_connections 2023-11-28T13:08:09.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.666+0000 7fb1b0fff700 1 --2- 172.21.15.156:0/2234900083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb194005c10 0x7fb194006020 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.666+0000 7fb1b0fff700 1 -- 172.21.15.156:0/2234900083 >> 172.21.15.156:0/2234900083 conn(0x7fb1ac0b06f0 msgr2=0x7fb1ac0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.666+0000 7fb1b0fff700 1 -- 172.21.15.156:0/2234900083 shutdown_connections 2023-11-28T13:08:09.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.666+0000 7fb1b0fff700 1 -- 172.21.15.156:0/2234900083 wait complete. 2023-11-28T13:08:09.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.665+0000 7f79e75ff700 1 -- 172.21.15.156:0/383360152 shutdown_connections 2023-11-28T13:08:09.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.666+0000 7f79e75ff700 1 -- 172.21.15.156:0/383360152 wait complete. 2023-11-28T13:08:09.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.667+0000 7f5d92ffd700 1 -- 172.21.15.156:0/2374934589 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5d8c012c60 con 0x7f5d9c0af480 2023-11-28T13:08:09.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.667+0000 7f9fd339d700 1 Processor -- start 2023-11-28T13:08:09.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.668+0000 7f9fd339d700 1 -- start start 2023-11-28T13:08:09.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.668+0000 7f5d92ffd700 1 --2- 172.21.15.156:0/2374934589 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5d80068340 0x7f5d8006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.668+0000 7fe355b9d700 1 -- 172.21.15.156:0/517149737 shutdown_connections 2023-11-28T13:08:09.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.668+0000 7fe355b9d700 1 -- 172.21.15.156:0/517149737 wait complete. 2023-11-28T13:08:09.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.669+0000 7efe7cd9d700 1 Processor -- start 2023-11-28T13:08:09.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.669+0000 7f5d92ffd700 1 -- 172.21.15.156:0/2374934589 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5d8c0862b0 con 0x7f5d9c0af480 2023-11-28T13:08:09.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.669+0000 7efe7cd9d700 1 -- start start 2023-11-28T13:08:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.669+0000 7fb1b0fff700 1 Processor -- start 2023-11-28T13:08:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.670+0000 7f5d92ffd700 1 --2- 172.21.15.156:0/2374934589 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x7f5d80070b70 0x7f5d80072f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.670+0000 7f5d92ffd700 1 -- 172.21.15.156:0/2374934589 --> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d80073630 con 0x7f5d80070b70 2023-11-28T13:08:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.670+0000 7f5da219b700 1 --2- 172.21.15.156:0/2374934589 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5d80068340 0x7f5d8006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.670+0000 7fb1b0fff700 1 -- start start 2023-11-28T13:08:09.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.670+0000 7f79e75ff700 1 Processor -- start 2023-11-28T13:08:09.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.671+0000 7f5da219b700 1 --2- 172.21.15.156:0/2374934589 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5d80068340 0x7f5d8006a7f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5d94007fb0 tx=0x7f5d94007560 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.671+0000 7f5da299c700 1 --2- 172.21.15.156:0/2374934589 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x7f5d80070b70 0x7f5d80072f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.671+0000 7efe7cd9d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe78158950 0x7efe78157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.671+0000 7efe7cd9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7815c8e0 0x7efe78156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.671+0000 7efe7cd9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe7815ae00 0x7efe780b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.671+0000 7f9fd339d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fc40e6d00 0x7f9fc404f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.671+0000 7f9fd339d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9fc4051150 0x7f9fc4036610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.671+0000 7f9fd339d700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc4036fb0 con 0x7f9fc4036b50 2023-11-28T13:08:09.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.671+0000 7f9fd339d700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc4048550 con 0x7f9fc4051150 2023-11-28T13:08:09.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.671+0000 7f9fd339d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc4033920 con 0x7f9fc40e6d00 2023-11-28T13:08:09.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.671+0000 7efe777fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe78158950 0x7efe78157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7efe777fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe78158950 0x7efe78157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36828/0 (socket says 172.21.15.156:36828) 2023-11-28T13:08:09.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7fb1b0fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb194005c10 0x7fb1ac1ee980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7fe355b9d700 1 Processor -- start 2023-11-28T13:08:09.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7efe76ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7815c8e0 0x7efe78156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7efe76ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7815c8e0 0x7efe78156b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:35004/0 (socket says 172.21.15.156:35004) 2023-11-28T13:08:09.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7f9fd2b9c700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f9fc4036b50 0x7f9fc4005f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:38514/0 (socket says 172.21.15.156:38514) 2023-11-28T13:08:09.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7f9fd2b9c700 1 -- 172.21.15.156:0/3819508759 learned_addr learned my addr 172.21.15.156:0/3819508759 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:09.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7fe355b9d700 1 -- start start 2023-11-28T13:08:09.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7fb1b0fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1ac14bbc0 0x7fb1ac0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7fb1b0fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1ac15ac90 0x7fb1ac0cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7fb1b0fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac1edf20 con 0x7fb1ac15ac90 2023-11-28T13:08:09.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7fb1b0fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac1eeec0 con 0x7fb194005c10 2023-11-28T13:08:09.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7fb1b0fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac15b600 con 0x7fb1ac14bbc0 2023-11-28T13:08:09.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7f5d92ffd700 1 -- 172.21.15.156:0/2374934589 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d8c00b3a0 con 0x7f5d9c0af480 2023-11-28T13:08:09.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7f9fd1399700 1 -- 172.21.15.156:0/3819508759 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281028065 0 0) 0x7f9fc4036fb0 con 0x7f9fc4036b50 2023-11-28T13:08:09.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7f9fd239b700 1 --2- 172.21.15.156:0/3819508759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fc40e6d00 0x7f9fc404f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7f9fd1399700 1 -- 172.21.15.156:0/3819508759 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fb80035b0 con 0x7f9fc4036b50 2023-11-28T13:08:09.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7fb1aad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1ac15ac90 0x7fb1ac0cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7fb1aad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1ac15ac90 0x7fb1ac0cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58840/0 (socket says 172.21.15.156:58840) 2023-11-28T13:08:09.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7fb1aad9d700 1 -- 172.21.15.156:0/1670121363 learned_addr learned my addr 172.21.15.156:0/1670121363 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:09.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7f9fd1399700 1 -- 172.21.15.156:0/3819508759 <== mon.1 v1:172.21.15.186:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2470617984 0 0) 0x7f9fb80035b0 con 0x7f9fc4036b50 2023-11-28T13:08:09.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.673+0000 7f9fd1399700 1 -- 172.21.15.156:0/3819508759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fc40e6d00 msgr2=0x7f9fc404f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:09.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7efe7cd9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe780b3290 con 0x7efe7815ae00 2023-11-28T13:08:09.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7efe7cd9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7815a970 con 0x7efe7815c8e0 2023-11-28T13:08:09.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.673+0000 7f9fd1399700 1 --2- 172.21.15.156:0/3819508759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fc40e6d00 0x7f9fc404f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.673+0000 7f9fd1399700 1 -- 172.21.15.156:0/3819508759 >> v2:172.21.15.186:3300/0 conn(0x7f9fc4051150 msgr2=0x7f9fc4036610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:09.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7efe7cd9d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe780c4f80 con 0x7efe78158950 2023-11-28T13:08:09.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.673+0000 7efe777fe700 1 -- 172.21.15.156:0/484838469 learned_addr learned my addr 172.21.15.156:0/484838469 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:09.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.673+0000 7fb1aa59c700 1 --2- 172.21.15.156:0/1670121363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb194005c10 0x7fb1ac1ee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.673+0000 7f79e75ff700 1 -- start start 2023-11-28T13:08:09.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.672+0000 7f9fd1b9a700 1 --2- 172.21.15.156:0/3819508759 >> v2:172.21.15.186:3300/0 conn(0x7f9fc4051150 0x7f9fc4036610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.673+0000 7fe355b9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3500b1080 0x7fe3501e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7fe355b9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3500cc660 0x7fe350154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7fe355b9d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe350158b80 0x7fe3501e1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7fe355b9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500b18c0 con 0x7fe3500b1080 2023-11-28T13:08:09.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7fe355b9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3501553a0 con 0x7fe3500cc660 2023-11-28T13:08:09.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7fe355b9d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3501e22a0 con 0x7fe350158b80 2023-11-28T13:08:09.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7fe354b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3500b1080 0x7fe3501e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.675+0000 7fe354b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3500b1080 0x7fe3501e3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58854/0 (socket says 172.21.15.156:58854) 2023-11-28T13:08:09.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.675+0000 7fe354b9b700 1 -- 172.21.15.156:0/2298853731 learned_addr learned my addr 172.21.15.156:0/2298853731 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:09.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.673+0000 7f5da299c700 1 --2- 172.21.15.156:0/2374934589 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x7f5d80070b70 0x7f5d80072f80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.673+0000 7efe76ffd700 1 -- 172.21.15.156:0/484838469 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe78158950 msgr2=0x7efe78157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.673+0000 7efe76ffd700 1 --2- 172.21.15.156:0/484838469 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe78158950 0x7efe78157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.673+0000 7efe76ffd700 1 -- 172.21.15.156:0/484838469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe7815ae00 msgr2=0x7efe780b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.673+0000 7efe76ffd700 1 --2- 172.21.15.156:0/484838469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe7815ae00 0x7efe780b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7efe76ffd700 1 -- 172.21.15.156:0/484838469 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c0094a0 con 0x7efe7815c8e0 2023-11-28T13:08:09.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.675+0000 7efe76ffd700 1 --2- 172.21.15.156:0/484838469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7815c8e0 0x7efe78156b60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe6c01a000 tx=0x7efe6c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.673+0000 7fb1aad9d700 1 -- 172.21.15.156:0/1670121363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1ac14bbc0 msgr2=0x7fb1ac0c7990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T13:08:09.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7fb1a9d9b700 1 --2- 172.21.15.156:0/1670121363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1ac14bbc0 0x7fb1ac0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7fb1aad9d700 1 --2- 172.21.15.156:0/1670121363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1ac14bbc0 0x7fb1ac0c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7fb1aad9d700 1 -- 172.21.15.156:0/1670121363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb194005c10 msgr2=0x7fb1ac1ee980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7fb1aad9d700 1 --2- 172.21.15.156:0/1670121363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb194005c10 0x7fb1ac1ee980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7fb1aad9d700 1 -- 172.21.15.156:0/1670121363 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac15cc20 con 0x7fb1ac15ac90 2023-11-28T13:08:09.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7fb1aa59c700 1 --2- 172.21.15.156:0/1670121363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb194005c10 0x7fb1ac1ee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:09.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7fb1aad9d700 1 --2- 172.21.15.156:0/1670121363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1ac15ac90 0x7fb1ac0cd6a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0cc610 tx=0x7fb1a400aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7f9fd1399700 1 --2- 172.21.15.156:0/3819508759 >> v2:172.21.15.186:3300/0 conn(0x7f9fc4051150 0x7f9fc4036610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7f9fd1399700 1 -- 172.21.15.156:0/3819508759 --> v1:172.21.15.186:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc4053c90 con 0x7f9fc4036b50 2023-11-28T13:08:09.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7f9fd1399700 1 -- 172.21.15.156:0/3819508759 <== mon.1 v1:172.21.15.186:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f9fcc0745c0 con 0x7f9fc4036b50 2023-11-28T13:08:09.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7f9fd1399700 1 -- 172.21.15.156:0/3819508759 >> v1:172.21.15.186:6789/0 conn(0x7f9fc4036b50 legacy=0x7f9fc4005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7f9fd1399700 1 --2- 172.21.15.156:0/3819508759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fc40e6d00 0x7f9fb8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7f9fd1399700 1 --2- 172.21.15.156:0/3819508759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fb8004030 0x7f9fb8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7f9fd1399700 1 --2- 172.21.15.156:0/3819508759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fb8004d90 0x7f9fb8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7f9fd1399700 1 -- 172.21.15.156:0/3819508759 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc4048550 con 0x7f9fb8004d90 2023-11-28T13:08:09.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7f9fd1399700 1 -- 172.21.15.156:0/3819508759 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc4033920 con 0x7f9fb8004030 2023-11-28T13:08:09.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.674+0000 7f9fd1399700 1 -- 172.21.15.156:0/3819508759 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb800bbd0 con 0x7f9fc40e6d00 2023-11-28T13:08:09.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.675+0000 7fe347fff700 1 --2- 172.21.15.156:0/2298853731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3500cc660 0x7fe350154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.675+0000 7f9fd239b700 1 --2- 172.21.15.156:0/3819508759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fc40e6d00 0x7f9fb8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.675+0000 7f74b8b9d700 1 Processor -- start 2023-11-28T13:08:09.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.675+0000 7f9fd2b9c700 1 --2- 172.21.15.156:0/3819508759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fb8004d90 0x7f9fb8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.675+0000 7efe5ffff700 1 -- 172.21.15.156:0/484838469 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c0038a0 con 0x7efe7815c8e0 2023-11-28T13:08:09.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.675+0000 7fe35539c700 1 --2- 172.21.15.156:0/2298853731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe350158b80 0x7fe3501e1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.675+0000 7f74b8b9d700 1 -- start start 2023-11-28T13:08:09.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.675+0000 7efe5ffff700 1 -- 172.21.15.156:0/484838469 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6c003a00 con 0x7efe7815c8e0 2023-11-28T13:08:09.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.675+0000 7efe5ffff700 1 -- 172.21.15.156:0/484838469 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c003db0 con 0x7efe7815c8e0 2023-11-28T13:08:09.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.676+0000 7f5d92ffd700 1 -- 172.21.15.156:0/2374934589 <== osd.1 v2:172.21.15.156:6800/132479 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5d80073630 con 0x7f5d80070b70 2023-11-28T13:08:09.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.676+0000 7fe35539c700 1 -- 172.21.15.156:0/2298853731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3500cc660 msgr2=0x7fe350154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.676+0000 7fe35539c700 1 --2- 172.21.15.156:0/2298853731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3500cc660 0x7fe350154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.676+0000 7fe35539c700 1 -- 172.21.15.156:0/2298853731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3500b1080 msgr2=0x7fe3501e3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.676+0000 7efe7cd9d700 1 -- 172.21.15.156:0/484838469 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe780cb2b0 con 0x7efe7815c8e0 2023-11-28T13:08:09.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.676+0000 7efe7cd9d700 1 -- 172.21.15.156:0/484838469 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe780cafd0 con 0x7efe7815c8e0 2023-11-28T13:08:09.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.676+0000 7f9fd2b9c700 1 -- 172.21.15.156:0/3819508759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fc40e6d00 msgr2=0x7f9fb8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:09.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.676+0000 7f9fd2b9c700 1 --2- 172.21.15.156:0/3819508759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fc40e6d00 0x7f9fb8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.676+0000 7f9fd2b9c700 1 -- 172.21.15.156:0/3819508759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fb8004030 msgr2=0x7f9fb8004850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T13:08:09.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.676+0000 7f9fd2b9c700 1 --2- 172.21.15.156:0/3819508759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fb8004030 0x7f9fb8004850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.676+0000 7f9fd2b9c700 1 -- 172.21.15.156:0/3819508759 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb80035b0 con 0x7f9fb8004d90 2023-11-28T13:08:09.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7fe35539c700 1 --2- 172.21.15.156:0/2298853731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3500b1080 0x7fe3501e3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7fe35539c700 1 -- 172.21.15.156:0/2298853731 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33800b7f0 con 0x7fe350158b80 2023-11-28T13:08:09.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7fe354b9b700 1 --2- 172.21.15.156:0/2298853731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3500b1080 0x7fe3501e3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:09.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7efe7cd9d700 1 -- 172.21.15.156:0/484838469 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efe58000fc0 con 0x7efe7815c8e0 2023-11-28T13:08:09.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7f79e75ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79d8004b40 0x7f79d8038320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7f9fd2b9c700 1 --2- 172.21.15.156:0/3819508759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fb8004d90 0x7f9fb8005200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc04f380 tx=0x7f9fb8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22b89c7481ab4b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7f79e75ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d8007ac0 0x7f79d8180520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7f9fd1b9a700 1 --2- 172.21.15.156:0/3819508759 >> v2:172.21.15.186:3300/0 conn(0x7f9fc4051150 0x7f9fc4036610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:09.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7f9fd1399700 1 -- 172.21.15.156:0/3819508759 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc072300 con 0x7f9fb8004d90 2023-11-28T13:08:09.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7f9fd1399700 1 -- 172.21.15.156:0/3819508759 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc072460 con 0x7f9fb8004d90 2023-11-28T13:08:09.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7f9fd1399700 1 -- 172.21.15.156:0/3819508759 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0727f0 con 0x7f9fb8004d90 2023-11-28T13:08:09.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7f79e75ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79d804f980 0x7f79d8180a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.675+0000 7f24b5fff700 1 Processor -- start 2023-11-28T13:08:09.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.675+0000 7f24b5fff700 1 -- start start 2023-11-28T13:08:09.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7f24b5fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f24b00ce480 0x7f24b00c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7f24b5fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b0161220 0x7f24b006b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7f24b5fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b0162210 con 0x7f24b0164e60 2023-11-28T13:08:09.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7f24b5fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b01649d0 con 0x7f24b00ce480 2023-11-28T13:08:09.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7f24b5fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b006c020 con 0x7f24b0161220 2023-11-28T13:08:09.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.677+0000 7f79e75ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8049500 con 0x7f79d8007ac0 2023-11-28T13:08:09.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.678+0000 7f24aeffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b0161220 0x7f24b006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.678+0000 7f24aeffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b0161220 0x7f24b006b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36878/0 (socket says 172.21.15.156:36878) 2023-11-28T13:08:09.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.678+0000 7f24aeffd700 1 -- 172.21.15.156:0/3934346949 learned_addr learned my addr 172.21.15.156:0/3934346949 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:09.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.678+0000 7fb19affd700 1 -- 172.21.15.156:0/1670121363 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4009710 con 0x7fb1ac15ac90 2023-11-28T13:08:09.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.678+0000 7f79e5b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79d804f980 0x7f79d8180a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.678+0000 7f79e5b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79d804f980 0x7f79d8180a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:35038/0 (socket says 172.21.15.156:35038) 2023-11-28T13:08:09.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.678+0000 7f79e5b9c700 1 -- 172.21.15.156:0/4036108884 learned_addr learned my addr 172.21.15.156:0/4036108884 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:09.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.678+0000 7fe35539c700 1 --2- 172.21.15.156:0/2298853731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe350158b80 0x7fe3501e1d60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe34800b410 tx=0x7fe34800b720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.678+0000 7f24aeffd700 1 -- 172.21.15.156:0/3934346949 >> v2:172.21.15.156:3300/0 conn(0x7f24b00ce480 msgr2=0x7f24b00c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:09.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7fb19affd700 1 -- 172.21.15.156:0/1670121363 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a4009870 con 0x7fb1ac15ac90 2023-11-28T13:08:09.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7f74b37fe700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f74ac0e6da0 0x7f74ac053780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:38530/0 (socket says 172.21.15.156:38530) 2023-11-28T13:08:09.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7f74b37fe700 1 -- 172.21.15.156:0/2084527835 learned_addr learned my addr 172.21.15.156:0/2084527835 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:09.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7fb19affd700 1 -- 172.21.15.156:0/1670121363 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4009c20 con 0x7fb1ac15ac90 2023-11-28T13:08:09.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7f24aeffd700 1 --2- 172.21.15.156:0/3934346949 >> v2:172.21.15.156:3300/0 conn(0x7f24b00ce480 0x7f24b00c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7f79e4b9a700 1 --2- 172.21.15.156:0/4036108884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d8007ac0 0x7f79d8180520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7f24aeffd700 1 -- 172.21.15.156:0/3934346949 >> v1:172.21.15.156:6789/0 conn(0x7f24b0164e60 legacy=0x7f24b00b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:09.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7fe34dffb700 1 -- 172.21.15.156:0/2298853731 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348011620 con 0x7fe350158b80 2023-11-28T13:08:09.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7f79e75ff700 1 -- 172.21.15.156:0/4036108884 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80cbc30 con 0x7f79d804f980 2023-11-28T13:08:09.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7f79e75ff700 1 -- 172.21.15.156:0/4036108884 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8049750 con 0x7f79d8004b40 2023-11-28T13:08:09.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7fe34dffb700 1 -- 172.21.15.156:0/2298853731 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe348011780 con 0x7fe350158b80 2023-11-28T13:08:09.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7fe34dffb700 1 -- 172.21.15.156:0/2298853731 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348011b30 con 0x7fe350158b80 2023-11-28T13:08:09.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7f9fd339d700 1 -- 172.21.15.156:0/3819508759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fb8004d90 msgr2=0x7f9fb8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7f9fd339d700 1 --2- 172.21.15.156:0/3819508759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fb8004d90 0x7f9fb8005200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc04f380 tx=0x7f9fb8005880 comp rx=0 tx=0).stop 2023-11-28T13:08:09.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7f9fd2b9c700 1 -- 172.21.15.156:0/3819508759 reap_dead start 2023-11-28T13:08:09.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7fe355b9d700 1 -- 172.21.15.156:0/2298853731 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35006c0e0 con 0x7fe350158b80 2023-11-28T13:08:09.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7fe355b9d700 1 -- 172.21.15.156:0/2298853731 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3500cb030 con 0x7fe350158b80 2023-11-28T13:08:09.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.680+0000 7fb1b0fff700 1 -- 172.21.15.156:0/1670121363 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ac15c940 con 0x7fb1ac15ac90 2023-11-28T13:08:09.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.680+0000 7fb1b0fff700 1 -- 172.21.15.156:0/1670121363 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ac0b1200 con 0x7fb1ac15ac90 2023-11-28T13:08:09.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.680+0000 7f79e5b9c700 1 -- 172.21.15.156:0/4036108884 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79d8004b40 msgr2=0x7f79d8038320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T13:08:09.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.680+0000 7f79e5b9c700 1 --2- 172.21.15.156:0/4036108884 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79d8004b40 0x7f79d8038320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.680+0000 7f79e5b9c700 1 -- 172.21.15.156:0/4036108884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d8007ac0 msgr2=0x7f79d8180520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.680+0000 7f79e5b9c700 1 --2- 172.21.15.156:0/4036108884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d8007ac0 0x7f79d8180520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.680+0000 7f79e5b9c700 1 -- 172.21.15.156:0/4036108884 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e004ec80 con 0x7f79d804f980 2023-11-28T13:08:09.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.680+0000 7f79e5b9c700 1 --2- 172.21.15.156:0/4036108884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79d804f980 0x7f79d8180a60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f79e008afd0 tx=0x7f79e0068d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.680+0000 7f79d5ffb700 1 -- 172.21.15.156:0/4036108884 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0072440 con 0x7f79d804f980 2023-11-28T13:08:09.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.680+0000 7f79e75ff700 1 -- 172.21.15.156:0/4036108884 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d8054a70 con 0x7f79d804f980 2023-11-28T13:08:09.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.680+0000 7f79d5ffb700 1 -- 172.21.15.156:0/4036108884 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e00725a0 con 0x7f79d804f980 2023-11-28T13:08:09.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.679+0000 7f24aeffd700 1 -- 172.21.15.156:0/3934346949 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b006bd60 con 0x7f24b0161220 2023-11-28T13:08:09.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.680+0000 7f9fd339d700 1 -- 172.21.15.156:0/3819508759 shutdown_connections 2023-11-28T13:08:09.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.680+0000 7f9fd339d700 1 -- 172.21.15.156:0/3819508759 >> 172.21.15.156:0/3819508759 conn(0x7f9fc4051610 msgr2=0x7f9fc40519e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.680+0000 7f74b8b9d700 1 --2- 172.21.15.156:0/2084527835 >> v2:172.21.15.186:3300/0 conn(0x7f74ac007390 0x7f74ac03ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.681+0000 7f74b8b9d700 1 --2- 172.21.15.156:0/2084527835 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74ac006e70 0x7f74ac03b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.680+0000 7fe355b9d700 1 -- 172.21.15.156:0/2298853731 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe35006c3c0 con 0x7fe350158b80 2023-11-28T13:08:09.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.681+0000 7f74b8b9d700 1 -- 172.21.15.156:0/2084527835 --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ac0e7170 con 0x7f74ac0e6da0 2023-11-28T13:08:09.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.681+0000 7f24aeffd700 1 --2- 172.21.15.156:0/3934346949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b0161220 0x7f24b006b820 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f24a40098f0 tx=0x7f24a4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96ed2d547ec3be48 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.681+0000 7f74b3fff700 1 --2- 172.21.15.156:0/2084527835 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74ac006e70 0x7f74ac03b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.681+0000 7f79d5ffb700 1 -- 172.21.15.156:0/4036108884 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0072910 con 0x7f79d804f980 2023-11-28T13:08:09.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.682+0000 7f79e75ff700 1 -- 172.21.15.156:0/4036108884 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d8051580 con 0x7f79d804f980 2023-11-28T13:08:09.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.682+0000 7f74b8b9d700 1 -- 172.21.15.156:0/2084527835 --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ac053e90 con 0x7f74ac007390 2023-11-28T13:08:09.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.682+0000 7fb198ff9700 1 -- 172.21.15.156:0/1670121363 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb184000fc0 con 0x7fb1ac15ac90 2023-11-28T13:08:09.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.682+0000 7f24ae7fc700 1 -- 172.21.15.156:0/3934346949 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a4004230 con 0x7f24b0161220 2023-11-28T13:08:09.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.682+0000 7f74b27fc700 1 -- 172.21.15.156:0/2084527835 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2945020336 0 0) 0x7f74ac0e7170 con 0x7f74ac0e6da0 2023-11-28T13:08:09.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.682+0000 7f74b8b9d700 1 -- 172.21.15.156:0/2084527835 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac007900 con 0x7f74ac006e70 2023-11-28T13:08:09.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.682+0000 7fe34dffb700 1 -- 172.21.15.156:0/2298853731 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe348021070 con 0x7fe350158b80 2023-11-28T13:08:09.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.682+0000 7f24ae7fc700 1 -- 172.21.15.156:0/3934346949 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a4004390 con 0x7f24b0161220 2023-11-28T13:08:09.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.682+0000 7f24ae7fc700 1 -- 172.21.15.156:0/3934346949 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a40036a0 con 0x7f24b0161220 2023-11-28T13:08:09.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.682+0000 7f74b27fc700 1 -- 172.21.15.156:0/2084527835 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74a40035b0 con 0x7f74ac0e6da0 2023-11-28T13:08:09.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.682+0000 7efe5ffff700 1 -- 172.21.15.156:0/484838469 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efe6c00f070 con 0x7efe7815c8e0 2023-11-28T13:08:09.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.683+0000 7f74b27fc700 1 -- 172.21.15.156:0/2084527835 <== mon.1 v1:172.21.15.186:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 22051275 0 0) 0x7f74a40035b0 con 0x7f74ac0e6da0 2023-11-28T13:08:09.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.683+0000 7f79e75ff700 1 -- 172.21.15.156:0/4036108884 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f79d8054c00 con 0x7f79d804f980 2023-11-28T13:08:09.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.683+0000 7f74b3fff700 1 -- 172.21.15.156:0/2084527835 >> v2:172.21.15.186:3300/0 conn(0x7f74ac007390 msgr2=0x7f74ac03ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:09.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.683+0000 7f74b3fff700 1 --2- 172.21.15.156:0/2084527835 >> v2:172.21.15.186:3300/0 conn(0x7f74ac007390 0x7f74ac03ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.683+0000 7f74b3fff700 1 -- 172.21.15.156:0/2084527835 >> v1:172.21.15.186:6789/0 conn(0x7f74ac0e6da0 legacy=0x7f74ac053780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.683+0000 7f24b5fff700 1 -- 172.21.15.156:0/3934346949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b0161220 msgr2=0x7f24b006b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.683+0000 7f24b5fff700 1 --2- 172.21.15.156:0/3934346949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b0161220 0x7f24b006b820 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f24a40098f0 tx=0x7f24a4009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:09.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.683+0000 7f74b3fff700 1 -- 172.21.15.156:0/2084527835 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac006390 con 0x7f74ac006e70 2023-11-28T13:08:09.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.683+0000 7f24b5fff700 1 -- 172.21.15.156:0/3934346949 shutdown_connections 2023-11-28T13:08:09.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7f24b5fff700 1 --2- 172.21.15.156:0/3934346949 >> v2:172.21.15.156:3300/0 conn(0x7f24b00ce480 0x7f24b00c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7f24b5fff700 1 --2- 172.21.15.156:0/3934346949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b0161220 0x7f24b006b820 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7f74b3fff700 1 --2- 172.21.15.156:0/2084527835 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74ac006e70 0x7f74ac03b060 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f74a8009840 tx=0x7f74a8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6823f82a8675e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7f24b5fff700 1 -- 172.21.15.156:0/3934346949 >> 172.21.15.156:0/3934346949 conn(0x7f24b00cb670 msgr2=0x7f24b00cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7f24b5fff700 1 -- 172.21.15.156:0/3934346949 shutdown_connections 2023-11-28T13:08:09.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7f24b5fff700 1 -- 172.21.15.156:0/3934346949 wait complete. 2023-11-28T13:08:09.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7efe5ffff700 1 --2- 172.21.15.156:0/484838469 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efe640683c0 0x7efe6406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7efe5ffff700 1 -- 172.21.15.156:0/484838469 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efe6c0864a0 con 0x7efe7815c8e0 2023-11-28T13:08:09.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7f9fd339d700 1 -- 172.21.15.156:0/3819508759 shutdown_connections 2023-11-28T13:08:09.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7f79d5ffb700 1 -- 172.21.15.156:0/4036108884 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f79e009c030 con 0x7f79d804f980 2023-11-28T13:08:09.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.683+0000 7fe34dffb700 1 --2- 172.21.15.156:0/2298853731 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe3300683c0 0x7fe33006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7fe34dffb700 1 -- 172.21.15.156:0/2298853731 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe3480858f0 con 0x7fe350158b80 2023-11-28T13:08:09.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7fe34dffb700 1 --2- 172.21.15.156:0/2298853731 >> [v2:172.21.15.156:6808/132471,v1:172.21.15.156:6809/132471] conn(0x7fe330070bf0 0x7fe330073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7fe34dffb700 1 -- 172.21.15.156:0/2298853731 --> [v2:172.21.15.156:6808/132471,v1:172.21.15.156:6809/132471] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe3300736b0 con 0x7fe330070bf0 2023-11-28T13:08:09.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7fe34dffb700 1 -- 172.21.15.156:0/2298853731 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3480520e0 con 0x7fe350158b80 2023-11-28T13:08:09.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7fe347fff700 1 --2- 172.21.15.156:0/2298853731 >> [v2:172.21.15.156:6808/132471,v1:172.21.15.156:6809/132471] conn(0x7fe330070bf0 0x7fe330073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7f74b27fc700 1 -- 172.21.15.156:0/2084527835 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a0001670 con 0x7f74ac006e70 2023-11-28T13:08:09.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7fe354b9b700 1 --2- 172.21.15.156:0/2298853731 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe3300683c0 0x7fe33006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.685+0000 7efe777fe700 1 --2- 172.21.15.156:0/484838469 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efe640683c0 0x7efe6406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.685+0000 7f74b27fc700 1 -- 172.21.15.156:0/2084527835 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a8004030 con 0x7f74ac006e70 2023-11-28T13:08:09.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.685+0000 7f74b27fc700 1 -- 172.21.15.156:0/2084527835 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a0001670 con 0x7f74ac006e70 2023-11-28T13:08:09.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.685+0000 7fb19affd700 1 -- 172.21.15.156:0/1670121363 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb1a401c030 con 0x7fb1ac15ac90 2023-11-28T13:08:09.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.685+0000 7f9fd339d700 1 -- 172.21.15.156:0/3819508759 wait complete. 2023-11-28T13:08:09.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.685+0000 7fe354b9b700 1 --2- 172.21.15.156:0/2298853731 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe3300683c0 0x7fe33006a870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe3500b49e0 tx=0x7fe34000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.684+0000 7efe5ffff700 1 --2- 172.21.15.156:0/484838469 >> [v2:172.21.15.156:6814/132475,v1:172.21.15.156:6816/132475] conn(0x7efe64070c70 0x7efe64073080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.686+0000 7efe777fe700 1 --2- 172.21.15.156:0/484838469 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efe640683c0 0x7efe6406a870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe68009c50 tx=0x7efe680092c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.686+0000 7f74b8b9d700 1 -- 172.21.15.156:0/2084527835 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74ac006e70 msgr2=0x7f74ac03b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.686+0000 7f74b8b9d700 1 --2- 172.21.15.156:0/2084527835 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74ac006e70 0x7f74ac03b060 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f74a8009840 tx=0x7f74a8009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:09.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.686+0000 7fb19affd700 1 --2- 172.21.15.156:0/1670121363 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb18c068340 0x7fb18c06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.686+0000 7fb19affd700 1 -- 172.21.15.156:0/1670121363 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb1a4085730 con 0x7fb1ac15ac90 2023-11-28T13:08:09.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.687+0000 7fb19affd700 1 --2- 172.21.15.156:0/1670121363 >> [v2:172.21.15.156:6821/132480,v1:172.21.15.156:6823/132480] conn(0x7fb18c070bf0 0x7fb18c073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.686+0000 7f74b8b9d700 1 -- 172.21.15.156:0/2084527835 shutdown_connections 2023-11-28T13:08:09.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.686+0000 7f74b8b9d700 1 --2- 172.21.15.156:0/2084527835 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74ac006e70 0x7f74ac03b060 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.686+0000 7f74b8b9d700 1 --2- 172.21.15.156:0/2084527835 >> v2:172.21.15.186:3300/0 conn(0x7f74ac007390 0x7f74ac03ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.686+0000 7f74b8b9d700 1 -- 172.21.15.156:0/2084527835 >> 172.21.15.156:0/2084527835 conn(0x7f74ac055b20 msgr2=0x7f74ac055ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.687+0000 7efe77fff700 1 --2- 172.21.15.156:0/484838469 >> [v2:172.21.15.156:6814/132475,v1:172.21.15.156:6816/132475] conn(0x7efe64070c70 0x7efe64073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.687+0000 7fb19affd700 1 -- 172.21.15.156:0/1670121363 --> [v2:172.21.15.156:6821/132480,v1:172.21.15.156:6823/132480] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb18c0736b0 con 0x7fb18c070bf0 2023-11-28T13:08:09.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.687+0000 7fb19affd700 1 -- 172.21.15.156:0/1670121363 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1a4085bb0 con 0x7fb1ac15ac90 2023-11-28T13:08:09.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.687+0000 7fb1aa59c700 1 --2- 172.21.15.156:0/1670121363 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb18c068340 0x7fb18c06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.687+0000 7f24b5fff700 1 Processor -- start 2023-11-28T13:08:09.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.687+0000 7f74b8b9d700 1 -- 172.21.15.156:0/2084527835 shutdown_connections 2023-11-28T13:08:09.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.687+0000 7f74b8b9d700 1 -- 172.21.15.156:0/2084527835 wait complete. 2023-11-28T13:08:09.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.687+0000 7efe5ffff700 1 -- 172.21.15.156:0/484838469 --> [v2:172.21.15.156:6814/132475,v1:172.21.15.156:6816/132475] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe64073730 con 0x7efe64070c70 2023-11-28T13:08:09.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.687+0000 7efe5ffff700 1 -- 172.21.15.156:0/484838469 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe6c086850 con 0x7efe7815c8e0 2023-11-28T13:08:09.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.687+0000 7fb1a9d9b700 1 --2- 172.21.15.156:0/1670121363 >> [v2:172.21.15.156:6821/132480,v1:172.21.15.156:6823/132480] conn(0x7fb18c070bf0 0x7fb18c073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.686+0000 7f79d5ffb700 1 --2- 172.21.15.156:0/4036108884 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f79c40682f0 0x7f79c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.686+0000 7f79d5ffb700 1 -- 172.21.15.156:0/4036108884 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f79e0101200 con 0x7f79d804f980 2023-11-28T13:08:09.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.687+0000 7f79d5ffb700 1 --2- 172.21.15.156:0/4036108884 >> [v2:172.21.15.186:6816/132937,v1:172.21.15.186:6817/132937] conn(0x7f79c40709b0 0x7f79c4072dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.687+0000 7f79d5ffb700 1 -- 172.21.15.156:0/4036108884 --> [v2:172.21.15.186:6816/132937,v1:172.21.15.186:6817/132937] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79c4073470 con 0x7f79c40709b0 2023-11-28T13:08:09.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.687+0000 7f79d5ffb700 1 -- 172.21.15.156:0/4036108884 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79e0101580 con 0x7f79d804f980 2023-11-28T13:08:09.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.688+0000 7f9fd339d700 1 Processor -- start 2023-11-28T13:08:09.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.688+0000 7f24b5fff700 1 -- start start 2023-11-28T13:08:09.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.688+0000 7f9fd339d700 1 -- start start 2023-11-28T13:08:09.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.688+0000 7f79e539b700 1 --2- 172.21.15.156:0/4036108884 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f79c40682f0 0x7f79c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.688+0000 7f79e4b9a700 1 --2- 172.21.15.156:0/4036108884 >> [v2:172.21.15.186:6816/132937,v1:172.21.15.186:6817/132937] conn(0x7f79c40709b0 0x7f79c4072dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7fe347fff700 1 --2- 172.21.15.156:0/2298853731 >> [v2:172.21.15.156:6808/132471,v1:172.21.15.156:6809/132471] conn(0x7fe330070bf0 0x7fe330073000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7f9fd339d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fb8004d90 0x7f9fc417c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7f9fd339d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fc40d7050 0x7f9fc40e12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7f9fd339d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fc40e17e0 0x7f9fc4039b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7f9fd339d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc4039750 con 0x7f9fc40e17e0 2023-11-28T13:08:09.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7f9fd339d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc40e1c50 con 0x7f9fc40d7050 2023-11-28T13:08:09.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7f9fd339d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc40e04a0 con 0x7f9fb8004d90 2023-11-28T13:08:09.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7fb1aa59c700 1 --2- 172.21.15.156:0/1670121363 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb18c068340 0x7fb18c06a7f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0c8bd0 tx=0x7fb19c005000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7f24b5fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24b00ce480 0x7f24b00b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7f24b5fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24b0161220 0x7f24b00b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7f24b5fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b0164e60 0x7f24b01630d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7f24b5fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b00b4280 con 0x7f24b0161220 2023-11-28T13:08:09.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7f24b5fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b00b3950 con 0x7f24b00ce480 2023-11-28T13:08:09.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7f24b5fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b00b3c30 con 0x7f24b0164e60 2023-11-28T13:08:09.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7f9fd239b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fb8004d90 0x7f9fc417c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7f9fd239b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fb8004d90 0x7f9fc417c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36892/0 (socket says 172.21.15.156:36892) 2023-11-28T13:08:09.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7f9fd239b700 1 -- 172.21.15.156:0/438367030 learned_addr learned my addr 172.21.15.156:0/438367030 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:09.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.689+0000 7f9fd1b9a700 1 --2- 172.21.15.156:0/438367030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fc40d7050 0x7f9fc40e12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.690+0000 7f79e4b9a700 1 --2- 172.21.15.156:0/4036108884 >> [v2:172.21.15.186:6816/132937,v1:172.21.15.186:6817/132937] conn(0x7f79c40709b0 0x7f79c4072dc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.690+0000 7f9fd2b9c700 1 --2- 172.21.15.156:0/438367030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fc40e17e0 0x7f9fc4039b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.690+0000 7f79e539b700 1 --2- 172.21.15.156:0/4036108884 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f79c40682f0 0x7f79c406a7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f79d000b850 tx=0x7f79d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.690+0000 7f24affff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b0164e60 0x7f24b01630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.690+0000 7f24affff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b0164e60 0x7f24b01630d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36906/0 (socket says 172.21.15.156:36906) 2023-11-28T13:08:09.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.690+0000 7f24affff700 1 -- 172.21.15.156:0/1630745392 learned_addr learned my addr 172.21.15.156:0/1630745392 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:09.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.690+0000 7fe34dffb700 1 -- 172.21.15.156:0/2298853731 <== osd.0 v2:172.21.15.156:6808/132471 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe3300736b0 con 0x7fe330070bf0 2023-11-28T13:08:09.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.690+0000 7f24aeffd700 1 --2- 172.21.15.156:0/1630745392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24b0161220 0x7f24b00b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.690+0000 7f9fd1b9a700 1 -- 172.21.15.156:0/438367030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fb8004d90 msgr2=0x7f9fc417c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.690+0000 7f9fd1b9a700 1 --2- 172.21.15.156:0/438367030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fb8004d90 0x7f9fc417c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.690+0000 7f9fd1b9a700 1 -- 172.21.15.156:0/438367030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fc40e17e0 msgr2=0x7f9fc4039b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.690+0000 7f9fd1b9a700 1 --2- 172.21.15.156:0/438367030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fc40e17e0 0x7f9fc4039b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.690+0000 7f9fd1b9a700 1 -- 172.21.15.156:0/438367030 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc4033ee0 con 0x7f9fc40d7050 2023-11-28T13:08:09.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.690+0000 7f9fd2b9c700 1 --2- 172.21.15.156:0/438367030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fc40e17e0 0x7f9fc4039b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:09.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.690+0000 7f24af7fe700 1 --2- 172.21.15.156:0/1630745392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24b00ce480 0x7f24b00b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.691+0000 7f74b8b9d700 1 Processor -- start 2023-11-28T13:08:09.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.691+0000 7f24affff700 1 -- 172.21.15.156:0/1630745392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24b00ce480 msgr2=0x7f24b00b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.691+0000 7f24affff700 1 --2- 172.21.15.156:0/1630745392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24b00ce480 0x7f24b00b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.691+0000 7f24affff700 1 -- 172.21.15.156:0/1630745392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24b0161220 msgr2=0x7f24b00b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.691+0000 7fb1a9d9b700 1 --2- 172.21.15.156:0/1670121363 >> [v2:172.21.15.156:6821/132480,v1:172.21.15.156:6823/132480] conn(0x7fb18c070bf0 0x7fb18c073000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.691+0000 7f24affff700 1 --2- 172.21.15.156:0/1630745392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24b0161220 0x7f24b00b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.691+0000 7f24affff700 1 -- 172.21.15.156:0/1630745392 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a4009550 con 0x7f24b0164e60 2023-11-28T13:08:09.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.691+0000 7f24aeffd700 1 --2- 172.21.15.156:0/1630745392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24b0161220 0x7f24b00b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:09.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.691+0000 7f79d5ffb700 1 -- 172.21.15.156:0/4036108884 <== osd.4 v2:172.21.15.186:6816/132937 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f79c4073470 con 0x7f79c40709b0 2023-11-28T13:08:09.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.691+0000 7f24affff700 1 --2- 172.21.15.156:0/1630745392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b0164e60 0x7f24b01630d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f24a800c7a0 tx=0x7f24a800cab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.691+0000 7f74b8b9d700 1 -- start start 2023-11-28T13:08:09.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.691+0000 7efe77fff700 1 --2- 172.21.15.156:0/484838469 >> [v2:172.21.15.156:6814/132475,v1:172.21.15.156:6816/132475] conn(0x7efe64070c70 0x7efe64073080 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.693+0000 7efe5ffff700 1 -- 172.21.15.156:0/484838469 <== osd.2 v2:172.21.15.156:6814/132475 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7efe64073730 con 0x7efe64070c70 2023-11-28T13:08:09.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.692+0000 7f2497fff700 1 -- 172.21.15.156:0/1630745392 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8003e10 con 0x7f24b0164e60 2023-11-28T13:08:09.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.692+0000 7f2497fff700 1 -- 172.21.15.156:0/1630745392 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a8005070 con 0x7f24b0164e60 2023-11-28T13:08:09.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.692+0000 7f2497fff700 1 -- 172.21.15.156:0/1630745392 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8005420 con 0x7f24b0164e60 2023-11-28T13:08:09.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.691+0000 7f9fd1b9a700 1 --2- 172.21.15.156:0/438367030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fc40d7050 0x7f9fc40e12a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00c7c0 tx=0x7f9fbc00cad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.692+0000 7f9fc2ffd700 1 -- 172.21.15.156:0/438367030 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc011800 con 0x7f9fc40d7050 2023-11-28T13:08:09.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.692+0000 7f9fc2ffd700 1 -- 172.21.15.156:0/438367030 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fbc011960 con 0x7f9fc40d7050 2023-11-28T13:08:09.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.692+0000 7f9fc2ffd700 1 -- 172.21.15.156:0/438367030 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc011d10 con 0x7f9fc40d7050 2023-11-28T13:08:09.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.692+0000 7f9fd339d700 1 -- 172.21.15.156:0/438367030 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc4033c00 con 0x7f9fc40d7050 2023-11-28T13:08:09.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.693+0000 7f9fd339d700 1 -- 172.21.15.156:0/438367030 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc40341c0 con 0x7f9fc40d7050 2023-11-28T13:08:09.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.693+0000 7f74b8b9d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74ac006e70 0x7f74ac0df170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.693+0000 7f74b8b9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74ac007390 0x7f74ac04e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.693+0000 7f74b8b9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74ac0e6da0 0x7f74ac0e89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.693+0000 7f74b8b9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac17cc10 con 0x7f74ac0e6da0 2023-11-28T13:08:09.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.693+0000 7f74b8b9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac17cef0 con 0x7f74ac007390 2023-11-28T13:08:09.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.693+0000 7f74b8b9d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac17d1d0 con 0x7f74ac006e70 2023-11-28T13:08:09.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.693+0000 7f74b3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74ac0e6da0 0x7f74ac0e89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.694+0000 7f74b3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74ac0e6da0 0x7f74ac0e89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58900/0 (socket says 172.21.15.156:58900) 2023-11-28T13:08:09.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.694+0000 7f74b3fff700 1 -- 172.21.15.156:0/167571413 learned_addr learned my addr 172.21.15.156:0/167571413 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:09.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.694+0000 7f24b5fff700 1 -- 172.21.15.156:0/1630745392 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b0161c50 con 0x7f24b0164e60 2023-11-28T13:08:09.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.692+0000 7fb19affd700 1 -- 172.21.15.156:0/1670121363 <== osd.3 v2:172.21.15.156:6821/132480 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb18c0736b0 con 0x7fb18c070bf0 2023-11-28T13:08:09.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.694+0000 7f24b5fff700 1 -- 172.21.15.156:0/1630745392 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b0161970 con 0x7f24b0164e60 2023-11-28T13:08:09.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.694+0000 7f74b37fe700 1 --2- 172.21.15.156:0/167571413 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74ac006e70 0x7f74ac0df170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.694+0000 7f74b3fff700 1 -- 172.21.15.156:0/167571413 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74ac006e70 msgr2=0x7f74ac0df170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.694+0000 7f74b3fff700 1 --2- 172.21.15.156:0/167571413 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f74ac006e70 0x7f74ac0df170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.694+0000 7f74b3fff700 1 -- 172.21.15.156:0/167571413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74ac007390 msgr2=0x7f74ac04e170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T13:08:09.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.694+0000 7f74b3fff700 1 --2- 172.21.15.156:0/167571413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74ac007390 0x7f74ac04e170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.694+0000 7f74b3fff700 1 -- 172.21.15.156:0/167571413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a80094a0 con 0x7f74ac0e6da0 2023-11-28T13:08:09.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.695+0000 7f74b3fff700 1 --2- 172.21.15.156:0/167571413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74ac0e6da0 0x7f74ac0e89b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f74a800a010 tx=0x7f74a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.695+0000 7f24b5fff700 1 -- 172.21.15.156:0/1630745392 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f24b0161f30 con 0x7f24b0164e60 2023-11-28T13:08:09.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.696+0000 7f749ffff700 1 -- 172.21.15.156:0/167571413 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8011630 con 0x7f74ac0e6da0 2023-11-28T13:08:09.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.696+0000 7f749ffff700 1 -- 172.21.15.156:0/167571413 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a801f070 con 0x7f74ac0e6da0 2023-11-28T13:08:09.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.696+0000 7f749ffff700 1 -- 172.21.15.156:0/167571413 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a80119d0 con 0x7f74ac0e6da0 2023-11-28T13:08:09.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.696+0000 7f74b8b9d700 1 -- 172.21.15.156:0/167571413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ac006950 con 0x7f74ac0e6da0 2023-11-28T13:08:09.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.696+0000 7f9fc2ffd700 1 -- 172.21.15.156:0/438367030 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9fbc021070 con 0x7f9fc40d7050 2023-11-28T13:08:09.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.696+0000 7f9fd339d700 1 -- 172.21.15.156:0/438367030 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9fc40053d0 con 0x7f9fc40d7050 2023-11-28T13:08:09.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.697+0000 7f9fc2ffd700 1 --2- 172.21.15.156:0/438367030 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9fb0068340 0x7f9fb006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.696+0000 7f74b8b9d700 1 -- 172.21.15.156:0/167571413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac006670 con 0x7f74ac0e6da0 2023-11-28T13:08:09.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.697+0000 7f74b8b9d700 1 -- 172.21.15.156:0/167571413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f74ac03df40 con 0x7f74ac0e6da0 2023-11-28T13:08:09.755 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.698+0000 7f9fc2ffd700 1 -- 172.21.15.156:0/438367030 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9fbc05de90 con 0x7f9fc40d7050 2023-11-28T13:08:09.755 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.698+0000 7f9fd239b700 1 --2- 172.21.15.156:0/438367030 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9fb0068340 0x7f9fb006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.755 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.699+0000 7f9fd239b700 1 --2- 172.21.15.156:0/438367030 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9fb0068340 0x7f9fb006a7f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8006c90 tx=0x7f9fc8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.755 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.699+0000 7f9fc2ffd700 1 --2- 172.21.15.156:0/438367030 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x7f9fb00718d0 0x7f9fb0073d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.755 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.699+0000 7f9fc2ffd700 1 -- 172.21.15.156:0/438367030 --> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9fb0074430 con 0x7f9fb00718d0 2023-11-28T13:08:09.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.699+0000 7f9fc2ffd700 1 -- 172.21.15.156:0/438367030 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fbc00a3d0 con 0x7f9fc40d7050 2023-11-28T13:08:09.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.698+0000 7f2497fff700 1 -- 172.21.15.156:0/1630745392 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f24a8010070 con 0x7f24b0164e60 2023-11-28T13:08:09.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.699+0000 7f2497fff700 1 --2- 172.21.15.156:0/1630745392 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f249c0682f0 0x7f249c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.699+0000 7f2497fff700 1 -- 172.21.15.156:0/1630745392 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f24a8085230 con 0x7f24b0164e60 2023-11-28T13:08:09.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.700+0000 7f2497fff700 1 --2- 172.21.15.156:0/1630745392 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x7f249c070b20 0x7f249c072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.700+0000 7f2497fff700 1 -- 172.21.15.156:0/1630745392 --> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f249c0735e0 con 0x7f249c070b20 2023-11-28T13:08:09.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.700+0000 7f2497fff700 1 -- 172.21.15.156:0/1630745392 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24a80855a0 con 0x7f24b0164e60 2023-11-28T13:08:09.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.700+0000 7f9fd2b9c700 1 --2- 172.21.15.156:0/438367030 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x7f9fb00718d0 0x7f9fb0073d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.700+0000 7f749ffff700 1 -- 172.21.15.156:0/167571413 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f74a8017020 con 0x7f74ac0e6da0 2023-11-28T13:08:09.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.700+0000 7f24af7fe700 1 --2- 172.21.15.156:0/1630745392 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f249c0682f0 0x7f249c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.701+0000 7f24aeffd700 1 --2- 172.21.15.156:0/1630745392 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x7f249c070b20 0x7f249c072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.701+0000 7f9fd2b9c700 1 --2- 172.21.15.156:0/438367030 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x7f9fb00718d0 0x7f9fb0073d60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.701+0000 7f24aeffd700 1 --2- 172.21.15.156:0/1630745392 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x7f249c070b20 0x7f249c072f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.702+0000 7f749ffff700 1 --2- 172.21.15.156:0/167571413 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f74980683c0 0x7f749806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.702+0000 7f24af7fe700 1 --2- 172.21.15.156:0/1630745392 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f249c0682f0 0x7f249c06a7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009970 tx=0x7f24a0009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.702+0000 7f749ffff700 1 -- 172.21.15.156:0/167571413 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f74a8021070 con 0x7f74ac0e6da0 2023-11-28T13:08:09.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.702+0000 7f749ffff700 1 --2- 172.21.15.156:0/167571413 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x7f7498070c70 0x7f7498073080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:09.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.702+0000 7f749ffff700 1 -- 172.21.15.156:0/167571413 --> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7498073730 con 0x7f7498070c70 2023-11-28T13:08:09.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.702+0000 7f749ffff700 1 -- 172.21.15.156:0/167571413 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74a8088110 con 0x7f74ac0e6da0 2023-11-28T13:08:09.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.702+0000 7f74b2ffd700 1 --2- 172.21.15.156:0/167571413 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x7f7498070c70 0x7f7498073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.702+0000 7f74b37fe700 1 --2- 172.21.15.156:0/167571413 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f74980683c0 0x7f749806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:09.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.703+0000 7f2497fff700 1 -- 172.21.15.156:0/1630745392 <== osd.5 v2:172.21.15.186:6800/132935 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f249c0735e0 con 0x7f249c070b20 2023-11-28T13:08:09.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.703+0000 7f74b37fe700 1 --2- 172.21.15.156:0/167571413 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f74980683c0 0x7f749806a870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f74a0000f60 tx=0x7f74a0000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.705+0000 7f9fc2ffd700 1 -- 172.21.15.156:0/438367030 <== osd.6 v2:172.21.15.186:6824/132946 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9fb0074430 con 0x7f9fb00718d0 2023-11-28T13:08:09.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.705+0000 7f74b2ffd700 1 --2- 172.21.15.156:0/167571413 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x7f7498070c70 0x7f7498073080 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:09.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.706+0000 7f749ffff700 1 -- 172.21.15.156:0/167571413 <== osd.7 v2:172.21.15.186:6808/132936 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7498073730 con 0x7f7498070c70 2023-11-28T13:08:09.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.717+0000 7f5da43ff700 1 -- 172.21.15.156:0/2374934589 --> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5d84002d40 con 0x7f5d80070b70 2023-11-28T13:08:09.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.720+0000 7f5d92ffd700 1 -- 172.21.15.156:0/2374934589 <== osd.1 v2:172.21.15.156:6800/132479 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5d84002d40 con 0x7f5d80070b70 2023-11-28T13:08:09.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.721+0000 7f5da43ff700 1 -- 172.21.15.156:0/2374934589 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x7f5d80070b70 msgr2=0x7f5d80072f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.721+0000 7f5da43ff700 1 --2- 172.21.15.156:0/2374934589 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x7f5d80070b70 0x7f5d80072f80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.721+0000 7f79e75ff700 1 -- 172.21.15.156:0/4036108884 --> [v2:172.21.15.186:6816/132937,v1:172.21.15.186:6817/132937] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f79d80cbd70 con 0x7f79c40709b0 2023-11-28T13:08:09.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.722+0000 7f79d5ffb700 1 -- 172.21.15.156:0/4036108884 <== osd.4 v2:172.21.15.186:6816/132937 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f79dc001560 con 0x7f79c40709b0 2023-11-28T13:08:09.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.723+0000 7f79e75ff700 1 -- 172.21.15.156:0/4036108884 >> [v2:172.21.15.186:6816/132937,v1:172.21.15.186:6817/132937] conn(0x7f79c40709b0 msgr2=0x7f79c4072dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.723+0000 7f79e75ff700 1 --2- 172.21.15.156:0/4036108884 >> [v2:172.21.15.186:6816/132937,v1:172.21.15.186:6817/132937] conn(0x7f79c40709b0 0x7f79c4072dc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.723+0000 7f79e75ff700 1 -- 172.21.15.156:0/4036108884 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f79c40682f0 msgr2=0x7f79c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.723+0000 7f79e75ff700 1 --2- 172.21.15.156:0/4036108884 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f79c40682f0 0x7f79c406a7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f79d000b850 tx=0x7f79d0008040 comp rx=0 tx=0).stop 2023-11-28T13:08:09.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.723+0000 7f79e75ff700 1 -- 172.21.15.156:0/4036108884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79d804f980 msgr2=0x7f79d8180a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.723+0000 7f79e75ff700 1 --2- 172.21.15.156:0/4036108884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79d804f980 0x7f79d8180a60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f79e008afd0 tx=0x7f79e0068d70 comp rx=0 tx=0).stop 2023-11-28T13:08:09.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.723+0000 7f79e5b9c700 1 -- 172.21.15.156:0/4036108884 reap_dead start 2023-11-28T13:08:09.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.725+0000 7f5da43ff700 1 -- 172.21.15.156:0/2374934589 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5d80068340 msgr2=0x7f5d8006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.725+0000 7f5da43ff700 1 --2- 172.21.15.156:0/2374934589 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5d80068340 0x7f5d8006a7f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5d94007fb0 tx=0x7f5d94007560 comp rx=0 tx=0).stop 2023-11-28T13:08:09.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.725+0000 7f5da43ff700 1 -- 172.21.15.156:0/2374934589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d9c0af480 msgr2=0x7f5d9c0af8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.725+0000 7f5da43ff700 1 --2- 172.21.15.156:0/2374934589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d9c0af480 0x7f5d9c0af8f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00d620 tx=0x7f5d8c00d9e0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.725+0000 7f5da299c700 1 -- 172.21.15.156:0/2374934589 reap_dead start 2023-11-28T13:08:09.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.725+0000 7f5da43ff700 1 -- 172.21.15.156:0/2374934589 shutdown_connections 2023-11-28T13:08:09.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.725+0000 7f5da43ff700 1 -- 172.21.15.156:0/2374934589 >> 172.21.15.156:0/2374934589 conn(0x7f5d9c0b06f0 msgr2=0x7f5d9c0cb650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.725+0000 7f79e75ff700 1 -- 172.21.15.156:0/4036108884 shutdown_connections 2023-11-28T13:08:09.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.725+0000 7f79e75ff700 1 -- 172.21.15.156:0/4036108884 >> 172.21.15.156:0/4036108884 conn(0x7f79d8051a60 msgr2=0x7f79d8006a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.725+0000 7f5da43ff700 1 -- 172.21.15.156:0/2374934589 shutdown_connections 2023-11-28T13:08:09.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.725+0000 7f5da43ff700 1 -- 172.21.15.156:0/2374934589 wait complete. 2023-11-28T13:08:09.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.725+0000 7f79e75ff700 1 -- 172.21.15.156:0/4036108884 shutdown_connections 2023-11-28T13:08:09.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.725+0000 7f79e75ff700 1 -- 172.21.15.156:0/4036108884 wait complete. 2023-11-28T13:08:09.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.727+0000 7fb198ff9700 1 -- 172.21.15.156:0/1670121363 --> [v2:172.21.15.156:6821/132480,v1:172.21.15.156:6823/132480] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb184002d40 con 0x7fb18c070bf0 2023-11-28T13:08:09.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.732+0000 7efe7cd9d700 1 -- 172.21.15.156:0/484838469 --> [v2:172.21.15.156:6814/132475,v1:172.21.15.156:6816/132475] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efe58002d40 con 0x7efe64070c70 2023-11-28T13:08:09.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.732+0000 7f24b5fff700 1 -- 172.21.15.156:0/1630745392 --> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f24b00745c0 con 0x7f249c070b20 2023-11-28T13:08:09.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.733+0000 7fe355b9d700 1 -- 172.21.15.156:0/2298853731 --> [v2:172.21.15.156:6808/132471,v1:172.21.15.156:6809/132471] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe350061e80 con 0x7fe330070bf0 2023-11-28T13:08:09.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.733+0000 7fb19affd700 1 -- 172.21.15.156:0/1670121363 <== osd.3 v2:172.21.15.156:6821/132480 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb184002d40 con 0x7fb18c070bf0 2023-11-28T13:08:09.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.733+0000 7efe5ffff700 1 -- 172.21.15.156:0/484838469 <== osd.2 v2:172.21.15.156:6814/132475 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efe58002d40 con 0x7efe64070c70 2023-11-28T13:08:09.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.734+0000 7efe7cd9d700 1 -- 172.21.15.156:0/484838469 >> [v2:172.21.15.156:6814/132475,v1:172.21.15.156:6816/132475] conn(0x7efe64070c70 msgr2=0x7efe64073080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.734+0000 7efe7cd9d700 1 --2- 172.21.15.156:0/484838469 >> [v2:172.21.15.156:6814/132475,v1:172.21.15.156:6816/132475] conn(0x7efe64070c70 0x7efe64073080 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.734+0000 7efe7cd9d700 1 -- 172.21.15.156:0/484838469 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efe640683c0 msgr2=0x7efe6406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.734+0000 7efe7cd9d700 1 --2- 172.21.15.156:0/484838469 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efe640683c0 0x7efe6406a870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe68009c50 tx=0x7efe680092c0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.734+0000 7efe7cd9d700 1 -- 172.21.15.156:0/484838469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7815c8e0 msgr2=0x7efe78156b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.734+0000 7efe7cd9d700 1 --2- 172.21.15.156:0/484838469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7815c8e0 0x7efe78156b60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe6c01a000 tx=0x7efe6c004750 comp rx=0 tx=0).stop 2023-11-28T13:08:09.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.734+0000 7fb198ff9700 1 -- 172.21.15.156:0/1670121363 >> [v2:172.21.15.156:6821/132480,v1:172.21.15.156:6823/132480] conn(0x7fb18c070bf0 msgr2=0x7fb18c073000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.734+0000 7fb198ff9700 1 --2- 172.21.15.156:0/1670121363 >> [v2:172.21.15.156:6821/132480,v1:172.21.15.156:6823/132480] conn(0x7fb18c070bf0 0x7fb18c073000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.734+0000 7fb198ff9700 1 -- 172.21.15.156:0/1670121363 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb18c068340 msgr2=0x7fb18c06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.734+0000 7fb198ff9700 1 --2- 172.21.15.156:0/1670121363 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb18c068340 0x7fb18c06a7f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0c8bd0 tx=0x7fb19c005000 comp rx=0 tx=0).stop 2023-11-28T13:08:09.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.734+0000 7fb198ff9700 1 -- 172.21.15.156:0/1670121363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1ac15ac90 msgr2=0x7fb1ac0cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.734+0000 7fb198ff9700 1 --2- 172.21.15.156:0/1670121363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1ac15ac90 0x7fb1ac0cd6a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0cc610 tx=0x7fb1a400aa70 comp rx=0 tx=0).stop 2023-11-28T13:08:09.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.735+0000 7efe77fff700 1 -- 172.21.15.156:0/484838469 reap_dead start 2023-11-28T13:08:09.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.735+0000 7fe34dffb700 1 -- 172.21.15.156:0/2298853731 <== osd.0 v2:172.21.15.156:6808/132471 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe350061e80 con 0x7fe330070bf0 2023-11-28T13:08:09.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.735+0000 7efe7cd9d700 1 -- 172.21.15.156:0/484838469 shutdown_connections 2023-11-28T13:08:09.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.735+0000 7efe7cd9d700 1 -- 172.21.15.156:0/484838469 >> 172.21.15.156:0/484838469 conn(0x7efe780cb510 msgr2=0x7efe78070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.735+0000 7efe7cd9d700 1 -- 172.21.15.156:0/484838469 shutdown_connections 2023-11-28T13:08:09.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.734+0000 7fb1aad9d700 1 -- 172.21.15.156:0/1670121363 reap_dead start 2023-11-28T13:08:09.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.735+0000 7f2497fff700 1 -- 172.21.15.156:0/1630745392 <== osd.5 v2:172.21.15.186:6800/132935 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f24b00745c0 con 0x7f249c070b20 2023-11-28T13:08:09.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.735+0000 7efe7cd9d700 1 -- 172.21.15.156:0/484838469 wait complete. 2023-11-28T13:08:09.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.735+0000 7fb198ff9700 1 -- 172.21.15.156:0/1670121363 shutdown_connections 2023-11-28T13:08:09.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.735+0000 7fb198ff9700 1 -- 172.21.15.156:0/1670121363 >> 172.21.15.156:0/1670121363 conn(0x7fb1ac0b06f0 msgr2=0x7fb1ac0ca990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.735+0000 7fb198ff9700 1 -- 172.21.15.156:0/1670121363 shutdown_connections 2023-11-28T13:08:09.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.736+0000 7f24b5fff700 1 -- 172.21.15.156:0/1630745392 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x7f249c070b20 msgr2=0x7f249c072f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.736+0000 7f24b5fff700 1 --2- 172.21.15.156:0/1630745392 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x7f249c070b20 0x7f249c072f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.736+0000 7fb198ff9700 1 -- 172.21.15.156:0/1670121363 wait complete. 2023-11-28T13:08:09.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.736+0000 7f24b5fff700 1 -- 172.21.15.156:0/1630745392 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f249c0682f0 msgr2=0x7f249c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.736+0000 7f24b5fff700 1 --2- 172.21.15.156:0/1630745392 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f249c0682f0 0x7f249c06a7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009970 tx=0x7f24a0009110 comp rx=0 tx=0).stop 2023-11-28T13:08:09.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.736+0000 7f24b5fff700 1 -- 172.21.15.156:0/1630745392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b0164e60 msgr2=0x7f24b01630d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.736+0000 7f24b5fff700 1 --2- 172.21.15.156:0/1630745392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24b0164e60 0x7f24b01630d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f24a800c7a0 tx=0x7f24a800cab0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.736+0000 7fe355b9d700 1 -- 172.21.15.156:0/2298853731 >> [v2:172.21.15.156:6808/132471,v1:172.21.15.156:6809/132471] conn(0x7fe330070bf0 msgr2=0x7fe330073000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.736+0000 7fe355b9d700 1 --2- 172.21.15.156:0/2298853731 >> [v2:172.21.15.156:6808/132471,v1:172.21.15.156:6809/132471] conn(0x7fe330070bf0 0x7fe330073000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.737+0000 7f9fd339d700 1 -- 172.21.15.156:0/438367030 --> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9fc4039890 con 0x7f9fb00718d0 2023-11-28T13:08:09.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.737+0000 7fe355b9d700 1 -- 172.21.15.156:0/2298853731 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe3300683c0 msgr2=0x7fe33006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.737+0000 7fe355b9d700 1 --2- 172.21.15.156:0/2298853731 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe3300683c0 0x7fe33006a870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe3500b49e0 tx=0x7fe34000b040 comp rx=0 tx=0).stop 2023-11-28T13:08:09.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.737+0000 7fe355b9d700 1 -- 172.21.15.156:0/2298853731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe350158b80 msgr2=0x7fe3501e1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.737+0000 7fe355b9d700 1 --2- 172.21.15.156:0/2298853731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe350158b80 0x7fe3501e1d60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe34800b410 tx=0x7fe34800b720 comp rx=0 tx=0).stop 2023-11-28T13:08:09.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.737+0000 7f24affff700 1 -- 172.21.15.156:0/1630745392 reap_dead start 2023-11-28T13:08:09.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.737+0000 7f24b5fff700 1 -- 172.21.15.156:0/1630745392 shutdown_connections 2023-11-28T13:08:09.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.737+0000 7f24b5fff700 1 -- 172.21.15.156:0/1630745392 >> 172.21.15.156:0/1630745392 conn(0x7f24b00cb670 msgr2=0x7f24b015eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.737+0000 7f24b5fff700 1 -- 172.21.15.156:0/1630745392 shutdown_connections 2023-11-28T13:08:09.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.737+0000 7f24b5fff700 1 -- 172.21.15.156:0/1630745392 wait complete. 2023-11-28T13:08:09.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.737+0000 7fe35539c700 1 -- 172.21.15.156:0/2298853731 reap_dead start 2023-11-28T13:08:09.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.737+0000 7fe355b9d700 1 -- 172.21.15.156:0/2298853731 shutdown_connections 2023-11-28T13:08:09.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.737+0000 7fe355b9d700 1 -- 172.21.15.156:0/2298853731 >> 172.21.15.156:0/2298853731 conn(0x7fe3500cb510 msgr2=0x7fe350158170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.737+0000 7fe355b9d700 1 -- 172.21.15.156:0/2298853731 shutdown_connections 2023-11-28T13:08:09.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.737+0000 7f9fc2ffd700 1 -- 172.21.15.156:0/438367030 <== osd.6 v2:172.21.15.186:6824/132946 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9fc4039890 con 0x7f9fb00718d0 2023-11-28T13:08:09.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.737+0000 7fe355b9d700 1 -- 172.21.15.156:0/2298853731 wait complete. 2023-11-28T13:08:09.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.738+0000 7f9fd339d700 1 -- 172.21.15.156:0/438367030 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x7f9fb00718d0 msgr2=0x7f9fb0073d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.738+0000 7f9fd339d700 1 --2- 172.21.15.156:0/438367030 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x7f9fb00718d0 0x7f9fb0073d60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.738+0000 7f9fd339d700 1 -- 172.21.15.156:0/438367030 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9fb0068340 msgr2=0x7f9fb006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.738+0000 7f9fd339d700 1 --2- 172.21.15.156:0/438367030 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9fb0068340 0x7f9fb006a7f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8006c90 tx=0x7f9fc8008040 comp rx=0 tx=0).stop 2023-11-28T13:08:09.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.738+0000 7f9fd339d700 1 -- 172.21.15.156:0/438367030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fc40d7050 msgr2=0x7f9fc40e12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.738+0000 7f9fd339d700 1 --2- 172.21.15.156:0/438367030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fc40d7050 0x7f9fc40e12a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00c7c0 tx=0x7f9fbc00cad0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.738+0000 7f9fd2b9c700 1 -- 172.21.15.156:0/438367030 reap_dead start 2023-11-28T13:08:09.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.738+0000 7f74b8b9d700 1 -- 172.21.15.156:0/167571413 --> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f74ac17cd50 con 0x7f7498070c70 2023-11-28T13:08:09.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.738+0000 7f9fd339d700 1 -- 172.21.15.156:0/438367030 shutdown_connections 2023-11-28T13:08:09.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.738+0000 7f9fd339d700 1 -- 172.21.15.156:0/438367030 >> 172.21.15.156:0/438367030 conn(0x7f9fc4051610 msgr2=0x7f9fc4048720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.738+0000 7f9fd339d700 1 -- 172.21.15.156:0/438367030 shutdown_connections 2023-11-28T13:08:09.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.738+0000 7f9fd339d700 1 -- 172.21.15.156:0/438367030 wait complete. 2023-11-28T13:08:09.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.739+0000 7f749ffff700 1 -- 172.21.15.156:0/167571413 <== osd.7 v2:172.21.15.186:6808/132936 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f74ac17cd50 con 0x7f7498070c70 2023-11-28T13:08:09.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.739+0000 7f74b8b9d700 1 -- 172.21.15.156:0/167571413 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x7f7498070c70 msgr2=0x7f7498073080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.739+0000 7f74b8b9d700 1 --2- 172.21.15.156:0/167571413 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x7f7498070c70 0x7f7498073080 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.742+0000 7f74b8b9d700 1 -- 172.21.15.156:0/167571413 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f74980683c0 msgr2=0x7f749806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.742+0000 7f74b8b9d700 1 --2- 172.21.15.156:0/167571413 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f74980683c0 0x7f749806a870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f74a0000f60 tx=0x7f74a0000ed0 comp rx=0 tx=0).stop 2023-11-28T13:08:09.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.742+0000 7f74b8b9d700 1 -- 172.21.15.156:0/167571413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74ac0e6da0 msgr2=0x7f74ac0e89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:09.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.742+0000 7f74b8b9d700 1 --2- 172.21.15.156:0/167571413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74ac0e6da0 0x7f74ac0e89b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f74a800a010 tx=0x7f74a8004750 comp rx=0 tx=0).stop 2023-11-28T13:08:09.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.742+0000 7f74b3fff700 1 -- 172.21.15.156:0/167571413 reap_dead start 2023-11-28T13:08:09.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.743+0000 7f74b8b9d700 1 -- 172.21.15.156:0/167571413 shutdown_connections 2023-11-28T13:08:09.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.743+0000 7f74b8b9d700 1 -- 172.21.15.156:0/167571413 >> 172.21.15.156:0/167571413 conn(0x7f74ac055b20 msgr2=0x7f74ac03e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:09.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.743+0000 7f74b8b9d700 1 -- 172.21.15.156:0/167571413 shutdown_connections 2023-11-28T13:08:09.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:09.743+0000 7f74b8b9d700 1 -- 172.21.15.156:0/167571413 wait complete. 2023-11-28T13:08:09.784 INFO:teuthology.orchestra.run.smithi156.stdout:55834574852 2023-11-28T13:08:09.784 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T13:08:09.785 INFO:teuthology.orchestra.run.smithi156.stdout:55834574853 2023-11-28T13:08:09.786 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T13:08:09.786 INFO:teuthology.orchestra.run.smithi156.stdout:55834574852 2023-11-28T13:08:09.787 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T13:08:09.787 INFO:teuthology.orchestra.run.smithi156.stdout:55834574853 2023-11-28T13:08:09.788 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T13:08:09.789 INFO:teuthology.orchestra.run.smithi156.stdout:55834574853 2023-11-28T13:08:09.789 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T13:08:09.790 INFO:teuthology.orchestra.run.smithi156.stdout:55834574853 2023-11-28T13:08:09.790 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T13:08:09.791 INFO:teuthology.orchestra.run.smithi156.stdout:55834574854 2023-11-28T13:08:09.791 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T13:08:09.792 INFO:teuthology.orchestra.run.smithi156.stdout:55834574854 2023-11-28T13:08:09.792 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T13:08:10.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.174+0000 7fbb2afff700 1 Processor -- start 2023-11-28T13:08:10.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.174+0000 7f57e0dff700 1 Processor -- start 2023-11-28T13:08:10.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.174+0000 7f57e0dff700 1 -- start start 2023-11-28T13:08:10.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.174+0000 7fbb2afff700 1 -- start start 2023-11-28T13:08:10.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.176+0000 7fbb2afff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbb24158890 0x7fbb240ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.176+0000 7fbb2afff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbb2406d4f0 0x7fbb240b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.176+0000 7fbb2afff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2415c490 con 0x7fbb240b0bb0 2023-11-28T13:08:10.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.176+0000 7fbb2afff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb240b1050 con 0x7fbb2406d4f0 2023-11-28T13:08:10.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.176+0000 7fbb2afff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb240ce650 con 0x7fbb24158890 2023-11-28T13:08:10.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.177+0000 7fbb28d9b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbb24158890 0x7fbb240ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.177+0000 7fbb23fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbb2406d4f0 0x7fbb240b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.177+0000 7fbb28d9b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbb24158890 0x7fbb240ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:35074/0 (socket says 172.21.15.156:35074) 2023-11-28T13:08:10.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.177+0000 7fbb28d9b700 1 -- 172.21.15.156:0/3606486337 learned_addr learned my addr 172.21.15.156:0/3606486337 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:10.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.177+0000 7f57e0dff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f57dc0b0670 0x7f57dc0b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.177+0000 7f57e0dff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57dc0b1000 con 0x7f57dc0cc180 2023-11-28T13:08:10.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.177+0000 7f57e0dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57dc158d30 con 0x7f57dc06d4f0 2023-11-28T13:08:10.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.177+0000 7f57e0dff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57dc15f710 con 0x7f57dc0b0670 2023-11-28T13:08:10.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.179+0000 7fbb237fe700 1 -- 172.21.15.156:0/3606486337 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4103088503 0 0) 0x7fbb2415c490 con 0x7fbb240b0bb0 2023-11-28T13:08:10.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.179+0000 7fbb237fe700 1 -- 172.21.15.156:0/3606486337 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb0c0035b0 con 0x7fbb240b0bb0 2023-11-28T13:08:10.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.180+0000 7fbb23fff700 1 -- 172.21.15.156:0/3606486337 >> v2:172.21.15.186:3300/0 conn(0x7fbb24158890 msgr2=0x7fbb240ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:10.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.180+0000 7fbb23fff700 1 --2- 172.21.15.156:0/3606486337 >> v2:172.21.15.186:3300/0 conn(0x7fbb24158890 0x7fbb240ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.180+0000 7fbb23fff700 1 -- 172.21.15.156:0/3606486337 >> v1:172.21.15.186:6789/0 conn(0x7fbb240b0bb0 legacy=0x7fbb240cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.180+0000 7fbb23fff700 1 -- 172.21.15.156:0/3606486337 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2415c730 con 0x7fbb2406d4f0 2023-11-28T13:08:10.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.180+0000 7fbb23fff700 1 --2- 172.21.15.156:0/3606486337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb2406d4f0 0x7fbb240b0670 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbb240b17e0 tx=0x7fbb1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5b6e0245b973fd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.179+0000 7f57dad9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f57dc0b0670 0x7f57dc0b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.179+0000 7f57dad9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f57dc0b0670 0x7f57dc0b0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58922/0 (socket says 172.21.15.156:58922) 2023-11-28T13:08:10.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.179+0000 7f57dad9d700 1 -- 172.21.15.156:0/1900344369 learned_addr learned my addr 172.21.15.156:0/1900344369 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:10.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.180+0000 7f57d959a700 1 -- 172.21.15.156:0/1900344369 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 639617193 0 0) 0x7f57dc158d30 con 0x7f57dc06d4f0 2023-11-28T13:08:10.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.180+0000 7f57d959a700 1 -- 172.21.15.156:0/1900344369 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57c80035b0 con 0x7f57dc06d4f0 2023-11-28T13:08:10.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.175+0000 7fbcd139d700 1 Processor -- start 2023-11-28T13:08:10.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.176+0000 7fbcd139d700 1 -- start start 2023-11-28T13:08:10.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.178+0000 7fbcd139d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbccc158950 0x7fbccc0b0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.178+0000 7fbcd139d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbccc0b0a50 0x7fbccc0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.178+0000 7fbcd139d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc0cab60 0x7fbccc0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.178+0000 7fbcd139d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbccc0b0ed0 con 0x7fbccc0b0a50 2023-11-28T13:08:10.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.178+0000 7fbcd139d700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbccc0ae1b0 con 0x7fbccc158950 2023-11-28T13:08:10.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.178+0000 7fbcd139d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc15c430 con 0x7fbccc0cab60 2023-11-28T13:08:10.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.179+0000 7fbccbfff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbccc158950 0x7fbccc0b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.179+0000 7fbccbfff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbccc158950 0x7fbccc0b0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:35080/0 (socket says 172.21.15.156:35080) 2023-11-28T13:08:10.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.179+0000 7fbccbfff700 1 -- 172.21.15.156:0/105709608 learned_addr learned my addr 172.21.15.156:0/105709608 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:10.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.179+0000 7fbccb7fe700 1 --2- 172.21.15.156:0/105709608 >> v2:172.21.15.156:3300/0 conn(0x7fbccc0b0a50 0x7fbccc0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.179+0000 7fbcd0b9c700 1 --2- 172.21.15.156:0/105709608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc0cab60 0x7fbccc0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.181+0000 7f57dad9d700 1 -- 172.21.15.156:0/1900344369 >> v1:172.21.15.186:6789/0 conn(0x7f57dc06d4f0 legacy=0x7f57dc0ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.181+0000 7f57dad9d700 1 -- 172.21.15.156:0/1900344369 >> v1:172.21.15.156:6789/0 conn(0x7f57dc0cc180 legacy=0x7f57dc158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.181+0000 7fbccbfff700 1 -- 172.21.15.156:0/105709608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc0cab60 msgr2=0x7fbccc0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:10.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.181+0000 7fbccbfff700 1 --2- 172.21.15.156:0/105709608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc0cab60 0x7fbccc0cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.181+0000 7fbccbfff700 1 -- 172.21.15.156:0/105709608 >> v2:172.21.15.156:3300/0 conn(0x7fbccc0b0a50 msgr2=0x7fbccc0ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:10.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.181+0000 7fbccbfff700 1 --2- 172.21.15.156:0/105709608 >> v2:172.21.15.156:3300/0 conn(0x7fbccc0b0a50 0x7fbccc0ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.181+0000 7fbccbfff700 1 -- 172.21.15.156:0/105709608 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc15c710 con 0x7fbccc158950 2023-11-28T13:08:10.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.181+0000 7fbb237fe700 1 -- 172.21.15.156:0/3606486337 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c00e070 con 0x7fbb2406d4f0 2023-11-28T13:08:10.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.181+0000 7fbb237fe700 1 -- 172.21.15.156:0/3606486337 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c00bb20 con 0x7fbb2406d4f0 2023-11-28T13:08:10.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.181+0000 7fbccb7fe700 1 --2- 172.21.15.156:0/105709608 >> v2:172.21.15.156:3300/0 conn(0x7fbccc0b0a50 0x7fbccc0ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:10.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.182+0000 7fbb237fe700 1 -- 172.21.15.156:0/3606486337 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c00be90 con 0x7fbb2406d4f0 2023-11-28T13:08:10.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.182+0000 7fbccbfff700 1 --2- 172.21.15.156:0/105709608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbccc158950 0x7fbccc0b0510 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0cfb40 tx=0x7fbcbc00d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc6dcbd2a70db90f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.182+0000 7f57dad9d700 1 -- 172.21.15.156:0/1900344369 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc15c730 con 0x7f57dc0b0670 2023-11-28T13:08:10.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.182+0000 7fbb2afff700 1 -- 172.21.15.156:0/3606486337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb2406d4f0 msgr2=0x7fbb240b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.182+0000 7fbb2afff700 1 --2- 172.21.15.156:0/3606486337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb2406d4f0 0x7fbb240b0670 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbb240b17e0 tx=0x7fbb1c00b330 comp rx=0 tx=0).stop 2023-11-28T13:08:10.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.182+0000 7fbccaffd700 1 -- 172.21.15.156:0/105709608 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc010070 con 0x7fbccc158950 2023-11-28T13:08:10.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.182+0000 7fbccaffd700 1 -- 172.21.15.156:0/105709608 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcbc00d9d0 con 0x7fbccc158950 2023-11-28T13:08:10.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.182+0000 7fbccaffd700 1 -- 172.21.15.156:0/105709608 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc00dd60 con 0x7fbccc158950 2023-11-28T13:08:10.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.183+0000 7f57dad9d700 1 --2- 172.21.15.156:0/1900344369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57dc0b0670 0x7f57dc0b0ac0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f57c40098f0 tx=0x7f57c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdcbccd0a6d562 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.183+0000 7fbb2afff700 1 -- 172.21.15.156:0/3606486337 shutdown_connections 2023-11-28T13:08:10.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.183+0000 7fbb2afff700 1 --2- 172.21.15.156:0/3606486337 >> v2:172.21.15.186:3300/0 conn(0x7fbb24158890 0x7fbb240ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.183+0000 7fbb2afff700 1 --2- 172.21.15.156:0/3606486337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb2406d4f0 0x7fbb240b0670 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.183+0000 7fbb2afff700 1 -- 172.21.15.156:0/3606486337 >> 172.21.15.156:0/3606486337 conn(0x7fbb240cb670 msgr2=0x7fbb240cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:10.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.183+0000 7fbb2afff700 1 -- 172.21.15.156:0/3606486337 shutdown_connections 2023-11-28T13:08:10.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.183+0000 7fbb2afff700 1 -- 172.21.15.156:0/3606486337 wait complete. 2023-11-28T13:08:10.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.183+0000 7f57d959a700 1 -- 172.21.15.156:0/1900344369 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c4004230 con 0x7f57dc0b0670 2023-11-28T13:08:10.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.184+0000 7fbcd139d700 1 -- 172.21.15.156:0/105709608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbccc158950 msgr2=0x7fbccc0b0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.184+0000 7fbcd139d700 1 --2- 172.21.15.156:0/105709608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbccc158950 0x7fbccc0b0510 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0cfb40 tx=0x7fbcbc00d330 comp rx=0 tx=0).stop 2023-11-28T13:08:10.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.184+0000 7fbcd139d700 1 -- 172.21.15.156:0/105709608 shutdown_connections 2023-11-28T13:08:10.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.184+0000 7fbcd139d700 1 --2- 172.21.15.156:0/105709608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc0cab60 0x7fbccc0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.184+0000 7fbcd139d700 1 --2- 172.21.15.156:0/105709608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbccc158950 0x7fbccc0b0510 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.184+0000 7fbcd139d700 1 --2- 172.21.15.156:0/105709608 >> v2:172.21.15.156:3300/0 conn(0x7fbccc0b0a50 0x7fbccc0ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.184+0000 7f57d959a700 1 -- 172.21.15.156:0/1900344369 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c4004390 con 0x7f57dc0b0670 2023-11-28T13:08:10.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.184+0000 7f57d959a700 1 -- 172.21.15.156:0/1900344369 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40036a0 con 0x7f57dc0b0670 2023-11-28T13:08:10.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.184+0000 7f57e0dff700 1 -- 172.21.15.156:0/1900344369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57dc0b0670 msgr2=0x7f57dc0b0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.184+0000 7f57e0dff700 1 --2- 172.21.15.156:0/1900344369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57dc0b0670 0x7f57dc0b0ac0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f57c40098f0 tx=0x7f57c4009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:10.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.185+0000 7fbcd139d700 1 -- 172.21.15.156:0/105709608 >> 172.21.15.156:0/105709608 conn(0x7fbccc0cb510 msgr2=0x7fbccc0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:10.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.185+0000 7f57e0dff700 1 -- 172.21.15.156:0/1900344369 shutdown_connections 2023-11-28T13:08:10.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.185+0000 7f57e0dff700 1 --2- 172.21.15.156:0/1900344369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57dc0b0670 0x7f57dc0b0ac0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.193 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.185+0000 7f57e0dff700 1 -- 172.21.15.156:0/1900344369 >> 172.21.15.156:0/1900344369 conn(0x7f57dc0cb670 msgr2=0x7f57dc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:10.193 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.186+0000 7f57e0dff700 1 -- 172.21.15.156:0/1900344369 shutdown_connections 2023-11-28T13:08:10.193 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.186+0000 7f57e0dff700 1 -- 172.21.15.156:0/1900344369 wait complete. 2023-11-28T13:08:10.193 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.186+0000 7fbb2afff700 1 Processor -- start 2023-11-28T13:08:10.193 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.186+0000 7fbb2afff700 1 -- start start 2023-11-28T13:08:10.193 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.188+0000 7fbb2afff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb2406d4f0 0x7fbb240b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.193 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.188+0000 7fbb2afff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb240b0bb0 0x7fbb241571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.188+0000 7fbb2afff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb24158890 0x7fbb24157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.188+0000 7fbb2afff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb241f8780 con 0x7fbb24158890 2023-11-28T13:08:10.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.188+0000 7fbb2afff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb241f7b70 con 0x7fbb2406d4f0 2023-11-28T13:08:10.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.188+0000 7fbb2afff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb241f7e50 con 0x7fbb240b0bb0 2023-11-28T13:08:10.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.189+0000 7fbcd139d700 1 -- 172.21.15.156:0/105709608 shutdown_connections 2023-11-28T13:08:10.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.189+0000 7fbb2959c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb24158890 0x7fbb24157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.189+0000 7fbb2959c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb24158890 0x7fbb24157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58924/0 (socket says 172.21.15.156:58924) 2023-11-28T13:08:10.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.189+0000 7f57e0dff700 1 Processor -- start 2023-11-28T13:08:10.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.189+0000 7fbb2959c700 1 -- 172.21.15.156:0/1322909212 learned_addr learned my addr 172.21.15.156:0/1322909212 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:10.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.190+0000 7f57e0dff700 1 -- start start 2023-11-28T13:08:10.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.190+0000 7fbb2959c700 1 -- 172.21.15.156:0/1322909212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb240b0bb0 msgr2=0x7fbb241571e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T13:08:10.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.190+0000 7fbb2959c700 1 --2- 172.21.15.156:0/1322909212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb240b0bb0 0x7fbb241571e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.190+0000 7fbb2959c700 1 -- 172.21.15.156:0/1322909212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb2406d4f0 msgr2=0x7fbb240b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.190+0000 7fbb2959c700 1 --2- 172.21.15.156:0/1322909212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb2406d4f0 0x7fbb240b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.190+0000 7fbb2959c700 1 -- 172.21.15.156:0/1322909212 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c009a70 con 0x7fbb24158890 2023-11-28T13:08:10.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.191+0000 7fbb2959c700 1 --2- 172.21.15.156:0/1322909212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb24158890 0x7fbb24157720 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800c500 tx=0x7fbb1800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.191+0000 7f57e0dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57dc06d4f0 0x7f57dc0c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.191+0000 7f57e0dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57dc0b0670 0x7f57dc157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.192+0000 7fbb217fa700 1 -- 172.21.15.156:0/1322909212 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb180052d0 con 0x7fbb24158890 2023-11-28T13:08:10.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.192+0000 7fbb217fa700 1 -- 172.21.15.156:0/1322909212 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb18005450 con 0x7fbb24158890 2023-11-28T13:08:10.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.192+0000 7fbb217fa700 1 -- 172.21.15.156:0/1322909212 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb180038c0 con 0x7fbb24158890 2023-11-28T13:08:10.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.189+0000 7fbcd139d700 1 -- 172.21.15.156:0/105709608 wait complete. 2023-11-28T13:08:10.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.192+0000 7fbcd139d700 1 Processor -- start 2023-11-28T13:08:10.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.192+0000 7f57e0dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57dc0cc180 0x7f57dc143e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.192+0000 7f57da59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57dc06d4f0 0x7f57dc0c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.192+0000 7f57da59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57dc06d4f0 0x7f57dc0c12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:35100/0 (socket says 172.21.15.156:35100) 2023-11-28T13:08:10.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.192+0000 7f57da59c700 1 -- 172.21.15.156:0/3524638297 learned_addr learned my addr 172.21.15.156:0/3524638297 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:10.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.192+0000 7f57d9d9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57dc0b0670 0x7f57dc157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.192+0000 7f57e0dff700 1 -- 172.21.15.156:0/3524638297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0c0e20 con 0x7f57dc0cc180 2023-11-28T13:08:10.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.192+0000 7f57e0dff700 1 -- 172.21.15.156:0/3524638297 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc14d300 con 0x7f57dc06d4f0 2023-11-28T13:08:10.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.192+0000 7f57e0dff700 1 -- 172.21.15.156:0/3524638297 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc14d5a0 con 0x7f57dc0b0670 2023-11-28T13:08:10.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.193+0000 7f57dad9d700 1 --2- 172.21.15.156:0/3524638297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57dc0cc180 0x7f57dc143e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.193+0000 7fbb2afff700 1 -- 172.21.15.156:0/1322909212 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2415ccf0 con 0x7fbb24158890 2023-11-28T13:08:10.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.194+0000 7fbb2afff700 1 -- 172.21.15.156:0/1322909212 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2415ca10 con 0x7fbb24158890 2023-11-28T13:08:10.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.194+0000 7f57dad9d700 1 -- 172.21.15.156:0/3524638297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57dc0b0670 msgr2=0x7f57dc157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.194+0000 7f57dad9d700 1 --2- 172.21.15.156:0/3524638297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57dc0b0670 0x7f57dc157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.194+0000 7f57dad9d700 1 -- 172.21.15.156:0/3524638297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57dc06d4f0 msgr2=0x7f57dc0c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.194+0000 7f57dad9d700 1 --2- 172.21.15.156:0/3524638297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57dc06d4f0 0x7f57dc0c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.194+0000 7f57dad9d700 1 -- 172.21.15.156:0/3524638297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c4009550 con 0x7f57dc0cc180 2023-11-28T13:08:10.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.194+0000 7f57dad9d700 1 --2- 172.21.15.156:0/3524638297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57dc0cc180 0x7f57dc143e70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f57c4015000 tx=0x7f57c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.196+0000 7fbb2afff700 1 -- 172.21.15.156:0/1322909212 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb240c9370 con 0x7fbb24158890 2023-11-28T13:08:10.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.195+0000 7f57d2ffd700 1 -- 172.21.15.156:0/3524638297 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40045d0 con 0x7f57dc0cc180 2023-11-28T13:08:10.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.196+0000 7f57da59c700 1 --2- 172.21.15.156:0/3524638297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57dc06d4f0 0x7f57dc0c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:10.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.196+0000 7f57d2ffd700 1 -- 172.21.15.156:0/3524638297 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c4011630 con 0x7f57dc0cc180 2023-11-28T13:08:10.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.196+0000 7f57e0dff700 1 -- 172.21.15.156:0/3524638297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57dc15ccf0 con 0x7f57dc0cc180 2023-11-28T13:08:10.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.196+0000 7f57d2ffd700 1 -- 172.21.15.156:0/3524638297 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40119e0 con 0x7f57dc0cc180 2023-11-28T13:08:10.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.196+0000 7f57d9d9b700 1 --2- 172.21.15.156:0/3524638297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57dc0b0670 0x7f57dc157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:10.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.197+0000 7f57e0dff700 1 -- 172.21.15.156:0/3524638297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57dc15ca10 con 0x7f57dc0cc180 2023-11-28T13:08:10.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.196+0000 7fbcd139d700 1 -- start start 2023-11-28T13:08:10.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.197+0000 7fbcd139d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbccc0b0a50 0x7fbccc0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.197+0000 7fbcd139d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc0cab60 0x7fbccc1fe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.197+0000 7fbcd139d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbccc158950 0x7fbccc0af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.197+0000 7fbcd139d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0b2a20 con 0x7fbccc0b0a50 2023-11-28T13:08:10.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.197+0000 7fbcd139d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0b2180 con 0x7fbccc158950 2023-11-28T13:08:10.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.198+0000 7fbcd139d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0b2460 con 0x7fbccc0cab60 2023-11-28T13:08:10.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.198+0000 7fbcd0b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbccc158950 0x7fbccc0af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.198+0000 7fbcd0b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbccc158950 0x7fbccc0af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:35110/0 (socket says 172.21.15.156:35110) 2023-11-28T13:08:10.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.198+0000 7fbcd0b9c700 1 -- 172.21.15.156:0/3253743166 learned_addr learned my addr 172.21.15.156:0/3253743166 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:10.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.198+0000 7fbccb7fe700 1 --2- 172.21.15.156:0/3253743166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc0cab60 0x7fbccc1fe7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.198+0000 7fbb217fa700 1 -- 172.21.15.156:0/1322909212 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbb18019030 con 0x7fbb24158890 2023-11-28T13:08:10.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.198+0000 7fbccbfff700 1 --2- 172.21.15.156:0/3253743166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbccc0b0a50 0x7fbccc0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.198+0000 7fbcd0b9c700 1 -- 172.21.15.156:0/3253743166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc0cab60 msgr2=0x7fbccc1fe7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.198+0000 7fbcd0b9c700 1 --2- 172.21.15.156:0/3253743166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc0cab60 0x7fbccc1fe7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.198+0000 7fbcd0b9c700 1 -- 172.21.15.156:0/3253743166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbccc0b0a50 msgr2=0x7fbccc0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.198+0000 7fbcd0b9c700 1 --2- 172.21.15.156:0/3253743166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbccc0b0a50 0x7fbccc0b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.198+0000 7fbcd0b9c700 1 -- 172.21.15.156:0/3253743166 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc00094a0 con 0x7fbccc158950 2023-11-28T13:08:10.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.199+0000 7fbcd0b9c700 1 --2- 172.21.15.156:0/3253743166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbccc158950 0x7fbccc0af540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400efc0 tx=0x7fbcc400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.199+0000 7fbcc8ff9700 1 -- 172.21.15.156:0/3253743166 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc400cd40 con 0x7fbccc158950 2023-11-28T13:08:10.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.200+0000 7fbcc8ff9700 1 -- 172.21.15.156:0/3253743166 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc400f040 con 0x7fbccc158950 2023-11-28T13:08:10.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.199+0000 7fbb217fa700 1 --2- 172.21.15.156:0/1322909212 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbb100683c0 0x7fbb1006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.200+0000 7fbcd139d700 1 -- 172.21.15.156:0/3253743166 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcbc0085d0 con 0x7fbccc158950 2023-11-28T13:08:10.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.199+0000 7fbb217fa700 1 -- 172.21.15.156:0/1322909212 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbb180850c0 con 0x7fbb24158890 2023-11-28T13:08:10.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.199+0000 7f57d0ff9700 1 -- 172.21.15.156:0/3524638297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57dc0c9370 con 0x7f57dc0cc180 2023-11-28T13:08:10.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.200+0000 7fbcc8ff9700 1 -- 172.21.15.156:0/3253743166 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40146c0 con 0x7fbccc158950 2023-11-28T13:08:10.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.201+0000 7fbcd139d700 1 -- 172.21.15.156:0/3253743166 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbccc15ccd0 con 0x7fbccc158950 2023-11-28T13:08:10.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.202+0000 7f9e103ff700 1 Processor -- start 2023-11-28T13:08:10.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.202+0000 7fbb28d9b700 1 --2- 172.21.15.156:0/1322909212 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbb100683c0 0x7fbb1006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.202+0000 7fcd0659d700 1 Processor -- start 2023-11-28T13:08:10.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.203+0000 7fbcb27fc700 1 -- 172.21.15.156:0/3253743166 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca40052f0 con 0x7fbccc158950 2023-11-28T13:08:10.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.203+0000 7fbb28d9b700 1 --2- 172.21.15.156:0/1322909212 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbb100683c0 0x7fbb1006a870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbb14006d50 tx=0x7fbb14006c80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.202+0000 7f9e103ff700 1 -- start start 2023-11-28T13:08:10.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.204+0000 7fbcc8ff9700 1 -- 172.21.15.156:0/3253743166 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbcc4014820 con 0x7fbccc158950 2023-11-28T13:08:10.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.204+0000 7f9e103ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9e080d0c20 0x7f9e080cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.204+0000 7f9e103ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e080c6030 0x7f9e080d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.202+0000 7fcd0659d700 1 -- start start 2023-11-28T13:08:10.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.204+0000 7fcd0659d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fcd00160c00 0x7fcd000ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.204+0000 7fcd0659d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd000adf00 0x7fcd000cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.204+0000 7fcd0659d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcd000ce080 0x7fcd001647e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.204+0000 7fcd0659d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd000ae370 con 0x7fcd000ce080 2023-11-28T13:08:10.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.204+0000 7fcd0659d700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd000cd2d0 con 0x7fcd00160c00 2023-11-28T13:08:10.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.205+0000 7fcd0659d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd000cc700 con 0x7fcd000adf00 2023-11-28T13:08:10.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.205+0000 7f9e103ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e080d1530 con 0x7f9e080c6580 2023-11-28T13:08:10.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.205+0000 7f9e103ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e080cf080 con 0x7f9e080d0c20 2023-11-28T13:08:10.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.205+0000 7f9e103ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e080b49e0 con 0x7f9e080c6030 2023-11-28T13:08:10.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.203+0000 7f9e7d19d700 1 Processor -- start 2023-11-28T13:08:10.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.203+0000 7f9e7d19d700 1 -- start start 2023-11-28T13:08:10.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.205+0000 7f9e7d19d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e78160fb0 0x7f9e780b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.205+0000 7f9e7d19d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9e780b3500 0x7f9e7816aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.205+0000 7f9e7d19d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e780c5ba0 con 0x7f9e780b29a0 2023-11-28T13:08:10.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.205+0000 7f9e7d19d700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e780b1fd0 con 0x7f9e780b3500 2023-11-28T13:08:10.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.205+0000 7f9e7d19d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78161380 con 0x7f9e78160fb0 2023-11-28T13:08:10.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.205+0000 7fcd04d9a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd000adf00 0x7fcd000cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.205+0000 7fcd04d9a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd000adf00 0x7fcd000cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36978/0 (socket says 172.21.15.156:36978) 2023-11-28T13:08:10.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.205+0000 7fcd04d9a700 1 -- 172.21.15.156:0/518876154 learned_addr learned my addr 172.21.15.156:0/518876154 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:10.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.206+0000 7fcd0559b700 1 --2- 172.21.15.156:0/518876154 >> v2:172.21.15.186:3300/0 conn(0x7fcd00160c00 0x7fcd000ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.206+0000 7fcd04d9a700 1 -- 172.21.15.156:0/518876154 >> v2:172.21.15.186:3300/0 conn(0x7fcd00160c00 msgr2=0x7fcd000ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:10.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.206+0000 7fcd04d9a700 1 --2- 172.21.15.156:0/518876154 >> v2:172.21.15.186:3300/0 conn(0x7fcd00160c00 0x7fcd000ad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.206+0000 7fcd04d9a700 1 -- 172.21.15.156:0/518876154 >> v2:172.21.15.156:3300/0 conn(0x7fcd000ce080 msgr2=0x7fcd001647e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:10.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.206+0000 7fcd04d9a700 1 --2- 172.21.15.156:0/518876154 >> v2:172.21.15.156:3300/0 conn(0x7fcd000ce080 0x7fcd001647e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.206+0000 7fcd04d9a700 1 -- 172.21.15.156:0/518876154 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd000cc9a0 con 0x7fcd000adf00 2023-11-28T13:08:10.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.206+0000 7f9e77fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e78160fb0 0x7f9e780b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.206+0000 7f9e77fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e78160fb0 0x7f9e780b2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36980/0 (socket says 172.21.15.156:36980) 2023-11-28T13:08:10.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.206+0000 7f9e77fff700 1 -- 172.21.15.156:0/1086668666 learned_addr learned my addr 172.21.15.156:0/1086668666 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:10.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.206+0000 7f9e7c99c700 1 --2- 172.21.15.156:0/1086668666 >> v2:172.21.15.186:3300/0 conn(0x7f9e780b3500 0x7f9e7816aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.206+0000 7fbcc8ff9700 1 --2- 172.21.15.156:0/3253743166 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcb80682f0 0x7fbcb806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.206+0000 7fbcc8ff9700 1 -- 172.21.15.156:0/3253743166 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbcc408e350 con 0x7fbccc158950 2023-11-28T13:08:10.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.206+0000 7fbccbfff700 1 --2- 172.21.15.156:0/3253743166 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcb80682f0 0x7fbcb806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.206+0000 7f9e0d99a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e080c6030 0x7f9e080d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.206+0000 7f9e0d99a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e080c6030 0x7f9e080d0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36982/0 (socket says 172.21.15.156:36982) 2023-11-28T13:08:10.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.206+0000 7f9e0d99a700 1 -- 172.21.15.156:0/832151156 learned_addr learned my addr 172.21.15.156:0/832151156 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:10.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.207+0000 7fcd04d9a700 1 --2- 172.21.15.156:0/518876154 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd000adf00 0x7fcd000cdb40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcd0006f350 tx=0x7fccf0009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4f7ee148e830c6d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.207+0000 7f9e7c99c700 1 -- 172.21.15.156:0/1086668666 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e78160fb0 msgr2=0x7f9e780b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:10.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.207+0000 7f9e7c99c700 1 --2- 172.21.15.156:0/1086668666 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e78160fb0 0x7f9e780b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.207+0000 7f9e7c99c700 1 -- 172.21.15.156:0/1086668666 >> v1:172.21.15.156:6789/0 conn(0x7f9e780b29a0 legacy=0x7f9e780b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:10.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.207+0000 7f9e7c99c700 1 -- 172.21.15.156:0/1086668666 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e780c6290 con 0x7f9e780b3500 2023-11-28T13:08:10.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.207+0000 7fccf7fff700 1 -- 172.21.15.156:0/518876154 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf000b870 con 0x7fcd000adf00 2023-11-28T13:08:10.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.207+0000 7f9e0d99a700 1 -- 172.21.15.156:0/832151156 >> v2:172.21.15.156:3300/0 conn(0x7f9e080d0c20 msgr2=0x7f9e080cf4a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T13:08:10.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.207+0000 7f9e0d99a700 1 --2- 172.21.15.156:0/832151156 >> v2:172.21.15.156:3300/0 conn(0x7f9e080d0c20 0x7f9e080cf4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.207+0000 7f9e0d99a700 1 -- 172.21.15.156:0/832151156 >> v1:172.21.15.156:6789/0 conn(0x7f9e080c6580 legacy=0x7f9e080b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:10.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.207+0000 7f9e0d99a700 1 -- 172.21.15.156:0/832151156 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e080ce400 con 0x7f9e080c6030 2023-11-28T13:08:10.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.207+0000 7f9e7c99c700 1 --2- 172.21.15.156:0/1086668666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e780b3500 0x7f9e7816aad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009d90 tx=0x7f9e7000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c155be9fc3580f68 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.207+0000 7fccf7fff700 1 -- 172.21.15.156:0/518876154 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf000b9d0 con 0x7fcd000adf00 2023-11-28T13:08:10.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.207+0000 7fbccbfff700 1 --2- 172.21.15.156:0/3253743166 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcb80682f0 0x7fbcb806a7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00d370 tx=0x7fbcbc012000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.207+0000 7fccf7fff700 1 -- 172.21.15.156:0/518876154 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf000bd60 con 0x7fcd000adf00 2023-11-28T13:08:10.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.207+0000 7f9e0d99a700 1 --2- 172.21.15.156:0/832151156 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e080c6030 0x7f9e080d0ff0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9e040098f0 tx=0x7f9e04009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69b0329336d18c25 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.208+0000 7fbb217fa700 1 -- 172.21.15.156:0/1322909212 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbb180518b0 con 0x7fbb24158890 2023-11-28T13:08:10.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.208+0000 7f9e0d199700 1 -- 172.21.15.156:0/832151156 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e04004230 con 0x7f9e080c6030 2023-11-28T13:08:10.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.208+0000 7f9e0d199700 1 -- 172.21.15.156:0/832151156 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e04004390 con 0x7f9e080c6030 2023-11-28T13:08:10.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.208+0000 7fcd0659d700 1 -- 172.21.15.156:0/518876154 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd000adf00 msgr2=0x7fcd000cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.208+0000 7fcd0659d700 1 --2- 172.21.15.156:0/518876154 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd000adf00 0x7fcd000cdb40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcd0006f350 tx=0x7fccf0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.208+0000 7fcd0659d700 1 -- 172.21.15.156:0/518876154 shutdown_connections 2023-11-28T13:08:10.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.208+0000 7fcd0659d700 1 --2- 172.21.15.156:0/518876154 >> v2:172.21.15.156:3300/0 conn(0x7fcd000ce080 0x7fcd001647e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.208+0000 7fcd0659d700 1 --2- 172.21.15.156:0/518876154 >> v2:172.21.15.186:3300/0 conn(0x7fcd00160c00 0x7fcd000ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.208+0000 7fcd0659d700 1 --2- 172.21.15.156:0/518876154 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd000adf00 0x7fcd000cdb40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.208+0000 7fcd0659d700 1 -- 172.21.15.156:0/518876154 >> 172.21.15.156:0/518876154 conn(0x7fcd000cb510 msgr2=0x7fcd000cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:10.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.208+0000 7fcd0659d700 1 -- 172.21.15.156:0/518876154 shutdown_connections 2023-11-28T13:08:10.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.209+0000 7fcd0659d700 1 -- 172.21.15.156:0/518876154 wait complete. 2023-11-28T13:08:10.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.209+0000 7f9e0d199700 1 -- 172.21.15.156:0/832151156 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e040036a0 con 0x7f9e080c6030 2023-11-28T13:08:10.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.209+0000 7f9e103ff700 1 -- 172.21.15.156:0/832151156 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e080c6030 msgr2=0x7f9e080d0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.209+0000 7f9e103ff700 1 --2- 172.21.15.156:0/832151156 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e080c6030 0x7f9e080d0ff0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9e040098f0 tx=0x7f9e04009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:10.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.209+0000 7f9e103ff700 1 -- 172.21.15.156:0/832151156 shutdown_connections 2023-11-28T13:08:10.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.209+0000 7f9e103ff700 1 --2- 172.21.15.156:0/832151156 >> v2:172.21.15.156:3300/0 conn(0x7f9e080d0c20 0x7f9e080cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.209+0000 7f9e103ff700 1 --2- 172.21.15.156:0/832151156 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e080c6030 0x7f9e080d0ff0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.209+0000 7f9e103ff700 1 -- 172.21.15.156:0/832151156 >> 172.21.15.156:0/832151156 conn(0x7f9e0809c3f0 msgr2=0x7f9e080d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:10.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.209+0000 7f9e103ff700 1 -- 172.21.15.156:0/832151156 shutdown_connections 2023-11-28T13:08:10.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.209+0000 7f9e103ff700 1 -- 172.21.15.156:0/832151156 wait complete. 2023-11-28T13:08:10.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.211+0000 7fbcc8ff9700 1 -- 172.21.15.156:0/3253743166 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbcc408b060 con 0x7fbccc158950 2023-11-28T13:08:10.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.211+0000 7f57d2ffd700 1 -- 172.21.15.156:0/3524638297 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f57c4017020 con 0x7f57dc0cc180 2023-11-28T13:08:10.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.211+0000 7f9e103ff700 1 Processor -- start 2023-11-28T13:08:10.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.211+0000 7f9e103ff700 1 -- start start 2023-11-28T13:08:10.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.212+0000 7f9e103ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e080c6030 0x7f9e080c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.212+0000 7f9e103ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e080c6580 0x7f9e080cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.212+0000 7f9e103ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e080d0c20 0x7f9e080cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.212+0000 7f9e103ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e080cb170 con 0x7f9e080d0c20 2023-11-28T13:08:10.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.212+0000 7f9e103ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e081ffad0 con 0x7f9e080c6030 2023-11-28T13:08:10.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.212+0000 7f9e103ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e080cb380 con 0x7f9e080c6580 2023-11-28T13:08:10.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.213+0000 7f9e0e19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e080c6030 0x7f9e080c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.213+0000 7f45fc1ff700 1 Processor -- start 2023-11-28T13:08:10.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.213+0000 7f45fc1ff700 1 -- start start 2023-11-28T13:08:10.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.211+0000 7f9e76ffd700 1 -- 172.21.15.156:0/1086668666 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7000e070 con 0x7f9e780b3500 2023-11-28T13:08:10.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.211+0000 7f9e76ffd700 1 -- 172.21.15.156:0/1086668666 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7000bae0 con 0x7f9e780b3500 2023-11-28T13:08:10.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.211+0000 7f9e76ffd700 1 -- 172.21.15.156:0/1086668666 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7000be90 con 0x7f9e780b3500 2023-11-28T13:08:10.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.212+0000 7f9e7d19d700 1 -- 172.21.15.156:0/1086668666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e780b3500 msgr2=0x7f9e7816aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.212+0000 7f9e7d19d700 1 --2- 172.21.15.156:0/1086668666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e780b3500 0x7f9e7816aad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009d90 tx=0x7f9e7000b330 comp rx=0 tx=0).stop 2023-11-28T13:08:10.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.213+0000 7f9e7d19d700 1 -- 172.21.15.156:0/1086668666 shutdown_connections 2023-11-28T13:08:10.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.213+0000 7f9e7d19d700 1 --2- 172.21.15.156:0/1086668666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e780b3500 0x7f9e7816aad0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.213+0000 7f9e7d19d700 1 --2- 172.21.15.156:0/1086668666 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e78160fb0 0x7f9e780b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.213+0000 7f9e7d19d700 1 -- 172.21.15.156:0/1086668666 >> 172.21.15.156:0/1086668666 conn(0x7f9e7809c1d0 msgr2=0x7f9e780d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:10.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.211+0000 7fcd0659d700 1 Processor -- start 2023-11-28T13:08:10.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.212+0000 7fcd0659d700 1 -- start start 2023-11-28T13:08:10.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.213+0000 7f9e0e19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e080c6030 0x7f9e080c9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:35134/0 (socket says 172.21.15.156:35134) 2023-11-28T13:08:10.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.213+0000 7f9e0e19b700 1 -- 172.21.15.156:0/1692853067 learned_addr learned my addr 172.21.15.156:0/1692853067 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:10.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.213+0000 7f9e0d99a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e080c6580 0x7f9e080cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.213+0000 7f9e0e19b700 1 -- 172.21.15.156:0/1692853067 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e080c6580 msgr2=0x7f9e080cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.213+0000 7f9e0e19b700 1 --2- 172.21.15.156:0/1692853067 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e080c6580 0x7f9e080cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.213+0000 7f9e0e19b700 1 -- 172.21.15.156:0/1692853067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e080d0c20 msgr2=0x7f9e080cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.213+0000 7f9e0e19b700 1 --2- 172.21.15.156:0/1692853067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e080d0c20 0x7f9e080cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.213+0000 7f9e0e19b700 1 -- 172.21.15.156:0/1692853067 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e04009550 con 0x7f9e080c6030 2023-11-28T13:08:10.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.213+0000 7fcd0659d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd000ce080 0x7fcd00142f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.214+0000 7fcd0659d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd00160c00 0x7fcd00143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.214+0000 7f9e0e19b700 1 --2- 172.21.15.156:0/1692853067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e080c6030 0x7f9e080c9390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000ba50 tx=0x7f9e0000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.214+0000 7f9e7d19d700 1 -- 172.21.15.156:0/1086668666 shutdown_connections 2023-11-28T13:08:10.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.214+0000 7fcd0559b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd000ce080 0x7fcd00142f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.214+0000 7fcd0559b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd000ce080 0x7fcd00142f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:36998/0 (socket says 172.21.15.156:36998) 2023-11-28T13:08:10.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.214+0000 7fcd0559b700 1 -- 172.21.15.156:0/1713603699 learned_addr learned my addr 172.21.15.156:0/1713603699 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:10.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.214+0000 7f9e7d19d700 1 -- 172.21.15.156:0/1086668666 wait complete. 2023-11-28T13:08:10.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.214+0000 7fcd0659d700 1 --2- 172.21.15.156:0/1713603699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd000b18c0 0x7fcd001449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.214+0000 7fcd0659d700 1 -- 172.21.15.156:0/1713603699 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd00143b40 con 0x7fcd000b18c0 2023-11-28T13:08:10.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.214+0000 7fcd0659d700 1 -- 172.21.15.156:0/1713603699 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd000b22a0 con 0x7fcd00160c00 2023-11-28T13:08:10.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.214+0000 7fcd0659d700 1 -- 172.21.15.156:0/1713603699 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd001434f0 con 0x7fcd000ce080 2023-11-28T13:08:10.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.214+0000 7fcd04d9a700 1 --2- 172.21.15.156:0/1713603699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd00160c00 0x7fcd00143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.215+0000 7fcd04d9a700 1 -- 172.21.15.156:0/1713603699 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd000ce080 msgr2=0x7fcd00142f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.215+0000 7fcd04d9a700 1 --2- 172.21.15.156:0/1713603699 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd000ce080 0x7fcd00142f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.215+0000 7fcd04d9a700 1 -- 172.21.15.156:0/1713603699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd000b18c0 msgr2=0x7fcd001449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.215+0000 7fcd04d9a700 1 --2- 172.21.15.156:0/1713603699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd000b18c0 0x7fcd001449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.215+0000 7fcd04d9a700 1 -- 172.21.15.156:0/1713603699 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf0009550 con 0x7fcd00160c00 2023-11-28T13:08:10.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.215+0000 7f45fc1ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f45f40cb210 0x7f45f406b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.215+0000 7fcd04d9a700 1 --2- 172.21.15.156:0/1713603699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd00160c00 0x7fcd00143fd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fccf0016000 tx=0x7fccf0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.214+0000 7f9dfeffd700 1 -- 172.21.15.156:0/1692853067 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0000c9b0 con 0x7f9e080c6030 2023-11-28T13:08:10.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.214+0000 7f9dfeffd700 1 -- 172.21.15.156:0/1692853067 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0000cb30 con 0x7f9e080c6030 2023-11-28T13:08:10.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.214+0000 7f9dfeffd700 1 -- 172.21.15.156:0/1692853067 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00014440 con 0x7f9e080c6030 2023-11-28T13:08:10.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.216+0000 7f9e103ff700 1 -- 172.21.15.156:0/1692853067 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e080ce9c0 con 0x7f9e080c6030 2023-11-28T13:08:10.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.216+0000 7f45fc1ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f45f40ce7d0 0x7f45f415c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.216+0000 7f9e103ff700 1 -- 172.21.15.156:0/1692853067 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e080ce6e0 con 0x7f9e080c6030 2023-11-28T13:08:10.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.216+0000 7f45f9f9b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f45f40cb210 0x7f45f406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.216+0000 7f45f9f9b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f45f40cb210 0x7f45f406b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:35164/0 (socket says 172.21.15.156:35164) 2023-11-28T13:08:10.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.216+0000 7f45f9f9b700 1 -- 172.21.15.156:0/2500334689 learned_addr learned my addr 172.21.15.156:0/2500334689 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:10.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.216+0000 7f45fc1ff700 1 -- 172.21.15.156:0/2500334689 --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f406bd60 con 0x7f45f40b0a30 2023-11-28T13:08:10.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.216+0000 7f45fc1ff700 1 -- 172.21.15.156:0/2500334689 --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f4159d30 con 0x7f45f40ce7d0 2023-11-28T13:08:10.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.216+0000 7f45fc1ff700 1 -- 172.21.15.156:0/2500334689 --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f41591b0 con 0x7f45f40cb210 2023-11-28T13:08:10.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.216+0000 7f45fa79c700 1 --2- 172.21.15.156:0/2500334689 >> v2:172.21.15.156:3300/0 conn(0x7f45f40ce7d0 0x7f45f415c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.217+0000 7f45f9f9b700 1 -- 172.21.15.156:0/2500334689 >> v2:172.21.15.156:3300/0 conn(0x7f45f40ce7d0 msgr2=0x7f45f415c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:10.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.217+0000 7f45f9f9b700 1 --2- 172.21.15.156:0/2500334689 >> v2:172.21.15.156:3300/0 conn(0x7f45f40ce7d0 0x7f45f415c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.217+0000 7f45f9f9b700 1 -- 172.21.15.156:0/2500334689 >> v1:172.21.15.186:6789/0 conn(0x7f45f40b0a30 legacy=0x7f45f40ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.217+0000 7f45f9f9b700 1 -- 172.21.15.156:0/2500334689 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f4159490 con 0x7f45f40cb210 2023-11-28T13:08:10.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.217+0000 7f57d2ffd700 1 --2- 172.21.15.156:0/3524638297 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57bc0682f0 0x7f57bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.217+0000 7f57da59c700 1 --2- 172.21.15.156:0/3524638297 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57bc0682f0 0x7f57bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.217+0000 7fccf5ffb700 1 -- 172.21.15.156:0/1713603699 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0004440 con 0x7fcd00160c00 2023-11-28T13:08:10.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.217+0000 7f45f9f9b700 1 --2- 172.21.15.156:0/2500334689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45f40cb210 0x7f45f406b820 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f45f4072390 tx=0x7f45f0007640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e220a7d71357433c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.217+0000 7fccf5ffb700 1 -- 172.21.15.156:0/1713603699 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf00045a0 con 0x7fcd00160c00 2023-11-28T13:08:10.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.217+0000 7fccf5ffb700 1 -- 172.21.15.156:0/1713603699 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf00038e0 con 0x7fcd00160c00 2023-11-28T13:08:10.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.218+0000 7f57d2ffd700 1 -- 172.21.15.156:0/3524638297 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57c405feb0 con 0x7f57dc0cc180 2023-11-28T13:08:10.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.218+0000 7f45f8f99700 1 -- 172.21.15.156:0/2500334689 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f000f070 con 0x7f45f40cb210 2023-11-28T13:08:10.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.218+0000 7f45f8f99700 1 -- 172.21.15.156:0/2500334689 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f0008bf0 con 0x7f45f40cb210 2023-11-28T13:08:10.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.218+0000 7f45f8f99700 1 -- 172.21.15.156:0/2500334689 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f000a0d0 con 0x7f45f40cb210 2023-11-28T13:08:10.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.218+0000 7f9e7d19d700 1 Processor -- start 2023-11-28T13:08:10.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.218+0000 7f9e7d19d700 1 -- start start 2023-11-28T13:08:10.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.218+0000 7fcd0659d700 1 -- 172.21.15.156:0/1713603699 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd000ccf60 con 0x7fcd00160c00 2023-11-28T13:08:10.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.218+0000 7fcd0659d700 1 -- 172.21.15.156:0/1713603699 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd000ccc80 con 0x7fcd00160c00 2023-11-28T13:08:10.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.219+0000 7f45fc1ff700 1 -- 172.21.15.156:0/2500334689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45f40cb210 msgr2=0x7f45f406b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.219+0000 7f45fc1ff700 1 --2- 172.21.15.156:0/2500334689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45f40cb210 0x7f45f406b820 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f45f4072390 tx=0x7f45f0007640 comp rx=0 tx=0).stop 2023-11-28T13:08:10.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.219+0000 7f45fc1ff700 1 -- 172.21.15.156:0/2500334689 shutdown_connections 2023-11-28T13:08:10.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.219+0000 7f45fc1ff700 1 --2- 172.21.15.156:0/2500334689 >> v2:172.21.15.156:3300/0 conn(0x7f45f40ce7d0 0x7f45f415c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.219+0000 7f45fc1ff700 1 --2- 172.21.15.156:0/2500334689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45f40cb210 0x7f45f406b820 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.220+0000 7f57da59c700 1 --2- 172.21.15.156:0/3524638297 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57bc0682f0 0x7f57bc06a7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f57cc007fd0 tx=0x7f57cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.219+0000 7f45fc1ff700 1 -- 172.21.15.156:0/2500334689 >> 172.21.15.156:0/2500334689 conn(0x7f45f40cb670 msgr2=0x7f45f40cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:10.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.220+0000 7f45fc1ff700 1 -- 172.21.15.156:0/2500334689 shutdown_connections 2023-11-28T13:08:10.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.220+0000 7fcd0659d700 1 -- 172.21.15.156:0/1713603699 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd00160fd0 con 0x7fcd00160c00 2023-11-28T13:08:10.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.220+0000 7f9e103ff700 1 -- 172.21.15.156:0/1692853067 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e080ceca0 con 0x7f9e080c6030 2023-11-28T13:08:10.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.221+0000 7f45fc1ff700 1 -- 172.21.15.156:0/2500334689 wait complete. 2023-11-28T13:08:10.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.220+0000 7f9e7d19d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e780b29a0 0x7f9e780a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.220+0000 7f9e7d19d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e780b3500 0x7f9e78154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.220+0000 7f9e7d19d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e78160fb0 0x7f9e780b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.220+0000 7f9e7d19d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78148c50 con 0x7f9e78160fb0 2023-11-28T13:08:10.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.220+0000 7f9e7d19d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e780a03b0 con 0x7f9e780b29a0 2023-11-28T13:08:10.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.220+0000 7f9e7d19d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7806f660 con 0x7f9e780b3500 2023-11-28T13:08:10.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.220+0000 7f9e7c99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e78160fb0 0x7f9e780b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.220+0000 7f9e7c99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e78160fb0 0x7f9e780b4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58990/0 (socket says 172.21.15.156:58990) 2023-11-28T13:08:10.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.220+0000 7f9e7c99c700 1 -- 172.21.15.156:0/4157794048 learned_addr learned my addr 172.21.15.156:0/4157794048 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:10.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.222+0000 7f9e7c99c700 1 -- 172.21.15.156:0/4157794048 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e780b3500 msgr2=0x7f9e78154350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T13:08:10.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.222+0000 7f9e7c99c700 1 --2- 172.21.15.156:0/4157794048 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e780b3500 0x7f9e78154350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.222+0000 7f9e7c99c700 1 -- 172.21.15.156:0/4157794048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e780b29a0 msgr2=0x7f9e780a0840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T13:08:10.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.223+0000 7f9dfeffd700 1 -- 172.21.15.156:0/1692853067 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9e0001b030 con 0x7f9e080c6030 2023-11-28T13:08:10.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.223+0000 7f9e7c99c700 1 --2- 172.21.15.156:0/4157794048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e780b29a0 0x7f9e780a0840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.223+0000 7f9e7c99c700 1 -- 172.21.15.156:0/4157794048 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e70009a70 con 0x7f9e78160fb0 2023-11-28T13:08:10.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.223+0000 7f9e7c99c700 1 --2- 172.21.15.156:0/4157794048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e78160fb0 0x7f9e780b4a50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9e70004000 tx=0x7f9e70004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.224+0000 7f45fc1ff700 1 Processor -- start 2023-11-28T13:08:10.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.224+0000 7f45fc1ff700 1 -- start start 2023-11-28T13:08:10.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.225+0000 7f9dfeffd700 1 --2- 172.21.15.156:0/1692853067 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9dec0683c0 0x7f9dec06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.225+0000 7f9dfeffd700 1 -- 172.21.15.156:0/1692853067 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e00085830 con 0x7f9e080c6030 2023-11-28T13:08:10.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.225+0000 7fccf5ffb700 1 -- 172.21.15.156:0/1713603699 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fccf0045070 con 0x7fcd00160c00 2023-11-28T13:08:10.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.226+0000 7f9e74ff9700 1 -- 172.21.15.156:0/4157794048 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e700046a0 con 0x7f9e78160fb0 2023-11-28T13:08:10.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.226+0000 7f9e74ff9700 1 -- 172.21.15.156:0/4157794048 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e70004800 con 0x7f9e78160fb0 2023-11-28T13:08:10.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.226+0000 7f9e74ff9700 1 -- 172.21.15.156:0/4157794048 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e70004970 con 0x7f9e78160fb0 2023-11-28T13:08:10.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.226+0000 7f9e7d19d700 1 -- 172.21.15.156:0/4157794048 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e780b1c60 con 0x7f9e78160fb0 2023-11-28T13:08:10.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.226+0000 7f9e7d19d700 1 -- 172.21.15.156:0/4157794048 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e780c5df0 con 0x7f9e78160fb0 2023-11-28T13:08:10.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.227+0000 7f9e0d99a700 1 --2- 172.21.15.156:0/1692853067 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9dec0683c0 0x7f9dec06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.227+0000 7f45fc1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45f40b0a30 0x7f45f40c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.227+0000 7f45fc1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45f40ce7d0 0x7f45f40b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.227+0000 7f45fc1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45f4149750 0x7f45f40c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.227+0000 7f45fc1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40b48f0 con 0x7f45f40b0a30 2023-11-28T13:08:10.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.227+0000 7f45fc1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40c6740 con 0x7f45f40ce7d0 2023-11-28T13:08:10.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.227+0000 7f45fc1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40c3710 con 0x7f45f4149750 2023-11-28T13:08:10.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.227+0000 7f57d2ffd700 1 -- 172.21.15.156:0/3524638297 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f57c4054160 con 0x7f57dc0cc180 2023-11-28T13:08:10.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.228+0000 7f45f979a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45f40ce7d0 0x7f45f40b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.228+0000 7f45f979a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45f40ce7d0 0x7f45f40b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:35176/0 (socket says 172.21.15.156:35176) 2023-11-28T13:08:10.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.228+0000 7f45f979a700 1 -- 172.21.15.156:0/1596935999 learned_addr learned my addr 172.21.15.156:0/1596935999 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:10.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.228+0000 7f45fa79c700 1 --2- 172.21.15.156:0/1596935999 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45f4149750 0x7f45f40c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.228+0000 7fccf5ffb700 1 --2- 172.21.15.156:0/1713603699 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcce40683c0 0x7fcce406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.228+0000 7fccf5ffb700 1 -- 172.21.15.156:0/1713603699 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fccf0087b90 con 0x7fcd00160c00 2023-11-28T13:08:10.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.228+0000 7f9e7d19d700 1 -- 172.21.15.156:0/4157794048 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e580052f0 con 0x7f9e78160fb0 2023-11-28T13:08:10.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.228+0000 7fcd0559b700 1 --2- 172.21.15.156:0/1713603699 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcce40683c0 0x7fcce406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.228+0000 7fccf5ffb700 1 -- 172.21.15.156:0/1713603699 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fccf00b25e0 con 0x7fcd00160c00 2023-11-28T13:08:10.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.228+0000 7f45f9f9b700 1 --2- 172.21.15.156:0/1596935999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45f40b0a30 0x7f45f40c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.228+0000 7f45f979a700 1 -- 172.21.15.156:0/1596935999 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45f4149750 msgr2=0x7f45f40c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.228+0000 7f45f979a700 1 --2- 172.21.15.156:0/1596935999 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45f4149750 0x7f45f40c4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.228+0000 7f45f979a700 1 -- 172.21.15.156:0/1596935999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45f40b0a30 msgr2=0x7f45f40c30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.228+0000 7f45f979a700 1 --2- 172.21.15.156:0/1596935999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45f40b0a30 0x7f45f40c30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.228+0000 7f45f979a700 1 -- 172.21.15.156:0/1596935999 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f0007040 con 0x7f45f40ce7d0 2023-11-28T13:08:10.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.228+0000 7f45f9f9b700 1 --2- 172.21.15.156:0/1596935999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45f40b0a30 0x7f45f40c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:10.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.229+0000 7f45f979a700 1 --2- 172.21.15.156:0/1596935999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45f40ce7d0 0x7f45f40b43b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45e800c450 tx=0x7f45e800c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.231+0000 7f9e0d99a700 1 --2- 172.21.15.156:0/1692853067 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9dec0683c0 0x7f9dec06a870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009130 tx=0x7f9e04008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.232+0000 7fcd0559b700 1 --2- 172.21.15.156:0/1713603699 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcce40683c0 0x7fcce406a870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00ba70 tx=0x7fccfc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.232+0000 7f45e67fc700 1 -- 172.21.15.156:0/1596935999 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e80052d0 con 0x7f45f40ce7d0 2023-11-28T13:08:10.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.232+0000 7f45e67fc700 1 -- 172.21.15.156:0/1596935999 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e8005430 con 0x7f45f40ce7d0 2023-11-28T13:08:10.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.232+0000 7f45e67fc700 1 -- 172.21.15.156:0/1596935999 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e80052d0 con 0x7f45f40ce7d0 2023-11-28T13:08:10.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.232+0000 7f45fc1ff700 1 -- 172.21.15.156:0/1596935999 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f4159a50 con 0x7f45f40ce7d0 2023-11-28T13:08:10.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.232+0000 7f45fc1ff700 1 -- 172.21.15.156:0/1596935999 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f4159770 con 0x7f45f40ce7d0 2023-11-28T13:08:10.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.233+0000 7f9e74ff9700 1 -- 172.21.15.156:0/4157794048 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9e7002f470 con 0x7f9e78160fb0 2023-11-28T13:08:10.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.233+0000 7f9dfeffd700 1 -- 172.21.15.156:0/1692853067 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9e0004d5e0 con 0x7f9e080c6030 2023-11-28T13:08:10.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.234+0000 7f9e74ff9700 1 --2- 172.21.15.156:0/4157794048 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9e640683c0 0x7f9e6406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.234+0000 7f9e74ff9700 1 -- 172.21.15.156:0/4157794048 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e70088830 con 0x7f9e78160fb0 2023-11-28T13:08:10.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.234+0000 7f9e77fff700 1 --2- 172.21.15.156:0/4157794048 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9e640683c0 0x7f9e6406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.234+0000 7f45fc1ff700 1 -- 172.21.15.156:0/1596935999 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f4158d60 con 0x7f45f40ce7d0 2023-11-28T13:08:10.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.235+0000 7f9e77fff700 1 --2- 172.21.15.156:0/4157794048 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9e640683c0 0x7f9e6406a870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9e680099c0 tx=0x7f9e680092c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.239+0000 7f0e493ff700 1 Processor -- start 2023-11-28T13:08:10.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.239+0000 7f0e493ff700 1 -- start start 2023-11-28T13:08:10.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.238+0000 7f45e67fc700 1 -- 172.21.15.156:0/1596935999 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f45e801a020 con 0x7f45f40ce7d0 2023-11-28T13:08:10.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.239+0000 7f45e67fc700 1 --2- 172.21.15.156:0/1596935999 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f45d8068380 0x7f45d806a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.239+0000 7f45e67fc700 1 -- 172.21.15.156:0/1596935999 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f45e8089960 con 0x7f45f40ce7d0 2023-11-28T13:08:10.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.240+0000 7f45e67fc700 1 -- 172.21.15.156:0/1596935999 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f45e8056200 con 0x7f45f40ce7d0 2023-11-28T13:08:10.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.240+0000 7f45f9f9b700 1 --2- 172.21.15.156:0/1596935999 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f45d8068380 0x7f45d806a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.240+0000 7f9e74ff9700 1 -- 172.21.15.156:0/4157794048 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9e70058a50 con 0x7f9e78160fb0 2023-11-28T13:08:10.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.241+0000 7f0e493ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e44158890 0x7f0e440ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.241+0000 7f0e493ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0e4406d4f0 0x7f0e440b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.242+0000 7f0e493ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0e440b0bb0 0x7f0e440cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.242+0000 7f0e493ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e440b1030 con 0x7f0e4406d4f0 2023-11-28T13:08:10.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.242+0000 7f0e493ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4415c490 con 0x7f0e440b0bb0 2023-11-28T13:08:10.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.242+0000 7f0e493ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440ce650 con 0x7f0e44158890 2023-11-28T13:08:10.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.242+0000 7f0e42ffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e44158890 0x7f0e440ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.242+0000 7f0e42ffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e44158890 0x7f0e440ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:37032/0 (socket says 172.21.15.156:37032) 2023-11-28T13:08:10.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.242+0000 7f0e42ffd700 1 -- 172.21.15.156:0/2868680880 learned_addr learned my addr 172.21.15.156:0/2868680880 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:10.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.243+0000 7f0e42ffd700 1 -- 172.21.15.156:0/2868680880 >> v2:172.21.15.186:3300/0 conn(0x7f0e440b0bb0 msgr2=0x7f0e440cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:10.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.243+0000 7f0e42ffd700 1 --2- 172.21.15.156:0/2868680880 >> v2:172.21.15.186:3300/0 conn(0x7f0e440b0bb0 0x7f0e440cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.243+0000 7f0e42ffd700 1 -- 172.21.15.156:0/2868680880 >> v2:172.21.15.156:3300/0 conn(0x7f0e4406d4f0 msgr2=0x7f0e440b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:10.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.243+0000 7f0e42ffd700 1 --2- 172.21.15.156:0/2868680880 >> v2:172.21.15.156:3300/0 conn(0x7f0e4406d4f0 0x7f0e440b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.243+0000 7f0e42ffd700 1 -- 172.21.15.156:0/2868680880 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4415c730 con 0x7f0e44158890 2023-11-28T13:08:10.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.243+0000 7f0e42ffd700 1 --2- 172.21.15.156:0/2868680880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e44158890 0x7f0e440ce110 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0e4406d4a0 tx=0x7f0e34009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b5d0ce886821905 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.245+0000 7f45f9f9b700 1 --2- 172.21.15.156:0/1596935999 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f45d8068380 0x7f45d806a830 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f45f4068870 tx=0x7f45f000a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.246+0000 7f0e41ffb700 1 -- 172.21.15.156:0/2868680880 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3400b870 con 0x7f0e44158890 2023-11-28T13:08:10.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.246+0000 7f0e41ffb700 1 -- 172.21.15.156:0/2868680880 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3400b9d0 con 0x7f0e44158890 2023-11-28T13:08:10.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.246+0000 7f0e41ffb700 1 -- 172.21.15.156:0/2868680880 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3400bd40 con 0x7f0e44158890 2023-11-28T13:08:10.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.247+0000 7f0e493ff700 1 -- 172.21.15.156:0/2868680880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e44158890 msgr2=0x7f0e440ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.247+0000 7f0e493ff700 1 --2- 172.21.15.156:0/2868680880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e44158890 0x7f0e440ce110 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0e4406d4a0 tx=0x7f0e34009bd0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.248+0000 7f0e493ff700 1 -- 172.21.15.156:0/2868680880 shutdown_connections 2023-11-28T13:08:10.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.248+0000 7f0e493ff700 1 --2- 172.21.15.156:0/2868680880 >> v2:172.21.15.186:3300/0 conn(0x7f0e440b0bb0 0x7f0e440cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.248+0000 7f0e493ff700 1 --2- 172.21.15.156:0/2868680880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e44158890 0x7f0e440ce110 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.248+0000 7f0e493ff700 1 --2- 172.21.15.156:0/2868680880 >> v2:172.21.15.156:3300/0 conn(0x7f0e4406d4f0 0x7f0e440b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.248+0000 7f0e493ff700 1 -- 172.21.15.156:0/2868680880 >> 172.21.15.156:0/2868680880 conn(0x7f0e440cb670 msgr2=0x7f0e440cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:10.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.248+0000 7f0e493ff700 1 -- 172.21.15.156:0/2868680880 shutdown_connections 2023-11-28T13:08:10.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.248+0000 7f0e493ff700 1 -- 172.21.15.156:0/2868680880 wait complete. 2023-11-28T13:08:10.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.252+0000 7f0e493ff700 1 Processor -- start 2023-11-28T13:08:10.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.252+0000 7f0e493ff700 1 -- start start 2023-11-28T13:08:10.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.254+0000 7f0e493ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e4406d4f0 0x7f0e441f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.254+0000 7f0e493ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e440b0bb0 0x7f0e440b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.255+0000 7f0e493ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e44158890 0x7f0e440b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.255+0000 7f0e427fc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e440b0bb0 0x7f0e440b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.255+0000 7f0e427fc700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e440b0bb0 0x7f0e440b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:37038/0 (socket says 172.21.15.156:37038) 2023-11-28T13:08:10.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.255+0000 7f0e427fc700 1 -- 172.21.15.156:0/2782006886 learned_addr learned my addr 172.21.15.156:0/2782006886 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:10.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.255+0000 7f0e493ff700 1 -- 172.21.15.156:0/2782006886 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e44200130 con 0x7f0e44158890 2023-11-28T13:08:10.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.256+0000 7f0e493ff700 1 -- 172.21.15.156:0/2782006886 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e441577c0 con 0x7f0e4406d4f0 2023-11-28T13:08:10.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.256+0000 7f0e493ff700 1 -- 172.21.15.156:0/2782006886 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e441ffe50 con 0x7f0e440b0bb0 2023-11-28T13:08:10.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.256+0000 7f0e42ffd700 1 --2- 172.21.15.156:0/2782006886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e4406d4f0 0x7f0e441f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.256+0000 7f0e437fe700 1 --2- 172.21.15.156:0/2782006886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e44158890 0x7f0e440b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.257+0000 7f0e42ffd700 1 -- 172.21.15.156:0/2782006886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e440b0bb0 msgr2=0x7f0e440b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.257+0000 7f0e42ffd700 1 --2- 172.21.15.156:0/2782006886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e440b0bb0 0x7f0e440b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.257+0000 7f0e42ffd700 1 -- 172.21.15.156:0/2782006886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e44158890 msgr2=0x7f0e440b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.257+0000 7f0e42ffd700 1 --2- 172.21.15.156:0/2782006886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e44158890 0x7f0e440b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.257+0000 7f0e42ffd700 1 -- 172.21.15.156:0/2782006886 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e34009550 con 0x7f0e4406d4f0 2023-11-28T13:08:10.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.257+0000 7f0e437fe700 1 --2- 172.21.15.156:0/2782006886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e44158890 0x7f0e440b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:10.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.258+0000 7f0e427fc700 1 --2- 172.21.15.156:0/2782006886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e440b0bb0 0x7f0e440b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:10.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.258+0000 7f0e42ffd700 1 --2- 172.21.15.156:0/2782006886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e4406d4f0 0x7f0e441f8f90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0e4406d4a0 tx=0x7f0e34004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.258+0000 7f0e2b7fe700 1 -- 172.21.15.156:0/2782006886 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34004500 con 0x7f0e4406d4f0 2023-11-28T13:08:10.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.258+0000 7f0e2b7fe700 1 -- 172.21.15.156:0/2782006886 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e340036a0 con 0x7f0e4406d4f0 2023-11-28T13:08:10.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.258+0000 7f0e2b7fe700 1 -- 172.21.15.156:0/2782006886 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34003a30 con 0x7f0e4406d4f0 2023-11-28T13:08:10.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.258+0000 7f0e493ff700 1 -- 172.21.15.156:0/2782006886 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4415ccf0 con 0x7f0e4406d4f0 2023-11-28T13:08:10.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.259+0000 7f0e493ff700 1 -- 172.21.15.156:0/2782006886 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4415ca10 con 0x7f0e4406d4f0 2023-11-28T13:08:10.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.260+0000 7f0e493ff700 1 -- 172.21.15.156:0/2782006886 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e440c9370 con 0x7f0e4406d4f0 2023-11-28T13:08:10.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.262+0000 7f0e2b7fe700 1 -- 172.21.15.156:0/2782006886 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0e3402d070 con 0x7f0e4406d4f0 2023-11-28T13:08:10.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.263+0000 7f0e2b7fe700 1 --2- 172.21.15.156:0/2782006886 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0e300682f0 0x7f0e3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.263+0000 7f0e2b7fe700 1 -- 172.21.15.156:0/2782006886 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0e34021030 con 0x7f0e4406d4f0 2023-11-28T13:08:10.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.267+0000 7f0e2b7fe700 1 -- 172.21.15.156:0/2782006886 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0e34053bb0 con 0x7f0e4406d4f0 2023-11-28T13:08:10.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.268+0000 7f0e427fc700 1 --2- 172.21.15.156:0/2782006886 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0e300682f0 0x7f0e3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.272+0000 7f0e427fc700 1 --2- 172.21.15.156:0/2782006886 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0e300682f0 0x7f0e3006a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e380098d0 tx=0x7f0e38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.657+0000 7fbcb27fc700 1 -- 172.21.15.156:0/3253743166 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fbca40055a0 con 0x7fbccc158950 2023-11-28T13:08:10.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.658+0000 7fbcc8ff9700 1 -- 172.21.15.156:0/3253743166 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbcc408b090 con 0x7fbccc158950 2023-11-28T13:08:10.659 INFO:teuthology.orchestra.run.smithi156.stdout:55834574852 2023-11-28T13:08:10.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.662+0000 7fbcb27fc700 1 -- 172.21.15.156:0/3253743166 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcb80682f0 msgr2=0x7fbcb806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.662+0000 7fbcb27fc700 1 --2- 172.21.15.156:0/3253743166 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcb80682f0 0x7fbcb806a7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00d370 tx=0x7fbcbc012000 comp rx=0 tx=0).stop 2023-11-28T13:08:10.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.662+0000 7fbcb27fc700 1 -- 172.21.15.156:0/3253743166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbccc158950 msgr2=0x7fbccc0af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.662+0000 7fbcb27fc700 1 --2- 172.21.15.156:0/3253743166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbccc158950 0x7fbccc0af540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400efc0 tx=0x7fbcc400c3a0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.663+0000 7fbcb27fc700 1 -- 172.21.15.156:0/3253743166 shutdown_connections 2023-11-28T13:08:10.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.663+0000 7fbcb27fc700 1 --2- 172.21.15.156:0/3253743166 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcb80682f0 0x7fbcb806a7a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.663+0000 7fbcb27fc700 1 --2- 172.21.15.156:0/3253743166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbccc158950 0x7fbccc0af540 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.663+0000 7fbcb27fc700 1 --2- 172.21.15.156:0/3253743166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbccc0b0a50 0x7fbccc0b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.663+0000 7fbcb27fc700 1 --2- 172.21.15.156:0/3253743166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbccc0cab60 0x7fbccc1fe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.663+0000 7fbcb27fc700 1 -- 172.21.15.156:0/3253743166 >> 172.21.15.156:0/3253743166 conn(0x7fbccc0cb510 msgr2=0x7fbccc0705c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:10.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.663+0000 7fbcb27fc700 1 -- 172.21.15.156:0/3253743166 shutdown_connections 2023-11-28T13:08:10.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.664+0000 7fbcb27fc700 1 -- 172.21.15.156:0/3253743166 wait complete. 2023-11-28T13:08:10.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.664+0000 7fbb2afff700 1 -- 172.21.15.156:0/1322909212 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbb240745c0 con 0x7fbb24158890 2023-11-28T13:08:10.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.665+0000 7fbb217fa700 1 -- 172.21.15.156:0/1322909212 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb18054ed0 con 0x7fbb24158890 2023-11-28T13:08:10.666 INFO:teuthology.orchestra.run.smithi156.stdout:55834574852 2023-11-28T13:08:10.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.669+0000 7fbb2afff700 1 -- 172.21.15.156:0/1322909212 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbb100683c0 msgr2=0x7fbb1006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.669+0000 7fbb2afff700 1 --2- 172.21.15.156:0/1322909212 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbb100683c0 0x7fbb1006a870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbb14006d50 tx=0x7fbb14006c80 comp rx=0 tx=0).stop 2023-11-28T13:08:10.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.669+0000 7fbb2afff700 1 -- 172.21.15.156:0/1322909212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb24158890 msgr2=0x7fbb24157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.669+0000 7fbb2afff700 1 --2- 172.21.15.156:0/1322909212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb24158890 0x7fbb24157720 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800c500 tx=0x7fbb1800c8c0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.670+0000 7fbb2afff700 1 -- 172.21.15.156:0/1322909212 shutdown_connections 2023-11-28T13:08:10.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.670+0000 7fbb2afff700 1 --2- 172.21.15.156:0/1322909212 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbb100683c0 0x7fbb1006a870 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbb14006d50 tx=0x7fbb14006c80 comp rx=0 tx=0).stop 2023-11-28T13:08:10.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.670+0000 7fbb2afff700 1 --2- 172.21.15.156:0/1322909212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb24158890 0x7fbb24157720 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.670+0000 7fbb2afff700 1 --2- 172.21.15.156:0/1322909212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb2406d4f0 0x7fbb240b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.670+0000 7fbb2afff700 1 --2- 172.21.15.156:0/1322909212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbb240b0bb0 0x7fbb241571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.670+0000 7fbb2afff700 1 -- 172.21.15.156:0/1322909212 >> 172.21.15.156:0/1322909212 conn(0x7fbb240cb670 msgr2=0x7fbb24155e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:10.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.670+0000 7f57d0ff9700 1 -- 172.21.15.156:0/3524638297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f57dc14d6e0 con 0x7f57dc0cc180 2023-11-28T13:08:10.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.670+0000 7fbb2afff700 1 -- 172.21.15.156:0/1322909212 shutdown_connections 2023-11-28T13:08:10.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.670+0000 7fbb2afff700 1 -- 172.21.15.156:0/1322909212 wait complete. 2023-11-28T13:08:10.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.671+0000 7f9e103ff700 1 -- 172.21.15.156:0/1692853067 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9e08068690 con 0x7f9e080c6030 2023-11-28T13:08:10.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.671+0000 7f9dfeffd700 1 -- 172.21.15.156:0/1692853067 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9e00019020 con 0x7f9e080c6030 2023-11-28T13:08:10.673 INFO:teuthology.orchestra.run.smithi156.stdout:55834574853 2023-11-28T13:08:10.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.671+0000 7f57d2ffd700 1 -- 172.21.15.156:0/3524638297 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f57dc14d6e0 con 0x7f57dc0cc180 2023-11-28T13:08:10.676 INFO:teuthology.orchestra.run.smithi156.stdout:55834574853 2023-11-28T13:08:10.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.676+0000 7f57d0ff9700 1 -- 172.21.15.156:0/3524638297 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57bc0682f0 msgr2=0x7f57bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.676+0000 7f57d0ff9700 1 --2- 172.21.15.156:0/3524638297 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57bc0682f0 0x7f57bc06a7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f57cc007fd0 tx=0x7f57cc009040 comp rx=0 tx=0).stop 2023-11-28T13:08:10.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.676+0000 7f57d0ff9700 1 -- 172.21.15.156:0/3524638297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57dc0cc180 msgr2=0x7f57dc143e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.676+0000 7f57d0ff9700 1 --2- 172.21.15.156:0/3524638297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57dc0cc180 0x7f57dc143e70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f57c4015000 tx=0x7f57c4004750 comp rx=0 tx=0).stop 2023-11-28T13:08:10.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.677+0000 7f57d0ff9700 1 -- 172.21.15.156:0/3524638297 shutdown_connections 2023-11-28T13:08:10.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.677+0000 7f57d0ff9700 1 --2- 172.21.15.156:0/3524638297 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57bc0682f0 0x7f57bc06a7a0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f57cc007fd0 tx=0x7f57cc009040 comp rx=0 tx=0).stop 2023-11-28T13:08:10.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.677+0000 7f57d0ff9700 1 --2- 172.21.15.156:0/3524638297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57dc0cc180 0x7f57dc143e70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.677+0000 7f57d0ff9700 1 --2- 172.21.15.156:0/3524638297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57dc06d4f0 0x7f57dc0c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.677+0000 7f57d0ff9700 1 --2- 172.21.15.156:0/3524638297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57dc0b0670 0x7f57dc157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.677+0000 7f57d0ff9700 1 -- 172.21.15.156:0/3524638297 >> 172.21.15.156:0/3524638297 conn(0x7f57dc0cb670 msgr2=0x7f57dc06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:10.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.678+0000 7f9e103ff700 1 -- 172.21.15.156:0/1692853067 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9dec0683c0 msgr2=0x7f9dec06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.678+0000 7f9e103ff700 1 --2- 172.21.15.156:0/1692853067 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9dec0683c0 0x7f9dec06a870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009130 tx=0x7f9e04008030 comp rx=0 tx=0).stop 2023-11-28T13:08:10.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.678+0000 7f9e103ff700 1 -- 172.21.15.156:0/1692853067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e080c6030 msgr2=0x7f9e080c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.678+0000 7f9e103ff700 1 --2- 172.21.15.156:0/1692853067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e080c6030 0x7f9e080c9390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000ba50 tx=0x7f9e0000bd60 comp rx=0 tx=0).stop 2023-11-28T13:08:10.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.678+0000 7f9e103ff700 1 -- 172.21.15.156:0/1692853067 shutdown_connections 2023-11-28T13:08:10.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.678+0000 7f9e103ff700 1 --2- 172.21.15.156:0/1692853067 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9dec0683c0 0x7f9dec06a870 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.679+0000 7f57d0ff9700 1 -- 172.21.15.156:0/3524638297 shutdown_connections 2023-11-28T13:08:10.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.679+0000 7f9e103ff700 1 --2- 172.21.15.156:0/1692853067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e080d0c20 0x7f9e080cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.679+0000 7f9e103ff700 1 --2- 172.21.15.156:0/1692853067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e080c6030 0x7f9e080c9390 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.679+0000 7f57d0ff9700 1 -- 172.21.15.156:0/3524638297 wait complete. 2023-11-28T13:08:10.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.679+0000 7f9e103ff700 1 --2- 172.21.15.156:0/1692853067 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e080c6580 0x7f9e080cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.679+0000 7f9e103ff700 1 -- 172.21.15.156:0/1692853067 >> 172.21.15.156:0/1692853067 conn(0x7f9e0809c3f0 msgr2=0x7f9e080b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:10.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.679+0000 7f9e103ff700 1 -- 172.21.15.156:0/1692853067 shutdown_connections 2023-11-28T13:08:10.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.679+0000 7f9e103ff700 1 -- 172.21.15.156:0/1692853067 wait complete. 2023-11-28T13:08:10.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.680+0000 7fcd0659d700 1 -- 172.21.15.156:0/1713603699 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcd00143630 con 0x7fcd00160c00 2023-11-28T13:08:10.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.681+0000 7fccf5ffb700 1 -- 172.21.15.156:0/1713603699 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fccf0015020 con 0x7fcd00160c00 2023-11-28T13:08:10.684 INFO:teuthology.orchestra.run.smithi156.stdout:55834574854 2023-11-28T13:08:10.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.686+0000 7f9e7d19d700 1 -- 172.21.15.156:0/4157794048 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9e58005870 con 0x7f9e78160fb0 2023-11-28T13:08:10.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.686+0000 7fcd0659d700 1 -- 172.21.15.156:0/1713603699 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcce40683c0 msgr2=0x7fcce406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.686+0000 7fcd0659d700 1 --2- 172.21.15.156:0/1713603699 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcce40683c0 0x7fcce406a870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00ba70 tx=0x7fccfc008040 comp rx=0 tx=0).stop 2023-11-28T13:08:10.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.686+0000 7fcd0659d700 1 -- 172.21.15.156:0/1713603699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd00160c00 msgr2=0x7fcd00143fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.686+0000 7fcd0659d700 1 --2- 172.21.15.156:0/1713603699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd00160c00 0x7fcd00143fd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fccf0016000 tx=0x7fccf0004750 comp rx=0 tx=0).stop 2023-11-28T13:08:10.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.687+0000 7fcd0659d700 1 -- 172.21.15.156:0/1713603699 shutdown_connections 2023-11-28T13:08:10.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.687+0000 7fcd0659d700 1 --2- 172.21.15.156:0/1713603699 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcce40683c0 0x7fcce406a870 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.687+0000 7fcd0659d700 1 --2- 172.21.15.156:0/1713603699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd000b18c0 0x7fcd001449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.687+0000 7fcd0659d700 1 --2- 172.21.15.156:0/1713603699 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd000ce080 0x7fcd00142f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.687+0000 7fcd0659d700 1 --2- 172.21.15.156:0/1713603699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd00160c00 0x7fcd00143fd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.687+0000 7fcd0659d700 1 -- 172.21.15.156:0/1713603699 >> 172.21.15.156:0/1713603699 conn(0x7fcd000cb510 msgr2=0x7fcd0006c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:10.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.687+0000 7fcd0659d700 1 -- 172.21.15.156:0/1713603699 shutdown_connections 2023-11-28T13:08:10.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.687+0000 7fcd0659d700 1 -- 172.21.15.156:0/1713603699 wait complete. 2023-11-28T13:08:10.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.687+0000 7f9e74ff9700 1 -- 172.21.15.156:0/4157794048 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9e700585e0 con 0x7f9e78160fb0 2023-11-28T13:08:10.689 INFO:teuthology.orchestra.run.smithi156.stdout:55834574853 2023-11-28T13:08:10.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.693+0000 7f9e7d19d700 1 -- 172.21.15.156:0/4157794048 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9e640683c0 msgr2=0x7f9e6406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.693+0000 7f9e7d19d700 1 --2- 172.21.15.156:0/4157794048 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9e640683c0 0x7f9e6406a870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9e680099c0 tx=0x7f9e680092c0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.693+0000 7f9e7d19d700 1 -- 172.21.15.156:0/4157794048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e78160fb0 msgr2=0x7f9e780b4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.693+0000 7f9e7d19d700 1 --2- 172.21.15.156:0/4157794048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e78160fb0 0x7f9e780b4a50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9e70004000 tx=0x7f9e70004ad0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.693+0000 7f9e7d19d700 1 -- 172.21.15.156:0/4157794048 shutdown_connections 2023-11-28T13:08:10.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.693+0000 7f9e7d19d700 1 --2- 172.21.15.156:0/4157794048 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9e640683c0 0x7f9e6406a870 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9e680099c0 tx=0x7f9e680092c0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.693+0000 7f9e7d19d700 1 --2- 172.21.15.156:0/4157794048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e78160fb0 0x7f9e780b4a50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.693+0000 7f9e7d19d700 1 --2- 172.21.15.156:0/4157794048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e780b29a0 0x7f9e780a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.693+0000 7f9e7d19d700 1 --2- 172.21.15.156:0/4157794048 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e780b3500 0x7f9e78154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.693+0000 7f9e7d19d700 1 -- 172.21.15.156:0/4157794048 >> 172.21.15.156:0/4157794048 conn(0x7f9e7809c1d0 msgr2=0x7f9e780afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:10.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.694+0000 7f9e7d19d700 1 -- 172.21.15.156:0/4157794048 shutdown_connections 2023-11-28T13:08:10.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.694+0000 7f9e7d19d700 1 -- 172.21.15.156:0/4157794048 wait complete. 2023-11-28T13:08:10.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.696+0000 7f45fc1ff700 1 -- 172.21.15.156:0/1596935999 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f45f40c3bd0 con 0x7f45f40ce7d0 2023-11-28T13:08:10.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.697+0000 7f45e67fc700 1 -- 172.21.15.156:0/1596935999 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f45e8059820 con 0x7f45f40ce7d0 2023-11-28T13:08:10.699 INFO:teuthology.orchestra.run.smithi156.stdout:55834574853 2023-11-28T13:08:10.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2023-11-28T13:08:10.702 DEBUG:teuthology.parallel:result is None 2023-11-28T13:08:10.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.703+0000 7f45fc1ff700 1 -- 172.21.15.156:0/1596935999 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f45d8068380 msgr2=0x7f45d806a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.703+0000 7f45fc1ff700 1 --2- 172.21.15.156:0/1596935999 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f45d8068380 0x7f45d806a830 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f45f4068870 tx=0x7f45f000a000 comp rx=0 tx=0).stop 2023-11-28T13:08:10.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.703+0000 7f45fc1ff700 1 -- 172.21.15.156:0/1596935999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45f40ce7d0 msgr2=0x7f45f40b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.703+0000 7f45fc1ff700 1 --2- 172.21.15.156:0/1596935999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45f40ce7d0 0x7f45f40b43b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45e800c450 tx=0x7f45e800c760 comp rx=0 tx=0).stop 2023-11-28T13:08:10.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.705+0000 7f45fc1ff700 1 -- 172.21.15.156:0/1596935999 shutdown_connections 2023-11-28T13:08:10.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.705+0000 7f45fc1ff700 1 --2- 172.21.15.156:0/1596935999 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f45d8068380 0x7f45d806a830 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.706+0000 7f45fc1ff700 1 --2- 172.21.15.156:0/1596935999 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f45f4149750 0x7f45f40c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.706+0000 7f45fc1ff700 1 --2- 172.21.15.156:0/1596935999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45f40b0a30 0x7f45f40c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.706+0000 7f45fc1ff700 1 --2- 172.21.15.156:0/1596935999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45f40ce7d0 0x7f45f40b43b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.706+0000 7f45fc1ff700 1 -- 172.21.15.156:0/1596935999 >> 172.21.15.156:0/1596935999 conn(0x7f45f40cb670 msgr2=0x7f45f4071630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:10.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.706+0000 7f45fc1ff700 1 -- 172.21.15.156:0/1596935999 shutdown_connections 2023-11-28T13:08:10.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.706+0000 7f45fc1ff700 1 -- 172.21.15.156:0/1596935999 wait complete. 2023-11-28T13:08:10.708 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.1 2023-11-28T13:08:10.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.708+0000 7f0e493ff700 1 -- 172.21.15.156:0/2782006886 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f0e440745c0 con 0x7f0e4406d4f0 2023-11-28T13:08:10.709 DEBUG:teuthology.parallel:result is None 2023-11-28T13:08:10.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.708+0000 7f0e2b7fe700 1 -- 172.21.15.156:0/2782006886 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0e340571d0 con 0x7f0e4406d4f0 2023-11-28T13:08:10.709 INFO:teuthology.orchestra.run.smithi156.stdout:55834574854 2023-11-28T13:08:10.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.712+0000 7f0e493ff700 1 -- 172.21.15.156:0/2782006886 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0e300682f0 msgr2=0x7f0e3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.712+0000 7f0e493ff700 1 --2- 172.21.15.156:0/2782006886 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0e300682f0 0x7f0e3006a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e380098d0 tx=0x7f0e38008040 comp rx=0 tx=0).stop 2023-11-28T13:08:10.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.712+0000 7f0e493ff700 1 -- 172.21.15.156:0/2782006886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e4406d4f0 msgr2=0x7f0e441f8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.712+0000 7f0e493ff700 1 --2- 172.21.15.156:0/2782006886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e4406d4f0 0x7f0e441f8f90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0e4406d4a0 tx=0x7f0e34004750 comp rx=0 tx=0).stop 2023-11-28T13:08:10.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.712+0000 7f0e493ff700 1 -- 172.21.15.156:0/2782006886 shutdown_connections 2023-11-28T13:08:10.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.712+0000 7f0e493ff700 1 --2- 172.21.15.156:0/2782006886 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0e300682f0 0x7f0e3006a7a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.712+0000 7f0e493ff700 1 --2- 172.21.15.156:0/2782006886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e44158890 0x7f0e440b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.712+0000 7f0e493ff700 1 --2- 172.21.15.156:0/2782006886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e4406d4f0 0x7f0e441f8f90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.712+0000 7f0e493ff700 1 --2- 172.21.15.156:0/2782006886 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e440b0bb0 0x7f0e440b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.712+0000 7f0e493ff700 1 -- 172.21.15.156:0/2782006886 >> 172.21.15.156:0/2782006886 conn(0x7f0e440cb670 msgr2=0x7f0e440cadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:10.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.712+0000 7f0e493ff700 1 -- 172.21.15.156:0/2782006886 shutdown_connections 2023-11-28T13:08:10.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.712+0000 7f0e493ff700 1 -- 172.21.15.156:0/2782006886 wait complete. 2023-11-28T13:08:10.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2023-11-28T13:08:10.715 DEBUG:teuthology.parallel:result is None 2023-11-28T13:08:10.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2023-11-28T13:08:10.717 DEBUG:teuthology.parallel:result is None 2023-11-28T13:08:10.726 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-11-28T13:08:10.726 DEBUG:teuthology.parallel:result is None 2023-11-28T13:08:10.730 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2023-11-28T13:08:10.730 DEBUG:teuthology.parallel:result is None 2023-11-28T13:08:10.732 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2023-11-28T13:08:10.732 DEBUG:teuthology.parallel:result is None 2023-11-28T13:08:10.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-11-28T13:08:10.735 DEBUG:teuthology.parallel:result is None 2023-11-28T13:08:10.736 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T13:08:10.736 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:08:10.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.971+0000 7f96a19ff700 1 Processor -- start 2023-11-28T13:08:10.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.972+0000 7f96a19ff700 1 -- start start 2023-11-28T13:08:10.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.973+0000 7f96a19ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f969c161340 0x7f969c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.973+0000 7f96a19ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f969c0d2320 0x7f969c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.973+0000 7f96a19ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969c0d1950 con 0x7f969c15c9b0 2023-11-28T13:08:10.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.973+0000 7f96a19ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969c0b5800 con 0x7f969c161340 2023-11-28T13:08:10.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.973+0000 7f96a19ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c0b5300 con 0x7f969c0d2320 2023-11-28T13:08:10.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.973+0000 7f969affd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f969c15c9b0 0x7f969c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:34648/0 (socket says 172.21.15.156:34648) 2023-11-28T13:08:10.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.973+0000 7f969affd700 1 -- 172.21.15.156:0/1488343147 learned_addr learned my addr 172.21.15.156:0/1488343147 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:10.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.973+0000 7f9693fff700 1 --2- 172.21.15.156:0/1488343147 >> v2:172.21.15.186:3300/0 conn(0x7f969c161340 0x7f969c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.973+0000 7f969b7fe700 1 --2- 172.21.15.156:0/1488343147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f969c0d2320 0x7f969c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.973+0000 7f969a7fc700 1 -- 172.21.15.156:0/1488343147 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004150450 0 0) 0x7f969c0d1950 con 0x7f969c15c9b0 2023-11-28T13:08:10.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.973+0000 7f969a7fc700 1 -- 172.21.15.156:0/1488343147 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96880035b0 con 0x7f969c15c9b0 2023-11-28T13:08:10.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969a7fc700 1 -- 172.21.15.156:0/1488343147 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2679312292 0 0) 0x7f96880035b0 con 0x7f969c15c9b0 2023-11-28T13:08:10.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969a7fc700 1 -- 172.21.15.156:0/1488343147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f969c0d2320 msgr2=0x7f969c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:10.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969a7fc700 1 --2- 172.21.15.156:0/1488343147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f969c0d2320 0x7f969c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969a7fc700 1 -- 172.21.15.156:0/1488343147 >> v2:172.21.15.186:3300/0 conn(0x7f969c161340 msgr2=0x7f969c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:10.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969a7fc700 1 --2- 172.21.15.156:0/1488343147 >> v2:172.21.15.186:3300/0 conn(0x7f969c161340 0x7f969c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969a7fc700 1 -- 172.21.15.156:0/1488343147 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c0b5520 con 0x7f969c15c9b0 2023-11-28T13:08:10.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969a7fc700 1 -- 172.21.15.156:0/1488343147 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f968c001670 con 0x7f969c15c9b0 2023-11-28T13:08:10.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969a7fc700 1 -- 172.21.15.156:0/1488343147 >> v1:172.21.15.156:6789/0 conn(0x7f969c15c9b0 legacy=0x7f969c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969a7fc700 1 --2- 172.21.15.156:0/1488343147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9688003a00 0x7f9688003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969a7fc700 1 --2- 172.21.15.156:0/1488343147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9688004430 0x7f96880048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969a7fc700 1 --2- 172.21.15.156:0/1488343147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f969c0d2320 0x7f9688005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969a7fc700 1 -- 172.21.15.156:0/1488343147 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c0b5800 con 0x7f9688003a00 2023-11-28T13:08:10.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969a7fc700 1 -- 172.21.15.156:0/1488343147 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c0b5300 con 0x7f9688004430 2023-11-28T13:08:10.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969a7fc700 1 -- 172.21.15.156:0/1488343147 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968800ba10 con 0x7f969c0d2320 2023-11-28T13:08:10.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969affd700 1 --2- 172.21.15.156:0/1488343147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9688003a00 0x7f9688003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f9693fff700 1 --2- 172.21.15.156:0/1488343147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9688004430 0x7f96880048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969b7fe700 1 --2- 172.21.15.156:0/1488343147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f969c0d2320 0x7f9688005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969affd700 1 -- 172.21.15.156:0/1488343147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f969c0d2320 msgr2=0x7f9688005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:10.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969affd700 1 --2- 172.21.15.156:0/1488343147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f969c0d2320 0x7f9688005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969affd700 1 -- 172.21.15.156:0/1488343147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9688004430 msgr2=0x7f96880048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:10.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969affd700 1 --2- 172.21.15.156:0/1488343147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9688004430 0x7f96880048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.974+0000 7f969affd700 1 -- 172.21.15.156:0/1488343147 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96880035b0 con 0x7f9688003a00 2023-11-28T13:08:10.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.975+0000 7f969affd700 1 --2- 172.21.15.156:0/1488343147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9688003a00 0x7f9688003e10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f968c00c3a0 tx=0x7f9688005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92261925bde41871 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.975+0000 7f969a7fc700 1 -- 172.21.15.156:0/1488343147 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c007c60 con 0x7f9688003a00 2023-11-28T13:08:10.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.975+0000 7f969a7fc700 1 -- 172.21.15.156:0/1488343147 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968c005960 con 0x7f9688003a00 2023-11-28T13:08:10.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.975+0000 7f969a7fc700 1 -- 172.21.15.156:0/1488343147 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c00cea0 con 0x7f9688003a00 2023-11-28T13:08:10.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.975+0000 7f96a19ff700 1 -- 172.21.15.156:0/1488343147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9688003a00 msgr2=0x7f9688003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.975+0000 7f96a19ff700 1 --2- 172.21.15.156:0/1488343147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9688003a00 0x7f9688003e10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f968c00c3a0 tx=0x7f9688005880 comp rx=0 tx=0).stop 2023-11-28T13:08:10.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.975+0000 7f969b7fe700 1 -- 172.21.15.156:0/1488343147 reap_dead start 2023-11-28T13:08:10.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.976+0000 7f96a19ff700 1 -- 172.21.15.156:0/1488343147 shutdown_connections 2023-11-28T13:08:10.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.976+0000 7f96a19ff700 1 -- 172.21.15.156:0/1488343147 >> 172.21.15.156:0/1488343147 conn(0x7f969c072600 msgr2=0x7f969c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:10.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.976+0000 7f96a19ff700 1 -- 172.21.15.156:0/1488343147 shutdown_connections 2023-11-28T13:08:10.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.976+0000 7f96a19ff700 1 -- 172.21.15.156:0/1488343147 wait complete. 2023-11-28T13:08:10.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.977+0000 7f96a19ff700 1 Processor -- start 2023-11-28T13:08:10.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.978+0000 7f96a19ff700 1 -- start start 2023-11-28T13:08:10.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.978+0000 7f96a19ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9688003a00 0x7f969c0c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.978+0000 7f96a19ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f969c153460 0x7f969c15eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.979+0000 7f96a19ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f969c1539a0 0x7f969c15f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.979+0000 7f96a19ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c1f6a80 con 0x7f969c153460 2023-11-28T13:08:10.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.979+0000 7f96a19ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c1f5e00 con 0x7f9688003a00 2023-11-28T13:08:10.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.979+0000 7f96a19ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c1f60e0 con 0x7f969c1539a0 2023-11-28T13:08:10.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.979+0000 7f9693fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f969c153460 0x7f969c15eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.979+0000 7f9693fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f969c153460 0x7f969c15eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59038/0 (socket says 172.21.15.156:59038) 2023-11-28T13:08:10.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.979+0000 7f9693fff700 1 -- 172.21.15.156:0/1844449519 learned_addr learned my addr 172.21.15.156:0/1844449519 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:10.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.979+0000 7f9693fff700 1 -- 172.21.15.156:0/1844449519 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f969c1539a0 msgr2=0x7f969c15f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.979+0000 7f9693fff700 1 --2- 172.21.15.156:0/1844449519 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f969c1539a0 0x7f969c15f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.979+0000 7f9693fff700 1 -- 172.21.15.156:0/1844449519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9688003a00 msgr2=0x7f969c0c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:10.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.979+0000 7f9693fff700 1 --2- 172.21.15.156:0/1844449519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9688003a00 0x7f969c0c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:10.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.979+0000 7f9693fff700 1 -- 172.21.15.156:0/1844449519 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c0d15e0 con 0x7f969c153460 2023-11-28T13:08:10.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.979+0000 7f9693fff700 1 --2- 172.21.15.156:0/1844449519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f969c153460 0x7f969c15eff0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9684011aa0 tx=0x7f9684011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.980+0000 7f9698ff9700 1 -- 172.21.15.156:0/1844449519 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968400f8a0 con 0x7f969c153460 2023-11-28T13:08:10.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.980+0000 7f9698ff9700 1 -- 172.21.15.156:0/1844449519 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968400fa20 con 0x7f969c153460 2023-11-28T13:08:10.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.980+0000 7f96a19ff700 1 -- 172.21.15.156:0/1844449519 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969c0d1300 con 0x7f969c153460 2023-11-28T13:08:10.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.980+0000 7f9698ff9700 1 -- 172.21.15.156:0/1844449519 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968400fdd0 con 0x7f969c153460 2023-11-28T13:08:10.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.980+0000 7f96a19ff700 1 -- 172.21.15.156:0/1844449519 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c15cd80 con 0x7f969c153460 2023-11-28T13:08:10.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.981+0000 7f96a19ff700 1 -- 172.21.15.156:0/1844449519 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969c0745c0 con 0x7f969c153460 2023-11-28T13:08:10.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.982+0000 7f9698ff9700 1 -- 172.21.15.156:0/1844449519 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9684014440 con 0x7f969c153460 2023-11-28T13:08:10.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.983+0000 7f9698ff9700 1 --2- 172.21.15.156:0/1844449519 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f967c0682f0 0x7f967c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:10.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.983+0000 7f9698ff9700 1 -- 172.21.15.156:0/1844449519 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9684086480 con 0x7f969c153460 2023-11-28T13:08:10.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.983+0000 7f969affd700 1 --2- 172.21.15.156:0/1844449519 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f967c0682f0 0x7f967c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:10.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.984+0000 7f969affd700 1 --2- 172.21.15.156:0/1844449519 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f967c0682f0 0x7f967c06a7a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f968c001050 tx=0x7f968c00f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:10.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:10.987+0000 7f9698ff9700 1 -- 172.21.15.156:0/1844449519 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9684052c50 con 0x7f969c153460 2023-11-28T13:08:11.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.182+0000 7f96a19ff700 1 -- 172.21.15.156:0/1844449519 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f969c0d08a0 con 0x7f967c0682f0 2023-11-28T13:08:11.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.192+0000 7f9698ff9700 1 -- 172.21.15.156:0/1844449519 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286837 (secure 0 0 0) 0x7f969c0d08a0 con 0x7f967c0682f0 2023-11-28T13:08:11.196 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:11.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.196+0000 7f96a19ff700 1 -- 172.21.15.156:0/1844449519 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f967c0682f0 msgr2=0x7f967c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:11.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.196+0000 7f96a19ff700 1 --2- 172.21.15.156:0/1844449519 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f967c0682f0 0x7f967c06a7a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f968c001050 tx=0x7f968c00f000 comp rx=0 tx=0).stop 2023-11-28T13:08:11.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.196+0000 7f96a19ff700 1 -- 172.21.15.156:0/1844449519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f969c153460 msgr2=0x7f969c15eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:11.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.196+0000 7f96a19ff700 1 --2- 172.21.15.156:0/1844449519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f969c153460 0x7f969c15eff0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9684011aa0 tx=0x7f9684011e60 comp rx=0 tx=0).stop 2023-11-28T13:08:11.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.196+0000 7f96a19ff700 1 -- 172.21.15.156:0/1844449519 shutdown_connections 2023-11-28T13:08:11.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.196+0000 7f96a19ff700 1 --2- 172.21.15.156:0/1844449519 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f967c0682f0 0x7f967c06a7a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.196+0000 7f96a19ff700 1 --2- 172.21.15.156:0/1844449519 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f969c1539a0 0x7f969c15f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.196+0000 7f96a19ff700 1 --2- 172.21.15.156:0/1844449519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9688003a00 0x7f969c0c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.196+0000 7f96a19ff700 1 --2- 172.21.15.156:0/1844449519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f969c153460 0x7f969c15eff0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.196+0000 7f96a19ff700 1 -- 172.21.15.156:0/1844449519 >> 172.21.15.156:0/1844449519 conn(0x7f969c072600 msgr2=0x7f969c15a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:11.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.197+0000 7f96a19ff700 1 -- 172.21.15.156:0/1844449519 shutdown_connections 2023-11-28T13:08:11.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.197+0000 7f96a19ff700 1 -- 172.21.15.156:0/1844449519 wait complete. 2023-11-28T13:08:11.200 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T13:08:11.215 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-11-28T13:08:10.015506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":82468,"kb_used_data":2916,"kb_used_omap":0,"kb_used_meta":79488,"kb_avail":754892252,"statfs":{"total":773094113280,"available":773009666048,"internally_reserved":0,"allocated":2985984,"data_stored":1820090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81395712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.458874"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371323+0000","last_change":"2023-11-28T13:08:03.085544+0000","last_active":"2023-11-28T13:08:04.371323+0000","last_peered":"2023-11-28T13:08:04.371323+0000","last_clean":"2023-11-28T13:08:04.371323+0000","last_became_active":"2023-11-28T13:08:03.085265+0000","last_became_peered":"2023-11-28T13:08:03.085265+0000","last_unstale":"2023-11-28T13:08:04.371323+0000","last_undegraded":"2023-11-28T13:08:04.371323+0000","last_fullsized":"2023-11-28T13:08:04.371323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:37.423803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069498+0000","last_change":"2023-11-28T13:08:03.084729+0000","last_active":"2023-11-28T13:08:04.069498+0000","last_peered":"2023-11-28T13:08:04.069498+0000","last_clean":"2023-11-28T13:08:04.069498+0000","last_became_active":"2023-11-28T13:08:03.084473+0000","last_became_peered":"2023-11-28T13:08:03.084473+0000","last_unstale":"2023-11-28T13:08:04.069498+0000","last_undegraded":"2023-11-28T13:08:04.069498+0000","last_fullsized":"2023-11-28T13:08:04.069498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:53.049281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371761+0000","last_change":"2023-11-28T13:08:03.086292+0000","last_active":"2023-11-28T13:08:04.371761+0000","last_peered":"2023-11-28T13:08:04.371761+0000","last_clean":"2023-11-28T13:08:04.371761+0000","last_became_active":"2023-11-28T13:08:03.085984+0000","last_became_peered":"2023-11-28T13:08:03.085984+0000","last_unstale":"2023-11-28T13:08:04.371761+0000","last_undegraded":"2023-11-28T13:08:04.371761+0000","last_fullsized":"2023-11-28T13:08:04.371761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:58.749445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372036+0000","last_change":"2023-11-28T13:08:03.080558+0000","last_active":"2023-11-28T13:08:04.372036+0000","last_peered":"2023-11-28T13:08:04.372036+0000","last_clean":"2023-11-28T13:08:04.372036+0000","last_became_active":"2023-11-28T13:08:03.080267+0000","last_became_peered":"2023-11-28T13:08:03.080267+0000","last_unstale":"2023-11-28T13:08:04.372036+0000","last_undegraded":"2023-11-28T13:08:04.372036+0000","last_fullsized":"2023-11-28T13:08:04.372036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:38.726877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372532+0000","last_change":"2023-11-28T13:08:03.081890+0000","last_active":"2023-11-28T13:08:04.372532+0000","last_peered":"2023-11-28T13:08:04.372532+0000","last_clean":"2023-11-28T13:08:04.372532+0000","last_became_active":"2023-11-28T13:08:03.081638+0000","last_became_peered":"2023-11-28T13:08:03.081638+0000","last_unstale":"2023-11-28T13:08:04.372532+0000","last_undegraded":"2023-11-28T13:08:04.372532+0000","last_fullsized":"2023-11-28T13:08:04.372532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:53.049281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371404+0000","last_change":"2023-11-28T13:08:03.081127+0000","last_active":"2023-11-28T13:08:04.371404+0000","last_peered":"2023-11-28T13:08:04.371404+0000","last_clean":"2023-11-28T13:08:04.371404+0000","last_became_active":"2023-11-28T13:08:03.077682+0000","last_became_peered":"2023-11-28T13:08:03.077682+0000","last_unstale":"2023-11-28T13:08:04.371404+0000","last_undegraded":"2023-11-28T13:08:04.371404+0000","last_fullsized":"2023-11-28T13:08:04.371404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:58.947317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070150+0000","last_change":"2023-11-28T13:08:03.099999+0000","last_active":"2023-11-28T13:08:04.070150+0000","last_peered":"2023-11-28T13:08:04.070150+0000","last_clean":"2023-11-28T13:08:04.070150+0000","last_became_active":"2023-11-28T13:08:03.099733+0000","last_became_peered":"2023-11-28T13:08:03.099733+0000","last_unstale":"2023-11-28T13:08:04.070150+0000","last_undegraded":"2023-11-28T13:08:04.070150+0000","last_fullsized":"2023-11-28T13:08:04.070150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:46.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069502+0000","last_change":"2023-11-28T13:08:03.094943+0000","last_active":"2023-11-28T13:08:04.069502+0000","last_peered":"2023-11-28T13:08:04.069502+0000","last_clean":"2023-11-28T13:08:04.069502+0000","last_became_active":"2023-11-28T13:08:03.094688+0000","last_became_peered":"2023-11-28T13:08:03.094688+0000","last_unstale":"2023-11-28T13:08:04.069502+0000","last_undegraded":"2023-11-28T13:08:04.069502+0000","last_fullsized":"2023-11-28T13:08:04.069502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:14.015315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372476+0000","last_change":"2023-11-28T13:08:03.083815+0000","last_active":"2023-11-28T13:08:04.372476+0000","last_peered":"2023-11-28T13:08:04.372476+0000","last_clean":"2023-11-28T13:08:04.372476+0000","last_became_active":"2023-11-28T13:08:03.083292+0000","last_became_peered":"2023-11-28T13:08:03.083292+0000","last_unstale":"2023-11-28T13:08:04.372476+0000","last_undegraded":"2023-11-28T13:08:04.372476+0000","last_fullsized":"2023-11-28T13:08:04.372476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:14.015315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070743+0000","last_change":"2023-11-28T13:08:03.083685+0000","last_active":"2023-11-28T13:08:04.070743+0000","last_peered":"2023-11-28T13:08:04.070743+0000","last_clean":"2023-11-28T13:08:04.070743+0000","last_became_active":"2023-11-28T13:08:03.077038+0000","last_became_peered":"2023-11-28T13:08:03.077038+0000","last_unstale":"2023-11-28T13:08:04.070743+0000","last_undegraded":"2023-11-28T13:08:04.070743+0000","last_fullsized":"2023-11-28T13:08:04.070743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:34.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371966+0000","last_change":"2023-11-28T13:08:03.085905+0000","last_active":"2023-11-28T13:08:04.371966+0000","last_peered":"2023-11-28T13:08:04.371966+0000","last_clean":"2023-11-28T13:08:04.371966+0000","last_became_active":"2023-11-28T13:08:03.085524+0000","last_became_peered":"2023-11-28T13:08:03.085524+0000","last_unstale":"2023-11-28T13:08:04.371966+0000","last_undegraded":"2023-11-28T13:08:04.371966+0000","last_fullsized":"2023-11-28T13:08:04.371966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:34.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371160+0000","last_change":"2023-11-28T13:08:03.091440+0000","last_active":"2023-11-28T13:08:04.371160+0000","last_peered":"2023-11-28T13:08:04.371160+0000","last_clean":"2023-11-28T13:08:04.371160+0000","last_became_active":"2023-11-28T13:08:03.091117+0000","last_became_peered":"2023-11-28T13:08:03.091117+0000","last_unstale":"2023-11-28T13:08:04.371160+0000","last_undegraded":"2023-11-28T13:08:04.371160+0000","last_fullsized":"2023-11-28T13:08:04.371160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:41.849774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371255+0000","last_change":"2023-11-28T13:08:03.080912+0000","last_active":"2023-11-28T13:08:04.371255+0000","last_peered":"2023-11-28T13:08:04.371255+0000","last_clean":"2023-11-28T13:08:04.371255+0000","last_became_active":"2023-11-28T13:08:03.080578+0000","last_became_peered":"2023-11-28T13:08:03.080578+0000","last_unstale":"2023-11-28T13:08:04.371255+0000","last_undegraded":"2023-11-28T13:08:04.371255+0000","last_fullsized":"2023-11-28T13:08:04.371255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:38.657150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069701+0000","last_change":"2023-11-28T13:08:03.083208+0000","last_active":"2023-11-28T13:08:04.069701+0000","last_peered":"2023-11-28T13:08:04.069701+0000","last_clean":"2023-11-28T13:08:04.069701+0000","last_became_active":"2023-11-28T13:08:03.082944+0000","last_became_peered":"2023-11-28T13:08:03.082944+0000","last_unstale":"2023-11-28T13:08:04.069701+0000","last_undegraded":"2023-11-28T13:08:04.069701+0000","last_fullsized":"2023-11-28T13:08:04.069701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:36.892676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070334+0000","last_change":"2023-11-28T13:08:03.098345+0000","last_active":"2023-11-28T13:08:04.070334+0000","last_peered":"2023-11-28T13:08:04.070334+0000","last_clean":"2023-11-28T13:08:04.070334+0000","last_became_active":"2023-11-28T13:08:03.098105+0000","last_became_peered":"2023-11-28T13:08:03.098105+0000","last_unstale":"2023-11-28T13:08:04.070334+0000","last_undegraded":"2023-11-28T13:08:04.070334+0000","last_fullsized":"2023-11-28T13:08:04.070334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:43.340474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371254+0000","last_change":"2023-11-28T13:08:03.076945+0000","last_active":"2023-11-28T13:08:04.371254+0000","last_peered":"2023-11-28T13:08:04.371254+0000","last_clean":"2023-11-28T13:08:04.371254+0000","last_became_active":"2023-11-28T13:08:03.076639+0000","last_became_peered":"2023-11-28T13:08:03.076639+0000","last_unstale":"2023-11-28T13:08:04.371254+0000","last_undegraded":"2023-11-28T13:08:04.371254+0000","last_fullsized":"2023-11-28T13:08:04.371254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:41.076578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372578+0000","last_change":"2023-11-28T13:08:03.088044+0000","last_active":"2023-11-28T13:08:04.372578+0000","last_peered":"2023-11-28T13:08:04.372578+0000","last_clean":"2023-11-28T13:08:04.372578+0000","last_became_active":"2023-11-28T13:08:03.087703+0000","last_became_peered":"2023-11-28T13:08:03.087703+0000","last_unstale":"2023-11-28T13:08:04.372578+0000","last_undegraded":"2023-11-28T13:08:04.372578+0000","last_fullsized":"2023-11-28T13:08:04.372578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:43.340474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372432+0000","last_change":"2023-11-28T13:08:03.077748+0000","last_active":"2023-11-28T13:08:04.372432+0000","last_peered":"2023-11-28T13:08:04.372432+0000","last_clean":"2023-11-28T13:08:04.372432+0000","last_became_active":"2023-11-28T13:08:03.077488+0000","last_became_peered":"2023-11-28T13:08:03.077488+0000","last_unstale":"2023-11-28T13:08:04.372432+0000","last_undegraded":"2023-11-28T13:08:04.372432+0000","last_fullsized":"2023-11-28T13:08:04.372432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:14.015315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371120+0000","last_change":"2023-11-28T13:08:03.079221+0000","last_active":"2023-11-28T13:08:04.371120+0000","last_peered":"2023-11-28T13:08:04.371120+0000","last_clean":"2023-11-28T13:08:04.371120+0000","last_became_active":"2023-11-28T13:08:03.077084+0000","last_became_peered":"2023-11-28T13:08:03.077084+0000","last_unstale":"2023-11-28T13:08:04.371120+0000","last_undegraded":"2023-11-28T13:08:04.371120+0000","last_fullsized":"2023-11-28T13:08:04.371120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:02.499466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371692+0000","last_change":"2023-11-28T13:08:03.083322+0000","last_active":"2023-11-28T13:08:04.371692+0000","last_peered":"2023-11-28T13:08:04.371692+0000","last_clean":"2023-11-28T13:08:04.371692+0000","last_became_active":"2023-11-28T13:08:03.083064+0000","last_became_peered":"2023-11-28T13:08:03.083064+0000","last_unstale":"2023-11-28T13:08:04.371692+0000","last_undegraded":"2023-11-28T13:08:04.371692+0000","last_fullsized":"2023-11-28T13:08:04.371692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:13.351182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070670+0000","last_change":"2023-11-28T13:08:03.095031+0000","last_active":"2023-11-28T13:08:04.070670+0000","last_peered":"2023-11-28T13:08:04.070670+0000","last_clean":"2023-11-28T13:08:04.070670+0000","last_became_active":"2023-11-28T13:08:03.094746+0000","last_became_peered":"2023-11-28T13:08:03.094746+0000","last_unstale":"2023-11-28T13:08:04.070670+0000","last_undegraded":"2023-11-28T13:08:04.070670+0000","last_fullsized":"2023-11-28T13:08:04.070670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:49.707036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372182+0000","last_change":"2023-11-28T13:08:03.082127+0000","last_active":"2023-11-28T13:08:04.372182+0000","last_peered":"2023-11-28T13:08:04.372182+0000","last_clean":"2023-11-28T13:08:04.372182+0000","last_became_active":"2023-11-28T13:08:03.081861+0000","last_became_peered":"2023-11-28T13:08:03.081861+0000","last_unstale":"2023-11-28T13:08:04.372182+0000","last_undegraded":"2023-11-28T13:08:04.372182+0000","last_fullsized":"2023-11-28T13:08:04.372182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:05:35.135974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064206+0000","last_change":"2023-11-28T13:08:03.083718+0000","last_active":"2023-11-28T13:08:04.064206+0000","last_peered":"2023-11-28T13:08:04.064206+0000","last_clean":"2023-11-28T13:08:04.064206+0000","last_became_active":"2023-11-28T13:08:03.082420+0000","last_became_peered":"2023-11-28T13:08:03.082420+0000","last_unstale":"2023-11-28T13:08:04.064206+0000","last_undegraded":"2023-11-28T13:08:04.064206+0000","last_fullsized":"2023-11-28T13:08:04.064206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:58.181120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070265+0000","last_change":"2023-11-28T13:08:03.095780+0000","last_active":"2023-11-28T13:08:04.070265+0000","last_peered":"2023-11-28T13:08:04.070265+0000","last_clean":"2023-11-28T13:08:04.070265+0000","last_became_active":"2023-11-28T13:08:03.095522+0000","last_became_peered":"2023-11-28T13:08:03.095522+0000","last_unstale":"2023-11-28T13:08:04.070265+0000","last_undegraded":"2023-11-28T13:08:04.070265+0000","last_fullsized":"2023-11-28T13:08:04.070265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:37:44.657148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371784+0000","last_change":"2023-11-28T13:08:03.081168+0000","last_active":"2023-11-28T13:08:04.371784+0000","last_peered":"2023-11-28T13:08:04.371784+0000","last_clean":"2023-11-28T13:08:04.371784+0000","last_became_active":"2023-11-28T13:08:03.080867+0000","last_became_peered":"2023-11-28T13:08:03.080867+0000","last_unstale":"2023-11-28T13:08:04.371784+0000","last_undegraded":"2023-11-28T13:08:04.371784+0000","last_fullsized":"2023-11-28T13:08:04.371784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:44.188797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371597+0000","last_change":"2023-11-28T13:08:03.074696+0000","last_active":"2023-11-28T13:08:04.371597+0000","last_peered":"2023-11-28T13:08:04.371597+0000","last_clean":"2023-11-28T13:08:04.371597+0000","last_became_active":"2023-11-28T13:08:03.074392+0000","last_became_peered":"2023-11-28T13:08:03.074392+0000","last_unstale":"2023-11-28T13:08:04.371597+0000","last_undegraded":"2023-11-28T13:08:04.371597+0000","last_fullsized":"2023-11-28T13:08:04.371597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:50.062766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070127+0000","last_change":"2023-11-28T13:08:03.092477+0000","last_active":"2023-11-28T13:08:04.070127+0000","last_peered":"2023-11-28T13:08:04.070127+0000","last_clean":"2023-11-28T13:08:04.070127+0000","last_became_active":"2023-11-28T13:08:03.092243+0000","last_became_peered":"2023-11-28T13:08:03.092243+0000","last_unstale":"2023-11-28T13:08:04.070127+0000","last_undegraded":"2023-11-28T13:08:04.070127+0000","last_fullsized":"2023-11-28T13:08:04.070127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:46.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371521+0000","last_change":"2023-11-28T13:08:02.582255+0000","last_active":"2023-11-28T13:08:04.371521+0000","last_peered":"2023-11-28T13:08:04.371521+0000","last_clean":"2023-11-28T13:08:04.371521+0000","last_became_active":"2023-11-28T13:08:02.581960+0000","last_became_peered":"2023-11-28T13:08:02.581960+0000","last_unstale":"2023-11-28T13:08:04.371521+0000","last_undegraded":"2023-11-28T13:08:04.371521+0000","last_fullsized":"2023-11-28T13:08:04.371521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:49.593563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064820+0000","last_change":"2023-11-28T13:08:03.080261+0000","last_active":"2023-11-28T13:08:04.064820+0000","last_peered":"2023-11-28T13:08:04.064820+0000","last_clean":"2023-11-28T13:08:04.064820+0000","last_became_active":"2023-11-28T13:08:03.079964+0000","last_became_peered":"2023-11-28T13:08:03.079964+0000","last_unstale":"2023-11-28T13:08:04.064820+0000","last_undegraded":"2023-11-28T13:08:04.064820+0000","last_fullsized":"2023-11-28T13:08:04.064820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:30.263143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371820+0000","last_change":"2023-11-28T13:08:02.575220+0000","last_active":"2023-11-28T13:08:04.371820+0000","last_peered":"2023-11-28T13:08:04.371820+0000","last_clean":"2023-11-28T13:08:04.371820+0000","last_became_active":"2023-11-28T13:08:02.574933+0000","last_became_peered":"2023-11-28T13:08:02.574933+0000","last_unstale":"2023-11-28T13:08:04.371820+0000","last_undegraded":"2023-11-28T13:08:04.371820+0000","last_fullsized":"2023-11-28T13:08:04.371820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:56.910035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372498+0000","last_change":"2023-11-28T13:08:03.077902+0000","last_active":"2023-11-28T13:08:04.372498+0000","last_peered":"2023-11-28T13:08:04.372498+0000","last_clean":"2023-11-28T13:08:04.372498+0000","last_became_active":"2023-11-28T13:08:03.077379+0000","last_became_peered":"2023-11-28T13:08:03.077379+0000","last_unstale":"2023-11-28T13:08:04.372498+0000","last_undegraded":"2023-11-28T13:08:04.372498+0000","last_fullsized":"2023-11-28T13:08:04.372498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:34.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064779+0000","last_change":"2023-11-28T13:08:02.574513+0000","last_active":"2023-11-28T13:08:04.064779+0000","last_peered":"2023-11-28T13:08:04.064779+0000","last_clean":"2023-11-28T13:08:04.064779+0000","last_became_active":"2023-11-28T13:08:02.573509+0000","last_became_peered":"2023-11-28T13:08:02.573509+0000","last_unstale":"2023-11-28T13:08:04.064779+0000","last_undegraded":"2023-11-28T13:08:04.064779+0000","last_fullsized":"2023-11-28T13:08:04.064779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:01.496804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372381+0000","last_change":"2023-11-28T13:08:03.091021+0000","last_active":"2023-11-28T13:08:04.372381+0000","last_peered":"2023-11-28T13:08:04.372381+0000","last_clean":"2023-11-28T13:08:04.372381+0000","last_became_active":"2023-11-28T13:08:03.090717+0000","last_became_peered":"2023-11-28T13:08:03.090717+0000","last_unstale":"2023-11-28T13:08:04.372381+0000","last_undegraded":"2023-11-28T13:08:04.372381+0000","last_fullsized":"2023-11-28T13:08:04.372381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:50.062766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372655+0000","last_change":"2023-11-28T13:08:02.580525+0000","last_active":"2023-11-28T13:08:04.372655+0000","last_peered":"2023-11-28T13:08:04.372655+0000","last_clean":"2023-11-28T13:08:04.372655+0000","last_became_active":"2023-11-28T13:08:02.580220+0000","last_became_peered":"2023-11-28T13:08:02.580220+0000","last_unstale":"2023-11-28T13:08:04.372655+0000","last_undegraded":"2023-11-28T13:08:04.372655+0000","last_fullsized":"2023-11-28T13:08:04.372655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:01.496804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064707+0000","last_change":"2023-11-28T13:08:03.080244+0000","last_active":"2023-11-28T13:08:04.064707+0000","last_peered":"2023-11-28T13:08:04.064707+0000","last_clean":"2023-11-28T13:08:04.064707+0000","last_became_active":"2023-11-28T13:08:03.079988+0000","last_became_peered":"2023-11-28T13:08:03.079988+0000","last_unstale":"2023-11-28T13:08:04.064707+0000","last_undegraded":"2023-11-28T13:08:04.064707+0000","last_fullsized":"2023-11-28T13:08:04.064707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:49.707036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070421+0000","last_change":"2023-11-28T13:08:02.573388+0000","last_active":"2023-11-28T13:08:04.070421+0000","last_peered":"2023-11-28T13:08:04.070421+0000","last_clean":"2023-11-28T13:08:04.070421+0000","last_became_active":"2023-11-28T13:08:02.573059+0000","last_became_peered":"2023-11-28T13:08:02.573059+0000","last_unstale":"2023-11-28T13:08:04.070421+0000","last_undegraded":"2023-11-28T13:08:04.070421+0000","last_fullsized":"2023-11-28T13:08:04.070421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:46:29.622650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064417+0000","last_change":"2023-11-28T13:08:03.088704+0000","last_active":"2023-11-28T13:08:04.064417+0000","last_peered":"2023-11-28T13:08:04.064417+0000","last_clean":"2023-11-28T13:08:04.064417+0000","last_became_active":"2023-11-28T13:08:03.088417+0000","last_became_peered":"2023-11-28T13:08:03.088417+0000","last_unstale":"2023-11-28T13:08:04.064417+0000","last_undegraded":"2023-11-28T13:08:04.064417+0000","last_fullsized":"2023-11-28T13:08:04.064417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:41.196768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070940+0000","last_change":"2023-11-28T13:08:02.576140+0000","last_active":"2023-11-28T13:08:04.070940+0000","last_peered":"2023-11-28T13:08:04.070940+0000","last_clean":"2023-11-28T13:08:04.070940+0000","last_became_active":"2023-11-28T13:08:02.575721+0000","last_became_peered":"2023-11-28T13:08:02.575721+0000","last_unstale":"2023-11-28T13:08:04.070940+0000","last_undegraded":"2023-11-28T13:08:04.070940+0000","last_fullsized":"2023-11-28T13:08:04.070940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:12.214372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371693+0000","last_change":"2023-11-28T13:08:03.079421+0000","last_active":"2023-11-28T13:08:04.371693+0000","last_peered":"2023-11-28T13:08:04.371693+0000","last_clean":"2023-11-28T13:08:04.371693+0000","last_became_active":"2023-11-28T13:08:03.078471+0000","last_became_peered":"2023-11-28T13:08:03.078471+0000","last_unstale":"2023-11-28T13:08:04.371693+0000","last_undegraded":"2023-11-28T13:08:04.371693+0000","last_fullsized":"2023-11-28T13:08:04.371693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:05:35.135974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371902+0000","last_change":"2023-11-28T13:08:02.581355+0000","last_active":"2023-11-28T13:08:04.371902+0000","last_peered":"2023-11-28T13:08:04.371902+0000","last_clean":"2023-11-28T13:08:04.371902+0000","last_became_active":"2023-11-28T13:08:02.581051+0000","last_became_peered":"2023-11-28T13:08:02.581051+0000","last_unstale":"2023-11-28T13:08:04.371902+0000","last_undegraded":"2023-11-28T13:08:04.371902+0000","last_fullsized":"2023-11-28T13:08:04.371902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:43.117989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371569+0000","last_change":"2023-11-28T13:08:03.085302+0000","last_active":"2023-11-28T13:08:04.371569+0000","last_peered":"2023-11-28T13:08:04.371569+0000","last_clean":"2023-11-28T13:08:04.371569+0000","last_became_active":"2023-11-28T13:08:03.085061+0000","last_became_peered":"2023-11-28T13:08:03.085061+0000","last_unstale":"2023-11-28T13:08:04.371569+0000","last_undegraded":"2023-11-28T13:08:04.371569+0000","last_fullsized":"2023-11-28T13:08:04.371569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:13.625233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070589+0000","last_change":"2023-11-28T13:08:02.584763+0000","last_active":"2023-11-28T13:08:04.070589+0000","last_peered":"2023-11-28T13:08:04.070589+0000","last_clean":"2023-11-28T13:08:04.070589+0000","last_became_active":"2023-11-28T13:08:02.583655+0000","last_became_peered":"2023-11-28T13:08:02.583655+0000","last_unstale":"2023-11-28T13:08:04.070589+0000","last_undegraded":"2023-11-28T13:08:04.070589+0000","last_fullsized":"2023-11-28T13:08:04.070589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:53:59.398155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064403+0000","last_change":"2023-11-28T13:08:03.095515+0000","last_active":"2023-11-28T13:08:04.064403+0000","last_peered":"2023-11-28T13:08:04.064403+0000","last_clean":"2023-11-28T13:08:04.064403+0000","last_became_active":"2023-11-28T13:08:03.095004+0000","last_became_peered":"2023-11-28T13:08:03.095004+0000","last_unstale":"2023-11-28T13:08:04.064403+0000","last_undegraded":"2023-11-28T13:08:04.064403+0000","last_fullsized":"2023-11-28T13:08:04.064403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:34.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372572+0000","last_change":"2023-11-28T13:08:02.580082+0000","last_active":"2023-11-28T13:08:04.372572+0000","last_peered":"2023-11-28T13:08:04.372572+0000","last_clean":"2023-11-28T13:08:04.372572+0000","last_became_active":"2023-11-28T13:08:02.579745+0000","last_became_peered":"2023-11-28T13:08:02.579745+0000","last_unstale":"2023-11-28T13:08:04.372572+0000","last_undegraded":"2023-11-28T13:08:04.372572+0000","last_fullsized":"2023-11-28T13:08:04.372572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:12.363239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070163+0000","last_change":"2023-11-28T13:08:03.082966+0000","last_active":"2023-11-28T13:08:04.070163+0000","last_peered":"2023-11-28T13:08:04.070163+0000","last_clean":"2023-11-28T13:08:04.070163+0000","last_became_active":"2023-11-28T13:08:03.082710+0000","last_became_peered":"2023-11-28T13:08:03.082710+0000","last_unstale":"2023-11-28T13:08:04.070163+0000","last_undegraded":"2023-11-28T13:08:04.070163+0000","last_fullsized":"2023-11-28T13:08:04.070163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:49.707036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070236+0000","last_change":"2023-11-28T13:08:02.585538+0000","last_active":"2023-11-28T13:08:04.070236+0000","last_peered":"2023-11-28T13:08:04.070236+0000","last_clean":"2023-11-28T13:08:04.070236+0000","last_became_active":"2023-11-28T13:08:02.584407+0000","last_became_peered":"2023-11-28T13:08:02.584407+0000","last_unstale":"2023-11-28T13:08:04.070236+0000","last_undegraded":"2023-11-28T13:08:04.070236+0000","last_fullsized":"2023-11-28T13:08:04.070236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:01.496804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372389+0000","last_change":"2023-11-28T13:08:03.077754+0000","last_active":"2023-11-28T13:08:04.372389+0000","last_peered":"2023-11-28T13:08:04.372389+0000","last_clean":"2023-11-28T13:08:04.372389+0000","last_became_active":"2023-11-28T13:08:03.077073+0000","last_became_peered":"2023-11-28T13:08:03.077073+0000","last_unstale":"2023-11-28T13:08:04.372389+0000","last_undegraded":"2023-11-28T13:08:04.372389+0000","last_fullsized":"2023-11-28T13:08:04.372389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:46.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371255+0000","last_change":"2023-11-28T13:08:02.574654+0000","last_active":"2023-11-28T13:08:04.371255+0000","last_peered":"2023-11-28T13:08:04.371255+0000","last_clean":"2023-11-28T13:08:04.371255+0000","last_became_active":"2023-11-28T13:08:02.574354+0000","last_became_peered":"2023-11-28T13:08:02.574354+0000","last_unstale":"2023-11-28T13:08:04.371255+0000","last_undegraded":"2023-11-28T13:08:04.371255+0000","last_fullsized":"2023-11-28T13:08:04.371255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:43.117989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064965+0000","last_change":"2023-11-28T13:08:03.084191+0000","last_active":"2023-11-28T13:08:04.064965+0000","last_peered":"2023-11-28T13:08:04.064965+0000","last_clean":"2023-11-28T13:08:04.064965+0000","last_became_active":"2023-11-28T13:08:03.083879+0000","last_became_peered":"2023-11-28T13:08:03.083879+0000","last_unstale":"2023-11-28T13:08:04.064965+0000","last_undegraded":"2023-11-28T13:08:04.064965+0000","last_fullsized":"2023-11-28T13:08:04.064965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:14.015315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.071066+0000","last_change":"2023-11-28T13:08:02.584107+0000","last_active":"2023-11-28T13:08:04.071066+0000","last_peered":"2023-11-28T13:08:04.071066+0000","last_clean":"2023-11-28T13:08:04.071066+0000","last_became_active":"2023-11-28T13:08:02.583702+0000","last_became_peered":"2023-11-28T13:08:02.583702+0000","last_unstale":"2023-11-28T13:08:04.071066+0000","last_undegraded":"2023-11-28T13:08:04.071066+0000","last_fullsized":"2023-11-28T13:08:04.071066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:37.724216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069638+0000","last_change":"2023-11-28T13:08:03.077169+0000","last_active":"2023-11-28T13:08:04.069638+0000","last_peered":"2023-11-28T13:08:04.069638+0000","last_clean":"2023-11-28T13:08:04.069638+0000","last_became_active":"2023-11-28T13:08:03.076875+0000","last_became_peered":"2023-11-28T13:08:03.076875+0000","last_unstale":"2023-11-28T13:08:04.069638+0000","last_undegraded":"2023-11-28T13:08:04.069638+0000","last_fullsized":"2023-11-28T13:08:04.069638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:37:44.657148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.071007+0000","last_change":"2023-11-28T13:08:02.584070+0000","last_active":"2023-11-28T13:08:04.071007+0000","last_peered":"2023-11-28T13:08:04.071007+0000","last_clean":"2023-11-28T13:08:04.071007+0000","last_became_active":"2023-11-28T13:08:02.583527+0000","last_became_peered":"2023-11-28T13:08:02.583527+0000","last_unstale":"2023-11-28T13:08:04.071007+0000","last_undegraded":"2023-11-28T13:08:04.071007+0000","last_fullsized":"2023-11-28T13:08:04.071007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:49.060104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371361+0000","last_change":"2023-11-28T13:08:03.082338+0000","last_active":"2023-11-28T13:08:04.371361+0000","last_peered":"2023-11-28T13:08:04.371361+0000","last_clean":"2023-11-28T13:08:04.371361+0000","last_became_active":"2023-11-28T13:08:03.082006+0000","last_became_peered":"2023-11-28T13:08:03.082006+0000","last_unstale":"2023-11-28T13:08:04.371361+0000","last_undegraded":"2023-11-28T13:08:04.371361+0000","last_fullsized":"2023-11-28T13:08:04.371361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:56:48.282770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070812+0000","last_change":"2023-11-28T13:08:02.587071+0000","last_active":"2023-11-28T13:08:04.070812+0000","last_peered":"2023-11-28T13:08:04.070812+0000","last_clean":"2023-11-28T13:08:04.070812+0000","last_became_active":"2023-11-28T13:08:02.586541+0000","last_became_peered":"2023-11-28T13:08:02.586541+0000","last_unstale":"2023-11-28T13:08:04.070812+0000","last_undegraded":"2023-11-28T13:08:04.070812+0000","last_fullsized":"2023-11-28T13:08:04.070812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:56.535608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372224+0000","last_change":"2023-11-28T13:08:03.081008+0000","last_active":"2023-11-28T13:08:04.372224+0000","last_peered":"2023-11-28T13:08:04.372224+0000","last_clean":"2023-11-28T13:08:04.372224+0000","last_became_active":"2023-11-28T13:08:03.080718+0000","last_became_peered":"2023-11-28T13:08:03.080718+0000","last_unstale":"2023-11-28T13:08:04.372224+0000","last_undegraded":"2023-11-28T13:08:04.372224+0000","last_fullsized":"2023-11-28T13:08:04.372224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:00.400817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069777+0000","last_change":"2023-11-28T13:08:02.577784+0000","last_active":"2023-11-28T13:08:04.069777+0000","last_peered":"2023-11-28T13:08:04.069777+0000","last_clean":"2023-11-28T13:08:04.069777+0000","last_became_active":"2023-11-28T13:08:02.577364+0000","last_became_peered":"2023-11-28T13:08:02.577364+0000","last_unstale":"2023-11-28T13:08:04.069777+0000","last_undegraded":"2023-11-28T13:08:04.069777+0000","last_fullsized":"2023-11-28T13:08:04.069777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:37.654488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372256+0000","last_change":"2023-11-28T13:08:03.075105+0000","last_active":"2023-11-28T13:08:04.372256+0000","last_peered":"2023-11-28T13:08:04.372256+0000","last_clean":"2023-11-28T13:08:04.372256+0000","last_became_active":"2023-11-28T13:08:03.074768+0000","last_became_peered":"2023-11-28T13:08:03.074768+0000","last_unstale":"2023-11-28T13:08:04.372256+0000","last_undegraded":"2023-11-28T13:08:04.372256+0000","last_fullsized":"2023-11-28T13:08:04.372256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:37:44.657148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064578+0000","last_change":"2023-11-28T13:08:02.582099+0000","last_active":"2023-11-28T13:08:04.064578+0000","last_peered":"2023-11-28T13:08:04.064578+0000","last_clean":"2023-11-28T13:08:04.064578+0000","last_became_active":"2023-11-28T13:08:02.581651+0000","last_became_peered":"2023-11-28T13:08:02.581651+0000","last_unstale":"2023-11-28T13:08:04.064578+0000","last_undegraded":"2023-11-28T13:08:04.064578+0000","last_fullsized":"2023-11-28T13:08:04.064578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:12.214372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371794+0000","last_change":"2023-11-28T13:08:03.088535+0000","last_active":"2023-11-28T13:08:04.371794+0000","last_peered":"2023-11-28T13:08:04.371794+0000","last_clean":"2023-11-28T13:08:04.371794+0000","last_became_active":"2023-11-28T13:08:03.088263+0000","last_became_peered":"2023-11-28T13:08:03.088263+0000","last_unstale":"2023-11-28T13:08:04.371794+0000","last_undegraded":"2023-11-28T13:08:04.371794+0000","last_fullsized":"2023-11-28T13:08:04.371794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:22.644742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372159+0000","last_change":"2023-11-28T13:08:02.584910+0000","last_active":"2023-11-28T13:08:04.372159+0000","last_peered":"2023-11-28T13:08:04.372159+0000","last_clean":"2023-11-28T13:08:04.372159+0000","last_became_active":"2023-11-28T13:08:02.584573+0000","last_became_peered":"2023-11-28T13:08:02.584573+0000","last_unstale":"2023-11-28T13:08:04.372159+0000","last_undegraded":"2023-11-28T13:08:04.372159+0000","last_fullsized":"2023-11-28T13:08:04.372159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:49.593563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372100+0000","last_change":"2023-11-28T13:08:03.083587+0000","last_active":"2023-11-28T13:08:04.372100+0000","last_peered":"2023-11-28T13:08:04.372100+0000","last_clean":"2023-11-28T13:08:04.372100+0000","last_became_active":"2023-11-28T13:08:03.081658+0000","last_became_peered":"2023-11-28T13:08:03.081658+0000","last_unstale":"2023-11-28T13:08:04.372100+0000","last_undegraded":"2023-11-28T13:08:04.372100+0000","last_fullsized":"2023-11-28T13:08:04.372100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:53.049281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371468+0000","last_change":"2023-11-28T13:08:02.582045+0000","last_active":"2023-11-28T13:08:04.371468+0000","last_peered":"2023-11-28T13:08:04.371468+0000","last_clean":"2023-11-28T13:08:04.371468+0000","last_became_active":"2023-11-28T13:08:02.581687+0000","last_became_peered":"2023-11-28T13:08:02.581687+0000","last_unstale":"2023-11-28T13:08:04.371468+0000","last_undegraded":"2023-11-28T13:08:04.371468+0000","last_fullsized":"2023-11-28T13:08:04.371468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:56.910035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372483+0000","last_change":"2023-11-28T13:08:03.090341+0000","last_active":"2023-11-28T13:08:04.372483+0000","last_peered":"2023-11-28T13:08:04.372483+0000","last_clean":"2023-11-28T13:08:04.372483+0000","last_became_active":"2023-11-28T13:08:03.086320+0000","last_became_peered":"2023-11-28T13:08:03.086320+0000","last_unstale":"2023-11-28T13:08:04.372483+0000","last_undegraded":"2023-11-28T13:08:04.372483+0000","last_fullsized":"2023-11-28T13:08:04.372483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:30.263143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064693+0000","last_change":"2023-11-28T13:08:02.577006+0000","last_active":"2023-11-28T13:08:04.064693+0000","last_peered":"2023-11-28T13:08:04.064693+0000","last_clean":"2023-11-28T13:08:04.064693+0000","last_became_active":"2023-11-28T13:08:02.576734+0000","last_became_peered":"2023-11-28T13:08:02.576734+0000","last_unstale":"2023-11-28T13:08:04.064693+0000","last_undegraded":"2023-11-28T13:08:04.064693+0000","last_fullsized":"2023-11-28T13:08:04.064693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:40.847112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372143+0000","last_change":"2023-11-28T13:08:03.080548+0000","last_active":"2023-11-28T13:08:04.372143+0000","last_peered":"2023-11-28T13:08:04.372143+0000","last_clean":"2023-11-28T13:08:04.372143+0000","last_became_active":"2023-11-28T13:08:03.077609+0000","last_became_peered":"2023-11-28T13:08:03.077609+0000","last_unstale":"2023-11-28T13:08:04.372143+0000","last_undegraded":"2023-11-28T13:08:04.372143+0000","last_fullsized":"2023-11-28T13:08:04.372143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:57.732415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064801+0000","last_change":"2023-11-28T13:08:02.579378+0000","last_active":"2023-11-28T13:08:04.064801+0000","last_peered":"2023-11-28T13:08:04.064801+0000","last_clean":"2023-11-28T13:08:04.064801+0000","last_became_active":"2023-11-28T13:08:02.579080+0000","last_became_peered":"2023-11-28T13:08:02.579080+0000","last_unstale":"2023-11-28T13:08:04.064801+0000","last_undegraded":"2023-11-28T13:08:04.064801+0000","last_fullsized":"2023-11-28T13:08:04.064801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:36.421141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.075126+0000","last_change":"2023-11-28T13:07:56.890378+0000","last_active":"2023-11-28T13:08:03.075126+0000","last_peered":"2023-11-28T13:08:03.075126+0000","last_clean":"2023-11-28T13:08:03.075126+0000","last_became_active":"2023-11-28T13:07:56.890058+0000","last_became_peered":"2023-11-28T13:07:56.890058+0000","last_unstale":"2023-11-28T13:08:03.075126+0000","last_undegraded":"2023-11-28T13:08:03.075126+0000","last_fullsized":"2023-11-28T13:08:03.075126+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:05.344602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064197+0000","last_change":"2023-11-28T13:08:03.084094+0000","last_active":"2023-11-28T13:08:04.064197+0000","last_peered":"2023-11-28T13:08:04.064197+0000","last_clean":"2023-11-28T13:08:04.064197+0000","last_became_active":"2023-11-28T13:08:03.083811+0000","last_became_peered":"2023-11-28T13:08:03.083811+0000","last_unstale":"2023-11-28T13:08:04.064197+0000","last_undegraded":"2023-11-28T13:08:04.064197+0000","last_fullsized":"2023-11-28T13:08:04.064197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:53.049281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.072799+0000","last_change":"2023-11-28T13:07:56.844509+0000","last_active":"2023-11-28T13:08:03.072799+0000","last_peered":"2023-11-28T13:08:03.072799+0000","last_clean":"2023-11-28T13:08:03.072799+0000","last_became_active":"2023-11-28T13:07:56.844081+0000","last_became_peered":"2023-11-28T13:07:56.844081+0000","last_unstale":"2023-11-28T13:08:03.072799+0000","last_undegraded":"2023-11-28T13:08:03.072799+0000","last_fullsized":"2023-11-28T13:08:03.072799+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:05.095447+0000","last_change":"2023-11-28T13:08:02.576708+0000","last_active":"2023-11-28T13:08:05.095447+0000","last_peered":"2023-11-28T13:08:05.095447+0000","last_clean":"2023-11-28T13:08:05.095447+0000","last_became_active":"2023-11-28T13:08:02.572927+0000","last_became_peered":"2023-11-28T13:08:02.572927+0000","last_unstale":"2023-11-28T13:08:05.095447+0000","last_undegraded":"2023-11-28T13:08:05.095447+0000","last_fullsized":"2023-11-28T13:08:05.095447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:43.117989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372278+0000","last_change":"2023-11-28T13:08:03.093540+0000","last_active":"2023-11-28T13:08:04.372278+0000","last_peered":"2023-11-28T13:08:04.372278+0000","last_clean":"2023-11-28T13:08:04.372278+0000","last_became_active":"2023-11-28T13:08:03.089698+0000","last_became_peered":"2023-11-28T13:08:03.089698+0000","last_unstale":"2023-11-28T13:08:04.372278+0000","last_undegraded":"2023-11-28T13:08:04.372278+0000","last_fullsized":"2023-11-28T13:08:04.372278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:29:46.171857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.062393+0000","last_change":"2023-11-28T13:07:56.844699+0000","last_active":"2023-11-28T13:08:03.062393+0000","last_peered":"2023-11-28T13:08:03.062393+0000","last_clean":"2023-11-28T13:08:03.062393+0000","last_became_active":"2023-11-28T13:07:56.844391+0000","last_became_peered":"2023-11-28T13:07:56.844391+0000","last_unstale":"2023-11-28T13:08:03.062393+0000","last_undegraded":"2023-11-28T13:08:03.062393+0000","last_fullsized":"2023-11-28T13:08:03.062393+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.074953+0000","last_change":"2023-11-28T13:07:57.844020+0000","last_active":"2023-11-28T13:08:03.074953+0000","last_peered":"2023-11-28T13:08:03.074953+0000","last_clean":"2023-11-28T13:08:03.074953+0000","last_became_active":"2023-11-28T13:07:57.842981+0000","last_became_peered":"2023-11-28T13:07:57.842981+0000","last_unstale":"2023-11-28T13:08:03.074953+0000","last_undegraded":"2023-11-28T13:08:03.074953+0000","last_fullsized":"2023-11-28T13:08:03.074953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_clean_scrub_stamp":"2023-11-28T13:07:56.825711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:44.973218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371955+0000","last_change":"2023-11-28T13:08:02.583813+0000","last_active":"2023-11-28T13:08:04.371955+0000","last_peered":"2023-11-28T13:08:04.371955+0000","last_clean":"2023-11-28T13:08:04.371955+0000","last_became_active":"2023-11-28T13:08:02.583528+0000","last_became_peered":"2023-11-28T13:08:02.583528+0000","last_unstale":"2023-11-28T13:08:04.371955+0000","last_undegraded":"2023-11-28T13:08:04.371955+0000","last_fullsized":"2023-11-28T13:08:04.371955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:49.593563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372411+0000","last_change":"2023-11-28T13:08:03.077611+0000","last_active":"2023-11-28T13:08:04.372411+0000","last_peered":"2023-11-28T13:08:04.372411+0000","last_clean":"2023-11-28T13:08:04.372411+0000","last_became_active":"2023-11-28T13:08:03.076746+0000","last_became_peered":"2023-11-28T13:08:03.076746+0000","last_unstale":"2023-11-28T13:08:04.372411+0000","last_undegraded":"2023-11-28T13:08:04.372411+0000","last_fullsized":"2023-11-28T13:08:04.372411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:49.707036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.075246+0000","last_change":"2023-11-28T13:07:57.306318+0000","last_active":"2023-11-28T13:08:03.075246+0000","last_peered":"2023-11-28T13:08:03.075246+0000","last_clean":"2023-11-28T13:08:03.075246+0000","last_became_active":"2023-11-28T13:07:57.305937+0000","last_became_peered":"2023-11-28T13:07:57.305937+0000","last_unstale":"2023-11-28T13:08:03.075246+0000","last_undegraded":"2023-11-28T13:08:03.075246+0000","last_fullsized":"2023-11-28T13:08:03.075246+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371445+0000","last_change":"2023-11-28T13:08:02.569783+0000","last_active":"2023-11-28T13:08:04.371445+0000","last_peered":"2023-11-28T13:08:04.371445+0000","last_clean":"2023-11-28T13:08:04.371445+0000","last_became_active":"2023-11-28T13:08:02.569505+0000","last_became_peered":"2023-11-28T13:08:02.569505+0000","last_unstale":"2023-11-28T13:08:04.371445+0000","last_undegraded":"2023-11-28T13:08:04.371445+0000","last_fullsized":"2023-11-28T13:08:04.371445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:12.363239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372117+0000","last_change":"2023-11-28T13:08:03.094969+0000","last_active":"2023-11-28T13:08:04.372117+0000","last_peered":"2023-11-28T13:08:04.372117+0000","last_clean":"2023-11-28T13:08:04.372117+0000","last_became_active":"2023-11-28T13:08:03.094723+0000","last_became_peered":"2023-11-28T13:08:03.094723+0000","last_unstale":"2023-11-28T13:08:04.372117+0000","last_undegraded":"2023-11-28T13:08:04.372117+0000","last_fullsized":"2023-11-28T13:08:04.372117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:37:44.657148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.074562+0000","last_change":"2023-11-28T13:07:57.306277+0000","last_active":"2023-11-28T13:08:03.074562+0000","last_peered":"2023-11-28T13:08:03.074562+0000","last_clean":"2023-11-28T13:08:03.074562+0000","last_became_active":"2023-11-28T13:07:57.305983+0000","last_became_peered":"2023-11-28T13:07:57.305983+0000","last_unstale":"2023-11-28T13:08:03.074562+0000","last_undegraded":"2023-11-28T13:08:03.074562+0000","last_fullsized":"2023-11-28T13:08:03.074562+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064565+0000","last_change":"2023-11-28T13:08:02.578428+0000","last_active":"2023-11-28T13:08:04.064565+0000","last_peered":"2023-11-28T13:08:04.064565+0000","last_clean":"2023-11-28T13:08:04.064565+0000","last_became_active":"2023-11-28T13:08:02.578147+0000","last_became_peered":"2023-11-28T13:08:02.578147+0000","last_unstale":"2023-11-28T13:08:04.064565+0000","last_undegraded":"2023-11-28T13:08:04.064565+0000","last_fullsized":"2023-11-28T13:08:04.064565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:43.117989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372191+0000","last_change":"2023-11-28T13:08:03.094387+0000","last_active":"2023-11-28T13:08:04.372191+0000","last_peered":"2023-11-28T13:08:04.372191+0000","last_clean":"2023-11-28T13:08:04.372191+0000","last_became_active":"2023-11-28T13:08:03.094117+0000","last_became_peered":"2023-11-28T13:08:03.094117+0000","last_unstale":"2023-11-28T13:08:04.372191+0000","last_undegraded":"2023-11-28T13:08:04.372191+0000","last_fullsized":"2023-11-28T13:08:04.372191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:38.387013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.068929+0000","last_change":"2023-11-28T13:07:57.306177+0000","last_active":"2023-11-28T13:08:03.068929+0000","last_peered":"2023-11-28T13:08:03.068929+0000","last_clean":"2023-11-28T13:08:03.068929+0000","last_became_active":"2023-11-28T13:07:57.305853+0000","last_became_peered":"2023-11-28T13:07:57.305853+0000","last_unstale":"2023-11-28T13:08:03.068929+0000","last_undegraded":"2023-11-28T13:08:03.068929+0000","last_fullsized":"2023-11-28T13:08:03.068929+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371895+0000","last_change":"2023-11-28T13:08:02.584788+0000","last_active":"2023-11-28T13:08:04.371895+0000","last_peered":"2023-11-28T13:08:04.371895+0000","last_clean":"2023-11-28T13:08:04.371895+0000","last_became_active":"2023-11-28T13:08:02.584281+0000","last_became_peered":"2023-11-28T13:08:02.584281+0000","last_unstale":"2023-11-28T13:08:04.371895+0000","last_undegraded":"2023-11-28T13:08:04.371895+0000","last_fullsized":"2023-11-28T13:08:04.371895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:40.847112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371995+0000","last_change":"2023-11-28T13:08:03.074701+0000","last_active":"2023-11-28T13:08:04.371995+0000","last_peered":"2023-11-28T13:08:04.371995+0000","last_clean":"2023-11-28T13:08:04.371995+0000","last_became_active":"2023-11-28T13:08:03.074336+0000","last_became_peered":"2023-11-28T13:08:03.074336+0000","last_unstale":"2023-11-28T13:08:04.371995+0000","last_undegraded":"2023-11-28T13:08:04.371995+0000","last_fullsized":"2023-11-28T13:08:04.371995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:38.387013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.074084+0000","last_change":"2023-11-28T13:07:57.306321+0000","last_active":"2023-11-28T13:08:03.074084+0000","last_peered":"2023-11-28T13:08:03.074084+0000","last_clean":"2023-11-28T13:08:03.074084+0000","last_became_active":"2023-11-28T13:07:57.305923+0000","last_became_peered":"2023-11-28T13:07:57.305923+0000","last_unstale":"2023-11-28T13:08:03.074084+0000","last_undegraded":"2023-11-28T13:08:03.074084+0000","last_fullsized":"2023-11-28T13:08:03.074084+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:41.104679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371927+0000","last_change":"2023-11-28T13:08:02.570790+0000","last_active":"2023-11-28T13:08:04.371927+0000","last_peered":"2023-11-28T13:08:04.371927+0000","last_clean":"2023-11-28T13:08:04.371927+0000","last_became_active":"2023-11-28T13:08:02.569907+0000","last_became_peered":"2023-11-28T13:08:02.569907+0000","last_unstale":"2023-11-28T13:08:04.371927+0000","last_undegraded":"2023-11-28T13:08:04.371927+0000","last_fullsized":"2023-11-28T13:08:04.371927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:46.379743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372610+0000","last_change":"2023-11-28T13:08:03.087530+0000","last_active":"2023-11-28T13:08:04.372610+0000","last_peered":"2023-11-28T13:08:04.372610+0000","last_clean":"2023-11-28T13:08:04.372610+0000","last_became_active":"2023-11-28T13:08:03.087191+0000","last_became_peered":"2023-11-28T13:08:03.087191+0000","last_unstale":"2023-11-28T13:08:04.372610+0000","last_undegraded":"2023-11-28T13:08:04.372610+0000","last_fullsized":"2023-11-28T13:08:04.372610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:38.726877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.074497+0000","last_change":"2023-11-28T13:07:56.844305+0000","last_active":"2023-11-28T13:08:03.074497+0000","last_peered":"2023-11-28T13:08:03.074497+0000","last_clean":"2023-11-28T13:08:03.074497+0000","last_became_active":"2023-11-28T13:07:56.843980+0000","last_became_peered":"2023-11-28T13:07:56.843980+0000","last_unstale":"2023-11-28T13:08:03.074497+0000","last_undegraded":"2023-11-28T13:08:03.074497+0000","last_fullsized":"2023-11-28T13:08:03.074497+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070218+0000","last_change":"2023-11-28T13:08:02.577243+0000","last_active":"2023-11-28T13:08:04.070218+0000","last_peered":"2023-11-28T13:08:04.070218+0000","last_clean":"2023-11-28T13:08:04.070218+0000","last_became_active":"2023-11-28T13:08:02.576748+0000","last_became_peered":"2023-11-28T13:08:02.576748+0000","last_unstale":"2023-11-28T13:08:04.070218+0000","last_undegraded":"2023-11-28T13:08:04.070218+0000","last_fullsized":"2023-11-28T13:08:04.070218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:01.496804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371797+0000","last_change":"2023-11-28T13:08:03.095221+0000","last_active":"2023-11-28T13:08:04.371797+0000","last_peered":"2023-11-28T13:08:04.371797+0000","last_clean":"2023-11-28T13:08:04.371797+0000","last_became_active":"2023-11-28T13:08:03.091841+0000","last_became_peered":"2023-11-28T13:08:03.091841+0000","last_unstale":"2023-11-28T13:08:04.371797+0000","last_undegraded":"2023-11-28T13:08:04.371797+0000","last_fullsized":"2023-11-28T13:08:04.371797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:43.340474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372325+0000","last_change":"2023-11-28T13:08:02.583653+0000","last_active":"2023-11-28T13:08:04.372325+0000","last_peered":"2023-11-28T13:08:04.372325+0000","last_clean":"2023-11-28T13:08:04.372325+0000","last_became_active":"2023-11-28T13:08:02.583397+0000","last_became_peered":"2023-11-28T13:08:02.583397+0000","last_unstale":"2023-11-28T13:08:04.372325+0000","last_undegraded":"2023-11-28T13:08:04.372325+0000","last_fullsized":"2023-11-28T13:08:04.372325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:40.847112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371135+0000","last_change":"2023-11-28T13:08:03.096716+0000","last_active":"2023-11-28T13:08:04.371135+0000","last_peered":"2023-11-28T13:08:04.371135+0000","last_clean":"2023-11-28T13:08:04.371135+0000","last_became_active":"2023-11-28T13:08:03.096239+0000","last_became_peered":"2023-11-28T13:08:03.096239+0000","last_unstale":"2023-11-28T13:08:04.371135+0000","last_undegraded":"2023-11-28T13:08:04.371135+0000","last_fullsized":"2023-11-28T13:08:04.371135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:13.217034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069581+0000","last_change":"2023-11-28T13:08:02.581525+0000","last_active":"2023-11-28T13:08:04.069581+0000","last_peered":"2023-11-28T13:08:04.069581+0000","last_clean":"2023-11-28T13:08:04.069581+0000","last_became_active":"2023-11-28T13:08:02.581253+0000","last_became_peered":"2023-11-28T13:08:02.581253+0000","last_unstale":"2023-11-28T13:08:04.069581+0000","last_undegraded":"2023-11-28T13:08:04.069581+0000","last_fullsized":"2023-11-28T13:08:04.069581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:36.421141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372111+0000","last_change":"2023-11-28T13:08:03.084538+0000","last_active":"2023-11-28T13:08:04.372111+0000","last_peered":"2023-11-28T13:08:04.372111+0000","last_clean":"2023-11-28T13:08:04.372111+0000","last_became_active":"2023-11-28T13:08:03.084256+0000","last_became_peered":"2023-11-28T13:08:03.084256+0000","last_unstale":"2023-11-28T13:08:04.372111+0000","last_undegraded":"2023-11-28T13:08:04.372111+0000","last_fullsized":"2023-11-28T13:08:04.372111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:46:46.224857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070254+0000","last_change":"2023-11-28T13:08:02.583812+0000","last_active":"2023-11-28T13:08:04.070254+0000","last_peered":"2023-11-28T13:08:04.070254+0000","last_clean":"2023-11-28T13:08:04.070254+0000","last_became_active":"2023-11-28T13:08:02.583517+0000","last_became_peered":"2023-11-28T13:08:02.583517+0000","last_unstale":"2023-11-28T13:08:04.070254+0000","last_undegraded":"2023-11-28T13:08:04.070254+0000","last_fullsized":"2023-11-28T13:08:04.070254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:12.214372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371726+0000","last_change":"2023-11-28T13:08:03.088006+0000","last_active":"2023-11-28T13:08:04.371726+0000","last_peered":"2023-11-28T13:08:04.371726+0000","last_clean":"2023-11-28T13:08:04.371726+0000","last_became_active":"2023-11-28T13:08:03.086226+0000","last_became_peered":"2023-11-28T13:08:03.086226+0000","last_unstale":"2023-11-28T13:08:04.371726+0000","last_undegraded":"2023-11-28T13:08:04.371726+0000","last_fullsized":"2023-11-28T13:08:04.371726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:08.985463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064555+0000","last_change":"2023-11-28T13:08:02.574322+0000","last_active":"2023-11-28T13:08:04.064555+0000","last_peered":"2023-11-28T13:08:04.064555+0000","last_clean":"2023-11-28T13:08:04.064555+0000","last_became_active":"2023-11-28T13:08:02.573611+0000","last_became_peered":"2023-11-28T13:08:02.573611+0000","last_unstale":"2023-11-28T13:08:04.064555+0000","last_undegraded":"2023-11-28T13:08:04.064555+0000","last_fullsized":"2023-11-28T13:08:04.064555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:37.654488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064305+0000","last_change":"2023-11-28T13:08:03.093751+0000","last_active":"2023-11-28T13:08:04.064305+0000","last_peered":"2023-11-28T13:08:04.064305+0000","last_clean":"2023-11-28T13:08:04.064305+0000","last_became_active":"2023-11-28T13:08:03.093493+0000","last_became_peered":"2023-11-28T13:08:03.093493+0000","last_unstale":"2023-11-28T13:08:04.064305+0000","last_undegraded":"2023-11-28T13:08:04.064305+0000","last_fullsized":"2023-11-28T13:08:04.064305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:46.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372733+0000","last_change":"2023-11-28T13:08:02.580561+0000","last_active":"2023-11-28T13:08:04.372733+0000","last_peered":"2023-11-28T13:08:04.372733+0000","last_clean":"2023-11-28T13:08:04.372733+0000","last_became_active":"2023-11-28T13:08:02.580273+0000","last_became_peered":"2023-11-28T13:08:02.580273+0000","last_unstale":"2023-11-28T13:08:04.372733+0000","last_undegraded":"2023-11-28T13:08:04.372733+0000","last_fullsized":"2023-11-28T13:08:04.372733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:37.654488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371182+0000","last_change":"2023-11-28T13:08:03.089735+0000","last_active":"2023-11-28T13:08:04.371182+0000","last_peered":"2023-11-28T13:08:04.371182+0000","last_clean":"2023-11-28T13:08:04.371182+0000","last_became_active":"2023-11-28T13:08:03.089492+0000","last_became_peered":"2023-11-28T13:08:03.089492+0000","last_unstale":"2023-11-28T13:08:04.371182+0000","last_undegraded":"2023-11-28T13:08:04.371182+0000","last_fullsized":"2023-11-28T13:08:04.371182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:41.196768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371414+0000","last_change":"2023-11-28T13:08:03.082551+0000","last_active":"2023-11-28T13:08:04.371414+0000","last_peered":"2023-11-28T13:08:04.371414+0000","last_clean":"2023-11-28T13:08:04.371414+0000","last_became_active":"2023-11-28T13:08:03.082220+0000","last_became_peered":"2023-11-28T13:08:03.082220+0000","last_unstale":"2023-11-28T13:08:04.371414+0000","last_undegraded":"2023-11-28T13:08:04.371414+0000","last_fullsized":"2023-11-28T13:08:04.371414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:57.538270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064240+0000","last_change":"2023-11-28T13:08:03.095511+0000","last_active":"2023-11-28T13:08:04.064240+0000","last_peered":"2023-11-28T13:08:04.064240+0000","last_clean":"2023-11-28T13:08:04.064240+0000","last_became_active":"2023-11-28T13:08:03.095214+0000","last_became_peered":"2023-11-28T13:08:03.095214+0000","last_unstale":"2023-11-28T13:08:04.064240+0000","last_undegraded":"2023-11-28T13:08:04.064240+0000","last_fullsized":"2023-11-28T13:08:04.064240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:13.351182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371631+0000","last_change":"2023-11-28T13:08:03.089514+0000","last_active":"2023-11-28T13:08:04.371631+0000","last_peered":"2023-11-28T13:08:04.371631+0000","last_clean":"2023-11-28T13:08:04.371631+0000","last_became_active":"2023-11-28T13:08:03.089266+0000","last_became_peered":"2023-11-28T13:08:03.089266+0000","last_unstale":"2023-11-28T13:08:04.371631+0000","last_undegraded":"2023-11-28T13:08:04.371631+0000","last_fullsized":"2023-11-28T13:08:04.371631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:58.181120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064673+0000","last_change":"2023-11-28T13:08:03.084369+0000","last_active":"2023-11-28T13:08:04.064673+0000","last_peered":"2023-11-28T13:08:04.064673+0000","last_clean":"2023-11-28T13:08:04.064673+0000","last_became_active":"2023-11-28T13:08:03.084081+0000","last_became_peered":"2023-11-28T13:08:03.084081+0000","last_unstale":"2023-11-28T13:08:04.064673+0000","last_undegraded":"2023-11-28T13:08:04.064673+0000","last_fullsized":"2023-11-28T13:08:04.064673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:13.625233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371927+0000","last_change":"2023-11-28T13:08:03.093572+0000","last_active":"2023-11-28T13:08:04.371927+0000","last_peered":"2023-11-28T13:08:04.371927+0000","last_clean":"2023-11-28T13:08:04.371927+0000","last_became_active":"2023-11-28T13:08:03.093306+0000","last_became_peered":"2023-11-28T13:08:03.093306+0000","last_unstale":"2023-11-28T13:08:04.371927+0000","last_undegraded":"2023-11-28T13:08:04.371927+0000","last_fullsized":"2023-11-28T13:08:04.371927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:36.892676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9084,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362756,"statfs":{"total":96636764160,"available":96627462144,"internally_reserved":0,"allocated":839680,"data_stored":671592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7964,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363876,"statfs":{"total":96636764160,"available":96628609024,"internally_reserved":0,"allocated":217088,"data_stored":79460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13152,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358688,"statfs":{"total":96636764160,"available":96623296512,"internally_reserved":0,"allocated":811008,"data_stored":669828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12124,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359716,"statfs":{"total":96636764160,"available":96624349184,"internally_reserved":0,"allocated":217088,"data_stored":79040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11996,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":11776,"kb_avail":94359844,"statfs":{"total":96636764160,"available":96624480256,"internally_reserved":0,"allocated":217088,"data_stored":79460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12058624},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12128,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359712,"statfs":{"total":96636764160,"available":96624345088,"internally_reserved":0,"allocated":221184,"data_stored":79550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8048,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363792,"statfs":{"total":96636764160,"available":96628523008,"internally_reserved":0,"allocated":237568,"data_stored":81064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7972,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363868,"statfs":{"total":96636764160,"available":96628600832,"internally_reserved":0,"allocated":225280,"data_stored":80096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:08:11.220 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:08:11.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.452+0000 7fe31a9ff700 1 Processor -- start 2023-11-28T13:08:11.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.452+0000 7fe31a9ff700 1 -- start start 2023-11-28T13:08:11.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.453+0000 7fe31a9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe31415c9b0 0x7fe3140b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:11.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.453+0000 7fe31a9ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3140d1950 con 0x7fe314161340 2023-11-28T13:08:11.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.453+0000 7fe31a9ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3140d24f0 con 0x7fe3140cd7c0 2023-11-28T13:08:11.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.453+0000 7fe31a9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3140b5800 con 0x7fe31415c9b0 2023-11-28T13:08:11.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.454+0000 7fe3137fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fe314161340 0x7fe3140d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:34658/0 (socket says 172.21.15.156:34658) 2023-11-28T13:08:11.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.454+0000 7fe3137fe700 1 -- 172.21.15.156:0/34958426 learned_addr learned my addr 172.21.15.156:0/34958426 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:11.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.454+0000 7fe313fff700 1 --2- 172.21.15.156:0/34958426 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe31415c9b0 0x7fe3140b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:11.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.454+0000 7fe312ffd700 1 -- 172.21.15.156:0/34958426 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259233258 0 0) 0x7fe3140d1950 con 0x7fe314161340 2023-11-28T13:08:11.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.454+0000 7fe312ffd700 1 -- 172.21.15.156:0/34958426 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2fc0035b0 con 0x7fe314161340 2023-11-28T13:08:11.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.454+0000 7fe312ffd700 1 -- 172.21.15.156:0/34958426 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794809618 0 0) 0x7fe3140d24f0 con 0x7fe3140cd7c0 2023-11-28T13:08:11.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.454+0000 7fe312ffd700 1 -- 172.21.15.156:0/34958426 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3140d1950 con 0x7fe3140cd7c0 2023-11-28T13:08:11.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.454+0000 7fe313fff700 1 -- 172.21.15.156:0/34958426 >> v1:172.21.15.186:6789/0 conn(0x7fe3140cd7c0 legacy=0x7fe3140b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:11.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.454+0000 7fe313fff700 1 -- 172.21.15.156:0/34958426 >> v1:172.21.15.156:6789/0 conn(0x7fe314161340 legacy=0x7fe3140d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:11.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.454+0000 7fe312ffd700 1 -- 172.21.15.156:0/34958426 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1721090915 0 0) 0x7fe2fc0035b0 con 0x7fe314161340 2023-11-28T13:08:11.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.454+0000 7fe313fff700 1 -- 172.21.15.156:0/34958426 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3140b5520 con 0x7fe31415c9b0 2023-11-28T13:08:11.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.455+0000 7fe313fff700 1 --2- 172.21.15.156:0/34958426 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe31415c9b0 0x7fe3140b4bf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe304009870 tx=0x7fe304009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=908e2ade090a5dfe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:11.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.455+0000 7fe312ffd700 1 -- 172.21.15.156:0/34958426 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe308001670 con 0x7fe31415c9b0 2023-11-28T13:08:11.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.455+0000 7fe312ffd700 1 -- 172.21.15.156:0/34958426 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe304004030 con 0x7fe31415c9b0 2023-11-28T13:08:11.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.455+0000 7fe312ffd700 1 -- 172.21.15.156:0/34958426 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3040043c0 con 0x7fe31415c9b0 2023-11-28T13:08:11.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.455+0000 7fe31a9ff700 1 -- 172.21.15.156:0/34958426 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe31415c9b0 msgr2=0x7fe3140b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:11.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.455+0000 7fe31a9ff700 1 --2- 172.21.15.156:0/34958426 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe31415c9b0 0x7fe3140b4bf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe304009870 tx=0x7fe304009b80 comp rx=0 tx=0).stop 2023-11-28T13:08:11.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.455+0000 7fe31a9ff700 1 -- 172.21.15.156:0/34958426 shutdown_connections 2023-11-28T13:08:11.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.455+0000 7fe31a9ff700 1 --2- 172.21.15.156:0/34958426 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe31415c9b0 0x7fe3140b4bf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.455+0000 7fe31a9ff700 1 -- 172.21.15.156:0/34958426 >> 172.21.15.156:0/34958426 conn(0x7fe314072600 msgr2=0x7fe3140cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:11.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.456+0000 7fe31a9ff700 1 -- 172.21.15.156:0/34958426 shutdown_connections 2023-11-28T13:08:11.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.456+0000 7fe31a9ff700 1 -- 172.21.15.156:0/34958426 wait complete. 2023-11-28T13:08:11.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.458+0000 7fe31a9ff700 1 Processor -- start 2023-11-28T13:08:11.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.458+0000 7fe31a9ff700 1 -- start start 2023-11-28T13:08:11.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.459+0000 7fe31a9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3140cd7c0 0x7fe3141e6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:11.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.459+0000 7fe31a9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe31415c9b0 0x7fe3141f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:11.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.459+0000 7fe31a9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe314161340 0x7fe3140cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:11.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.459+0000 7fe31a9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3141e7450 con 0x7fe3140cd7c0 2023-11-28T13:08:11.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.459+0000 7fe31a9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3141e7240 con 0x7fe31415c9b0 2023-11-28T13:08:11.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.459+0000 7fe31a9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3140cd2c0 con 0x7fe314161340 2023-11-28T13:08:11.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.459+0000 7fe313fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3140cd7c0 0x7fe3141e6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:11.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.460+0000 7fe313fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3140cd7c0 0x7fe3141e6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59044/0 (socket says 172.21.15.156:59044) 2023-11-28T13:08:11.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.460+0000 7fe313fff700 1 -- 172.21.15.156:0/2020166488 learned_addr learned my addr 172.21.15.156:0/2020166488 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:11.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.460+0000 7fe3137fe700 1 --2- 172.21.15.156:0/2020166488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe31415c9b0 0x7fe3141f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:11.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.460+0000 7fe318f9c700 1 --2- 172.21.15.156:0/2020166488 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe314161340 0x7fe3140cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:11.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.460+0000 7fe313fff700 1 -- 172.21.15.156:0/2020166488 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe314161340 msgr2=0x7fe3140cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:11.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.460+0000 7fe313fff700 1 --2- 172.21.15.156:0/2020166488 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe314161340 0x7fe3140cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.460+0000 7fe313fff700 1 -- 172.21.15.156:0/2020166488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe31415c9b0 msgr2=0x7fe3141f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:11.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.460+0000 7fe313fff700 1 --2- 172.21.15.156:0/2020166488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe31415c9b0 0x7fe3141f2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.460+0000 7fe313fff700 1 -- 172.21.15.156:0/2020166488 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3040094a0 con 0x7fe3140cd7c0 2023-11-28T13:08:11.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.460+0000 7fe313fff700 1 --2- 172.21.15.156:0/2020166488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3140cd7c0 0x7fe3141e6d00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe3040090b0 tx=0x7fe304004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:11.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.460+0000 7fe310ff9700 1 -- 172.21.15.156:0/2020166488 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304003df0 con 0x7fe3140cd7c0 2023-11-28T13:08:11.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.460+0000 7fe310ff9700 1 -- 172.21.15.156:0/2020166488 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30401f070 con 0x7fe3140cd7c0 2023-11-28T13:08:11.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.460+0000 7fe31a9ff700 1 -- 172.21.15.156:0/2020166488 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3140d15e0 con 0x7fe3140cd7c0 2023-11-28T13:08:11.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.460+0000 7fe310ff9700 1 -- 172.21.15.156:0/2020166488 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304011870 con 0x7fe3140cd7c0 2023-11-28T13:08:11.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.461+0000 7fe31a9ff700 1 -- 172.21.15.156:0/2020166488 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3140d1300 con 0x7fe3140cd7c0 2023-11-28T13:08:11.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.461+0000 7fe31a9ff700 1 -- 172.21.15.156:0/2020166488 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3140b5130 con 0x7fe3140cd7c0 2023-11-28T13:08:11.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.462+0000 7fe310ff9700 1 -- 172.21.15.156:0/2020166488 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe30402a020 con 0x7fe3140cd7c0 2023-11-28T13:08:11.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.463+0000 7fe310ff9700 1 --2- 172.21.15.156:0/2020166488 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe3000682f0 0x7fe30006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:11.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.463+0000 7fe310ff9700 1 -- 172.21.15.156:0/2020166488 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe3040872b0 con 0x7fe3140cd7c0 2023-11-28T13:08:11.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.464+0000 7fe3137fe700 1 --2- 172.21.15.156:0/2020166488 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe3000682f0 0x7fe30006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:11.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.464+0000 7fe3137fe700 1 --2- 172.21.15.156:0/2020166488 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe3000682f0 0x7fe30006a7a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe308007fd0 tx=0x7fe3080079e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:11.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.467+0000 7fe310ff9700 1 -- 172.21.15.156:0/2020166488 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe304053a20 con 0x7fe3140cd7c0 2023-11-28T13:08:11.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.654+0000 7fe31a9ff700 1 -- 172.21.15.156:0/2020166488 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3140745c0 con 0x7fe3000682f0 2023-11-28T13:08:11.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.663+0000 7fe310ff9700 1 -- 172.21.15.156:0/2020166488 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286837 (secure 0 0 0) 0x7fe3140745c0 con 0x7fe3000682f0 2023-11-28T13:08:11.667 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:11.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.667+0000 7fe31a9ff700 1 -- 172.21.15.156:0/2020166488 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe3000682f0 msgr2=0x7fe30006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:11.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.667+0000 7fe31a9ff700 1 --2- 172.21.15.156:0/2020166488 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe3000682f0 0x7fe30006a7a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe308007fd0 tx=0x7fe3080079e0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.667+0000 7fe31a9ff700 1 -- 172.21.15.156:0/2020166488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3140cd7c0 msgr2=0x7fe3141e6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:11.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.667+0000 7fe31a9ff700 1 --2- 172.21.15.156:0/2020166488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3140cd7c0 0x7fe3141e6d00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe3040090b0 tx=0x7fe304004750 comp rx=0 tx=0).stop 2023-11-28T13:08:11.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.667+0000 7fe31a9ff700 1 -- 172.21.15.156:0/2020166488 shutdown_connections 2023-11-28T13:08:11.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.667+0000 7fe31a9ff700 1 --2- 172.21.15.156:0/2020166488 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe3000682f0 0x7fe30006a7a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.667+0000 7fe31a9ff700 1 --2- 172.21.15.156:0/2020166488 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe314161340 0x7fe3140cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.667+0000 7fe31a9ff700 1 --2- 172.21.15.156:0/2020166488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3140cd7c0 0x7fe3141e6d00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.667+0000 7fe31a9ff700 1 --2- 172.21.15.156:0/2020166488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe31415c9b0 0x7fe3141f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.667+0000 7fe31a9ff700 1 -- 172.21.15.156:0/2020166488 >> 172.21.15.156:0/2020166488 conn(0x7fe314072600 msgr2=0x7fe3140cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:11.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.667+0000 7fe31a9ff700 1 -- 172.21.15.156:0/2020166488 shutdown_connections 2023-11-28T13:08:11.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.667+0000 7fe31a9ff700 1 -- 172.21.15.156:0/2020166488 wait complete. 2023-11-28T13:08:11.671 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T13:08:11.686 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-11-28T13:08:10.015506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":82468,"kb_used_data":2916,"kb_used_omap":0,"kb_used_meta":79488,"kb_avail":754892252,"statfs":{"total":773094113280,"available":773009666048,"internally_reserved":0,"allocated":2985984,"data_stored":1820090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81395712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.458874"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371323+0000","last_change":"2023-11-28T13:08:03.085544+0000","last_active":"2023-11-28T13:08:04.371323+0000","last_peered":"2023-11-28T13:08:04.371323+0000","last_clean":"2023-11-28T13:08:04.371323+0000","last_became_active":"2023-11-28T13:08:03.085265+0000","last_became_peered":"2023-11-28T13:08:03.085265+0000","last_unstale":"2023-11-28T13:08:04.371323+0000","last_undegraded":"2023-11-28T13:08:04.371323+0000","last_fullsized":"2023-11-28T13:08:04.371323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:37.423803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069498+0000","last_change":"2023-11-28T13:08:03.084729+0000","last_active":"2023-11-28T13:08:04.069498+0000","last_peered":"2023-11-28T13:08:04.069498+0000","last_clean":"2023-11-28T13:08:04.069498+0000","last_became_active":"2023-11-28T13:08:03.084473+0000","last_became_peered":"2023-11-28T13:08:03.084473+0000","last_unstale":"2023-11-28T13:08:04.069498+0000","last_undegraded":"2023-11-28T13:08:04.069498+0000","last_fullsized":"2023-11-28T13:08:04.069498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:53.049281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371761+0000","last_change":"2023-11-28T13:08:03.086292+0000","last_active":"2023-11-28T13:08:04.371761+0000","last_peered":"2023-11-28T13:08:04.371761+0000","last_clean":"2023-11-28T13:08:04.371761+0000","last_became_active":"2023-11-28T13:08:03.085984+0000","last_became_peered":"2023-11-28T13:08:03.085984+0000","last_unstale":"2023-11-28T13:08:04.371761+0000","last_undegraded":"2023-11-28T13:08:04.371761+0000","last_fullsized":"2023-11-28T13:08:04.371761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:30:58.749445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372036+0000","last_change":"2023-11-28T13:08:03.080558+0000","last_active":"2023-11-28T13:08:04.372036+0000","last_peered":"2023-11-28T13:08:04.372036+0000","last_clean":"2023-11-28T13:08:04.372036+0000","last_became_active":"2023-11-28T13:08:03.080267+0000","last_became_peered":"2023-11-28T13:08:03.080267+0000","last_unstale":"2023-11-28T13:08:04.372036+0000","last_undegraded":"2023-11-28T13:08:04.372036+0000","last_fullsized":"2023-11-28T13:08:04.372036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:38.726877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372532+0000","last_change":"2023-11-28T13:08:03.081890+0000","last_active":"2023-11-28T13:08:04.372532+0000","last_peered":"2023-11-28T13:08:04.372532+0000","last_clean":"2023-11-28T13:08:04.372532+0000","last_became_active":"2023-11-28T13:08:03.081638+0000","last_became_peered":"2023-11-28T13:08:03.081638+0000","last_unstale":"2023-11-28T13:08:04.372532+0000","last_undegraded":"2023-11-28T13:08:04.372532+0000","last_fullsized":"2023-11-28T13:08:04.372532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:53.049281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371404+0000","last_change":"2023-11-28T13:08:03.081127+0000","last_active":"2023-11-28T13:08:04.371404+0000","last_peered":"2023-11-28T13:08:04.371404+0000","last_clean":"2023-11-28T13:08:04.371404+0000","last_became_active":"2023-11-28T13:08:03.077682+0000","last_became_peered":"2023-11-28T13:08:03.077682+0000","last_unstale":"2023-11-28T13:08:04.371404+0000","last_undegraded":"2023-11-28T13:08:04.371404+0000","last_fullsized":"2023-11-28T13:08:04.371404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:58.947317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070150+0000","last_change":"2023-11-28T13:08:03.099999+0000","last_active":"2023-11-28T13:08:04.070150+0000","last_peered":"2023-11-28T13:08:04.070150+0000","last_clean":"2023-11-28T13:08:04.070150+0000","last_became_active":"2023-11-28T13:08:03.099733+0000","last_became_peered":"2023-11-28T13:08:03.099733+0000","last_unstale":"2023-11-28T13:08:04.070150+0000","last_undegraded":"2023-11-28T13:08:04.070150+0000","last_fullsized":"2023-11-28T13:08:04.070150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:46.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069502+0000","last_change":"2023-11-28T13:08:03.094943+0000","last_active":"2023-11-28T13:08:04.069502+0000","last_peered":"2023-11-28T13:08:04.069502+0000","last_clean":"2023-11-28T13:08:04.069502+0000","last_became_active":"2023-11-28T13:08:03.094688+0000","last_became_peered":"2023-11-28T13:08:03.094688+0000","last_unstale":"2023-11-28T13:08:04.069502+0000","last_undegraded":"2023-11-28T13:08:04.069502+0000","last_fullsized":"2023-11-28T13:08:04.069502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:14.015315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372476+0000","last_change":"2023-11-28T13:08:03.083815+0000","last_active":"2023-11-28T13:08:04.372476+0000","last_peered":"2023-11-28T13:08:04.372476+0000","last_clean":"2023-11-28T13:08:04.372476+0000","last_became_active":"2023-11-28T13:08:03.083292+0000","last_became_peered":"2023-11-28T13:08:03.083292+0000","last_unstale":"2023-11-28T13:08:04.372476+0000","last_undegraded":"2023-11-28T13:08:04.372476+0000","last_fullsized":"2023-11-28T13:08:04.372476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:14.015315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070743+0000","last_change":"2023-11-28T13:08:03.083685+0000","last_active":"2023-11-28T13:08:04.070743+0000","last_peered":"2023-11-28T13:08:04.070743+0000","last_clean":"2023-11-28T13:08:04.070743+0000","last_became_active":"2023-11-28T13:08:03.077038+0000","last_became_peered":"2023-11-28T13:08:03.077038+0000","last_unstale":"2023-11-28T13:08:04.070743+0000","last_undegraded":"2023-11-28T13:08:04.070743+0000","last_fullsized":"2023-11-28T13:08:04.070743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:34.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371966+0000","last_change":"2023-11-28T13:08:03.085905+0000","last_active":"2023-11-28T13:08:04.371966+0000","last_peered":"2023-11-28T13:08:04.371966+0000","last_clean":"2023-11-28T13:08:04.371966+0000","last_became_active":"2023-11-28T13:08:03.085524+0000","last_became_peered":"2023-11-28T13:08:03.085524+0000","last_unstale":"2023-11-28T13:08:04.371966+0000","last_undegraded":"2023-11-28T13:08:04.371966+0000","last_fullsized":"2023-11-28T13:08:04.371966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:34.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371160+0000","last_change":"2023-11-28T13:08:03.091440+0000","last_active":"2023-11-28T13:08:04.371160+0000","last_peered":"2023-11-28T13:08:04.371160+0000","last_clean":"2023-11-28T13:08:04.371160+0000","last_became_active":"2023-11-28T13:08:03.091117+0000","last_became_peered":"2023-11-28T13:08:03.091117+0000","last_unstale":"2023-11-28T13:08:04.371160+0000","last_undegraded":"2023-11-28T13:08:04.371160+0000","last_fullsized":"2023-11-28T13:08:04.371160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:41.849774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371255+0000","last_change":"2023-11-28T13:08:03.080912+0000","last_active":"2023-11-28T13:08:04.371255+0000","last_peered":"2023-11-28T13:08:04.371255+0000","last_clean":"2023-11-28T13:08:04.371255+0000","last_became_active":"2023-11-28T13:08:03.080578+0000","last_became_peered":"2023-11-28T13:08:03.080578+0000","last_unstale":"2023-11-28T13:08:04.371255+0000","last_undegraded":"2023-11-28T13:08:04.371255+0000","last_fullsized":"2023-11-28T13:08:04.371255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:38.657150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069701+0000","last_change":"2023-11-28T13:08:03.083208+0000","last_active":"2023-11-28T13:08:04.069701+0000","last_peered":"2023-11-28T13:08:04.069701+0000","last_clean":"2023-11-28T13:08:04.069701+0000","last_became_active":"2023-11-28T13:08:03.082944+0000","last_became_peered":"2023-11-28T13:08:03.082944+0000","last_unstale":"2023-11-28T13:08:04.069701+0000","last_undegraded":"2023-11-28T13:08:04.069701+0000","last_fullsized":"2023-11-28T13:08:04.069701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:36.892676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070334+0000","last_change":"2023-11-28T13:08:03.098345+0000","last_active":"2023-11-28T13:08:04.070334+0000","last_peered":"2023-11-28T13:08:04.070334+0000","last_clean":"2023-11-28T13:08:04.070334+0000","last_became_active":"2023-11-28T13:08:03.098105+0000","last_became_peered":"2023-11-28T13:08:03.098105+0000","last_unstale":"2023-11-28T13:08:04.070334+0000","last_undegraded":"2023-11-28T13:08:04.070334+0000","last_fullsized":"2023-11-28T13:08:04.070334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:43.340474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371254+0000","last_change":"2023-11-28T13:08:03.076945+0000","last_active":"2023-11-28T13:08:04.371254+0000","last_peered":"2023-11-28T13:08:04.371254+0000","last_clean":"2023-11-28T13:08:04.371254+0000","last_became_active":"2023-11-28T13:08:03.076639+0000","last_became_peered":"2023-11-28T13:08:03.076639+0000","last_unstale":"2023-11-28T13:08:04.371254+0000","last_undegraded":"2023-11-28T13:08:04.371254+0000","last_fullsized":"2023-11-28T13:08:04.371254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:41.076578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372578+0000","last_change":"2023-11-28T13:08:03.088044+0000","last_active":"2023-11-28T13:08:04.372578+0000","last_peered":"2023-11-28T13:08:04.372578+0000","last_clean":"2023-11-28T13:08:04.372578+0000","last_became_active":"2023-11-28T13:08:03.087703+0000","last_became_peered":"2023-11-28T13:08:03.087703+0000","last_unstale":"2023-11-28T13:08:04.372578+0000","last_undegraded":"2023-11-28T13:08:04.372578+0000","last_fullsized":"2023-11-28T13:08:04.372578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:43.340474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372432+0000","last_change":"2023-11-28T13:08:03.077748+0000","last_active":"2023-11-28T13:08:04.372432+0000","last_peered":"2023-11-28T13:08:04.372432+0000","last_clean":"2023-11-28T13:08:04.372432+0000","last_became_active":"2023-11-28T13:08:03.077488+0000","last_became_peered":"2023-11-28T13:08:03.077488+0000","last_unstale":"2023-11-28T13:08:04.372432+0000","last_undegraded":"2023-11-28T13:08:04.372432+0000","last_fullsized":"2023-11-28T13:08:04.372432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:14.015315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371120+0000","last_change":"2023-11-28T13:08:03.079221+0000","last_active":"2023-11-28T13:08:04.371120+0000","last_peered":"2023-11-28T13:08:04.371120+0000","last_clean":"2023-11-28T13:08:04.371120+0000","last_became_active":"2023-11-28T13:08:03.077084+0000","last_became_peered":"2023-11-28T13:08:03.077084+0000","last_unstale":"2023-11-28T13:08:04.371120+0000","last_undegraded":"2023-11-28T13:08:04.371120+0000","last_fullsized":"2023-11-28T13:08:04.371120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:02.499466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371692+0000","last_change":"2023-11-28T13:08:03.083322+0000","last_active":"2023-11-28T13:08:04.371692+0000","last_peered":"2023-11-28T13:08:04.371692+0000","last_clean":"2023-11-28T13:08:04.371692+0000","last_became_active":"2023-11-28T13:08:03.083064+0000","last_became_peered":"2023-11-28T13:08:03.083064+0000","last_unstale":"2023-11-28T13:08:04.371692+0000","last_undegraded":"2023-11-28T13:08:04.371692+0000","last_fullsized":"2023-11-28T13:08:04.371692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:13.351182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070670+0000","last_change":"2023-11-28T13:08:03.095031+0000","last_active":"2023-11-28T13:08:04.070670+0000","last_peered":"2023-11-28T13:08:04.070670+0000","last_clean":"2023-11-28T13:08:04.070670+0000","last_became_active":"2023-11-28T13:08:03.094746+0000","last_became_peered":"2023-11-28T13:08:03.094746+0000","last_unstale":"2023-11-28T13:08:04.070670+0000","last_undegraded":"2023-11-28T13:08:04.070670+0000","last_fullsized":"2023-11-28T13:08:04.070670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:49.707036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372182+0000","last_change":"2023-11-28T13:08:03.082127+0000","last_active":"2023-11-28T13:08:04.372182+0000","last_peered":"2023-11-28T13:08:04.372182+0000","last_clean":"2023-11-28T13:08:04.372182+0000","last_became_active":"2023-11-28T13:08:03.081861+0000","last_became_peered":"2023-11-28T13:08:03.081861+0000","last_unstale":"2023-11-28T13:08:04.372182+0000","last_undegraded":"2023-11-28T13:08:04.372182+0000","last_fullsized":"2023-11-28T13:08:04.372182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:05:35.135974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064206+0000","last_change":"2023-11-28T13:08:03.083718+0000","last_active":"2023-11-28T13:08:04.064206+0000","last_peered":"2023-11-28T13:08:04.064206+0000","last_clean":"2023-11-28T13:08:04.064206+0000","last_became_active":"2023-11-28T13:08:03.082420+0000","last_became_peered":"2023-11-28T13:08:03.082420+0000","last_unstale":"2023-11-28T13:08:04.064206+0000","last_undegraded":"2023-11-28T13:08:04.064206+0000","last_fullsized":"2023-11-28T13:08:04.064206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:58.181120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070265+0000","last_change":"2023-11-28T13:08:03.095780+0000","last_active":"2023-11-28T13:08:04.070265+0000","last_peered":"2023-11-28T13:08:04.070265+0000","last_clean":"2023-11-28T13:08:04.070265+0000","last_became_active":"2023-11-28T13:08:03.095522+0000","last_became_peered":"2023-11-28T13:08:03.095522+0000","last_unstale":"2023-11-28T13:08:04.070265+0000","last_undegraded":"2023-11-28T13:08:04.070265+0000","last_fullsized":"2023-11-28T13:08:04.070265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:37:44.657148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371784+0000","last_change":"2023-11-28T13:08:03.081168+0000","last_active":"2023-11-28T13:08:04.371784+0000","last_peered":"2023-11-28T13:08:04.371784+0000","last_clean":"2023-11-28T13:08:04.371784+0000","last_became_active":"2023-11-28T13:08:03.080867+0000","last_became_peered":"2023-11-28T13:08:03.080867+0000","last_unstale":"2023-11-28T13:08:04.371784+0000","last_undegraded":"2023-11-28T13:08:04.371784+0000","last_fullsized":"2023-11-28T13:08:04.371784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:44.188797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371597+0000","last_change":"2023-11-28T13:08:03.074696+0000","last_active":"2023-11-28T13:08:04.371597+0000","last_peered":"2023-11-28T13:08:04.371597+0000","last_clean":"2023-11-28T13:08:04.371597+0000","last_became_active":"2023-11-28T13:08:03.074392+0000","last_became_peered":"2023-11-28T13:08:03.074392+0000","last_unstale":"2023-11-28T13:08:04.371597+0000","last_undegraded":"2023-11-28T13:08:04.371597+0000","last_fullsized":"2023-11-28T13:08:04.371597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:50.062766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070127+0000","last_change":"2023-11-28T13:08:03.092477+0000","last_active":"2023-11-28T13:08:04.070127+0000","last_peered":"2023-11-28T13:08:04.070127+0000","last_clean":"2023-11-28T13:08:04.070127+0000","last_became_active":"2023-11-28T13:08:03.092243+0000","last_became_peered":"2023-11-28T13:08:03.092243+0000","last_unstale":"2023-11-28T13:08:04.070127+0000","last_undegraded":"2023-11-28T13:08:04.070127+0000","last_fullsized":"2023-11-28T13:08:04.070127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:46.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371521+0000","last_change":"2023-11-28T13:08:02.582255+0000","last_active":"2023-11-28T13:08:04.371521+0000","last_peered":"2023-11-28T13:08:04.371521+0000","last_clean":"2023-11-28T13:08:04.371521+0000","last_became_active":"2023-11-28T13:08:02.581960+0000","last_became_peered":"2023-11-28T13:08:02.581960+0000","last_unstale":"2023-11-28T13:08:04.371521+0000","last_undegraded":"2023-11-28T13:08:04.371521+0000","last_fullsized":"2023-11-28T13:08:04.371521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:49.593563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064820+0000","last_change":"2023-11-28T13:08:03.080261+0000","last_active":"2023-11-28T13:08:04.064820+0000","last_peered":"2023-11-28T13:08:04.064820+0000","last_clean":"2023-11-28T13:08:04.064820+0000","last_became_active":"2023-11-28T13:08:03.079964+0000","last_became_peered":"2023-11-28T13:08:03.079964+0000","last_unstale":"2023-11-28T13:08:04.064820+0000","last_undegraded":"2023-11-28T13:08:04.064820+0000","last_fullsized":"2023-11-28T13:08:04.064820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:30.263143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371820+0000","last_change":"2023-11-28T13:08:02.575220+0000","last_active":"2023-11-28T13:08:04.371820+0000","last_peered":"2023-11-28T13:08:04.371820+0000","last_clean":"2023-11-28T13:08:04.371820+0000","last_became_active":"2023-11-28T13:08:02.574933+0000","last_became_peered":"2023-11-28T13:08:02.574933+0000","last_unstale":"2023-11-28T13:08:04.371820+0000","last_undegraded":"2023-11-28T13:08:04.371820+0000","last_fullsized":"2023-11-28T13:08:04.371820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:56.910035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372498+0000","last_change":"2023-11-28T13:08:03.077902+0000","last_active":"2023-11-28T13:08:04.372498+0000","last_peered":"2023-11-28T13:08:04.372498+0000","last_clean":"2023-11-28T13:08:04.372498+0000","last_became_active":"2023-11-28T13:08:03.077379+0000","last_became_peered":"2023-11-28T13:08:03.077379+0000","last_unstale":"2023-11-28T13:08:04.372498+0000","last_undegraded":"2023-11-28T13:08:04.372498+0000","last_fullsized":"2023-11-28T13:08:04.372498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:34.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064779+0000","last_change":"2023-11-28T13:08:02.574513+0000","last_active":"2023-11-28T13:08:04.064779+0000","last_peered":"2023-11-28T13:08:04.064779+0000","last_clean":"2023-11-28T13:08:04.064779+0000","last_became_active":"2023-11-28T13:08:02.573509+0000","last_became_peered":"2023-11-28T13:08:02.573509+0000","last_unstale":"2023-11-28T13:08:04.064779+0000","last_undegraded":"2023-11-28T13:08:04.064779+0000","last_fullsized":"2023-11-28T13:08:04.064779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:01.496804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372381+0000","last_change":"2023-11-28T13:08:03.091021+0000","last_active":"2023-11-28T13:08:04.372381+0000","last_peered":"2023-11-28T13:08:04.372381+0000","last_clean":"2023-11-28T13:08:04.372381+0000","last_became_active":"2023-11-28T13:08:03.090717+0000","last_became_peered":"2023-11-28T13:08:03.090717+0000","last_unstale":"2023-11-28T13:08:04.372381+0000","last_undegraded":"2023-11-28T13:08:04.372381+0000","last_fullsized":"2023-11-28T13:08:04.372381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:50.062766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372655+0000","last_change":"2023-11-28T13:08:02.580525+0000","last_active":"2023-11-28T13:08:04.372655+0000","last_peered":"2023-11-28T13:08:04.372655+0000","last_clean":"2023-11-28T13:08:04.372655+0000","last_became_active":"2023-11-28T13:08:02.580220+0000","last_became_peered":"2023-11-28T13:08:02.580220+0000","last_unstale":"2023-11-28T13:08:04.372655+0000","last_undegraded":"2023-11-28T13:08:04.372655+0000","last_fullsized":"2023-11-28T13:08:04.372655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:01.496804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064707+0000","last_change":"2023-11-28T13:08:03.080244+0000","last_active":"2023-11-28T13:08:04.064707+0000","last_peered":"2023-11-28T13:08:04.064707+0000","last_clean":"2023-11-28T13:08:04.064707+0000","last_became_active":"2023-11-28T13:08:03.079988+0000","last_became_peered":"2023-11-28T13:08:03.079988+0000","last_unstale":"2023-11-28T13:08:04.064707+0000","last_undegraded":"2023-11-28T13:08:04.064707+0000","last_fullsized":"2023-11-28T13:08:04.064707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:49.707036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070421+0000","last_change":"2023-11-28T13:08:02.573388+0000","last_active":"2023-11-28T13:08:04.070421+0000","last_peered":"2023-11-28T13:08:04.070421+0000","last_clean":"2023-11-28T13:08:04.070421+0000","last_became_active":"2023-11-28T13:08:02.573059+0000","last_became_peered":"2023-11-28T13:08:02.573059+0000","last_unstale":"2023-11-28T13:08:04.070421+0000","last_undegraded":"2023-11-28T13:08:04.070421+0000","last_fullsized":"2023-11-28T13:08:04.070421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:46:29.622650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064417+0000","last_change":"2023-11-28T13:08:03.088704+0000","last_active":"2023-11-28T13:08:04.064417+0000","last_peered":"2023-11-28T13:08:04.064417+0000","last_clean":"2023-11-28T13:08:04.064417+0000","last_became_active":"2023-11-28T13:08:03.088417+0000","last_became_peered":"2023-11-28T13:08:03.088417+0000","last_unstale":"2023-11-28T13:08:04.064417+0000","last_undegraded":"2023-11-28T13:08:04.064417+0000","last_fullsized":"2023-11-28T13:08:04.064417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:41.196768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070940+0000","last_change":"2023-11-28T13:08:02.576140+0000","last_active":"2023-11-28T13:08:04.070940+0000","last_peered":"2023-11-28T13:08:04.070940+0000","last_clean":"2023-11-28T13:08:04.070940+0000","last_became_active":"2023-11-28T13:08:02.575721+0000","last_became_peered":"2023-11-28T13:08:02.575721+0000","last_unstale":"2023-11-28T13:08:04.070940+0000","last_undegraded":"2023-11-28T13:08:04.070940+0000","last_fullsized":"2023-11-28T13:08:04.070940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:12.214372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371693+0000","last_change":"2023-11-28T13:08:03.079421+0000","last_active":"2023-11-28T13:08:04.371693+0000","last_peered":"2023-11-28T13:08:04.371693+0000","last_clean":"2023-11-28T13:08:04.371693+0000","last_became_active":"2023-11-28T13:08:03.078471+0000","last_became_peered":"2023-11-28T13:08:03.078471+0000","last_unstale":"2023-11-28T13:08:04.371693+0000","last_undegraded":"2023-11-28T13:08:04.371693+0000","last_fullsized":"2023-11-28T13:08:04.371693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:05:35.135974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371902+0000","last_change":"2023-11-28T13:08:02.581355+0000","last_active":"2023-11-28T13:08:04.371902+0000","last_peered":"2023-11-28T13:08:04.371902+0000","last_clean":"2023-11-28T13:08:04.371902+0000","last_became_active":"2023-11-28T13:08:02.581051+0000","last_became_peered":"2023-11-28T13:08:02.581051+0000","last_unstale":"2023-11-28T13:08:04.371902+0000","last_undegraded":"2023-11-28T13:08:04.371902+0000","last_fullsized":"2023-11-28T13:08:04.371902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:43.117989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371569+0000","last_change":"2023-11-28T13:08:03.085302+0000","last_active":"2023-11-28T13:08:04.371569+0000","last_peered":"2023-11-28T13:08:04.371569+0000","last_clean":"2023-11-28T13:08:04.371569+0000","last_became_active":"2023-11-28T13:08:03.085061+0000","last_became_peered":"2023-11-28T13:08:03.085061+0000","last_unstale":"2023-11-28T13:08:04.371569+0000","last_undegraded":"2023-11-28T13:08:04.371569+0000","last_fullsized":"2023-11-28T13:08:04.371569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:13.625233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070589+0000","last_change":"2023-11-28T13:08:02.584763+0000","last_active":"2023-11-28T13:08:04.070589+0000","last_peered":"2023-11-28T13:08:04.070589+0000","last_clean":"2023-11-28T13:08:04.070589+0000","last_became_active":"2023-11-28T13:08:02.583655+0000","last_became_peered":"2023-11-28T13:08:02.583655+0000","last_unstale":"2023-11-28T13:08:04.070589+0000","last_undegraded":"2023-11-28T13:08:04.070589+0000","last_fullsized":"2023-11-28T13:08:04.070589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:53:59.398155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064403+0000","last_change":"2023-11-28T13:08:03.095515+0000","last_active":"2023-11-28T13:08:04.064403+0000","last_peered":"2023-11-28T13:08:04.064403+0000","last_clean":"2023-11-28T13:08:04.064403+0000","last_became_active":"2023-11-28T13:08:03.095004+0000","last_became_peered":"2023-11-28T13:08:03.095004+0000","last_unstale":"2023-11-28T13:08:04.064403+0000","last_undegraded":"2023-11-28T13:08:04.064403+0000","last_fullsized":"2023-11-28T13:08:04.064403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:25:34.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372572+0000","last_change":"2023-11-28T13:08:02.580082+0000","last_active":"2023-11-28T13:08:04.372572+0000","last_peered":"2023-11-28T13:08:04.372572+0000","last_clean":"2023-11-28T13:08:04.372572+0000","last_became_active":"2023-11-28T13:08:02.579745+0000","last_became_peered":"2023-11-28T13:08:02.579745+0000","last_unstale":"2023-11-28T13:08:04.372572+0000","last_undegraded":"2023-11-28T13:08:04.372572+0000","last_fullsized":"2023-11-28T13:08:04.372572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:12.363239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070163+0000","last_change":"2023-11-28T13:08:03.082966+0000","last_active":"2023-11-28T13:08:04.070163+0000","last_peered":"2023-11-28T13:08:04.070163+0000","last_clean":"2023-11-28T13:08:04.070163+0000","last_became_active":"2023-11-28T13:08:03.082710+0000","last_became_peered":"2023-11-28T13:08:03.082710+0000","last_unstale":"2023-11-28T13:08:04.070163+0000","last_undegraded":"2023-11-28T13:08:04.070163+0000","last_fullsized":"2023-11-28T13:08:04.070163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:49.707036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070236+0000","last_change":"2023-11-28T13:08:02.585538+0000","last_active":"2023-11-28T13:08:04.070236+0000","last_peered":"2023-11-28T13:08:04.070236+0000","last_clean":"2023-11-28T13:08:04.070236+0000","last_became_active":"2023-11-28T13:08:02.584407+0000","last_became_peered":"2023-11-28T13:08:02.584407+0000","last_unstale":"2023-11-28T13:08:04.070236+0000","last_undegraded":"2023-11-28T13:08:04.070236+0000","last_fullsized":"2023-11-28T13:08:04.070236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:01.496804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372389+0000","last_change":"2023-11-28T13:08:03.077754+0000","last_active":"2023-11-28T13:08:04.372389+0000","last_peered":"2023-11-28T13:08:04.372389+0000","last_clean":"2023-11-28T13:08:04.372389+0000","last_became_active":"2023-11-28T13:08:03.077073+0000","last_became_peered":"2023-11-28T13:08:03.077073+0000","last_unstale":"2023-11-28T13:08:04.372389+0000","last_undegraded":"2023-11-28T13:08:04.372389+0000","last_fullsized":"2023-11-28T13:08:04.372389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:46.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371255+0000","last_change":"2023-11-28T13:08:02.574654+0000","last_active":"2023-11-28T13:08:04.371255+0000","last_peered":"2023-11-28T13:08:04.371255+0000","last_clean":"2023-11-28T13:08:04.371255+0000","last_became_active":"2023-11-28T13:08:02.574354+0000","last_became_peered":"2023-11-28T13:08:02.574354+0000","last_unstale":"2023-11-28T13:08:04.371255+0000","last_undegraded":"2023-11-28T13:08:04.371255+0000","last_fullsized":"2023-11-28T13:08:04.371255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:43.117989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064965+0000","last_change":"2023-11-28T13:08:03.084191+0000","last_active":"2023-11-28T13:08:04.064965+0000","last_peered":"2023-11-28T13:08:04.064965+0000","last_clean":"2023-11-28T13:08:04.064965+0000","last_became_active":"2023-11-28T13:08:03.083879+0000","last_became_peered":"2023-11-28T13:08:03.083879+0000","last_unstale":"2023-11-28T13:08:04.064965+0000","last_undegraded":"2023-11-28T13:08:04.064965+0000","last_fullsized":"2023-11-28T13:08:04.064965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:14.015315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.071066+0000","last_change":"2023-11-28T13:08:02.584107+0000","last_active":"2023-11-28T13:08:04.071066+0000","last_peered":"2023-11-28T13:08:04.071066+0000","last_clean":"2023-11-28T13:08:04.071066+0000","last_became_active":"2023-11-28T13:08:02.583702+0000","last_became_peered":"2023-11-28T13:08:02.583702+0000","last_unstale":"2023-11-28T13:08:04.071066+0000","last_undegraded":"2023-11-28T13:08:04.071066+0000","last_fullsized":"2023-11-28T13:08:04.071066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:37.724216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069638+0000","last_change":"2023-11-28T13:08:03.077169+0000","last_active":"2023-11-28T13:08:04.069638+0000","last_peered":"2023-11-28T13:08:04.069638+0000","last_clean":"2023-11-28T13:08:04.069638+0000","last_became_active":"2023-11-28T13:08:03.076875+0000","last_became_peered":"2023-11-28T13:08:03.076875+0000","last_unstale":"2023-11-28T13:08:04.069638+0000","last_undegraded":"2023-11-28T13:08:04.069638+0000","last_fullsized":"2023-11-28T13:08:04.069638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:37:44.657148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.071007+0000","last_change":"2023-11-28T13:08:02.584070+0000","last_active":"2023-11-28T13:08:04.071007+0000","last_peered":"2023-11-28T13:08:04.071007+0000","last_clean":"2023-11-28T13:08:04.071007+0000","last_became_active":"2023-11-28T13:08:02.583527+0000","last_became_peered":"2023-11-28T13:08:02.583527+0000","last_unstale":"2023-11-28T13:08:04.071007+0000","last_undegraded":"2023-11-28T13:08:04.071007+0000","last_fullsized":"2023-11-28T13:08:04.071007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:49.060104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371361+0000","last_change":"2023-11-28T13:08:03.082338+0000","last_active":"2023-11-28T13:08:04.371361+0000","last_peered":"2023-11-28T13:08:04.371361+0000","last_clean":"2023-11-28T13:08:04.371361+0000","last_became_active":"2023-11-28T13:08:03.082006+0000","last_became_peered":"2023-11-28T13:08:03.082006+0000","last_unstale":"2023-11-28T13:08:04.371361+0000","last_undegraded":"2023-11-28T13:08:04.371361+0000","last_fullsized":"2023-11-28T13:08:04.371361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:56:48.282770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070812+0000","last_change":"2023-11-28T13:08:02.587071+0000","last_active":"2023-11-28T13:08:04.070812+0000","last_peered":"2023-11-28T13:08:04.070812+0000","last_clean":"2023-11-28T13:08:04.070812+0000","last_became_active":"2023-11-28T13:08:02.586541+0000","last_became_peered":"2023-11-28T13:08:02.586541+0000","last_unstale":"2023-11-28T13:08:04.070812+0000","last_undegraded":"2023-11-28T13:08:04.070812+0000","last_fullsized":"2023-11-28T13:08:04.070812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:56.535608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372224+0000","last_change":"2023-11-28T13:08:03.081008+0000","last_active":"2023-11-28T13:08:04.372224+0000","last_peered":"2023-11-28T13:08:04.372224+0000","last_clean":"2023-11-28T13:08:04.372224+0000","last_became_active":"2023-11-28T13:08:03.080718+0000","last_became_peered":"2023-11-28T13:08:03.080718+0000","last_unstale":"2023-11-28T13:08:04.372224+0000","last_undegraded":"2023-11-28T13:08:04.372224+0000","last_fullsized":"2023-11-28T13:08:04.372224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:00.400817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069777+0000","last_change":"2023-11-28T13:08:02.577784+0000","last_active":"2023-11-28T13:08:04.069777+0000","last_peered":"2023-11-28T13:08:04.069777+0000","last_clean":"2023-11-28T13:08:04.069777+0000","last_became_active":"2023-11-28T13:08:02.577364+0000","last_became_peered":"2023-11-28T13:08:02.577364+0000","last_unstale":"2023-11-28T13:08:04.069777+0000","last_undegraded":"2023-11-28T13:08:04.069777+0000","last_fullsized":"2023-11-28T13:08:04.069777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:37.654488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372256+0000","last_change":"2023-11-28T13:08:03.075105+0000","last_active":"2023-11-28T13:08:04.372256+0000","last_peered":"2023-11-28T13:08:04.372256+0000","last_clean":"2023-11-28T13:08:04.372256+0000","last_became_active":"2023-11-28T13:08:03.074768+0000","last_became_peered":"2023-11-28T13:08:03.074768+0000","last_unstale":"2023-11-28T13:08:04.372256+0000","last_undegraded":"2023-11-28T13:08:04.372256+0000","last_fullsized":"2023-11-28T13:08:04.372256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:37:44.657148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064578+0000","last_change":"2023-11-28T13:08:02.582099+0000","last_active":"2023-11-28T13:08:04.064578+0000","last_peered":"2023-11-28T13:08:04.064578+0000","last_clean":"2023-11-28T13:08:04.064578+0000","last_became_active":"2023-11-28T13:08:02.581651+0000","last_became_peered":"2023-11-28T13:08:02.581651+0000","last_unstale":"2023-11-28T13:08:04.064578+0000","last_undegraded":"2023-11-28T13:08:04.064578+0000","last_fullsized":"2023-11-28T13:08:04.064578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:12.214372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371794+0000","last_change":"2023-11-28T13:08:03.088535+0000","last_active":"2023-11-28T13:08:04.371794+0000","last_peered":"2023-11-28T13:08:04.371794+0000","last_clean":"2023-11-28T13:08:04.371794+0000","last_became_active":"2023-11-28T13:08:03.088263+0000","last_became_peered":"2023-11-28T13:08:03.088263+0000","last_unstale":"2023-11-28T13:08:04.371794+0000","last_undegraded":"2023-11-28T13:08:04.371794+0000","last_fullsized":"2023-11-28T13:08:04.371794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:22.644742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372159+0000","last_change":"2023-11-28T13:08:02.584910+0000","last_active":"2023-11-28T13:08:04.372159+0000","last_peered":"2023-11-28T13:08:04.372159+0000","last_clean":"2023-11-28T13:08:04.372159+0000","last_became_active":"2023-11-28T13:08:02.584573+0000","last_became_peered":"2023-11-28T13:08:02.584573+0000","last_unstale":"2023-11-28T13:08:04.372159+0000","last_undegraded":"2023-11-28T13:08:04.372159+0000","last_fullsized":"2023-11-28T13:08:04.372159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:49.593563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372100+0000","last_change":"2023-11-28T13:08:03.083587+0000","last_active":"2023-11-28T13:08:04.372100+0000","last_peered":"2023-11-28T13:08:04.372100+0000","last_clean":"2023-11-28T13:08:04.372100+0000","last_became_active":"2023-11-28T13:08:03.081658+0000","last_became_peered":"2023-11-28T13:08:03.081658+0000","last_unstale":"2023-11-28T13:08:04.372100+0000","last_undegraded":"2023-11-28T13:08:04.372100+0000","last_fullsized":"2023-11-28T13:08:04.372100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:53.049281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371468+0000","last_change":"2023-11-28T13:08:02.582045+0000","last_active":"2023-11-28T13:08:04.371468+0000","last_peered":"2023-11-28T13:08:04.371468+0000","last_clean":"2023-11-28T13:08:04.371468+0000","last_became_active":"2023-11-28T13:08:02.581687+0000","last_became_peered":"2023-11-28T13:08:02.581687+0000","last_unstale":"2023-11-28T13:08:04.371468+0000","last_undegraded":"2023-11-28T13:08:04.371468+0000","last_fullsized":"2023-11-28T13:08:04.371468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:56.910035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372483+0000","last_change":"2023-11-28T13:08:03.090341+0000","last_active":"2023-11-28T13:08:04.372483+0000","last_peered":"2023-11-28T13:08:04.372483+0000","last_clean":"2023-11-28T13:08:04.372483+0000","last_became_active":"2023-11-28T13:08:03.086320+0000","last_became_peered":"2023-11-28T13:08:03.086320+0000","last_unstale":"2023-11-28T13:08:04.372483+0000","last_undegraded":"2023-11-28T13:08:04.372483+0000","last_fullsized":"2023-11-28T13:08:04.372483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:30.263143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064693+0000","last_change":"2023-11-28T13:08:02.577006+0000","last_active":"2023-11-28T13:08:04.064693+0000","last_peered":"2023-11-28T13:08:04.064693+0000","last_clean":"2023-11-28T13:08:04.064693+0000","last_became_active":"2023-11-28T13:08:02.576734+0000","last_became_peered":"2023-11-28T13:08:02.576734+0000","last_unstale":"2023-11-28T13:08:04.064693+0000","last_undegraded":"2023-11-28T13:08:04.064693+0000","last_fullsized":"2023-11-28T13:08:04.064693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:40.847112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372143+0000","last_change":"2023-11-28T13:08:03.080548+0000","last_active":"2023-11-28T13:08:04.372143+0000","last_peered":"2023-11-28T13:08:04.372143+0000","last_clean":"2023-11-28T13:08:04.372143+0000","last_became_active":"2023-11-28T13:08:03.077609+0000","last_became_peered":"2023-11-28T13:08:03.077609+0000","last_unstale":"2023-11-28T13:08:04.372143+0000","last_undegraded":"2023-11-28T13:08:04.372143+0000","last_fullsized":"2023-11-28T13:08:04.372143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:57.732415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064801+0000","last_change":"2023-11-28T13:08:02.579378+0000","last_active":"2023-11-28T13:08:04.064801+0000","last_peered":"2023-11-28T13:08:04.064801+0000","last_clean":"2023-11-28T13:08:04.064801+0000","last_became_active":"2023-11-28T13:08:02.579080+0000","last_became_peered":"2023-11-28T13:08:02.579080+0000","last_unstale":"2023-11-28T13:08:04.064801+0000","last_undegraded":"2023-11-28T13:08:04.064801+0000","last_fullsized":"2023-11-28T13:08:04.064801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:36.421141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.075126+0000","last_change":"2023-11-28T13:07:56.890378+0000","last_active":"2023-11-28T13:08:03.075126+0000","last_peered":"2023-11-28T13:08:03.075126+0000","last_clean":"2023-11-28T13:08:03.075126+0000","last_became_active":"2023-11-28T13:07:56.890058+0000","last_became_peered":"2023-11-28T13:07:56.890058+0000","last_unstale":"2023-11-28T13:08:03.075126+0000","last_undegraded":"2023-11-28T13:08:03.075126+0000","last_fullsized":"2023-11-28T13:08:03.075126+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:05.344602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064197+0000","last_change":"2023-11-28T13:08:03.084094+0000","last_active":"2023-11-28T13:08:04.064197+0000","last_peered":"2023-11-28T13:08:04.064197+0000","last_clean":"2023-11-28T13:08:04.064197+0000","last_became_active":"2023-11-28T13:08:03.083811+0000","last_became_peered":"2023-11-28T13:08:03.083811+0000","last_unstale":"2023-11-28T13:08:04.064197+0000","last_undegraded":"2023-11-28T13:08:04.064197+0000","last_fullsized":"2023-11-28T13:08:04.064197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:53.049281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.072799+0000","last_change":"2023-11-28T13:07:56.844509+0000","last_active":"2023-11-28T13:08:03.072799+0000","last_peered":"2023-11-28T13:08:03.072799+0000","last_clean":"2023-11-28T13:08:03.072799+0000","last_became_active":"2023-11-28T13:07:56.844081+0000","last_became_peered":"2023-11-28T13:07:56.844081+0000","last_unstale":"2023-11-28T13:08:03.072799+0000","last_undegraded":"2023-11-28T13:08:03.072799+0000","last_fullsized":"2023-11-28T13:08:03.072799+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:05.095447+0000","last_change":"2023-11-28T13:08:02.576708+0000","last_active":"2023-11-28T13:08:05.095447+0000","last_peered":"2023-11-28T13:08:05.095447+0000","last_clean":"2023-11-28T13:08:05.095447+0000","last_became_active":"2023-11-28T13:08:02.572927+0000","last_became_peered":"2023-11-28T13:08:02.572927+0000","last_unstale":"2023-11-28T13:08:05.095447+0000","last_undegraded":"2023-11-28T13:08:05.095447+0000","last_fullsized":"2023-11-28T13:08:05.095447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:43.117989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372278+0000","last_change":"2023-11-28T13:08:03.093540+0000","last_active":"2023-11-28T13:08:04.372278+0000","last_peered":"2023-11-28T13:08:04.372278+0000","last_clean":"2023-11-28T13:08:04.372278+0000","last_became_active":"2023-11-28T13:08:03.089698+0000","last_became_peered":"2023-11-28T13:08:03.089698+0000","last_unstale":"2023-11-28T13:08:04.372278+0000","last_undegraded":"2023-11-28T13:08:04.372278+0000","last_fullsized":"2023-11-28T13:08:04.372278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:29:46.171857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.062393+0000","last_change":"2023-11-28T13:07:56.844699+0000","last_active":"2023-11-28T13:08:03.062393+0000","last_peered":"2023-11-28T13:08:03.062393+0000","last_clean":"2023-11-28T13:08:03.062393+0000","last_became_active":"2023-11-28T13:07:56.844391+0000","last_became_peered":"2023-11-28T13:07:56.844391+0000","last_unstale":"2023-11-28T13:08:03.062393+0000","last_undegraded":"2023-11-28T13:08:03.062393+0000","last_fullsized":"2023-11-28T13:08:03.062393+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.074953+0000","last_change":"2023-11-28T13:07:57.844020+0000","last_active":"2023-11-28T13:08:03.074953+0000","last_peered":"2023-11-28T13:08:03.074953+0000","last_clean":"2023-11-28T13:08:03.074953+0000","last_became_active":"2023-11-28T13:07:57.842981+0000","last_became_peered":"2023-11-28T13:07:57.842981+0000","last_unstale":"2023-11-28T13:08:03.074953+0000","last_undegraded":"2023-11-28T13:08:03.074953+0000","last_fullsized":"2023-11-28T13:08:03.074953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_clean_scrub_stamp":"2023-11-28T13:07:56.825711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:44.973218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371955+0000","last_change":"2023-11-28T13:08:02.583813+0000","last_active":"2023-11-28T13:08:04.371955+0000","last_peered":"2023-11-28T13:08:04.371955+0000","last_clean":"2023-11-28T13:08:04.371955+0000","last_became_active":"2023-11-28T13:08:02.583528+0000","last_became_peered":"2023-11-28T13:08:02.583528+0000","last_unstale":"2023-11-28T13:08:04.371955+0000","last_undegraded":"2023-11-28T13:08:04.371955+0000","last_fullsized":"2023-11-28T13:08:04.371955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:49.593563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372411+0000","last_change":"2023-11-28T13:08:03.077611+0000","last_active":"2023-11-28T13:08:04.372411+0000","last_peered":"2023-11-28T13:08:04.372411+0000","last_clean":"2023-11-28T13:08:04.372411+0000","last_became_active":"2023-11-28T13:08:03.076746+0000","last_became_peered":"2023-11-28T13:08:03.076746+0000","last_unstale":"2023-11-28T13:08:04.372411+0000","last_undegraded":"2023-11-28T13:08:04.372411+0000","last_fullsized":"2023-11-28T13:08:04.372411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:49.707036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.075246+0000","last_change":"2023-11-28T13:07:57.306318+0000","last_active":"2023-11-28T13:08:03.075246+0000","last_peered":"2023-11-28T13:08:03.075246+0000","last_clean":"2023-11-28T13:08:03.075246+0000","last_became_active":"2023-11-28T13:07:57.305937+0000","last_became_peered":"2023-11-28T13:07:57.305937+0000","last_unstale":"2023-11-28T13:08:03.075246+0000","last_undegraded":"2023-11-28T13:08:03.075246+0000","last_fullsized":"2023-11-28T13:08:03.075246+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371445+0000","last_change":"2023-11-28T13:08:02.569783+0000","last_active":"2023-11-28T13:08:04.371445+0000","last_peered":"2023-11-28T13:08:04.371445+0000","last_clean":"2023-11-28T13:08:04.371445+0000","last_became_active":"2023-11-28T13:08:02.569505+0000","last_became_peered":"2023-11-28T13:08:02.569505+0000","last_unstale":"2023-11-28T13:08:04.371445+0000","last_undegraded":"2023-11-28T13:08:04.371445+0000","last_fullsized":"2023-11-28T13:08:04.371445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:12.363239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372117+0000","last_change":"2023-11-28T13:08:03.094969+0000","last_active":"2023-11-28T13:08:04.372117+0000","last_peered":"2023-11-28T13:08:04.372117+0000","last_clean":"2023-11-28T13:08:04.372117+0000","last_became_active":"2023-11-28T13:08:03.094723+0000","last_became_peered":"2023-11-28T13:08:03.094723+0000","last_unstale":"2023-11-28T13:08:04.372117+0000","last_undegraded":"2023-11-28T13:08:04.372117+0000","last_fullsized":"2023-11-28T13:08:04.372117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:37:44.657148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.074562+0000","last_change":"2023-11-28T13:07:57.306277+0000","last_active":"2023-11-28T13:08:03.074562+0000","last_peered":"2023-11-28T13:08:03.074562+0000","last_clean":"2023-11-28T13:08:03.074562+0000","last_became_active":"2023-11-28T13:07:57.305983+0000","last_became_peered":"2023-11-28T13:07:57.305983+0000","last_unstale":"2023-11-28T13:08:03.074562+0000","last_undegraded":"2023-11-28T13:08:03.074562+0000","last_fullsized":"2023-11-28T13:08:03.074562+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064565+0000","last_change":"2023-11-28T13:08:02.578428+0000","last_active":"2023-11-28T13:08:04.064565+0000","last_peered":"2023-11-28T13:08:04.064565+0000","last_clean":"2023-11-28T13:08:04.064565+0000","last_became_active":"2023-11-28T13:08:02.578147+0000","last_became_peered":"2023-11-28T13:08:02.578147+0000","last_unstale":"2023-11-28T13:08:04.064565+0000","last_undegraded":"2023-11-28T13:08:04.064565+0000","last_fullsized":"2023-11-28T13:08:04.064565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:43.117989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372191+0000","last_change":"2023-11-28T13:08:03.094387+0000","last_active":"2023-11-28T13:08:04.372191+0000","last_peered":"2023-11-28T13:08:04.372191+0000","last_clean":"2023-11-28T13:08:04.372191+0000","last_became_active":"2023-11-28T13:08:03.094117+0000","last_became_peered":"2023-11-28T13:08:03.094117+0000","last_unstale":"2023-11-28T13:08:04.372191+0000","last_undegraded":"2023-11-28T13:08:04.372191+0000","last_fullsized":"2023-11-28T13:08:04.372191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:38.387013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.068929+0000","last_change":"2023-11-28T13:07:57.306177+0000","last_active":"2023-11-28T13:08:03.068929+0000","last_peered":"2023-11-28T13:08:03.068929+0000","last_clean":"2023-11-28T13:08:03.068929+0000","last_became_active":"2023-11-28T13:07:57.305853+0000","last_became_peered":"2023-11-28T13:07:57.305853+0000","last_unstale":"2023-11-28T13:08:03.068929+0000","last_undegraded":"2023-11-28T13:08:03.068929+0000","last_fullsized":"2023-11-28T13:08:03.068929+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371895+0000","last_change":"2023-11-28T13:08:02.584788+0000","last_active":"2023-11-28T13:08:04.371895+0000","last_peered":"2023-11-28T13:08:04.371895+0000","last_clean":"2023-11-28T13:08:04.371895+0000","last_became_active":"2023-11-28T13:08:02.584281+0000","last_became_peered":"2023-11-28T13:08:02.584281+0000","last_unstale":"2023-11-28T13:08:04.371895+0000","last_undegraded":"2023-11-28T13:08:04.371895+0000","last_fullsized":"2023-11-28T13:08:04.371895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:40.847112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371995+0000","last_change":"2023-11-28T13:08:03.074701+0000","last_active":"2023-11-28T13:08:04.371995+0000","last_peered":"2023-11-28T13:08:04.371995+0000","last_clean":"2023-11-28T13:08:04.371995+0000","last_became_active":"2023-11-28T13:08:03.074336+0000","last_became_peered":"2023-11-28T13:08:03.074336+0000","last_unstale":"2023-11-28T13:08:04.371995+0000","last_undegraded":"2023-11-28T13:08:04.371995+0000","last_fullsized":"2023-11-28T13:08:04.371995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:38.387013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.074084+0000","last_change":"2023-11-28T13:07:57.306321+0000","last_active":"2023-11-28T13:08:03.074084+0000","last_peered":"2023-11-28T13:08:03.074084+0000","last_clean":"2023-11-28T13:08:03.074084+0000","last_became_active":"2023-11-28T13:07:57.305923+0000","last_became_peered":"2023-11-28T13:07:57.305923+0000","last_unstale":"2023-11-28T13:08:03.074084+0000","last_undegraded":"2023-11-28T13:08:03.074084+0000","last_fullsized":"2023-11-28T13:08:03.074084+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:41.104679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371927+0000","last_change":"2023-11-28T13:08:02.570790+0000","last_active":"2023-11-28T13:08:04.371927+0000","last_peered":"2023-11-28T13:08:04.371927+0000","last_clean":"2023-11-28T13:08:04.371927+0000","last_became_active":"2023-11-28T13:08:02.569907+0000","last_became_peered":"2023-11-28T13:08:02.569907+0000","last_unstale":"2023-11-28T13:08:04.371927+0000","last_undegraded":"2023-11-28T13:08:04.371927+0000","last_fullsized":"2023-11-28T13:08:04.371927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:46.379743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372610+0000","last_change":"2023-11-28T13:08:03.087530+0000","last_active":"2023-11-28T13:08:04.372610+0000","last_peered":"2023-11-28T13:08:04.372610+0000","last_clean":"2023-11-28T13:08:04.372610+0000","last_became_active":"2023-11-28T13:08:03.087191+0000","last_became_peered":"2023-11-28T13:08:03.087191+0000","last_unstale":"2023-11-28T13:08:04.372610+0000","last_undegraded":"2023-11-28T13:08:04.372610+0000","last_fullsized":"2023-11-28T13:08:04.372610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:38.726877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-28T13:08:03.074497+0000","last_change":"2023-11-28T13:07:56.844305+0000","last_active":"2023-11-28T13:08:03.074497+0000","last_peered":"2023-11-28T13:08:03.074497+0000","last_clean":"2023-11-28T13:08:03.074497+0000","last_became_active":"2023-11-28T13:07:56.843980+0000","last_became_peered":"2023-11-28T13:07:56.843980+0000","last_unstale":"2023-11-28T13:08:03.074497+0000","last_undegraded":"2023-11-28T13:08:03.074497+0000","last_fullsized":"2023-11-28T13:08:03.074497+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070218+0000","last_change":"2023-11-28T13:08:02.577243+0000","last_active":"2023-11-28T13:08:04.070218+0000","last_peered":"2023-11-28T13:08:04.070218+0000","last_clean":"2023-11-28T13:08:04.070218+0000","last_became_active":"2023-11-28T13:08:02.576748+0000","last_became_peered":"2023-11-28T13:08:02.576748+0000","last_unstale":"2023-11-28T13:08:04.070218+0000","last_undegraded":"2023-11-28T13:08:04.070218+0000","last_fullsized":"2023-11-28T13:08:04.070218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:01.496804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371797+0000","last_change":"2023-11-28T13:08:03.095221+0000","last_active":"2023-11-28T13:08:04.371797+0000","last_peered":"2023-11-28T13:08:04.371797+0000","last_clean":"2023-11-28T13:08:04.371797+0000","last_became_active":"2023-11-28T13:08:03.091841+0000","last_became_peered":"2023-11-28T13:08:03.091841+0000","last_unstale":"2023-11-28T13:08:04.371797+0000","last_undegraded":"2023-11-28T13:08:04.371797+0000","last_fullsized":"2023-11-28T13:08:04.371797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:43.340474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372325+0000","last_change":"2023-11-28T13:08:02.583653+0000","last_active":"2023-11-28T13:08:04.372325+0000","last_peered":"2023-11-28T13:08:04.372325+0000","last_clean":"2023-11-28T13:08:04.372325+0000","last_became_active":"2023-11-28T13:08:02.583397+0000","last_became_peered":"2023-11-28T13:08:02.583397+0000","last_unstale":"2023-11-28T13:08:04.372325+0000","last_undegraded":"2023-11-28T13:08:04.372325+0000","last_fullsized":"2023-11-28T13:08:04.372325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:40.847112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371135+0000","last_change":"2023-11-28T13:08:03.096716+0000","last_active":"2023-11-28T13:08:04.371135+0000","last_peered":"2023-11-28T13:08:04.371135+0000","last_clean":"2023-11-28T13:08:04.371135+0000","last_became_active":"2023-11-28T13:08:03.096239+0000","last_became_peered":"2023-11-28T13:08:03.096239+0000","last_unstale":"2023-11-28T13:08:04.371135+0000","last_undegraded":"2023-11-28T13:08:04.371135+0000","last_fullsized":"2023-11-28T13:08:04.371135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:13.217034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.069581+0000","last_change":"2023-11-28T13:08:02.581525+0000","last_active":"2023-11-28T13:08:04.069581+0000","last_peered":"2023-11-28T13:08:04.069581+0000","last_clean":"2023-11-28T13:08:04.069581+0000","last_became_active":"2023-11-28T13:08:02.581253+0000","last_became_peered":"2023-11-28T13:08:02.581253+0000","last_unstale":"2023-11-28T13:08:04.069581+0000","last_undegraded":"2023-11-28T13:08:04.069581+0000","last_fullsized":"2023-11-28T13:08:04.069581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:36.421141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372111+0000","last_change":"2023-11-28T13:08:03.084538+0000","last_active":"2023-11-28T13:08:04.372111+0000","last_peered":"2023-11-28T13:08:04.372111+0000","last_clean":"2023-11-28T13:08:04.372111+0000","last_became_active":"2023-11-28T13:08:03.084256+0000","last_became_peered":"2023-11-28T13:08:03.084256+0000","last_unstale":"2023-11-28T13:08:04.372111+0000","last_undegraded":"2023-11-28T13:08:04.372111+0000","last_fullsized":"2023-11-28T13:08:04.372111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:46:46.224857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.070254+0000","last_change":"2023-11-28T13:08:02.583812+0000","last_active":"2023-11-28T13:08:04.070254+0000","last_peered":"2023-11-28T13:08:04.070254+0000","last_clean":"2023-11-28T13:08:04.070254+0000","last_became_active":"2023-11-28T13:08:02.583517+0000","last_became_peered":"2023-11-28T13:08:02.583517+0000","last_unstale":"2023-11-28T13:08:04.070254+0000","last_undegraded":"2023-11-28T13:08:04.070254+0000","last_fullsized":"2023-11-28T13:08:04.070254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:12.214372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371726+0000","last_change":"2023-11-28T13:08:03.088006+0000","last_active":"2023-11-28T13:08:04.371726+0000","last_peered":"2023-11-28T13:08:04.371726+0000","last_clean":"2023-11-28T13:08:04.371726+0000","last_became_active":"2023-11-28T13:08:03.086226+0000","last_became_peered":"2023-11-28T13:08:03.086226+0000","last_unstale":"2023-11-28T13:08:04.371726+0000","last_undegraded":"2023-11-28T13:08:04.371726+0000","last_fullsized":"2023-11-28T13:08:04.371726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:08.985463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064555+0000","last_change":"2023-11-28T13:08:02.574322+0000","last_active":"2023-11-28T13:08:04.064555+0000","last_peered":"2023-11-28T13:08:04.064555+0000","last_clean":"2023-11-28T13:08:04.064555+0000","last_became_active":"2023-11-28T13:08:02.573611+0000","last_became_peered":"2023-11-28T13:08:02.573611+0000","last_unstale":"2023-11-28T13:08:04.064555+0000","last_undegraded":"2023-11-28T13:08:04.064555+0000","last_fullsized":"2023-11-28T13:08:04.064555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:37.654488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064305+0000","last_change":"2023-11-28T13:08:03.093751+0000","last_active":"2023-11-28T13:08:04.064305+0000","last_peered":"2023-11-28T13:08:04.064305+0000","last_clean":"2023-11-28T13:08:04.064305+0000","last_became_active":"2023-11-28T13:08:03.093493+0000","last_became_peered":"2023-11-28T13:08:03.093493+0000","last_unstale":"2023-11-28T13:08:04.064305+0000","last_undegraded":"2023-11-28T13:08:04.064305+0000","last_fullsized":"2023-11-28T13:08:04.064305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:46.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.372733+0000","last_change":"2023-11-28T13:08:02.580561+0000","last_active":"2023-11-28T13:08:04.372733+0000","last_peered":"2023-11-28T13:08:04.372733+0000","last_clean":"2023-11-28T13:08:04.372733+0000","last_became_active":"2023-11-28T13:08:02.580273+0000","last_became_peered":"2023-11-28T13:08:02.580273+0000","last_unstale":"2023-11-28T13:08:04.372733+0000","last_undegraded":"2023-11-28T13:08:04.372733+0000","last_fullsized":"2023-11-28T13:08:04.372733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:01.549805+0000","last_clean_scrub_stamp":"2023-11-28T13:08:01.549805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:37.654488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371182+0000","last_change":"2023-11-28T13:08:03.089735+0000","last_active":"2023-11-28T13:08:04.371182+0000","last_peered":"2023-11-28T13:08:04.371182+0000","last_clean":"2023-11-28T13:08:04.371182+0000","last_became_active":"2023-11-28T13:08:03.089492+0000","last_became_peered":"2023-11-28T13:08:03.089492+0000","last_unstale":"2023-11-28T13:08:04.371182+0000","last_undegraded":"2023-11-28T13:08:04.371182+0000","last_fullsized":"2023-11-28T13:08:04.371182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:41.196768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371414+0000","last_change":"2023-11-28T13:08:03.082551+0000","last_active":"2023-11-28T13:08:04.371414+0000","last_peered":"2023-11-28T13:08:04.371414+0000","last_clean":"2023-11-28T13:08:04.371414+0000","last_became_active":"2023-11-28T13:08:03.082220+0000","last_became_peered":"2023-11-28T13:08:03.082220+0000","last_unstale":"2023-11-28T13:08:04.371414+0000","last_undegraded":"2023-11-28T13:08:04.371414+0000","last_fullsized":"2023-11-28T13:08:04.371414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:57.538270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064240+0000","last_change":"2023-11-28T13:08:03.095511+0000","last_active":"2023-11-28T13:08:04.064240+0000","last_peered":"2023-11-28T13:08:04.064240+0000","last_clean":"2023-11-28T13:08:04.064240+0000","last_became_active":"2023-11-28T13:08:03.095214+0000","last_became_peered":"2023-11-28T13:08:03.095214+0000","last_unstale":"2023-11-28T13:08:04.064240+0000","last_undegraded":"2023-11-28T13:08:04.064240+0000","last_fullsized":"2023-11-28T13:08:04.064240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:13.351182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371631+0000","last_change":"2023-11-28T13:08:03.089514+0000","last_active":"2023-11-28T13:08:04.371631+0000","last_peered":"2023-11-28T13:08:04.371631+0000","last_clean":"2023-11-28T13:08:04.371631+0000","last_became_active":"2023-11-28T13:08:03.089266+0000","last_became_peered":"2023-11-28T13:08:03.089266+0000","last_unstale":"2023-11-28T13:08:04.371631+0000","last_undegraded":"2023-11-28T13:08:04.371631+0000","last_fullsized":"2023-11-28T13:08:04.371631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:58.181120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.064673+0000","last_change":"2023-11-28T13:08:03.084369+0000","last_active":"2023-11-28T13:08:04.064673+0000","last_peered":"2023-11-28T13:08:04.064673+0000","last_clean":"2023-11-28T13:08:04.064673+0000","last_became_active":"2023-11-28T13:08:03.084081+0000","last_became_peered":"2023-11-28T13:08:03.084081+0000","last_unstale":"2023-11-28T13:08:04.064673+0000","last_undegraded":"2023-11-28T13:08:04.064673+0000","last_fullsized":"2023-11-28T13:08:04.064673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:13.625233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T13:08:04.371927+0000","last_change":"2023-11-28T13:08:03.093572+0000","last_active":"2023-11-28T13:08:04.371927+0000","last_peered":"2023-11-28T13:08:04.371927+0000","last_clean":"2023-11-28T13:08:04.371927+0000","last_became_active":"2023-11-28T13:08:03.093306+0000","last_became_peered":"2023-11-28T13:08:03.093306+0000","last_unstale":"2023-11-28T13:08:04.371927+0000","last_undegraded":"2023-11-28T13:08:04.371927+0000","last_fullsized":"2023-11-28T13:08:04.371927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:08:02.552467+0000","last_clean_scrub_stamp":"2023-11-28T13:08:02.552467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:36.892676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9084,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362756,"statfs":{"total":96636764160,"available":96627462144,"internally_reserved":0,"allocated":839680,"data_stored":671592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7964,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363876,"statfs":{"total":96636764160,"available":96628609024,"internally_reserved":0,"allocated":217088,"data_stored":79460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13152,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358688,"statfs":{"total":96636764160,"available":96623296512,"internally_reserved":0,"allocated":811008,"data_stored":669828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12124,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359716,"statfs":{"total":96636764160,"available":96624349184,"internally_reserved":0,"allocated":217088,"data_stored":79040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11996,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":11776,"kb_avail":94359844,"statfs":{"total":96636764160,"available":96624480256,"internally_reserved":0,"allocated":217088,"data_stored":79460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12058624},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12128,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":11904,"kb_avail":94359712,"statfs":{"total":96636764160,"available":96624345088,"internally_reserved":0,"allocated":221184,"data_stored":79550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12189696},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8048,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363792,"statfs":{"total":96636764160,"available":96628523008,"internally_reserved":0,"allocated":237568,"data_stored":81064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7972,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363868,"statfs":{"total":96636764160,"available":96628600832,"internally_reserved":0,"allocated":225280,"data_stored":80096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:08:11.690 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T13:08:11.691 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-28T13:08:11.691 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-28T13:08:11.691 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:08:11.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.922+0000 7f5dc19ff700 1 Processor -- start 2023-11-28T13:08:11.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.922+0000 7f5dc19ff700 1 -- start start 2023-11-28T13:08:11.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.923+0000 7f5dc19ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5dbc0d01b0 0x7f5dbc0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:11.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.923+0000 7f5dc19ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5dbc0b20b0 0x7f5dbc0d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:11.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.923+0000 7f5dc19ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5dbc0b29b0 0x7f5dbc0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:11.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.923+0000 7f5dc19ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dbc15cec0 con 0x7f5dbc0b20b0 2023-11-28T13:08:11.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.923+0000 7f5dc19ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dbc0b0230 con 0x7f5dbc0b29b0 2023-11-28T13:08:11.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.923+0000 7f5dc19ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0b4570 con 0x7f5dbc0d01b0 2023-11-28T13:08:11.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.924+0000 7f5dba7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5dbc0b20b0 0x7f5dbc0d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:11.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.924+0000 7f5dba7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5dbc0b20b0 0x7f5dbc0d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59052/0 (socket says 172.21.15.156:59052) 2023-11-28T13:08:11.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.924+0000 7f5dba7fc700 1 -- 172.21.15.156:0/326044560 learned_addr learned my addr 172.21.15.156:0/326044560 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:11.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.924+0000 7f5dbaffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5dbc0d01b0 0x7f5dbc0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:11.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.924+0000 7f5dbb7fe700 1 --2- 172.21.15.156:0/326044560 >> v2:172.21.15.186:3300/0 conn(0x7f5dbc0b29b0 0x7f5dbc0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:11.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.924+0000 7f5dba7fc700 1 -- 172.21.15.156:0/326044560 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5dbc0d01b0 msgr2=0x7f5dbc0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:11.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.924+0000 7f5dba7fc700 1 --2- 172.21.15.156:0/326044560 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5dbc0d01b0 0x7f5dbc0d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.924+0000 7f5dba7fc700 1 -- 172.21.15.156:0/326044560 >> v2:172.21.15.186:3300/0 conn(0x7f5dbc0b29b0 msgr2=0x7f5dbc0b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:11.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.924+0000 7f5dba7fc700 1 --2- 172.21.15.156:0/326044560 >> v2:172.21.15.186:3300/0 conn(0x7f5dbc0b29b0 0x7f5dbc0b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.924+0000 7f5dba7fc700 1 -- 172.21.15.156:0/326044560 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc0b4850 con 0x7f5dbc0b20b0 2023-11-28T13:08:11.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.924+0000 7f5dbb7fe700 1 --2- 172.21.15.156:0/326044560 >> v2:172.21.15.186:3300/0 conn(0x7f5dbc0b29b0 0x7f5dbc0b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:11.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.924+0000 7f5dba7fc700 1 --2- 172.21.15.156:0/326044560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dbc0b20b0 0x7f5dbc0d1160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc066e80 tx=0x7f5db0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5798ba6b798c4606 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:11.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.925+0000 7f5db9ffb700 1 -- 172.21.15.156:0/326044560 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0011830 con 0x7f5dbc0b20b0 2023-11-28T13:08:11.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.925+0000 7f5db9ffb700 1 -- 172.21.15.156:0/326044560 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db0011990 con 0x7f5dbc0b20b0 2023-11-28T13:08:11.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.925+0000 7f5db9ffb700 1 -- 172.21.15.156:0/326044560 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0011d20 con 0x7f5dbc0b20b0 2023-11-28T13:08:11.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.925+0000 7f5dc19ff700 1 -- 172.21.15.156:0/326044560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dbc0b20b0 msgr2=0x7f5dbc0d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:11.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.925+0000 7f5dc19ff700 1 --2- 172.21.15.156:0/326044560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dbc0b20b0 0x7f5dbc0d1160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc066e80 tx=0x7f5db0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.925+0000 7f5dc19ff700 1 -- 172.21.15.156:0/326044560 shutdown_connections 2023-11-28T13:08:11.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.925+0000 7f5dc19ff700 1 --2- 172.21.15.156:0/326044560 >> v2:172.21.15.186:3300/0 conn(0x7f5dbc0b29b0 0x7f5dbc0b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.925+0000 7f5dc19ff700 1 --2- 172.21.15.156:0/326044560 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5dbc0d01b0 0x7f5dbc0d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.925+0000 7f5dc19ff700 1 --2- 172.21.15.156:0/326044560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dbc0b20b0 0x7f5dbc0d1160 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.925+0000 7f5dc19ff700 1 -- 172.21.15.156:0/326044560 >> 172.21.15.156:0/326044560 conn(0x7f5dbc09c3f0 msgr2=0x7f5dbc0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:11.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.925+0000 7f5dc19ff700 1 -- 172.21.15.156:0/326044560 shutdown_connections 2023-11-28T13:08:11.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.925+0000 7f5dc19ff700 1 -- 172.21.15.156:0/326044560 wait complete. 2023-11-28T13:08:11.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.927+0000 7f5dc19ff700 1 Processor -- start 2023-11-28T13:08:11.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.927+0000 7f5dc19ff700 1 -- start start 2023-11-28T13:08:11.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.928+0000 7f5dc19ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5dbc0b20b0 0x7f5dbc155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:11.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.928+0000 7f5dc19ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5dbc0b29b0 0x7f5dbc0cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:11.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.928+0000 7f5dc19ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dbc0d01b0 0x7f5dbc0b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:11.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.928+0000 7f5dc19ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0b7c60 con 0x7f5dbc0d01b0 2023-11-28T13:08:11.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.928+0000 7f5dc19ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0cc1c0 con 0x7f5dbc0b29b0 2023-11-28T13:08:11.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.928+0000 7f5dc19ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0b7eb0 con 0x7f5dbc0b20b0 2023-11-28T13:08:11.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.928+0000 7f5dbb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dbc0d01b0 0x7f5dbc0b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:11.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.929+0000 7f5dba7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5dbc0b29b0 0x7f5dbc0cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:11.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.929+0000 7f5dbaffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5dbc0b20b0 0x7f5dbc155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:11.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.929+0000 7f5dba7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5dbc0b29b0 0x7f5dbc0cb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:35240/0 (socket says 172.21.15.156:35240) 2023-11-28T13:08:11.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.929+0000 7f5dba7fc700 1 -- 172.21.15.156:0/2213435526 learned_addr learned my addr 172.21.15.156:0/2213435526 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:11.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.929+0000 7f5dbb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dbc0d01b0 0x7f5dbc0b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59058/0 (socket says 172.21.15.156:59058) 2023-11-28T13:08:11.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.929+0000 7f5dba7fc700 1 -- 172.21.15.156:0/2213435526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5dbc0b20b0 msgr2=0x7f5dbc155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:11.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.929+0000 7f5dba7fc700 1 --2- 172.21.15.156:0/2213435526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5dbc0b20b0 0x7f5dbc155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.929+0000 7f5dba7fc700 1 -- 172.21.15.156:0/2213435526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dbc0d01b0 msgr2=0x7f5dbc0b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:11.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.929+0000 7f5dba7fc700 1 --2- 172.21.15.156:0/2213435526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dbc0d01b0 0x7f5dbc0b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:11.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.929+0000 7f5dba7fc700 1 -- 172.21.15.156:0/2213435526 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db0009550 con 0x7f5dbc0b29b0 2023-11-28T13:08:11.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.929+0000 7f5dbb7fe700 1 --2- 172.21.15.156:0/2213435526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dbc0d01b0 0x7f5dbc0b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:11.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.929+0000 7f5dba7fc700 1 --2- 172.21.15.156:0/2213435526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5dbc0b29b0 0x7f5dbc0cb870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5db0000c00 tx=0x7f5db0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:11.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.929+0000 7f5da3fff700 1 -- 172.21.15.156:0/2213435526 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0004240 con 0x7f5dbc0b29b0 2023-11-28T13:08:11.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.930+0000 7f5da3fff700 1 -- 172.21.15.156:0/2213435526 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db001f070 con 0x7f5dbc0b29b0 2023-11-28T13:08:11.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.930+0000 7f5dc19ff700 1 -- 172.21.15.156:0/2213435526 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dbc0afec0 con 0x7f5dbc0b29b0 2023-11-28T13:08:11.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.930+0000 7f5da3fff700 1 -- 172.21.15.156:0/2213435526 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db00045c0 con 0x7f5dbc0b29b0 2023-11-28T13:08:11.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.930+0000 7f5dc19ff700 1 -- 172.21.15.156:0/2213435526 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dbc0afbe0 con 0x7f5dbc0b29b0 2023-11-28T13:08:11.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.931+0000 7f5dc19ff700 1 -- 172.21.15.156:0/2213435526 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dbc0b2600 con 0x7f5dbc0b29b0 2023-11-28T13:08:11.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.932+0000 7f5da3fff700 1 -- 172.21.15.156:0/2213435526 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5db002a020 con 0x7f5dbc0b29b0 2023-11-28T13:08:11.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.933+0000 7f5da3fff700 1 --2- 172.21.15.156:0/2213435526 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5da80682f0 0x7f5da806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:11.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.933+0000 7f5da3fff700 1 -- 172.21.15.156:0/2213435526 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5db0087230 con 0x7f5dbc0b29b0 2023-11-28T13:08:11.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.933+0000 7f5dbaffd700 1 --2- 172.21.15.156:0/2213435526 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5da80682f0 0x7f5da806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:11.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.934+0000 7f5dbaffd700 1 --2- 172.21.15.156:0/2213435526 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5da80682f0 0x7f5da806a7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc0b7b90 tx=0x7f5dac009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:11.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:11.938+0000 7f5da3fff700 1 -- 172.21.15.156:0/2213435526 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5db00539a0 con 0x7f5dbc0b29b0 2023-11-28T13:08:12.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.170+0000 7f5dc19ff700 1 -- 172.21.15.156:0/2213435526 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5dbc068690 con 0x7f5dbc0b29b0 2023-11-28T13:08:12.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.171+0000 7f5da3fff700 1 -- 172.21.15.156:0/2213435526 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5db0056fc0 con 0x7f5dbc0b29b0 2023-11-28T13:08:12.171 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:12.171 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-28T13:08:12.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.173+0000 7f5dc19ff700 1 -- 172.21.15.156:0/2213435526 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5da80682f0 msgr2=0x7f5da806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:12.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.173+0000 7f5dc19ff700 1 --2- 172.21.15.156:0/2213435526 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5da80682f0 0x7f5da806a7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc0b7b90 tx=0x7f5dac009210 comp rx=0 tx=0).stop 2023-11-28T13:08:12.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.173+0000 7f5dc19ff700 1 -- 172.21.15.156:0/2213435526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5dbc0b29b0 msgr2=0x7f5dbc0cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:12.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.173+0000 7f5dc19ff700 1 --2- 172.21.15.156:0/2213435526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5dbc0b29b0 0x7f5dbc0cb870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5db0000c00 tx=0x7f5db0004750 comp rx=0 tx=0).stop 2023-11-28T13:08:12.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.173+0000 7f5dc19ff700 1 -- 172.21.15.156:0/2213435526 shutdown_connections 2023-11-28T13:08:12.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.173+0000 7f5dc19ff700 1 --2- 172.21.15.156:0/2213435526 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5da80682f0 0x7f5da806a7a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.173+0000 7f5dc19ff700 1 --2- 172.21.15.156:0/2213435526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dbc0d01b0 0x7f5dbc0b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.173+0000 7f5dc19ff700 1 --2- 172.21.15.156:0/2213435526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5dbc0b20b0 0x7f5dbc155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.173+0000 7f5dc19ff700 1 --2- 172.21.15.156:0/2213435526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5dbc0b29b0 0x7f5dbc0cb870 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.173+0000 7f5dc19ff700 1 -- 172.21.15.156:0/2213435526 >> 172.21.15.156:0/2213435526 conn(0x7f5dbc09c3f0 msgr2=0x7f5dbc0b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:12.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.173+0000 7f5dc19ff700 1 -- 172.21.15.156:0/2213435526 shutdown_connections 2023-11-28T13:08:12.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.173+0000 7f5dc19ff700 1 -- 172.21.15.156:0/2213435526 wait complete. 2023-11-28T13:08:12.192 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-28T13:08:12.193 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:08:12.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.389+0000 7f0ffa7ff700 1 Processor -- start 2023-11-28T13:08:12.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.390+0000 7f0ffa7ff700 1 -- start start 2023-11-28T13:08:12.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.391+0000 7f0ffa7ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0ff4161340 0x7f0ff40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:12.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.391+0000 7f0ffa7ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0ff40d2320 0x7f0ff40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:12.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.391+0000 7f0ffa7ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff40d1950 con 0x7f0ff415c9b0 2023-11-28T13:08:12.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.391+0000 7f0ffa7ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff40b5800 con 0x7f0ff4161340 2023-11-28T13:08:12.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.391+0000 7f0ffa7ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff40b5300 con 0x7f0ff40d2320 2023-11-28T13:08:12.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.391+0000 7f0ff37fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0ff4161340 0x7f0ff40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:12.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.391+0000 7f0ff37fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0ff4161340 0x7f0ff40d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59064/0 (socket says 172.21.15.156:59064) 2023-11-28T13:08:12.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.391+0000 7f0ff37fe700 1 -- 172.21.15.156:0/335749785 learned_addr learned my addr 172.21.15.156:0/335749785 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:12.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.391+0000 7f0ff8d9c700 1 --2- 172.21.15.156:0/335749785 >> v2:172.21.15.186:3300/0 conn(0x7f0ff40d2320 0x7f0ff40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:12.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.391+0000 7f0ff37fe700 1 -- 172.21.15.156:0/335749785 >> v2:172.21.15.186:3300/0 conn(0x7f0ff40d2320 msgr2=0x7f0ff40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:12.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.391+0000 7f0ff37fe700 1 --2- 172.21.15.156:0/335749785 >> v2:172.21.15.186:3300/0 conn(0x7f0ff40d2320 0x7f0ff40cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.391+0000 7f0ff37fe700 1 -- 172.21.15.156:0/335749785 >> v1:172.21.15.186:6789/0 conn(0x7f0ff415c9b0 legacy=0x7f0ff40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:12.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.392+0000 7f0ff37fe700 1 -- 172.21.15.156:0/335749785 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff40b5520 con 0x7f0ff4161340 2023-11-28T13:08:12.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.392+0000 7f0ff37fe700 1 --2- 172.21.15.156:0/335749785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff4161340 0x7f0ff40d1de0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0ff406c170 tx=0x7f0fe8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74dcefa2d6b81a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:12.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.392+0000 7f0ff2ffd700 1 -- 172.21.15.156:0/335749785 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe800b860 con 0x7f0ff4161340 2023-11-28T13:08:12.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.393+0000 7f0ff2ffd700 1 -- 172.21.15.156:0/335749785 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe800b9c0 con 0x7f0ff4161340 2023-11-28T13:08:12.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.393+0000 7f0ff2ffd700 1 -- 172.21.15.156:0/335749785 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe800bd50 con 0x7f0ff4161340 2023-11-28T13:08:12.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.393+0000 7f0ffa7ff700 1 -- 172.21.15.156:0/335749785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff4161340 msgr2=0x7f0ff40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:12.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.393+0000 7f0ffa7ff700 1 --2- 172.21.15.156:0/335749785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff4161340 0x7f0ff40d1de0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0ff406c170 tx=0x7f0fe8009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:12.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.394+0000 7f0ffa7ff700 1 -- 172.21.15.156:0/335749785 shutdown_connections 2023-11-28T13:08:12.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.394+0000 7f0ffa7ff700 1 --2- 172.21.15.156:0/335749785 >> v2:172.21.15.186:3300/0 conn(0x7f0ff40d2320 0x7f0ff40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.394+0000 7f0ffa7ff700 1 --2- 172.21.15.156:0/335749785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff4161340 0x7f0ff40d1de0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.394+0000 7f0ffa7ff700 1 -- 172.21.15.156:0/335749785 >> 172.21.15.156:0/335749785 conn(0x7f0ff4072600 msgr2=0x7f0ff40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:12.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.394+0000 7f0ffa7ff700 1 -- 172.21.15.156:0/335749785 shutdown_connections 2023-11-28T13:08:12.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.394+0000 7f0ffa7ff700 1 -- 172.21.15.156:0/335749785 wait complete. 2023-11-28T13:08:12.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.396+0000 7f0ffa7ff700 1 Processor -- start 2023-11-28T13:08:12.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.396+0000 7f0ffa7ff700 1 -- start start 2023-11-28T13:08:12.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.397+0000 7f0ffa7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ff40d2320 0x7f0ff41e80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:12.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.397+0000 7f0ffa7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff415c9b0 0x7f0ff4156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:12.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.397+0000 7f0ffa7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ff4161340 0x7f0ff415f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:12.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.397+0000 7f0ffa7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff4157ba0 con 0x7f0ff415c9b0 2023-11-28T13:08:12.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.397+0000 7f0ffa7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41e7c50 con 0x7f0ff4161340 2023-11-28T13:08:12.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.397+0000 7f0ffa7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41e7040 con 0x7f0ff40d2320 2023-11-28T13:08:12.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.397+0000 7f0ff37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff415c9b0 0x7f0ff4156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:12.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.397+0000 7f0ff37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff415c9b0 0x7f0ff4156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59068/0 (socket says 172.21.15.156:59068) 2023-11-28T13:08:12.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.397+0000 7f0ff37fe700 1 -- 172.21.15.156:0/3637432563 learned_addr learned my addr 172.21.15.156:0/3637432563 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:12.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.398+0000 7f0ff3fff700 1 --2- 172.21.15.156:0/3637432563 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ff40d2320 0x7f0ff41e80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:12.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.398+0000 7f0ff8d9c700 1 --2- 172.21.15.156:0/3637432563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ff4161340 0x7f0ff415f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:12.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.398+0000 7f0ff37fe700 1 -- 172.21.15.156:0/3637432563 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ff40d2320 msgr2=0x7f0ff41e80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:12.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.398+0000 7f0ff37fe700 1 --2- 172.21.15.156:0/3637432563 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ff40d2320 0x7f0ff41e80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.398+0000 7f0ff37fe700 1 -- 172.21.15.156:0/3637432563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ff4161340 msgr2=0x7f0ff415f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:12.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.398+0000 7f0ff37fe700 1 --2- 172.21.15.156:0/3637432563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ff4161340 0x7f0ff415f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.398+0000 7f0ff37fe700 1 -- 172.21.15.156:0/3637432563 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe8009550 con 0x7f0ff415c9b0 2023-11-28T13:08:12.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.398+0000 7f0ff37fe700 1 --2- 172.21.15.156:0/3637432563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff415c9b0 0x7f0ff4156a60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8009870 tx=0x7f0fe8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:12.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.398+0000 7f0ff0ff9700 1 -- 172.21.15.156:0/3637432563 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8004340 con 0x7f0ff415c9b0 2023-11-28T13:08:12.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.398+0000 7f0ff0ff9700 1 -- 172.21.15.156:0/3637432563 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe801f070 con 0x7f0ff415c9b0 2023-11-28T13:08:12.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.398+0000 7f0ffa7ff700 1 -- 172.21.15.156:0/3637432563 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff40d15e0 con 0x7f0ff415c9b0 2023-11-28T13:08:12.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.398+0000 7f0ff0ff9700 1 -- 172.21.15.156:0/3637432563 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe80036a0 con 0x7f0ff415c9b0 2023-11-28T13:08:12.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.398+0000 7f0ffa7ff700 1 -- 172.21.15.156:0/3637432563 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff40d1300 con 0x7f0ff415c9b0 2023-11-28T13:08:12.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.399+0000 7f0ffa7ff700 1 -- 172.21.15.156:0/3637432563 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff415cd80 con 0x7f0ff415c9b0 2023-11-28T13:08:12.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.400+0000 7f0ff0ff9700 1 -- 172.21.15.156:0/3637432563 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0fe802a020 con 0x7f0ff415c9b0 2023-11-28T13:08:12.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.401+0000 7f0ff0ff9700 1 --2- 172.21.15.156:0/3637432563 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0fe00682f0 0x7f0fe006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:12.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.401+0000 7f0ff0ff9700 1 -- 172.21.15.156:0/3637432563 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0fe8087360 con 0x7f0ff415c9b0 2023-11-28T13:08:12.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.401+0000 7f0ff3fff700 1 --2- 172.21.15.156:0/3637432563 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0fe00682f0 0x7f0fe006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:12.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.402+0000 7f0ff3fff700 1 --2- 172.21.15.156:0/3637432563 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0fe00682f0 0x7f0fe006a7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4001620 tx=0x7f0fe4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:12.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.405+0000 7f0ff0ff9700 1 -- 172.21.15.156:0/3637432563 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0fe8053a50 con 0x7f0ff415c9b0 2023-11-28T13:08:12.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.604+0000 7f0ffa7ff700 1 -- 172.21.15.156:0/3637432563 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ff40745c0 con 0x7f0ff415c9b0 2023-11-28T13:08:12.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.606+0000 7f0ff0ff9700 1 -- 172.21.15.156:0/3637432563 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17897 (secure 0 0 0) 0x7f0fe8057070 con 0x7f0ff415c9b0 2023-11-28T13:08:12.611 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:12.611 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":21,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:08:04.060053+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:08:00.903485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:08:02.037854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:08:12.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.608+0000 7f0ffa7ff700 1 -- 172.21.15.156:0/3637432563 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0fe00682f0 msgr2=0x7f0fe006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:12.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.608+0000 7f0ffa7ff700 1 --2- 172.21.15.156:0/3637432563 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0fe00682f0 0x7f0fe006a7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4001620 tx=0x7f0fe4009040 comp rx=0 tx=0).stop 2023-11-28T13:08:12.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.608+0000 7f0ffa7ff700 1 -- 172.21.15.156:0/3637432563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff415c9b0 msgr2=0x7f0ff4156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:12.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.608+0000 7f0ffa7ff700 1 --2- 172.21.15.156:0/3637432563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff415c9b0 0x7f0ff4156a60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8009870 tx=0x7f0fe8004750 comp rx=0 tx=0).stop 2023-11-28T13:08:12.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.608+0000 7f0ffa7ff700 1 -- 172.21.15.156:0/3637432563 shutdown_connections 2023-11-28T13:08:12.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.608+0000 7f0ffa7ff700 1 --2- 172.21.15.156:0/3637432563 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0fe00682f0 0x7f0fe006a7a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.608+0000 7f0ffa7ff700 1 --2- 172.21.15.156:0/3637432563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ff4161340 0x7f0ff415f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.608+0000 7f0ffa7ff700 1 --2- 172.21.15.156:0/3637432563 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ff40d2320 0x7f0ff41e80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.608+0000 7f0ffa7ff700 1 --2- 172.21.15.156:0/3637432563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff415c9b0 0x7f0ff4156a60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.608+0000 7f0ffa7ff700 1 -- 172.21.15.156:0/3637432563 >> 172.21.15.156:0/3637432563 conn(0x7f0ff4072600 msgr2=0x7f0ff40b8010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:12.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.608+0000 7f0ffa7ff700 1 -- 172.21.15.156:0/3637432563 shutdown_connections 2023-11-28T13:08:12.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.609+0000 7f0ffa7ff700 1 -- 172.21.15.156:0/3637432563 wait complete. 2023-11-28T13:08:12.628 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:08:00.903485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:08:02.037854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:08:12.628 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:08:12.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.829+0000 7f4938bff700 1 Processor -- start 2023-11-28T13:08:12.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.829+0000 7f4938bff700 1 -- start start 2023-11-28T13:08:12.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.830+0000 7f4938bff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f49340d1330 0x7f49340b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:12.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.830+0000 7f4938bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f49340b20b0 0x7f49340b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:12.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.830+0000 7f4938bff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49340b0230 con 0x7f49340d01b0 2023-11-28T13:08:12.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.830+0000 7f4938bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493415cec0 con 0x7f49340b20b0 2023-11-28T13:08:12.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.830+0000 7f4938bff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49340b4570 con 0x7f49340d1330 2023-11-28T13:08:12.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.830+0000 7f4932d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f49340b20b0 0x7f49340b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:12.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.830+0000 7f4932d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f49340b20b0 0x7f49340b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59076/0 (socket says 172.21.15.156:59076) 2023-11-28T13:08:12.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.830+0000 7f4932d9d700 1 -- 172.21.15.156:0/715195519 learned_addr learned my addr 172.21.15.156:0/715195519 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:12.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.830+0000 7f492bfff700 1 --2- 172.21.15.156:0/715195519 >> v2:172.21.15.186:3300/0 conn(0x7f49340d1330 0x7f49340b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:12.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.831+0000 7f4932d9d700 1 -- 172.21.15.156:0/715195519 >> v2:172.21.15.186:3300/0 conn(0x7f49340d1330 msgr2=0x7f49340b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:12.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.831+0000 7f4932d9d700 1 --2- 172.21.15.156:0/715195519 >> v2:172.21.15.186:3300/0 conn(0x7f49340d1330 0x7f49340b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.831+0000 7f4932d9d700 1 -- 172.21.15.156:0/715195519 >> v1:172.21.15.186:6789/0 conn(0x7f49340d01b0 legacy=0x7f49340d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:12.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.831+0000 7f4932d9d700 1 -- 172.21.15.156:0/715195519 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49340b4850 con 0x7f49340b20b0 2023-11-28T13:08:12.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.831+0000 7f492bfff700 1 --2- 172.21.15.156:0/715195519 >> v2:172.21.15.186:3300/0 conn(0x7f49340d1330 0x7f49340b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:12.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.831+0000 7f4932d9d700 1 --2- 172.21.15.156:0/715195519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49340b20b0 0x7f49340b0c00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f49340ca0b0 tx=0x7f492c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8735c45b1907a98a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:12.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.831+0000 7f4931d9b700 1 -- 172.21.15.156:0/715195519 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c00e070 con 0x7f49340b20b0 2023-11-28T13:08:12.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.831+0000 7f4931d9b700 1 -- 172.21.15.156:0/715195519 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c00b9c0 con 0x7f49340b20b0 2023-11-28T13:08:12.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.831+0000 7f4931d9b700 1 -- 172.21.15.156:0/715195519 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c00bd70 con 0x7f49340b20b0 2023-11-28T13:08:12.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.832+0000 7f4938bff700 1 -- 172.21.15.156:0/715195519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49340b20b0 msgr2=0x7f49340b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:12.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.832+0000 7f4938bff700 1 --2- 172.21.15.156:0/715195519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49340b20b0 0x7f49340b0c00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f49340ca0b0 tx=0x7f492c00b330 comp rx=0 tx=0).stop 2023-11-28T13:08:12.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.832+0000 7f4938bff700 1 -- 172.21.15.156:0/715195519 shutdown_connections 2023-11-28T13:08:12.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.832+0000 7f4938bff700 1 --2- 172.21.15.156:0/715195519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49340b20b0 0x7f49340b0c00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.832+0000 7f4938bff700 1 --2- 172.21.15.156:0/715195519 >> v2:172.21.15.186:3300/0 conn(0x7f49340d1330 0x7f49340b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.832+0000 7f4938bff700 1 -- 172.21.15.156:0/715195519 >> 172.21.15.156:0/715195519 conn(0x7f493409c3f0 msgr2=0x7f49340d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:12.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.832+0000 7f4938bff700 1 -- 172.21.15.156:0/715195519 shutdown_connections 2023-11-28T13:08:12.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.832+0000 7f4938bff700 1 -- 172.21.15.156:0/715195519 wait complete. 2023-11-28T13:08:12.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.834+0000 7f4938bff700 1 Processor -- start 2023-11-28T13:08:12.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.834+0000 7f4938bff700 1 -- start start 2023-11-28T13:08:12.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.835+0000 7f4938bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49340b20b0 0x7f493415b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:12.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.835+0000 7f4938bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49340d01b0 0x7f49340a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:12.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.835+0000 7f4938bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49340d1330 0x7f49341f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:12.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.835+0000 7f4938bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493409eb60 con 0x7f49340d1330 2023-11-28T13:08:12.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.835+0000 7f4938bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493409df50 con 0x7f49340d01b0 2023-11-28T13:08:12.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.835+0000 7f4938bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493409e230 con 0x7f49340b20b0 2023-11-28T13:08:12.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.835+0000 7f4932d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49340d1330 0x7f49341f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:12.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.835+0000 7f4932d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49340d1330 0x7f49341f5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59088/0 (socket says 172.21.15.156:59088) 2023-11-28T13:08:12.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.835+0000 7f4932d9d700 1 -- 172.21.15.156:0/2302719766 learned_addr learned my addr 172.21.15.156:0/2302719766 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:12.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.835+0000 7f492bfff700 1 --2- 172.21.15.156:0/2302719766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49340d01b0 0x7f49340a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:12.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.835+0000 7f493259c700 1 --2- 172.21.15.156:0/2302719766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49340b20b0 0x7f493415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:12.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.835+0000 7f4932d9d700 1 -- 172.21.15.156:0/2302719766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49340b20b0 msgr2=0x7f493415b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:12.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.835+0000 7f4932d9d700 1 --2- 172.21.15.156:0/2302719766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49340b20b0 0x7f493415b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.835+0000 7f4932d9d700 1 -- 172.21.15.156:0/2302719766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49340d01b0 msgr2=0x7f49340a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:12.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.835+0000 7f4932d9d700 1 --2- 172.21.15.156:0/2302719766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49340d01b0 0x7f49340a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:12.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.835+0000 7f4932d9d700 1 -- 172.21.15.156:0/2302719766 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c009b30 con 0x7f49340d1330 2023-11-28T13:08:12.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.836+0000 7f4932d9d700 1 --2- 172.21.15.156:0/2302719766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49340d1330 0x7f49341f5740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f492c012000 tx=0x7f492c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:12.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.836+0000 7f492affd700 1 -- 172.21.15.156:0/2302719766 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c004880 con 0x7f49340d1330 2023-11-28T13:08:12.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.836+0000 7f492affd700 1 -- 172.21.15.156:0/2302719766 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c0099b0 con 0x7f49340d1330 2023-11-28T13:08:12.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.836+0000 7f4938bff700 1 -- 172.21.15.156:0/2302719766 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49340afec0 con 0x7f49340d1330 2023-11-28T13:08:12.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.836+0000 7f492affd700 1 -- 172.21.15.156:0/2302719766 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c01c680 con 0x7f49340d1330 2023-11-28T13:08:12.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.836+0000 7f4938bff700 1 -- 172.21.15.156:0/2302719766 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49340afbe0 con 0x7f49340d1330 2023-11-28T13:08:12.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.837+0000 7f4938bff700 1 -- 172.21.15.156:0/2302719766 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49340b2600 con 0x7f49340d1330 2023-11-28T13:08:12.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.838+0000 7f492affd700 1 -- 172.21.15.156:0/2302719766 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f492c022030 con 0x7f49340d1330 2023-11-28T13:08:12.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.839+0000 7f492affd700 1 --2- 172.21.15.156:0/2302719766 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f49140682f0 0x7f491406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:12.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.839+0000 7f492affd700 1 -- 172.21.15.156:0/2302719766 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f492c0873e0 con 0x7f49340d1330 2023-11-28T13:08:12.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.839+0000 7f493259c700 1 --2- 172.21.15.156:0/2302719766 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f49140682f0 0x7f491406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:12.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.840+0000 7f493259c700 1 --2- 172.21.15.156:0/2302719766 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f49140682f0 0x7f491406a7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f49340c9b50 tx=0x7f4924009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:12.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:12.843+0000 7f492affd700 1 -- 172.21.15.156:0/2302719766 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f492c053c40 con 0x7f49340d1330 2023-11-28T13:08:13.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.034+0000 7f4938bff700 1 -- 172.21.15.156:0/2302719766 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4934068690 con 0x7f49340d1330 2023-11-28T13:08:13.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.034+0000 7f492affd700 1 -- 172.21.15.156:0/2302719766 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f492c057260 con 0x7f49340d1330 2023-11-28T13:08:13.036 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:08:13.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.037+0000 7f4938bff700 1 -- 172.21.15.156:0/2302719766 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f49140682f0 msgr2=0x7f491406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:13.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.037+0000 7f4938bff700 1 --2- 172.21.15.156:0/2302719766 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f49140682f0 0x7f491406a7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f49340c9b50 tx=0x7f4924009040 comp rx=0 tx=0).stop 2023-11-28T13:08:13.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.037+0000 7f4938bff700 1 -- 172.21.15.156:0/2302719766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49340d1330 msgr2=0x7f49341f5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:13.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.037+0000 7f4938bff700 1 --2- 172.21.15.156:0/2302719766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49340d1330 0x7f49341f5740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f492c012000 tx=0x7f492c004ad0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.037+0000 7f4938bff700 1 -- 172.21.15.156:0/2302719766 shutdown_connections 2023-11-28T13:08:13.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.037+0000 7f4938bff700 1 --2- 172.21.15.156:0/2302719766 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f49140682f0 0x7f491406a7a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.037+0000 7f4938bff700 1 --2- 172.21.15.156:0/2302719766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49340d1330 0x7f49341f5740 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.037+0000 7f4938bff700 1 --2- 172.21.15.156:0/2302719766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49340b20b0 0x7f493415b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.037+0000 7f4938bff700 1 --2- 172.21.15.156:0/2302719766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49340d01b0 0x7f49340a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.037+0000 7f4938bff700 1 -- 172.21.15.156:0/2302719766 >> 172.21.15.156:0/2302719766 conn(0x7f493409c3f0 msgr2=0x7f493406b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:13.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.037+0000 7f4938bff700 1 -- 172.21.15.156:0/2302719766 shutdown_connections 2023-11-28T13:08:13.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.037+0000 7f4938bff700 1 -- 172.21.15.156:0/2302719766 wait complete. 2023-11-28T13:08:13.057 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:08:13.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.259+0000 7ff0b11ff700 1 Processor -- start 2023-11-28T13:08:13.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.259+0000 7ff0b11ff700 1 -- start start 2023-11-28T13:08:13.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.260+0000 7ff0b11ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff0ac0d01b0 0x7ff0ac0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:13.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.260+0000 7ff0b11ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff0ac0b16d0 0x7ff0ac0d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:13.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.260+0000 7ff0b11ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0ac0b0230 con 0x7ff0ac0b20b0 2023-11-28T13:08:13.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.260+0000 7ff0b11ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0ac15cec0 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.260+0000 7ff0b11ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0ac0b4570 con 0x7ff0ac0b16d0 2023-11-28T13:08:13.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.260+0000 7ff0aad9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff0ac0d01b0 0x7ff0ac0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:13.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.260+0000 7ff0aad9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff0ac0d01b0 0x7ff0ac0d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59096/0 (socket says 172.21.15.156:59096) 2023-11-28T13:08:13.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.260+0000 7ff0aad9d700 1 -- 172.21.15.156:0/2923026852 learned_addr learned my addr 172.21.15.156:0/2923026852 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:13.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.261+0000 7ff0ab59e700 1 --2- 172.21.15.156:0/2923026852 >> v2:172.21.15.186:3300/0 conn(0x7ff0ac0b16d0 0x7ff0ac0d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:13.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.261+0000 7ff0aad9d700 1 -- 172.21.15.156:0/2923026852 >> v2:172.21.15.186:3300/0 conn(0x7ff0ac0b16d0 msgr2=0x7ff0ac0d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:13.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.261+0000 7ff0aad9d700 1 --2- 172.21.15.156:0/2923026852 >> v2:172.21.15.186:3300/0 conn(0x7ff0ac0b16d0 0x7ff0ac0d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.261+0000 7ff0aad9d700 1 -- 172.21.15.156:0/2923026852 >> v1:172.21.15.186:6789/0 conn(0x7ff0ac0b20b0 legacy=0x7ff0ac0b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:13.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.261+0000 7ff0aad9d700 1 -- 172.21.15.156:0/2923026852 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac0b4850 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.261+0000 7ff0aad9d700 1 --2- 172.21.15.156:0/2923026852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ac0d01b0 0x7ff0ac0d0c20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0cd630 tx=0x7ff09c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6ee2c20c192ca5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:13.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.261+0000 7ff0a9d9b700 1 -- 172.21.15.156:0/2923026852 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00b870 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.261+0000 7ff0a9d9b700 1 -- 172.21.15.156:0/2923026852 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c00b9d0 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.261+0000 7ff0a9d9b700 1 -- 172.21.15.156:0/2923026852 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00bd60 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.262+0000 7ff0b11ff700 1 -- 172.21.15.156:0/2923026852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ac0d01b0 msgr2=0x7ff0ac0d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.262+0000 7ff0b11ff700 1 --2- 172.21.15.156:0/2923026852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ac0d01b0 0x7ff0ac0d0c20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0cd630 tx=0x7ff09c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.262+0000 7ff0b11ff700 1 -- 172.21.15.156:0/2923026852 shutdown_connections 2023-11-28T13:08:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.262+0000 7ff0b11ff700 1 --2- 172.21.15.156:0/2923026852 >> v2:172.21.15.186:3300/0 conn(0x7ff0ac0b16d0 0x7ff0ac0d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.262+0000 7ff0b11ff700 1 --2- 172.21.15.156:0/2923026852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ac0d01b0 0x7ff0ac0d0c20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.262+0000 7ff0b11ff700 1 -- 172.21.15.156:0/2923026852 >> 172.21.15.156:0/2923026852 conn(0x7ff0ac09c3f0 msgr2=0x7ff0ac0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:13.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.262+0000 7ff0b11ff700 1 -- 172.21.15.156:0/2923026852 shutdown_connections 2023-11-28T13:08:13.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.262+0000 7ff0b11ff700 1 -- 172.21.15.156:0/2923026852 wait complete. 2023-11-28T13:08:13.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.264+0000 7ff0b11ff700 1 Processor -- start 2023-11-28T13:08:13.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.264+0000 7ff0b11ff700 1 -- start start 2023-11-28T13:08:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.265+0000 7ff0b11ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ac0b16d0 0x7ff0ac0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.265+0000 7ff0b11ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0ac0b20b0 0x7ff0ac1e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.265+0000 7ff0b11ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ac0d01b0 0x7ff0ac1f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.265+0000 7ff0b11ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac1e40f0 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.265+0000 7ff0b11ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac0cce70 con 0x7ff0ac0b20b0 2023-11-28T13:08:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.265+0000 7ff0b11ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac1f2b40 con 0x7ff0ac0b16d0 2023-11-28T13:08:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.265+0000 7ff0ab59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ac0d01b0 0x7ff0ac1f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.265+0000 7ff0ab59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ac0d01b0 0x7ff0ac1f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59102/0 (socket says 172.21.15.156:59102) 2023-11-28T13:08:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.265+0000 7ff0ab59e700 1 -- 172.21.15.156:0/231209311 learned_addr learned my addr 172.21.15.156:0/231209311 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.265+0000 7ff0aa59c700 1 --2- 172.21.15.156:0/231209311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0ac0b20b0 0x7ff0ac1e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.265+0000 7ff0aad9d700 1 --2- 172.21.15.156:0/231209311 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ac0b16d0 0x7ff0ac0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.265+0000 7ff0ab59e700 1 -- 172.21.15.156:0/231209311 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ac0b16d0 msgr2=0x7ff0ac0cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.265+0000 7ff0ab59e700 1 --2- 172.21.15.156:0/231209311 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ac0b16d0 0x7ff0ac0cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.265+0000 7ff0ab59e700 1 -- 172.21.15.156:0/231209311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0ac0b20b0 msgr2=0x7ff0ac1e4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.265+0000 7ff0ab59e700 1 --2- 172.21.15.156:0/231209311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0ac0b20b0 0x7ff0ac1e4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.265+0000 7ff0ab59e700 1 -- 172.21.15.156:0/231209311 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c009550 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.265+0000 7ff0ab59e700 1 --2- 172.21.15.156:0/231209311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ac0d01b0 0x7ff0ac1f20a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4007e40 tx=0x7ff0a400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.266+0000 7ff0937fe700 1 -- 172.21.15.156:0/231209311 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a400ed30 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.266+0000 7ff0937fe700 1 -- 172.21.15.156:0/231209311 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a4014420 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.266+0000 7ff0b11ff700 1 -- 172.21.15.156:0/231209311 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ac0afec0 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.266+0000 7ff0937fe700 1 -- 172.21.15.156:0/231209311 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40147d0 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.266+0000 7ff0b11ff700 1 -- 172.21.15.156:0/231209311 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ac0afbe0 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.267+0000 7ff0b11ff700 1 -- 172.21.15.156:0/231209311 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0ac0b2600 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.268+0000 7ff0937fe700 1 -- 172.21.15.156:0/231209311 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff0a4012050 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.269+0000 7ff0937fe700 1 --2- 172.21.15.156:0/231209311 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff0980682f0 0x7ff09806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.269+0000 7ff0937fe700 1 -- 172.21.15.156:0/231209311 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff0a40862d0 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.269+0000 7ff0aad9d700 1 --2- 172.21.15.156:0/231209311 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff0980682f0 0x7ff09806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.270+0000 7ff0aad9d700 1 --2- 172.21.15.156:0/231209311 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff0980682f0 0x7ff09806a7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff09c000c00 tx=0x7ff09c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:13.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.273+0000 7ff0937fe700 1 -- 172.21.15.156:0/231209311 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff0a4052a40 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.466+0000 7ff0b11ff700 1 -- 172.21.15.156:0/231209311 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0ac068690 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.467+0000 7ff0937fe700 1 -- 172.21.15.156:0/231209311 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff0a4056060 con 0x7ff0ac0d01b0 2023-11-28T13:08:13.468 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:08:13.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.469+0000 7ff0b11ff700 1 -- 172.21.15.156:0/231209311 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff0980682f0 msgr2=0x7ff09806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:13.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.469+0000 7ff0b11ff700 1 --2- 172.21.15.156:0/231209311 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff0980682f0 0x7ff09806a7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff09c000c00 tx=0x7ff09c008030 comp rx=0 tx=0).stop 2023-11-28T13:08:13.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.469+0000 7ff0b11ff700 1 -- 172.21.15.156:0/231209311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ac0d01b0 msgr2=0x7ff0ac1f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:13.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.469+0000 7ff0b11ff700 1 --2- 172.21.15.156:0/231209311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ac0d01b0 0x7ff0ac1f20a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4007e40 tx=0x7ff0a400e3c0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.469+0000 7ff0b11ff700 1 -- 172.21.15.156:0/231209311 shutdown_connections 2023-11-28T13:08:13.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.469+0000 7ff0b11ff700 1 --2- 172.21.15.156:0/231209311 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff0980682f0 0x7ff09806a7a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.469+0000 7ff0b11ff700 1 --2- 172.21.15.156:0/231209311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ac0d01b0 0x7ff0ac1f20a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.469+0000 7ff0b11ff700 1 --2- 172.21.15.156:0/231209311 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ac0b16d0 0x7ff0ac0cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.469+0000 7ff0b11ff700 1 --2- 172.21.15.156:0/231209311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0ac0b20b0 0x7ff0ac1e4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.469+0000 7ff0b11ff700 1 -- 172.21.15.156:0/231209311 >> 172.21.15.156:0/231209311 conn(0x7ff0ac09c3f0 msgr2=0x7ff0ac06b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:13.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.470+0000 7ff0b11ff700 1 -- 172.21.15.156:0/231209311 shutdown_connections 2023-11-28T13:08:13.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.470+0000 7ff0b11ff700 1 -- 172.21.15.156:0/231209311 wait complete. 2023-11-28T13:08:13.488 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:08:13.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.686+0000 7fd9671ff700 1 Processor -- start 2023-11-28T13:08:13.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.686+0000 7fd9671ff700 1 -- start start 2023-11-28T13:08:13.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.687+0000 7fd9671ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd96015c9b0 0x7fd9600b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:13.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.687+0000 7fd9671ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd960161340 0x7fd9600d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:13.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.687+0000 7fd9671ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd9600d2320 0x7fd9600cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:13.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.687+0000 7fd9671ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9600b5800 con 0x7fd960161340 2023-11-28T13:08:13.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.687+0000 7fd9671ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9600d1950 con 0x7fd9600d2320 2023-11-28T13:08:13.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.687+0000 7fd9671ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600b5520 con 0x7fd96015c9b0 2023-11-28T13:08:13.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.688+0000 7fd95ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd960161340 0x7fd9600d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:13.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.688+0000 7fd95ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd960161340 0x7fd9600d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59114/0 (socket says 172.21.15.156:59114) 2023-11-28T13:08:13.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.688+0000 7fd95ffff700 1 -- 172.21.15.156:0/1782779176 learned_addr learned my addr 172.21.15.156:0/1782779176 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:13.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.688+0000 7fd964f9b700 1 --2- 172.21.15.156:0/1782779176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd96015c9b0 0x7fd9600b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:13.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.688+0000 7fd96579c700 1 --2- 172.21.15.156:0/1782779176 >> v2:172.21.15.186:3300/0 conn(0x7fd9600d2320 0x7fd9600cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:13.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.688+0000 7fd95ffff700 1 -- 172.21.15.156:0/1782779176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd96015c9b0 msgr2=0x7fd9600b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:13.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.688+0000 7fd95ffff700 1 --2- 172.21.15.156:0/1782779176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd96015c9b0 0x7fd9600b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.688+0000 7fd95ffff700 1 -- 172.21.15.156:0/1782779176 >> v2:172.21.15.186:3300/0 conn(0x7fd9600d2320 msgr2=0x7fd9600cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:13.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.688+0000 7fd95ffff700 1 --2- 172.21.15.156:0/1782779176 >> v2:172.21.15.186:3300/0 conn(0x7fd9600d2320 0x7fd9600cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.688+0000 7fd95ffff700 1 -- 172.21.15.156:0/1782779176 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9600d1300 con 0x7fd960161340 2023-11-28T13:08:13.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.689+0000 7fd95ffff700 1 --2- 172.21.15.156:0/1782779176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd960161340 0x7fd9600d1de0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd96008c4d0 tx=0x7fd94c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3244807f99530f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:13.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.689+0000 7fd95f7fe700 1 -- 172.21.15.156:0/1782779176 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c011830 con 0x7fd960161340 2023-11-28T13:08:13.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.689+0000 7fd95f7fe700 1 -- 172.21.15.156:0/1782779176 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94c011990 con 0x7fd960161340 2023-11-28T13:08:13.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.689+0000 7fd95f7fe700 1 -- 172.21.15.156:0/1782779176 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c011d20 con 0x7fd960161340 2023-11-28T13:08:13.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.689+0000 7fd9671ff700 1 -- 172.21.15.156:0/1782779176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd960161340 msgr2=0x7fd9600d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:13.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.689+0000 7fd9671ff700 1 --2- 172.21.15.156:0/1782779176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd960161340 0x7fd9600d1de0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd96008c4d0 tx=0x7fd94c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.689+0000 7fd9671ff700 1 -- 172.21.15.156:0/1782779176 shutdown_connections 2023-11-28T13:08:13.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.689+0000 7fd9671ff700 1 --2- 172.21.15.156:0/1782779176 >> v2:172.21.15.186:3300/0 conn(0x7fd9600d2320 0x7fd9600cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.689+0000 7fd9671ff700 1 --2- 172.21.15.156:0/1782779176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd96015c9b0 0x7fd9600b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.689+0000 7fd9671ff700 1 --2- 172.21.15.156:0/1782779176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd960161340 0x7fd9600d1de0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.689+0000 7fd9671ff700 1 -- 172.21.15.156:0/1782779176 >> 172.21.15.156:0/1782779176 conn(0x7fd960072600 msgr2=0x7fd9600cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:13.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.690+0000 7fd9671ff700 1 -- 172.21.15.156:0/1782779176 shutdown_connections 2023-11-28T13:08:13.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.690+0000 7fd9671ff700 1 -- 172.21.15.156:0/1782779176 wait complete. 2023-11-28T13:08:13.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.691+0000 7fd9671ff700 1 Processor -- start 2023-11-28T13:08:13.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.691+0000 7fd9671ff700 1 -- start start 2023-11-28T13:08:13.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.692+0000 7fd9671ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9600d2320 0x7fd9600ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:13.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.692+0000 7fd9671ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd96015c9b0 0x7fd9600c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:13.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.692+0000 7fd9671ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd960161340 0x7fd96015ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:13.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.692+0000 7fd9671ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600c7bf0 con 0x7fd9600d2320 2023-11-28T13:08:13.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.692+0000 7fd9671ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600cd360 con 0x7fd960161340 2023-11-28T13:08:13.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.692+0000 7fd9671ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600cd640 con 0x7fd96015c9b0 2023-11-28T13:08:13.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.693+0000 7fd964f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9600d2320 0x7fd9600ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:13.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.693+0000 7fd964f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9600d2320 0x7fd9600ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59130/0 (socket says 172.21.15.156:59130) 2023-11-28T13:08:13.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.693+0000 7fd964f9b700 1 -- 172.21.15.156:0/323075848 learned_addr learned my addr 172.21.15.156:0/323075848 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:13.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.693+0000 7fd95ffff700 1 --2- 172.21.15.156:0/323075848 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd96015c9b0 0x7fd9600c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:13.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.693+0000 7fd96579c700 1 --2- 172.21.15.156:0/323075848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd960161340 0x7fd96015ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:13.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.693+0000 7fd964f9b700 1 -- 172.21.15.156:0/323075848 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd96015c9b0 msgr2=0x7fd9600c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:13.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.693+0000 7fd964f9b700 1 --2- 172.21.15.156:0/323075848 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd96015c9b0 0x7fd9600c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.693+0000 7fd964f9b700 1 -- 172.21.15.156:0/323075848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd960161340 msgr2=0x7fd96015ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:13.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.693+0000 7fd964f9b700 1 --2- 172.21.15.156:0/323075848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd960161340 0x7fd96015ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.693+0000 7fd964f9b700 1 -- 172.21.15.156:0/323075848 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94c009550 con 0x7fd9600d2320 2023-11-28T13:08:13.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.693+0000 7fd96579c700 1 --2- 172.21.15.156:0/323075848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd960161340 0x7fd96015ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:13.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.693+0000 7fd964f9b700 1 --2- 172.21.15.156:0/323075848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9600d2320 0x7fd9600ccc80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd95400e870 tx=0x7fd95400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:13.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.693+0000 7fd95d7fa700 1 -- 172.21.15.156:0/323075848 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd954009590 con 0x7fd9600d2320 2023-11-28T13:08:13.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.694+0000 7fd95d7fa700 1 -- 172.21.15.156:0/323075848 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd954013680 con 0x7fd9600d2320 2023-11-28T13:08:13.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.694+0000 7fd95d7fa700 1 -- 172.21.15.156:0/323075848 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9540139f0 con 0x7fd9600d2320 2023-11-28T13:08:13.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.694+0000 7fd9671ff700 1 -- 172.21.15.156:0/323075848 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9600b5130 con 0x7fd9600d2320 2023-11-28T13:08:13.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.694+0000 7fd9671ff700 1 -- 172.21.15.156:0/323075848 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9600d15e0 con 0x7fd9600d2320 2023-11-28T13:08:13.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.695+0000 7fd9671ff700 1 -- 172.21.15.156:0/323075848 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9600b52c0 con 0x7fd9600d2320 2023-11-28T13:08:13.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.696+0000 7fd95d7fa700 1 -- 172.21.15.156:0/323075848 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd954019030 con 0x7fd9600d2320 2023-11-28T13:08:13.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.697+0000 7fd95d7fa700 1 --2- 172.21.15.156:0/323075848 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd9440682f0 0x7fd94406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:13.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.697+0000 7fd95d7fa700 1 -- 172.21.15.156:0/323075848 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd954085ad0 con 0x7fd9600d2320 2023-11-28T13:08:13.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.697+0000 7fd95ffff700 1 --2- 172.21.15.156:0/323075848 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd9440682f0 0x7fd94406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:13.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.698+0000 7fd95ffff700 1 --2- 172.21.15.156:0/323075848 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd9440682f0 0x7fd94406a7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009c10 tx=0x7fd94c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:13.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.700+0000 7fd95d7fa700 1 -- 172.21.15.156:0/323075848 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd954052240 con 0x7fd9600d2320 2023-11-28T13:08:13.890 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.888+0000 7fd9671ff700 1 -- 172.21.15.156:0/323075848 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd9600745c0 con 0x7fd9600d2320 2023-11-28T13:08:13.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.889+0000 7fd95d7fa700 1 -- 172.21.15.156:0/323075848 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd954059e60 con 0x7fd9600d2320 2023-11-28T13:08:13.891 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T13:08:13.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.891+0000 7fd9671ff700 1 -- 172.21.15.156:0/323075848 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd9440682f0 msgr2=0x7fd94406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:13.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.891+0000 7fd9671ff700 1 --2- 172.21.15.156:0/323075848 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd9440682f0 0x7fd94406a7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009c10 tx=0x7fd94c008030 comp rx=0 tx=0).stop 2023-11-28T13:08:13.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.891+0000 7fd9671ff700 1 -- 172.21.15.156:0/323075848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9600d2320 msgr2=0x7fd9600ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:13.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.891+0000 7fd9671ff700 1 --2- 172.21.15.156:0/323075848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9600d2320 0x7fd9600ccc80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd95400e870 tx=0x7fd95400eb80 comp rx=0 tx=0).stop 2023-11-28T13:08:13.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.891+0000 7fd9671ff700 1 -- 172.21.15.156:0/323075848 shutdown_connections 2023-11-28T13:08:13.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.891+0000 7fd9671ff700 1 --2- 172.21.15.156:0/323075848 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd9440682f0 0x7fd94406a7a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.891+0000 7fd9671ff700 1 --2- 172.21.15.156:0/323075848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd960161340 0x7fd96015ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.891+0000 7fd9671ff700 1 --2- 172.21.15.156:0/323075848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9600d2320 0x7fd9600ccc80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.891+0000 7fd9671ff700 1 --2- 172.21.15.156:0/323075848 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd96015c9b0 0x7fd9600c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:13.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.891+0000 7fd9671ff700 1 -- 172.21.15.156:0/323075848 >> 172.21.15.156:0/323075848 conn(0x7fd960072600 msgr2=0x7fd96015a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:13.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.892+0000 7fd9671ff700 1 -- 172.21.15.156:0/323075848 shutdown_connections 2023-11-28T13:08:13.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:13.892+0000 7fd9671ff700 1 -- 172.21.15.156:0/323075848 wait complete. 2023-11-28T13:08:13.912 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:08:14.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.109+0000 7f4d071ff700 1 Processor -- start 2023-11-28T13:08:14.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.109+0000 7f4d071ff700 1 -- start start 2023-11-28T13:08:14.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.110+0000 7f4d071ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4d000cd7c0 0x7f4d000cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:14.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.110+0000 7f4d071ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d000d1950 con 0x7f4d00161340 2023-11-28T13:08:14.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.110+0000 7f4d071ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d000ce140 con 0x7f4d0015c9b0 2023-11-28T13:08:14.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.110+0000 7f4d071ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d000b5800 con 0x7f4d000cd7c0 2023-11-28T13:08:14.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.110+0000 7f4d0579c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4d000cd7c0 0x7f4d000cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:14.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.110+0000 7f4d0579c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4d000cd7c0 0x7f4d000cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59136/0 (socket says 172.21.15.156:59136) 2023-11-28T13:08:14.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.110+0000 7f4d0579c700 1 -- 172.21.15.156:0/219686547 learned_addr learned my addr 172.21.15.156:0/219686547 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:14.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.110+0000 7f4cfffff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4d00161340 0x7f4d000d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:34660/0 (socket says 172.21.15.156:34660) 2023-11-28T13:08:14.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.111+0000 7f4d0579c700 1 -- 172.21.15.156:0/219686547 >> v1:172.21.15.186:6789/0 conn(0x7f4d0015c9b0 legacy=0x7f4d000b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:14.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.111+0000 7f4d0579c700 1 -- 172.21.15.156:0/219686547 >> v1:172.21.15.156:6789/0 conn(0x7f4d00161340 legacy=0x7f4d000d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:14.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.111+0000 7f4d0579c700 1 -- 172.21.15.156:0/219686547 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d000d24f0 con 0x7f4d000cd7c0 2023-11-28T13:08:14.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.111+0000 7f4cff7fe700 1 -- 172.21.15.156:0/219686547 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057109660 0 0) 0x7f4d000d1950 con 0x7f4d00161340 2023-11-28T13:08:14.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.111+0000 7f4d0579c700 1 --2- 172.21.15.156:0/219686547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d000cd7c0 0x7f4d000cdc00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009e50 tx=0x7f4cf800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0298bcb2e8f1229 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:14.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.111+0000 7f4cff7fe700 1 -- 172.21.15.156:0/219686547 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf800e070 con 0x7f4d000cd7c0 2023-11-28T13:08:14.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.111+0000 7f4cff7fe700 1 -- 172.21.15.156:0/219686547 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf800b9c0 con 0x7f4d000cd7c0 2023-11-28T13:08:14.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.111+0000 7f4cff7fe700 1 -- 172.21.15.156:0/219686547 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf800bd70 con 0x7f4d000cd7c0 2023-11-28T13:08:14.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.112+0000 7f4d071ff700 1 -- 172.21.15.156:0/219686547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d000cd7c0 msgr2=0x7f4d000cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:14.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.112+0000 7f4d071ff700 1 --2- 172.21.15.156:0/219686547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d000cd7c0 0x7f4d000cdc00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009e50 tx=0x7f4cf800b330 comp rx=0 tx=0).stop 2023-11-28T13:08:14.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.112+0000 7f4d071ff700 1 -- 172.21.15.156:0/219686547 shutdown_connections 2023-11-28T13:08:14.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.112+0000 7f4d071ff700 1 --2- 172.21.15.156:0/219686547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d000cd7c0 0x7f4d000cdc00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.112+0000 7f4d071ff700 1 -- 172.21.15.156:0/219686547 >> 172.21.15.156:0/219686547 conn(0x7f4d00072600 msgr2=0x7f4d000cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:14.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.112+0000 7f4d071ff700 1 -- 172.21.15.156:0/219686547 shutdown_connections 2023-11-28T13:08:14.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.112+0000 7f4d071ff700 1 -- 172.21.15.156:0/219686547 wait complete. 2023-11-28T13:08:14.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.114+0000 7f4d071ff700 1 Processor -- start 2023-11-28T13:08:14.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.114+0000 7f4d071ff700 1 -- start start 2023-11-28T13:08:14.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4d071ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000cd7c0 0x7f4d000b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:14.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4d071ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d0015c9b0 0x7f4d001f2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:14.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4d071ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d00161340 0x7f4d000c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:14.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4d071ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d001f3810 con 0x7f4d00161340 2023-11-28T13:08:14.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4d071ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d001f2f70 con 0x7f4d000cd7c0 2023-11-28T13:08:14.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4d071ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d001f3250 con 0x7f4d0015c9b0 2023-11-28T13:08:14.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4d0579c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d00161340 0x7f4d000c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:14.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4d0579c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d00161340 0x7f4d000c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59148/0 (socket says 172.21.15.156:59148) 2023-11-28T13:08:14.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4d0579c700 1 -- 172.21.15.156:0/1324221465 learned_addr learned my addr 172.21.15.156:0/1324221465 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:14.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4cfffff700 1 --2- 172.21.15.156:0/1324221465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d0015c9b0 0x7f4d001f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:14.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4d0579c700 1 -- 172.21.15.156:0/1324221465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d0015c9b0 msgr2=0x7f4d001f2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:14.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4d04f9b700 1 --2- 172.21.15.156:0/1324221465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000cd7c0 0x7f4d000b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:14.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4d0579c700 1 --2- 172.21.15.156:0/1324221465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d0015c9b0 0x7f4d001f2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4d0579c700 1 -- 172.21.15.156:0/1324221465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000cd7c0 msgr2=0x7f4d000b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:14.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4d0579c700 1 --2- 172.21.15.156:0/1324221465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000cd7c0 0x7f4d000b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4d0579c700 1 -- 172.21.15.156:0/1324221465 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf8009b30 con 0x7f4d00161340 2023-11-28T13:08:14.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4cfffff700 1 --2- 172.21.15.156:0/1324221465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d0015c9b0 0x7f4d001f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:14.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.115+0000 7f4d04f9b700 1 --2- 172.21.15.156:0/1324221465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000cd7c0 0x7f4d000b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:14.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.116+0000 7f4d0579c700 1 --2- 172.21.15.156:0/1324221465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d00161340 0x7f4d000c7950 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800b730 tx=0x7f4cf8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:14.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.116+0000 7f4cfd7fa700 1 -- 172.21.15.156:0/1324221465 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf801b630 con 0x7f4d00161340 2023-11-28T13:08:14.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.116+0000 7f4cfd7fa700 1 -- 172.21.15.156:0/1324221465 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf801b7b0 con 0x7f4d00161340 2023-11-28T13:08:14.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.116+0000 7f4cfd7fa700 1 -- 172.21.15.156:0/1324221465 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf801bb60 con 0x7f4d00161340 2023-11-28T13:08:14.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.116+0000 7f4d071ff700 1 -- 172.21.15.156:0/1324221465 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d000d1300 con 0x7f4d00161340 2023-11-28T13:08:14.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.116+0000 7f4d071ff700 1 -- 172.21.15.156:0/1324221465 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d000b5520 con 0x7f4d00161340 2023-11-28T13:08:14.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.117+0000 7f4d071ff700 1 -- 172.21.15.156:0/1324221465 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d000d15e0 con 0x7f4d00161340 2023-11-28T13:08:14.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.118+0000 7f4cfd7fa700 1 -- 172.21.15.156:0/1324221465 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4cf802d070 con 0x7f4d00161340 2023-11-28T13:08:14.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.119+0000 7f4cfd7fa700 1 --2- 172.21.15.156:0/1324221465 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4ce40682f0 0x7f4ce406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:14.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.119+0000 7f4cfd7fa700 1 -- 172.21.15.156:0/1324221465 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4cf8087640 con 0x7f4d00161340 2023-11-28T13:08:14.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.119+0000 7f4d04f9b700 1 --2- 172.21.15.156:0/1324221465 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4ce40682f0 0x7f4ce406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:14.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.120+0000 7f4d04f9b700 1 --2- 172.21.15.156:0/1324221465 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4ce40682f0 0x7f4ce406a7a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4d000c67a0 tx=0x7f4cf4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:14.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.122+0000 7f4cfd7fa700 1 -- 172.21.15.156:0/1324221465 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4cf8053db0 con 0x7f4d00161340 2023-11-28T13:08:14.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.312+0000 7f4d071ff700 1 -- 172.21.15.156:0/1324221465 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d000745c0 con 0x7f4d00161340 2023-11-28T13:08:14.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.312+0000 7f4cfd7fa700 1 -- 172.21.15.156:0/1324221465 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4cf8021020 con 0x7f4d00161340 2023-11-28T13:08:14.424 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T13:08:14.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.315+0000 7f4d071ff700 1 -- 172.21.15.156:0/1324221465 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4ce40682f0 msgr2=0x7f4ce406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:14.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.315+0000 7f4d071ff700 1 --2- 172.21.15.156:0/1324221465 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4ce40682f0 0x7f4ce406a7a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4d000c67a0 tx=0x7f4cf4009040 comp rx=0 tx=0).stop 2023-11-28T13:08:14.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.315+0000 7f4d071ff700 1 -- 172.21.15.156:0/1324221465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d00161340 msgr2=0x7f4d000c7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:14.426 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.315+0000 7f4d071ff700 1 --2- 172.21.15.156:0/1324221465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d00161340 0x7f4d000c7950 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800b730 tx=0x7f4cf8004ad0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.315+0000 7f4d071ff700 1 -- 172.21.15.156:0/1324221465 shutdown_connections 2023-11-28T13:08:14.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.315+0000 7f4d071ff700 1 --2- 172.21.15.156:0/1324221465 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4ce40682f0 0x7f4ce406a7a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.315+0000 7f4d071ff700 1 --2- 172.21.15.156:0/1324221465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d00161340 0x7f4d000c7950 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.315+0000 7f4d071ff700 1 --2- 172.21.15.156:0/1324221465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000cd7c0 0x7f4d000b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.315+0000 7f4d071ff700 1 --2- 172.21.15.156:0/1324221465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d0015c9b0 0x7f4d001f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.315+0000 7f4d071ff700 1 -- 172.21.15.156:0/1324221465 >> 172.21.15.156:0/1324221465 conn(0x7f4d00072600 msgr2=0x7f4d00160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:14.428 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.315+0000 7f4d071ff700 1 -- 172.21.15.156:0/1324221465 shutdown_connections 2023-11-28T13:08:14.428 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.315+0000 7f4d071ff700 1 -- 172.21.15.156:0/1324221465 wait complete. 2023-11-28T13:08:14.429 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:08:14.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.626+0000 7f633cdff700 1 Processor -- start 2023-11-28T13:08:14.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.626+0000 7f633cdff700 1 -- start start 2023-11-28T13:08:14.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.627+0000 7f633cdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63380d01b0 0x7f63380d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:14.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.627+0000 7f633cdff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f63380b16d0 0x7f63380d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:14.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.627+0000 7f633cdff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63380b0230 con 0x7f63380b20b0 2023-11-28T13:08:14.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.627+0000 7f633cdff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633815cec0 con 0x7f63380b16d0 2023-11-28T13:08:14.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.627+0000 7f633cdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63380b4570 con 0x7f63380d01b0 2023-11-28T13:08:14.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.627+0000 7f6336d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f63380b16d0 0x7f63380d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:14.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.627+0000 7f6336d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f63380b16d0 0x7f63380d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59164/0 (socket says 172.21.15.156:59164) 2023-11-28T13:08:14.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.627+0000 7f6336d9d700 1 -- 172.21.15.156:0/1918195941 learned_addr learned my addr 172.21.15.156:0/1918195941 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:14.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.627+0000 7f633659c700 1 --2- 172.21.15.156:0/1918195941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63380d01b0 0x7f63380d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:14.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.628+0000 7f6336d9d700 1 -- 172.21.15.156:0/1918195941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63380d01b0 msgr2=0x7f63380d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:14.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.628+0000 7f6336d9d700 1 --2- 172.21.15.156:0/1918195941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63380d01b0 0x7f63380d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.628+0000 7f6336d9d700 1 -- 172.21.15.156:0/1918195941 >> v1:172.21.15.186:6789/0 conn(0x7f63380b20b0 legacy=0x7f63380b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:14.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.628+0000 7f6336d9d700 1 -- 172.21.15.156:0/1918195941 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63380b4850 con 0x7f63380b16d0 2023-11-28T13:08:14.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.628+0000 7f633659c700 1 --2- 172.21.15.156:0/1918195941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63380d01b0 0x7f63380d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:14.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.628+0000 7f6336d9d700 1 --2- 172.21.15.156:0/1918195941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63380b16d0 0x7f63380d1160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6330009e80 tx=0x7f633000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c953e857a2294ecb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:14.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.628+0000 7f633559a700 1 -- 172.21.15.156:0/1918195941 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633000e070 con 0x7f63380b16d0 2023-11-28T13:08:14.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.628+0000 7f633559a700 1 -- 172.21.15.156:0/1918195941 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633000b980 con 0x7f63380b16d0 2023-11-28T13:08:14.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.628+0000 7f633559a700 1 -- 172.21.15.156:0/1918195941 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633000bd30 con 0x7f63380b16d0 2023-11-28T13:08:14.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.629+0000 7f633cdff700 1 -- 172.21.15.156:0/1918195941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63380b16d0 msgr2=0x7f63380d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:14.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.629+0000 7f633cdff700 1 --2- 172.21.15.156:0/1918195941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63380b16d0 0x7f63380d1160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6330009e80 tx=0x7f633000b330 comp rx=0 tx=0).stop 2023-11-28T13:08:14.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.629+0000 7f633cdff700 1 -- 172.21.15.156:0/1918195941 shutdown_connections 2023-11-28T13:08:14.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.629+0000 7f633cdff700 1 --2- 172.21.15.156:0/1918195941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63380b16d0 0x7f63380d1160 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.629+0000 7f633cdff700 1 --2- 172.21.15.156:0/1918195941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63380d01b0 0x7f63380d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.629+0000 7f633cdff700 1 -- 172.21.15.156:0/1918195941 >> 172.21.15.156:0/1918195941 conn(0x7f633809c3f0 msgr2=0x7f63380d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:14.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.629+0000 7f633cdff700 1 -- 172.21.15.156:0/1918195941 shutdown_connections 2023-11-28T13:08:14.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.629+0000 7f633cdff700 1 -- 172.21.15.156:0/1918195941 wait complete. 2023-11-28T13:08:14.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.631+0000 7f633cdff700 1 Processor -- start 2023-11-28T13:08:14.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.631+0000 7f633cdff700 1 -- start start 2023-11-28T13:08:14.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.633+0000 7f633cdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63380b16d0 0x7f633815af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:14.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.633+0000 7f633cdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63380b20b0 0x7f63380b7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:14.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.633+0000 7f633cdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63380d01b0 0x7f63380cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:14.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.633+0000 7f633cdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63380b84b0 con 0x7f63380b16d0 2023-11-28T13:08:14.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.633+0000 7f633cdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63380b81d0 con 0x7f63380d01b0 2023-11-28T13:08:14.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.633+0000 7f633cdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633815d1a0 con 0x7f63380b20b0 2023-11-28T13:08:14.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.633+0000 7f633659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63380b16d0 0x7f633815af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:14.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.633+0000 7f633659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63380b16d0 0x7f633815af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59168/0 (socket says 172.21.15.156:59168) 2023-11-28T13:08:14.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.633+0000 7f633659c700 1 -- 172.21.15.156:0/1562071501 learned_addr learned my addr 172.21.15.156:0/1562071501 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.633+0000 7f633659c700 1 -- 172.21.15.156:0/1562071501 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63380b20b0 msgr2=0x7f63380b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.633+0000 7f633659c700 1 --2- 172.21.15.156:0/1562071501 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63380b20b0 0x7f63380b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.633+0000 7f633659c700 1 -- 172.21.15.156:0/1562071501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63380d01b0 msgr2=0x7f63380cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.633+0000 7f633659c700 1 --2- 172.21.15.156:0/1562071501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63380d01b0 0x7f63380cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.633+0000 7f633659c700 1 -- 172.21.15.156:0/1562071501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6330009b30 con 0x7f63380b16d0 2023-11-28T13:08:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.633+0000 7f633659c700 1 --2- 172.21.15.156:0/1562071501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63380b16d0 0x7f633815af00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f632800e810 tx=0x7f632800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.634+0000 7f6326ffd700 1 -- 172.21.15.156:0/1562071501 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632800c020 con 0x7f63380b16d0 2023-11-28T13:08:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.634+0000 7f6326ffd700 1 -- 172.21.15.156:0/1562071501 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6328004250 con 0x7f63380b16d0 2023-11-28T13:08:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.634+0000 7f633cdff700 1 -- 172.21.15.156:0/1562071501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63380afec0 con 0x7f63380b16d0 2023-11-28T13:08:14.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.634+0000 7f6326ffd700 1 -- 172.21.15.156:0/1562071501 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63280036a0 con 0x7f63380b16d0 2023-11-28T13:08:14.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.634+0000 7f633cdff700 1 -- 172.21.15.156:0/1562071501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63380afbe0 con 0x7f63380b16d0 2023-11-28T13:08:14.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.635+0000 7f633cdff700 1 -- 172.21.15.156:0/1562071501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63380b2600 con 0x7f63380b16d0 2023-11-28T13:08:14.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.636+0000 7f6326ffd700 1 -- 172.21.15.156:0/1562071501 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6328003800 con 0x7f63380b16d0 2023-11-28T13:08:14.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.636+0000 7f6326ffd700 1 --2- 172.21.15.156:0/1562071501 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63180682f0 0x7f631806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:14.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.637+0000 7f6326ffd700 1 -- 172.21.15.156:0/1562071501 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6328085a60 con 0x7f63380b16d0 2023-11-28T13:08:14.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.637+0000 7f6335d9b700 1 --2- 172.21.15.156:0/1562071501 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63180682f0 0x7f631806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:14.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.638+0000 7f6335d9b700 1 --2- 172.21.15.156:0/1562071501 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63180682f0 0x7f631806a7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f632c000f60 tx=0x7f632c007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:14.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.640+0000 7f6326ffd700 1 -- 172.21.15.156:0/1562071501 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f63280521d0 con 0x7f63380b16d0 2023-11-28T13:08:14.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.875+0000 7f633cdff700 1 -- 172.21.15.156:0/1562071501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6338068690 con 0x7f63380b16d0 2023-11-28T13:08:14.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.876+0000 7f6326ffd700 1 -- 172.21.15.156:0/1562071501 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4651 (secure 0 0 0) 0x7f632800f040 con 0x7f63380b16d0 2023-11-28T13:08:14.878 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:14.878 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1283906891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1283906891},{"type":"v1","addr":"172.21.15.156:6835","nonce":1283906891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1738881422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1738881422},{"type":"v1","addr":"172.21.15.156:6833","nonce":1738881422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2235657629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2235657629},{"type":"v1","addr":"172.21.15.186:6836","nonce":2235657629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:08:03.058116+0000","modified":"2023-11-28T13:08:06.077600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/402651964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":402651964},{"type":"v1","addr":"172.21.15.186:6837","nonce":402651964}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:08:14.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.878+0000 7f633cdff700 1 -- 172.21.15.156:0/1562071501 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63180682f0 msgr2=0x7f631806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:14.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.878+0000 7f633cdff700 1 --2- 172.21.15.156:0/1562071501 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63180682f0 0x7f631806a7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f632c000f60 tx=0x7f632c007a40 comp rx=0 tx=0).stop 2023-11-28T13:08:14.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.878+0000 7f633cdff700 1 -- 172.21.15.156:0/1562071501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63380b16d0 msgr2=0x7f633815af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:14.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.878+0000 7f633cdff700 1 --2- 172.21.15.156:0/1562071501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63380b16d0 0x7f633815af00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f632800e810 tx=0x7f632800ebd0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.878+0000 7f633cdff700 1 -- 172.21.15.156:0/1562071501 shutdown_connections 2023-11-28T13:08:14.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.878+0000 7f633cdff700 1 --2- 172.21.15.156:0/1562071501 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63180682f0 0x7f631806a7a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.878+0000 7f633cdff700 1 --2- 172.21.15.156:0/1562071501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63380d01b0 0x7f63380cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.878+0000 7f633cdff700 1 --2- 172.21.15.156:0/1562071501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63380b16d0 0x7f633815af00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.878+0000 7f633cdff700 1 --2- 172.21.15.156:0/1562071501 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63380b20b0 0x7f63380b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:14.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.878+0000 7f633cdff700 1 -- 172.21.15.156:0/1562071501 >> 172.21.15.156:0/1562071501 conn(0x7f633809c3f0 msgr2=0x7f63380b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:14.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.878+0000 7f633cdff700 1 -- 172.21.15.156:0/1562071501 shutdown_connections 2023-11-28T13:08:14.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:14.878+0000 7f633cdff700 1 -- 172.21.15.156:0/1562071501 wait complete. 2023-11-28T13:08:14.881 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 8 2023-11-28T13:08:15.301 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T13:08:03.058116+0000', 'modified': '2023-11-28T13:08:06.077600+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4278}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4278': {'gid': 4278, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.186:6837/402651964', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6835', 'nonce': 402651964}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 402651964}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T13:08:15.301 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T13:08:15.302 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T13:08:15.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.499+0000 7f3982fff700 1 Processor -- start 2023-11-28T13:08:15.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.500+0000 7f3982fff700 1 -- start start 2023-11-28T13:08:15.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.501+0000 7f3982fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f397c161340 0x7f397c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.501+0000 7f3982fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f397c0d1950 con 0x7f397c0d2320 2023-11-28T13:08:15.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.501+0000 7f3982fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f397c0b5520 con 0x7f397c15c9b0 2023-11-28T13:08:15.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.501+0000 7f3982fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f397c0b5800 con 0x7f397c161340 2023-11-28T13:08:15.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.501+0000 7f397bfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f397c161340 0x7f397c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:15.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.501+0000 7f398159c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f397c0d2320 0x7f397c0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:34672/0 (socket says 172.21.15.156:34672) 2023-11-28T13:08:15.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.501+0000 7f398159c700 1 -- 172.21.15.156:0/188995561 learned_addr learned my addr 172.21.15.156:0/188995561 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:15.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.501+0000 7f397bfff700 1 -- 172.21.15.156:0/188995561 >> v1:172.21.15.186:6789/0 conn(0x7f397c15c9b0 legacy=0x7f397c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:15.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.501+0000 7f397b7fe700 1 -- 172.21.15.156:0/188995561 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314883684 0 0) 0x7f397c0d1950 con 0x7f397c0d2320 2023-11-28T13:08:15.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.502+0000 7f397bfff700 1 -- 172.21.15.156:0/188995561 >> v1:172.21.15.156:6789/0 conn(0x7f397c0d2320 legacy=0x7f397c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:15.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.502+0000 7f397bfff700 1 -- 172.21.15.156:0/188995561 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c0d1300 con 0x7f397c161340 2023-11-28T13:08:15.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.502+0000 7f397bfff700 1 --2- 172.21.15.156:0/188995561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f397c161340 0x7f397c0d1de0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3968009840 tx=0x7f3968009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70796a29a46da81c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:15.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.502+0000 7f397b7fe700 1 -- 172.21.15.156:0/188995561 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968011830 con 0x7f397c161340 2023-11-28T13:08:15.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.502+0000 7f397b7fe700 1 -- 172.21.15.156:0/188995561 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3968011990 con 0x7f397c161340 2023-11-28T13:08:15.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.502+0000 7f397b7fe700 1 -- 172.21.15.156:0/188995561 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968011d20 con 0x7f397c161340 2023-11-28T13:08:15.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.503+0000 7f3982fff700 1 -- 172.21.15.156:0/188995561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f397c161340 msgr2=0x7f397c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:15.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.503+0000 7f3982fff700 1 --2- 172.21.15.156:0/188995561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f397c161340 0x7f397c0d1de0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3968009840 tx=0x7f3968009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:15.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.503+0000 7f3982fff700 1 -- 172.21.15.156:0/188995561 shutdown_connections 2023-11-28T13:08:15.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.503+0000 7f3982fff700 1 --2- 172.21.15.156:0/188995561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f397c161340 0x7f397c0d1de0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.503+0000 7f3982fff700 1 -- 172.21.15.156:0/188995561 >> 172.21.15.156:0/188995561 conn(0x7f397c072600 msgr2=0x7f397c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:15.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.503+0000 7f3982fff700 1 -- 172.21.15.156:0/188995561 shutdown_connections 2023-11-28T13:08:15.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.503+0000 7f3982fff700 1 -- 172.21.15.156:0/188995561 wait complete. 2023-11-28T13:08:15.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.505+0000 7f3982fff700 1 Processor -- start 2023-11-28T13:08:15.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.505+0000 7f3982fff700 1 -- start start 2023-11-28T13:08:15.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.506+0000 7f3982fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f397c0d2320 0x7f397c15e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.507+0000 7f3982fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f397c15c9b0 0x7f397c06a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.507+0000 7f3982fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f397c161340 0x7f397c06ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.507+0000 7f3982fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c0ca3e0 con 0x7f397c161340 2023-11-28T13:08:15.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.507+0000 7f3982fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c0c8970 con 0x7f397c15c9b0 2023-11-28T13:08:15.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.507+0000 7f3982fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397c0c7d60 con 0x7f397c0d2320 2023-11-28T13:08:15.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.507+0000 7f398159c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f397c161340 0x7f397c06ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:15.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.507+0000 7f398159c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f397c161340 0x7f397c06ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59190/0 (socket says 172.21.15.156:59190) 2023-11-28T13:08:15.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.507+0000 7f398159c700 1 -- 172.21.15.156:0/15527896 learned_addr learned my addr 172.21.15.156:0/15527896 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:15.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.507+0000 7f397bfff700 1 --2- 172.21.15.156:0/15527896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f397c15c9b0 0x7f397c06a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:15.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.507+0000 7f3980d9b700 1 --2- 172.21.15.156:0/15527896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f397c0d2320 0x7f397c15e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:15.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.507+0000 7f398159c700 1 -- 172.21.15.156:0/15527896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f397c0d2320 msgr2=0x7f397c15e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:15.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.507+0000 7f398159c700 1 --2- 172.21.15.156:0/15527896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f397c0d2320 0x7f397c15e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.507+0000 7f398159c700 1 -- 172.21.15.156:0/15527896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f397c15c9b0 msgr2=0x7f397c06a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:15.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.507+0000 7f398159c700 1 --2- 172.21.15.156:0/15527896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f397c15c9b0 0x7f397c06a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.507+0000 7f398159c700 1 -- 172.21.15.156:0/15527896 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39680094a0 con 0x7f397c161340 2023-11-28T13:08:15.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.507+0000 7f397bfff700 1 --2- 172.21.15.156:0/15527896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f397c15c9b0 0x7f397c06a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:15.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.507+0000 7f398159c700 1 --2- 172.21.15.156:0/15527896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f397c161340 0x7f397c06ae40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f397400cc70 tx=0x7f397400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:15.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.508+0000 7f39797fa700 1 -- 172.21.15.156:0/15527896 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397400ab50 con 0x7f397c161340 2023-11-28T13:08:15.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.508+0000 7f39797fa700 1 -- 172.21.15.156:0/15527896 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397400acd0 con 0x7f397c161340 2023-11-28T13:08:15.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.508+0000 7f3982fff700 1 -- 172.21.15.156:0/15527896 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397c15cd80 con 0x7f397c161340 2023-11-28T13:08:15.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.508+0000 7f39797fa700 1 -- 172.21.15.156:0/15527896 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974007310 con 0x7f397c161340 2023-11-28T13:08:15.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.508+0000 7f3982fff700 1 -- 172.21.15.156:0/15527896 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397c0d15e0 con 0x7f397c161340 2023-11-28T13:08:15.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.510+0000 7f39797fa700 1 -- 172.21.15.156:0/15527896 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3974010070 con 0x7f397c161340 2023-11-28T13:08:15.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.511+0000 7f39797fa700 1 --2- 172.21.15.156:0/15527896 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39600682f0 0x7f396006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.511+0000 7f39797fa700 1 -- 172.21.15.156:0/15527896 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3974085760 con 0x7f397c161340 2023-11-28T13:08:15.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.511+0000 7f3980d9b700 1 --2- 172.21.15.156:0/15527896 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39600682f0 0x7f396006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:15.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.512+0000 7f3980d9b700 1 --2- 172.21.15.156:0/15527896 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39600682f0 0x7f396006a7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f397c15f940 tx=0x7f3970009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:15.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.513+0000 7f399695db80 1 Processor -- start 2023-11-28T13:08:15.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.513+0000 7f399695db80 1 -- start start 2023-11-28T13:08:15.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.514+0000 7f399695db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf91810 0x55e32cf67a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.514+0000 7f399695db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf8f130 0x55e32cf91be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.514+0000 7f399695db80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 0x55e32cfad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.514+0000 7f399695db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32ce51bc0 con 0x55e32cf8f130 2023-11-28T13:08:15.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.514+0000 7f399695db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cec0350 con 0x55e32cf91810 2023-11-28T13:08:15.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.514+0000 7f399695db80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e32ceb9880 con 0x55e32cf92120 2023-11-28T13:08:15.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.515+0000 7f397bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf8f130 0x55e32cf91be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:15.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.515+0000 7f397bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf8f130 0x55e32cf91be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59198/0 (socket says 172.21.15.156:59198) 2023-11-28T13:08:15.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.515+0000 7f397bfff700 1 -- 172.21.15.156:0/825082068 learned_addr learned my addr 172.21.15.156:0/825082068 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:15.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.515+0000 7f397bfff700 1 -- 172.21.15.156:0/825082068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 msgr2=0x55e32cfad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:15.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.515+0000 7f397bfff700 1 --2- 172.21.15.156:0/825082068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 0x55e32cfad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.515+0000 7f397bfff700 1 -- 172.21.15.156:0/825082068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf91810 msgr2=0x55e32cf67a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:15.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.515+0000 7f397bfff700 1 --2- 172.21.15.156:0/825082068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf91810 0x55e32cf67a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.515+0000 7f397bfff700 1 -- 172.21.15.156:0/825082068 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e32cd3b5e0 con 0x55e32cf8f130 2023-11-28T13:08:15.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.515+0000 7f397bfff700 1 --2- 172.21.15.156:0/825082068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf8f130 0x55e32cf91be0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3968013000 tx=0x7f3968008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae9b8f881395dd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:15.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.515+0000 7f39667fc700 1 -- 172.21.15.156:0/825082068 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968011bc0 con 0x55e32cf8f130 2023-11-28T13:08:15.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.515+0000 7f39667fc700 1 -- 172.21.15.156:0/825082068 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3968004040 con 0x55e32cf8f130 2023-11-28T13:08:15.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.516+0000 7f39667fc700 1 -- 172.21.15.156:0/825082068 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39680043f0 con 0x55e32cf8f130 2023-11-28T13:08:15.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.516+0000 7f399695db80 1 -- 172.21.15.156:0/825082068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf8f130 msgr2=0x55e32cf91be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:15.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.516+0000 7f399695db80 1 --2- 172.21.15.156:0/825082068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf8f130 0x55e32cf91be0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3968013000 tx=0x7f3968008030 comp rx=0 tx=0).stop 2023-11-28T13:08:15.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.516+0000 7f399695db80 1 -- 172.21.15.156:0/825082068 shutdown_connections 2023-11-28T13:08:15.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.516+0000 7f399695db80 1 --2- 172.21.15.156:0/825082068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 0x55e32cfad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.516+0000 7f399695db80 1 --2- 172.21.15.156:0/825082068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf91810 0x55e32cf67a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.516+0000 7f399695db80 1 --2- 172.21.15.156:0/825082068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf8f130 0x55e32cf91be0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.516+0000 7f399695db80 1 -- 172.21.15.156:0/825082068 >> 172.21.15.156:0/825082068 conn(0x55e32cf64d60 msgr2=0x55e32cf656f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:15.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.516+0000 7f399695db80 1 -- 172.21.15.156:0/825082068 shutdown_connections 2023-11-28T13:08:15.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.516+0000 7f399695db80 1 -- 172.21.15.156:0/825082068 wait complete. 2023-11-28T13:08:15.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.517+0000 7f399695db80 1 -- start start 2023-11-28T13:08:15.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.517+0000 7f399695db80 1 Processor -- start 2023-11-28T13:08:15.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.518+0000 7f399695db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf8f130 0x55e32cf89be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.518+0000 7f399695db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf91810 0x55e32cfb47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.518+0000 7f399695db80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 0x55e32cf60ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.518+0000 7f399695db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cd3d8d0 con 0x55e32cf91810 2023-11-28T13:08:15.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.518+0000 7f399695db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cfa47f0 con 0x55e32cf8f130 2023-11-28T13:08:15.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.518+0000 7f399695db80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e32cfa5a80 con 0x55e32cf92120 2023-11-28T13:08:15.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.518+0000 7f399695db80 10 client.0 _ll_register_callbacks cb 0x55e32cd5f240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:08:15.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.519+0000 7f397bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf91810 0x55e32cfb47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:15.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.519+0000 7f397bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf91810 0x55e32cfb47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59200/0 (socket says 172.21.15.156:59200) 2023-11-28T13:08:15.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.519+0000 7f397bfff700 1 -- 172.21.15.156:0/2732524973 learned_addr learned my addr 172.21.15.156:0/2732524973 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:15.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.519+0000 7f3980d9b700 1 --2- 172.21.15.156:0/2732524973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf8f130 0x55e32cf89be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:15.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.519+0000 7f398159c700 1 --2- 172.21.15.156:0/2732524973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 0x55e32cf60ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:15.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.519+0000 7f397bfff700 1 -- 172.21.15.156:0/2732524973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 msgr2=0x55e32cf60ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:15.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.519+0000 7f397bfff700 1 --2- 172.21.15.156:0/2732524973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 0x55e32cf60ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.519+0000 7f397bfff700 1 -- 172.21.15.156:0/2732524973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf8f130 msgr2=0x55e32cf89be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:15.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.519+0000 7f397bfff700 1 --2- 172.21.15.156:0/2732524973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf8f130 0x55e32cf89be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.519+0000 7f397bfff700 1 -- 172.21.15.156:0/2732524973 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397c0d1300 con 0x55e32cf91810 2023-11-28T13:08:15.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.519+0000 7f398159c700 1 --2- 172.21.15.156:0/2732524973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 0x55e32cf60ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:15.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.519+0000 7f3980d9b700 1 --2- 172.21.15.156:0/2732524973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf8f130 0x55e32cf89be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:15.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.519+0000 7f397bfff700 1 --2- 172.21.15.156:0/2732524973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf91810 0x55e32cfb47a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3968011960 tx=0x7f39680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4bc6c754a2ee37 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:15.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.519+0000 7f395b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:08:15.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.519+0000 7f395b7fe700 1 -- 172.21.15.156:0/2732524973 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968004220 con 0x55e32cf91810 2023-11-28T13:08:15.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.519+0000 7f395b7fe700 1 -- 172.21.15.156:0/2732524973 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3968004380 con 0x55e32cf91810 2023-11-28T13:08:15.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.519+0000 7f399695db80 1 -- 172.21.15.156:0/2732524973 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e32ce87140 con 0x55e32cf91810 2023-11-28T13:08:15.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.520+0000 7f395b7fe700 1 -- 172.21.15.156:0/2732524973 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3968004540 con 0x55e32cf91810 2023-11-28T13:08:15.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.520+0000 7f399695db80 10 client.4566 fetch_fsmap learned FSMap version 8 2023-11-28T13:08:15.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.520+0000 7f399695db80 1 -- 172.21.15.156:0/2732524973 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55e32cce7d90 con 0x55e32cf91810 2023-11-28T13:08:15.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.520+0000 7f395b7fe700 1 -- 172.21.15.156:0/2732524973 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e32ce87140 con 0x55e32cf91810 2023-11-28T13:08:15.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.520+0000 7f395b7fe700 1 -- 172.21.15.156:0/2732524973 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f396801b800 con 0x55e32cf91810 2023-11-28T13:08:15.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.520+0000 7f399695db80 10 client.4566 fetch_fsmap finished waiting for FSMap version 8 2023-11-28T13:08:15.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.520+0000 7f399695db80 10 client.4566 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T13:08:15.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.520+0000 7f399695db80 20 client.4566 populate_metadata read hostname 'smithi156' 2023-11-28T13:08:15.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.520+0000 7f399695db80 1 --2- 172.21.15.156:0/2732524973 >> [v2:172.21.15.186:6835/402651964,v1:172.21.15.186:6837/402651964] conn(0x55e32cfa6110 0x55e32cfa64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.520+0000 7f399695db80 4 client.4566 mds_command: new command op to 4278 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:08:15.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.520+0000 7f399695db80 1 -- 172.21.15.156:0/2732524973 --> [v2:172.21.15.186:6835/402651964,v1:172.21.15.186:6837/402651964] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e32cd3df10 con 0x55e32cfa6110 2023-11-28T13:08:15.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.520+0000 7f3980d9b700 1 --2- 172.21.15.156:0/2732524973 >> [v2:172.21.15.186:6835/402651964,v1:172.21.15.186:6837/402651964] conn(0x55e32cfa6110 0x55e32cfa64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:15.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.521+0000 7f3980d9b700 1 --2- 172.21.15.156:0/2732524973 >> [v2:172.21.15.186:6835/402651964,v1:172.21.15.186:6837/402651964] conn(0x55e32cfa6110 0x55e32cfa64e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d0b00c10693676b server_cookie=2e714cc8670b7e65 in_seq=0 out_seq=0 2023-11-28T13:08:15.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.521+0000 7f395b7fe700 10 client.4566 ms_handle_connect on v2:172.21.15.186:6835/402651964 2023-11-28T13:08:15.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.522+0000 7f395b7fe700 1 -- 172.21.15.156:0/2732524973 <== mds.0 v2:172.21.15.186:6835/402651964 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55e32cd3df10 con 0x55e32cfa6110 2023-11-28T13:08:15.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.522+0000 7f395b7fe700 10 client.4566 handle_command_reply: tid=0 2023-11-28T13:08:15.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.522+0000 7f399695db80 1 client.4566 shutdown 2023-11-28T13:08:15.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.522+0000 7f399695db80 1 -- 172.21.15.156:0/2732524973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf91810 msgr2=0x55e32cfb47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:15.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.522+0000 7f399695db80 1 --2- 172.21.15.156:0/2732524973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf91810 0x55e32cfb47a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3968011960 tx=0x7f39680047c0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.522+0000 7f399695db80 1 -- 172.21.15.156:0/2732524973 shutdown_connections 2023-11-28T13:08:15.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.522+0000 7f399695db80 1 --2- 172.21.15.156:0/2732524973 >> [v2:172.21.15.186:6835/402651964,v1:172.21.15.186:6837/402651964] conn(0x55e32cfa6110 0x55e32cfa64e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.522+0000 7f399695db80 1 --2- 172.21.15.156:0/2732524973 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 0x55e32cf60ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.522+0000 7f399695db80 1 --2- 172.21.15.156:0/2732524973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf8f130 0x55e32cf89be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.522+0000 7f395b7fe700 0 client.4566 ms_handle_reset on v2:172.21.15.186:6835/402651964 2023-11-28T13:08:15.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.522+0000 7f399695db80 1 --2- 172.21.15.156:0/2732524973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf91810 0x55e32cfb47a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.522+0000 7f399695db80 1 -- 172.21.15.156:0/2732524973 >> 172.21.15.156:0/2732524973 conn(0x55e32cf64d60 msgr2=0x55e32cf68a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:15.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.522+0000 7f399695db80 1 -- 172.21.15.156:0/2732524973 shutdown_connections 2023-11-28T13:08:15.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.522+0000 7f399695db80 1 -- 172.21.15.156:0/2732524973 wait complete. 2023-11-28T13:08:15.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.523+0000 7f399695db80 20 client.4566 trim_cache size 0 max 16384 2023-11-28T13:08:15.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.540+0000 7f399695db80 1 Processor -- start 2023-11-28T13:08:15.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.540+0000 7f399695db80 1 -- start start 2023-11-28T13:08:15.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.541+0000 7f399695db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf8f130 0x55e32cf88dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.541+0000 7f399695db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf91810 0x55e32cfbb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.541+0000 7f399695db80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 0x55e32cfbbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.541+0000 7f399695db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cfb3800 con 0x55e32cf91810 2023-11-28T13:08:15.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.541+0000 7f399695db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cfa5c90 con 0x55e32cf8f130 2023-11-28T13:08:15.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.541+0000 7f399695db80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e32cf68730 con 0x55e32cf92120 2023-11-28T13:08:15.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.541+0000 7f397bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf91810 0x55e32cfbb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:15.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.541+0000 7f397bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf91810 0x55e32cfbb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59202/0 (socket says 172.21.15.156:59202) 2023-11-28T13:08:15.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.541+0000 7f397bfff700 1 -- 172.21.15.156:0/2305508925 learned_addr learned my addr 172.21.15.156:0/2305508925 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:15.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.541+0000 7f3980d9b700 1 --2- 172.21.15.156:0/2305508925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf8f130 0x55e32cf88dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:15.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.541+0000 7f398159c700 1 --2- 172.21.15.156:0/2305508925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 0x55e32cfbbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:15.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.541+0000 7f397bfff700 1 -- 172.21.15.156:0/2305508925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 msgr2=0x55e32cfbbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:15.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.541+0000 7f397bfff700 1 --2- 172.21.15.156:0/2305508925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 0x55e32cfbbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.541+0000 7f397bfff700 1 -- 172.21.15.156:0/2305508925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf8f130 msgr2=0x55e32cf88dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:15.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.541+0000 7f397bfff700 1 --2- 172.21.15.156:0/2305508925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf8f130 0x55e32cf88dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.541+0000 7f397bfff700 1 -- 172.21.15.156:0/2305508925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e32ce80140 con 0x55e32cf91810 2023-11-28T13:08:15.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.541+0000 7f3980d9b700 1 --2- 172.21.15.156:0/2305508925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf8f130 0x55e32cf88dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:15.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.542+0000 7f397bfff700 1 --2- 172.21.15.156:0/2305508925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf91810 0x55e32cfbb7d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3968009840 tx=0x7f3968013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcb687867449dd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:15.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.542+0000 7f3966ffd700 1 -- 172.21.15.156:0/2305508925 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968006d20 con 0x55e32cf91810 2023-11-28T13:08:15.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.542+0000 7f3966ffd700 1 -- 172.21.15.156:0/2305508925 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396801bd10 con 0x55e32cf91810 2023-11-28T13:08:15.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.542+0000 7f3966ffd700 1 -- 172.21.15.156:0/2305508925 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39680038a0 con 0x55e32cf91810 2023-11-28T13:08:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.542+0000 7f399695db80 1 -- 172.21.15.156:0/2305508925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf91810 msgr2=0x55e32cfbb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.542+0000 7f399695db80 1 --2- 172.21.15.156:0/2305508925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf91810 0x55e32cfbb7d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3968009840 tx=0x7f3968013040 comp rx=0 tx=0).stop 2023-11-28T13:08:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.542+0000 7f399695db80 1 -- 172.21.15.156:0/2305508925 shutdown_connections 2023-11-28T13:08:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.542+0000 7f399695db80 1 --2- 172.21.15.156:0/2305508925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 0x55e32cfbbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.542+0000 7f399695db80 1 --2- 172.21.15.156:0/2305508925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf8f130 0x55e32cf88dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.542+0000 7f399695db80 1 --2- 172.21.15.156:0/2305508925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf91810 0x55e32cfbb7d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.542+0000 7f399695db80 1 -- 172.21.15.156:0/2305508925 >> 172.21.15.156:0/2305508925 conn(0x55e32cf64d60 msgr2=0x55e32cf677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:15.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.542+0000 7f399695db80 1 -- 172.21.15.156:0/2305508925 shutdown_connections 2023-11-28T13:08:15.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.543+0000 7f399695db80 1 -- 172.21.15.156:0/2305508925 wait complete. 2023-11-28T13:08:15.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.543+0000 7f399695db80 1 -- start start 2023-11-28T13:08:15.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.543+0000 7f399695db80 1 Processor -- start 2023-11-28T13:08:15.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.545+0000 7f399695db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf8f130 0x55e32cf60640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.545+0000 7f399695db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf91810 0x55e32cf60b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.545+0000 7f399695db80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 0x55e32cf66230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.545+0000 7f399695db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cf67290 con 0x55e32cf8f130 2023-11-28T13:08:15.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.545+0000 7f399695db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32cfc4db0 con 0x55e32cf91810 2023-11-28T13:08:15.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.545+0000 7f399695db80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e32cfbb310 con 0x55e32cf92120 2023-11-28T13:08:15.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.545+0000 7f399695db80 10 client.0 _ll_register_callbacks cb 0x55e32cd5f240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:08:15.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.545+0000 7f3980d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf8f130 0x55e32cf60640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:15.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.545+0000 7f3980d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf8f130 0x55e32cf60640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59218/0 (socket says 172.21.15.156:59218) 2023-11-28T13:08:15.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.545+0000 7f3980d9b700 1 -- 172.21.15.156:0/2883230281 learned_addr learned my addr 172.21.15.156:0/2883230281 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:15.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.545+0000 7f397bfff700 1 --2- 172.21.15.156:0/2883230281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf91810 0x55e32cf60b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:15.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.545+0000 7f398159c700 1 --2- 172.21.15.156:0/2883230281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 0x55e32cf66230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:15.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.545+0000 7f3980d9b700 1 -- 172.21.15.156:0/2883230281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 msgr2=0x55e32cf66230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:15.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.545+0000 7f3980d9b700 1 --2- 172.21.15.156:0/2883230281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 0x55e32cf66230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.545+0000 7f3980d9b700 1 -- 172.21.15.156:0/2883230281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf91810 msgr2=0x55e32cf60b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:15.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.545+0000 7f3980d9b700 1 --2- 172.21.15.156:0/2883230281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf91810 0x55e32cf60b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.545+0000 7f3980d9b700 1 -- 172.21.15.156:0/2883230281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e32cce7d90 con 0x55e32cf8f130 2023-11-28T13:08:15.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.546+0000 7f3980d9b700 1 --2- 172.21.15.156:0/2883230281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf8f130 0x55e32cf60640 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3970000f60 tx=0x7f3970001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343dbf9db2e7f68f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:15.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.546+0000 7f395b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:08:15.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.546+0000 7f395b7fe700 1 -- 172.21.15.156:0/2883230281 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970023a70 con 0x55e32cf8f130 2023-11-28T13:08:15.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.546+0000 7f395b7fe700 1 -- 172.21.15.156:0/2883230281 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3970023bd0 con 0x55e32cf8f130 2023-11-28T13:08:15.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.546+0000 7f399695db80 1 -- 172.21.15.156:0/2883230281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e32ce77160 con 0x55e32cf8f130 2023-11-28T13:08:15.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.546+0000 7f395b7fe700 1 -- 172.21.15.156:0/2883230281 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3970023d90 con 0x55e32cf8f130 2023-11-28T13:08:15.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.546+0000 7f399695db80 10 client.4572 fetch_fsmap learned FSMap version 8 2023-11-28T13:08:15.558 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.546+0000 7f399695db80 1 -- 172.21.15.156:0/2883230281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55e32cd0e200 con 0x55e32cf8f130 2023-11-28T13:08:15.558 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.546+0000 7f395b7fe700 1 -- 172.21.15.156:0/2883230281 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e32ce77160 con 0x55e32cf8f130 2023-11-28T13:08:15.558 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.547+0000 7f395b7fe700 1 -- 172.21.15.156:0/2883230281 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f3970014450 con 0x55e32cf8f130 2023-11-28T13:08:15.558 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.547+0000 7f399695db80 10 client.4572 fetch_fsmap finished waiting for FSMap version 8 2023-11-28T13:08:15.558 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.547+0000 7f399695db80 10 client.4572 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T13:08:15.558 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.547+0000 7f399695db80 20 client.4572 populate_metadata read hostname 'smithi156' 2023-11-28T13:08:15.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.547+0000 7f399695db80 1 --2- 172.21.15.156:0/2883230281 >> [v2:172.21.15.186:6835/402651964,v1:172.21.15.186:6837/402651964] conn(0x55e32cf66770 0x55e32d0115d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:15.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.547+0000 7f399695db80 4 client.4572 mds_command: new command op to 4278 tid=0[{"prefix": "status"}] 2023-11-28T13:08:15.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.547+0000 7f399695db80 1 -- 172.21.15.156:0/2883230281 --> [v2:172.21.15.186:6835/402651964,v1:172.21.15.186:6837/402651964] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e32ce61170 con 0x55e32cf66770 2023-11-28T13:08:15.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.547+0000 7f397bfff700 1 --2- 172.21.15.156:0/2883230281 >> [v2:172.21.15.186:6835/402651964,v1:172.21.15.186:6837/402651964] conn(0x55e32cf66770 0x55e32d0115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:15.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.547+0000 7f395b7fe700 10 client.4572 ms_handle_connect on v2:172.21.15.186:6835/402651964 2023-11-28T13:08:15.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.547+0000 7f397bfff700 1 --2- 172.21.15.156:0/2883230281 >> [v2:172.21.15.186:6835/402651964,v1:172.21.15.186:6837/402651964] conn(0x55e32cf66770 0x55e32d0115d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=854c8f1d355f89a8 server_cookie=d4c3971116dac8a5 in_seq=0 out_seq=0 2023-11-28T13:08:15.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.548+0000 7f395b7fe700 1 -- 172.21.15.156:0/2883230281 <== mds.0 v2:172.21.15.186:6835/402651964 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55e32ce61170 con 0x55e32cf66770 2023-11-28T13:08:15.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.548+0000 7f395b7fe700 10 client.4572 handle_command_reply: tid=0 2023-11-28T13:08:15.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.548+0000 7f399695db80 1 client.4572 shutdown 2023-11-28T13:08:15.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.548+0000 7f399695db80 1 -- 172.21.15.156:0/2883230281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf8f130 msgr2=0x55e32cf60640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:15.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.548+0000 7f399695db80 1 --2- 172.21.15.156:0/2883230281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf8f130 0x55e32cf60640 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3970000f60 tx=0x7f3970001120 comp rx=0 tx=0).stop 2023-11-28T13:08:15.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.548+0000 7f399695db80 1 -- 172.21.15.156:0/2883230281 shutdown_connections 2023-11-28T13:08:15.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.548+0000 7f399695db80 1 --2- 172.21.15.156:0/2883230281 >> [v2:172.21.15.186:6835/402651964,v1:172.21.15.186:6837/402651964] conn(0x55e32cf66770 0x55e32d0115d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.548+0000 7f399695db80 1 --2- 172.21.15.156:0/2883230281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55e32cf92120 0x55e32cf66230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.548+0000 7f399695db80 1 --2- 172.21.15.156:0/2883230281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e32cf8f130 0x55e32cf60640 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.548+0000 7f399695db80 1 --2- 172.21.15.156:0/2883230281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e32cf91810 0x55e32cf60b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.548+0000 7f395b7fe700 0 client.4572 ms_handle_reset on v2:172.21.15.186:6835/402651964 2023-11-28T13:08:15.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.548+0000 7f399695db80 1 -- 172.21.15.156:0/2883230281 >> 172.21.15.156:0/2883230281 conn(0x55e32cf64d60 msgr2=0x55e32cf65670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:15.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.548+0000 7f399695db80 1 -- 172.21.15.156:0/2883230281 shutdown_connections 2023-11-28T13:08:15.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.548+0000 7f399695db80 1 -- 172.21.15.156:0/2883230281 wait complete. 2023-11-28T13:08:15.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.548+0000 7f399695db80 20 client.4572 trim_cache size 0 max 16384 2023-11-28T13:08:15.562 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:08:15.562 INFO:teuthology.orchestra.run.smithi156.stdout: "cluster_fsid": "12d9e849-dc60-46f0-8e8e-9e291633f61b", 2023-11-28T13:08:15.562 INFO:teuthology.orchestra.run.smithi156.stdout: "whoami": 0, 2023-11-28T13:08:15.562 INFO:teuthology.orchestra.run.smithi156.stdout: "id": 4278, 2023-11-28T13:08:15.562 INFO:teuthology.orchestra.run.smithi156.stdout: "want_state": "up:active", 2023-11-28T13:08:15.563 INFO:teuthology.orchestra.run.smithi156.stdout: "state": "up:active", 2023-11-28T13:08:15.563 INFO:teuthology.orchestra.run.smithi156.stdout: "fs_name": "cephfs", 2023-11-28T13:08:15.563 INFO:teuthology.orchestra.run.smithi156.stdout: "rank_uptime": 12.488581536, 2023-11-28T13:08:15.563 INFO:teuthology.orchestra.run.smithi156.stdout: "mdsmap_epoch": 8, 2023-11-28T13:08:15.563 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_epoch": 21, 2023-11-28T13:08:15.563 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_epoch_barrier": 21, 2023-11-28T13:08:15.563 INFO:teuthology.orchestra.run.smithi156.stdout: "uptime": 18.507700466999999 2023-11-28T13:08:15.564 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:08:15.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.549+0000 7f3982fff700 1 -- 172.21.15.156:0/15527896 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39600682f0 msgr2=0x7f396006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:15.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.549+0000 7f3982fff700 1 --2- 172.21.15.156:0/15527896 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39600682f0 0x7f396006a7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f397c15f940 tx=0x7f3970009040 comp rx=0 tx=0).stop 2023-11-28T13:08:15.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.549+0000 7f3982fff700 1 -- 172.21.15.156:0/15527896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f397c161340 msgr2=0x7f397c06ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:15.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.549+0000 7f3982fff700 1 --2- 172.21.15.156:0/15527896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f397c161340 0x7f397c06ae40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f397400cc70 tx=0x7f397400cf80 comp rx=0 tx=0).stop 2023-11-28T13:08:15.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.549+0000 7f3982fff700 1 -- 172.21.15.156:0/15527896 shutdown_connections 2023-11-28T13:08:15.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.549+0000 7f3982fff700 1 --2- 172.21.15.156:0/15527896 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39600682f0 0x7f396006a7a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.549+0000 7f3982fff700 1 --2- 172.21.15.156:0/15527896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f397c161340 0x7f397c06ae40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.549+0000 7f3982fff700 1 --2- 172.21.15.156:0/15527896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f397c0d2320 0x7f397c15e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.549+0000 7f3982fff700 1 --2- 172.21.15.156:0/15527896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f397c15c9b0 0x7f397c06a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:15.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.549+0000 7f3982fff700 1 -- 172.21.15.156:0/15527896 >> 172.21.15.156:0/15527896 conn(0x7f397c072600 msgr2=0x7f397c15f990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:15.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.549+0000 7f3982fff700 1 -- 172.21.15.156:0/15527896 shutdown_connections 2023-11-28T13:08:15.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:15.549+0000 7f3982fff700 1 -- 172.21.15.156:0/15527896 wait complete. 2023-11-28T13:08:15.568 INFO:teuthology.run_tasks:Running task kclient... 2023-11-28T13:08:15.590 INFO:tasks.kclient:Mounting kernel clients... 2023-11-28T13:08:15.591 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-28T13:08:15.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:15.591 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T13:08:15.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:15.637 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link delete ceph-brx 2023-11-28T13:08:15.748 INFO:teuthology.orchestra.run.smithi156.stderr:Cannot find device "ceph-brx" 2023-11-28T13:08:15.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:08:15.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:15.752 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T13:08:15.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:15.796 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link delete ceph-brx 2023-11-28T13:08:15.907 INFO:teuthology.orchestra.run.smithi156.stderr:Cannot find device "ceph-brx" 2023-11-28T13:08:15.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:08:15.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:15.910 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2023-11-28T13:08:15.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:15.959 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link delete ceph-brx 2023-11-28T13:08:16.034 INFO:teuthology.orchestra.run.smithi186.stderr:Cannot find device "ceph-brx" 2023-11-28T13:08:16.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:08:16.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:16.038 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2023-11-28T13:08:16.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:16.083 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link delete ceph-brx 2023-11-28T13:08:16.193 INFO:teuthology.orchestra.run.smithi186.stderr:Cannot find device "ceph-brx" 2023-11-28T13:08:16.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:08:16.196 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-11-28T13:08:16.196 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T13:08:16.196 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T13:08:16.196 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:08:16.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.391+0000 7f1303fff700 1 Processor -- start 2023-11-28T13:08:16.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.391+0000 7f1303fff700 1 -- start start 2023-11-28T13:08:16.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.392+0000 7f1303fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12fc161340 0x7f12fc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.392+0000 7f1303fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12fc0d1950 con 0x7f12fc15c9b0 2023-11-28T13:08:16.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.392+0000 7f1303fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12fc0b5520 con 0x7f12fc0d2320 2023-11-28T13:08:16.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.392+0000 7f1303fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc0b5800 con 0x7f12fc161340 2023-11-28T13:08:16.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.393+0000 7f1301d9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f12fc15c9b0 0x7f12fc0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53590/0 (socket says 172.21.15.156:53590) 2023-11-28T13:08:16.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.393+0000 7f1301d9b700 1 -- 172.21.15.156:0/3212860094 learned_addr learned my addr 172.21.15.156:0/3212860094 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:16.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.393+0000 7f130159a700 1 --2- 172.21.15.156:0/3212860094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12fc161340 0x7f12fc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.393+0000 7f1300d99700 1 -- 172.21.15.156:0/3212860094 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703537445 0 0) 0x7f12fc0d1950 con 0x7f12fc15c9b0 2023-11-28T13:08:16.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.393+0000 7f1300d99700 1 -- 172.21.15.156:0/3212860094 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12e80035b0 con 0x7f12fc15c9b0 2023-11-28T13:08:16.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.393+0000 7f1300d99700 1 -- 172.21.15.156:0/3212860094 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095479607 0 0) 0x7f12fc0b5520 con 0x7f12fc0d2320 2023-11-28T13:08:16.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.393+0000 7f1300d99700 1 -- 172.21.15.156:0/3212860094 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12fc0d1950 con 0x7f12fc0d2320 2023-11-28T13:08:16.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.393+0000 7f130159a700 1 -- 172.21.15.156:0/3212860094 >> v1:172.21.15.186:6789/0 conn(0x7f12fc0d2320 legacy=0x7f12fc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.393+0000 7f130159a700 1 -- 172.21.15.156:0/3212860094 >> v1:172.21.15.156:6789/0 conn(0x7f12fc15c9b0 legacy=0x7f12fc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.393+0000 7f130159a700 1 -- 172.21.15.156:0/3212860094 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc0d1300 con 0x7f12fc161340 2023-11-28T13:08:16.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.393+0000 7f130159a700 1 --2- 172.21.15.156:0/3212860094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12fc161340 0x7f12fc0d1de0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009870 tx=0x7f12ec009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cec1353ba21a3a44 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:16.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.394+0000 7f1300d99700 1 -- 172.21.15.156:0/3212860094 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec011830 con 0x7f12fc161340 2023-11-28T13:08:16.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.394+0000 7f1300d99700 1 -- 172.21.15.156:0/3212860094 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ec011990 con 0x7f12fc161340 2023-11-28T13:08:16.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.394+0000 7f1300d99700 1 -- 172.21.15.156:0/3212860094 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec011d20 con 0x7f12fc161340 2023-11-28T13:08:16.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.394+0000 7f1303fff700 1 -- 172.21.15.156:0/3212860094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12fc161340 msgr2=0x7f12fc0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.394+0000 7f1303fff700 1 --2- 172.21.15.156:0/3212860094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12fc161340 0x7f12fc0d1de0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009870 tx=0x7f12ec009b80 comp rx=0 tx=0).stop 2023-11-28T13:08:16.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.394+0000 7f1303fff700 1 -- 172.21.15.156:0/3212860094 shutdown_connections 2023-11-28T13:08:16.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.394+0000 7f1303fff700 1 --2- 172.21.15.156:0/3212860094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12fc161340 0x7f12fc0d1de0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.394+0000 7f1303fff700 1 -- 172.21.15.156:0/3212860094 >> 172.21.15.156:0/3212860094 conn(0x7f12fc072600 msgr2=0x7f12fc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:16.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.394+0000 7f1303fff700 1 -- 172.21.15.156:0/3212860094 shutdown_connections 2023-11-28T13:08:16.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.395+0000 7f1303fff700 1 -- 172.21.15.156:0/3212860094 wait complete. 2023-11-28T13:08:16.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.396+0000 7f1303fff700 1 Processor -- start 2023-11-28T13:08:16.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.396+0000 7f1303fff700 1 -- start start 2023-11-28T13:08:16.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.397+0000 7f1303fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12fc0d2320 0x7f12fc0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.397+0000 7f1303fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12fc15c9b0 0x7f12fc153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.397+0000 7f1303fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12fc161340 0x7f12fc0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.397+0000 7f1303fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc0cb590 con 0x7f12fc15c9b0 2023-11-28T13:08:16.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.397+0000 7f1303fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc154110 con 0x7f12fc161340 2023-11-28T13:08:16.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.397+0000 7f1303fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc0cac60 con 0x7f12fc0d2320 2023-11-28T13:08:16.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.398+0000 7f130159a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12fc15c9b0 0x7f12fc153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.398+0000 7f130159a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12fc15c9b0 0x7f12fc153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46736/0 (socket says 172.21.15.156:46736) 2023-11-28T13:08:16.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.398+0000 7f130159a700 1 -- 172.21.15.156:0/2035613762 learned_addr learned my addr 172.21.15.156:0/2035613762 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:16.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.398+0000 7f130259c700 1 --2- 172.21.15.156:0/2035613762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12fc161340 0x7f12fc0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.398+0000 7f1301d9b700 1 --2- 172.21.15.156:0/2035613762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12fc0d2320 0x7f12fc0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.398+0000 7f130159a700 1 -- 172.21.15.156:0/2035613762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12fc0d2320 msgr2=0x7f12fc0cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.398+0000 7f130159a700 1 --2- 172.21.15.156:0/2035613762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12fc0d2320 0x7f12fc0cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.398+0000 7f130159a700 1 -- 172.21.15.156:0/2035613762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12fc161340 msgr2=0x7f12fc0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.398+0000 7f130159a700 1 --2- 172.21.15.156:0/2035613762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12fc161340 0x7f12fc0c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.398+0000 7f130159a700 1 -- 172.21.15.156:0/2035613762 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec0094a0 con 0x7f12fc15c9b0 2023-11-28T13:08:16.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.398+0000 7f1301d9b700 1 --2- 172.21.15.156:0/2035613762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12fc0d2320 0x7f12fc0cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:16.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.398+0000 7f130159a700 1 --2- 172.21.15.156:0/2035613762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12fc15c9b0 0x7f12fc153bd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0097c0 tx=0x7f12ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:16.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.398+0000 7f12f27fc700 1 -- 172.21.15.156:0/2035613762 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec01a970 con 0x7f12fc15c9b0 2023-11-28T13:08:16.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.398+0000 7f12f27fc700 1 -- 172.21.15.156:0/2035613762 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ec01aad0 con 0x7f12fc15c9b0 2023-11-28T13:08:16.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.398+0000 7f12f27fc700 1 -- 172.21.15.156:0/2035613762 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec01ae40 con 0x7f12fc15c9b0 2023-11-28T13:08:16.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.398+0000 7f1303fff700 1 -- 172.21.15.156:0/2035613762 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc15cd80 con 0x7f12fc15c9b0 2023-11-28T13:08:16.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.399+0000 7f1303fff700 1 -- 172.21.15.156:0/2035613762 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12fc0d15e0 con 0x7f12fc15c9b0 2023-11-28T13:08:16.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.400+0000 7f1303fff700 1 -- 172.21.15.156:0/2035613762 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12fc15cf10 con 0x7f12fc15c9b0 2023-11-28T13:08:16.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.400+0000 7f12f27fc700 1 -- 172.21.15.156:0/2035613762 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f12ec00f050 con 0x7f12fc15c9b0 2023-11-28T13:08:16.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.402+0000 7f12f27fc700 1 --2- 172.21.15.156:0/2035613762 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f12e00682f0 0x7f12e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.402+0000 7f12f27fc700 1 -- 172.21.15.156:0/2035613762 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f12ec0870c0 con 0x7f12fc15c9b0 2023-11-28T13:08:16.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.402+0000 7f1301d9b700 1 --2- 172.21.15.156:0/2035613762 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f12e00682f0 0x7f12e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.403+0000 7f1301d9b700 1 --2- 172.21.15.156:0/2035613762 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f12e00682f0 0x7f12e006a7a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f12f8000f60 tx=0x7f12f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:16.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.406+0000 7f12f27fc700 1 -- 172.21.15.156:0/2035613762 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f12ec053830 con 0x7f12fc15c9b0 2023-11-28T13:08:16.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.644+0000 7f1303fff700 1 -- 172.21.15.156:0/2035613762 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f12fc0cb740 con 0x7f12fc15c9b0 2023-11-28T13:08:16.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.644+0000 7f12f27fc700 1 -- 172.21.15.156:0/2035613762 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f12fc0cb740 con 0x7f12fc15c9b0 2023-11-28T13:08:16.646 INFO:teuthology.orchestra.run.smithi156.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T13:08:16.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.647+0000 7f1303fff700 1 -- 172.21.15.156:0/2035613762 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f12e00682f0 msgr2=0x7f12e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.647+0000 7f1303fff700 1 --2- 172.21.15.156:0/2035613762 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f12e00682f0 0x7f12e006a7a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f12f8000f60 tx=0x7f12f8009040 comp rx=0 tx=0).stop 2023-11-28T13:08:16.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.647+0000 7f1303fff700 1 -- 172.21.15.156:0/2035613762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12fc15c9b0 msgr2=0x7f12fc153bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.647+0000 7f1303fff700 1 --2- 172.21.15.156:0/2035613762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12fc15c9b0 0x7f12fc153bd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0097c0 tx=0x7f12ec004750 comp rx=0 tx=0).stop 2023-11-28T13:08:16.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.647+0000 7f1303fff700 1 -- 172.21.15.156:0/2035613762 shutdown_connections 2023-11-28T13:08:16.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.647+0000 7f1303fff700 1 --2- 172.21.15.156:0/2035613762 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f12e00682f0 0x7f12e006a7a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.647+0000 7f1303fff700 1 --2- 172.21.15.156:0/2035613762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12fc161340 0x7f12fc0c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.647+0000 7f1303fff700 1 --2- 172.21.15.156:0/2035613762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12fc0d2320 0x7f12fc0cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.647+0000 7f1303fff700 1 --2- 172.21.15.156:0/2035613762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12fc15c9b0 0x7f12fc153bd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.647+0000 7f1303fff700 1 -- 172.21.15.156:0/2035613762 >> 172.21.15.156:0/2035613762 conn(0x7f12fc072600 msgr2=0x7f12fc06a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:16.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.647+0000 7f1303fff700 1 -- 172.21.15.156:0/2035613762 shutdown_connections 2023-11-28T13:08:16.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:16.647+0000 7f1303fff700 1 -- 172.21.15.156:0/2035613762 wait complete. 2023-11-28T13:08:16.667 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T13:08:16.667 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:08:16.667 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-11-28T13:08:16.667 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T13:08:16.667 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T13:08:16.668 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T13:08:16.668 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T13:08:16.668 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:08:16.668 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:08:16.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:16.669 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-11-28T13:08:16.712 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:08:16.713 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:08:16.713 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:08:16.713 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:16.713 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-28T13:08:16.713 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:16.713 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:08:16.713 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:16.713 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f0 2023-11-28T13:08:16.713 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:08:16.713 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:16.713 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f1 2023-11-28T13:08:16.714 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:08:16.714 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:16.714 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:08:16.714 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:16.714 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T13:08:16.714 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42167sec preferred_lft 42167sec 2023-11-28T13:08:16.714 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::d27c:fc8f:2674:dd68/64 scope link noprefixroute 2023-11-28T13:08:16.714 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:16.715 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T13:08:16.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:16.716 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T13:08:16.716 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T13:08:16.716 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link add name ceph-brx type bridge 2023-11-28T13:08:16.716 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip addr flush dev ceph-brx 2023-11-28T13:08:16.716 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set ceph-brx up 2023-11-28T13:08:16.716 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T13:08:16.716 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T13:08:17.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:17.195 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:08:17.268 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-28T13:08:17.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:17.272 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-11-28T13:08:17.335 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-11-28T13:08:17.335 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:08:17.335 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T13:08:17.335 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T13:08:17.336 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:08:17.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:17.337 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T13:08:17.337 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T13:08:17.337 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T13:08:17.337 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T13:08:17.337 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T13:08:17.337 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T13:08:18.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:18.046 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T13:08:18.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:18.091 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list-id 2023-11-28T13:08:18.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:18.174 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T13:08:18.175 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T13:08:18.175 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:08:18.175 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-28T13:08:18.175 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T13:08:18.359 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-28T13:08:18.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:18.359 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T13:08:18.359 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T13:08:18.359 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-28T13:08:18.359 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T13:08:18.359 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-28T13:08:18.359 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-28T13:08:18.360 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-28T13:08:18.360 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T13:08:18.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:18.678 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T13:08:18.678 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T13:08:18.678 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.0 up 2023-11-28T13:08:18.678 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T13:08:18.678 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T13:08:18.851 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:08:18.851 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T13:08:18.895 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:08:18.895 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:08:18.895 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:08:18.978 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T13:08:18.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:18.978 DEBUG:teuthology.orchestra.run.smithi156:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:08:19.132 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:08:19.132 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:08:19.132 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T13:08:19.595 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:08:19.595 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:08:19.595 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T13:08:19.595 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T13:08:19.595 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:08:19.621 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:08:19.621 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:08:19.621 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T13:08:19.621 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T13:08:19.621 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:08:19.621 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790" 2023-11-28T13:08:19.622 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2023-11-28T13:08:19.622 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=12d9e849-dc60-46f0-8e8e-9e291633f61b" will pass to kernel 2023-11-28T13:08:19.629 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T13:08:19.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:19.630 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:08:19.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:19.712 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:08:19.835 INFO:teuthology.orchestra.run.smithi156.stdout:client4567 2023-11-28T13:08:19.836 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:08:19.836 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:08:19.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:19.886 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:08:20.042 INFO:teuthology.orchestra.run.smithi156.stdout:client4567 2023-11-28T13:08:20.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:20.043 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:08:20.152 INFO:teuthology.orchestra.run.smithi156.stdout:client4567 2023-11-28T13:08:20.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:20.153 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/12d9e849-dc60-46f0-8e8e-9e291633f61b.client4567/status') 2023-11-28T13:08:20.284 INFO:teuthology.orchestra.run.smithi156.stdout:instance: client.4567 (3)172.21.15.156:0/3592179051 2023-11-28T13:08:20.284 INFO:teuthology.orchestra.run.smithi156.stdout:blocklisted: false 2023-11-28T13:08:20.284 INFO:teuthology.orchestra.run.smithi156.stderr:0+1 records in 2023-11-28T13:08:20.285 INFO:teuthology.orchestra.run.smithi156.stderr:0+1 records out 2023-11-28T13:08:20.285 INFO:teuthology.orchestra.run.smithi156.stderr:71 bytes copied, 5.5221e-05 s, 1.3 MB/s 2023-11-28T13:08:20.287 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-11-28T13:08:20.287 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T13:08:20.287 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T13:08:20.288 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:08:20.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.487+0000 7f4fdebff700 1 Processor -- start 2023-11-28T13:08:20.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.487+0000 7f4fdebff700 1 -- start start 2023-11-28T13:08:20.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.488+0000 7f4fdebff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4fd815c9b0 0x7f4fd80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:20.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.488+0000 7f4fdebff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4fd80cd7c0 0x7f4fd80cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:20.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.488+0000 7f4fdebff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd80d1950 con 0x7f4fd8161340 2023-11-28T13:08:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.488+0000 7f4fdebff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd80b5800 con 0x7f4fd815c9b0 2023-11-28T13:08:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.488+0000 7f4fdebff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd80ce140 con 0x7f4fd80cd7c0 2023-11-28T13:08:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.488+0000 7f4fdc99b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4fd815c9b0 0x7f4fd80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.488+0000 7f4fd7fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4fd8161340 0x7f4fd80d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53592/0 (socket says 172.21.15.156:53592) 2023-11-28T13:08:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.488+0000 7f4fd7fff700 1 -- 172.21.15.156:0/4174472591 learned_addr learned my addr 172.21.15.156:0/4174472591 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.488+0000 7f4fdd19c700 1 --2- 172.21.15.156:0/4174472591 >> v2:172.21.15.186:3300/0 conn(0x7f4fd80cd7c0 0x7f4fd80cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.489+0000 7f4fdc99b700 1 -- 172.21.15.156:0/4174472591 >> v2:172.21.15.186:3300/0 conn(0x7f4fd80cd7c0 msgr2=0x7f4fd80cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.489+0000 7f4fdc99b700 1 --2- 172.21.15.156:0/4174472591 >> v2:172.21.15.186:3300/0 conn(0x7f4fd80cd7c0 0x7f4fd80cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:20.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.489+0000 7f4fdc99b700 1 -- 172.21.15.156:0/4174472591 >> v1:172.21.15.156:6789/0 conn(0x7f4fd8161340 legacy=0x7f4fd80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:20.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.489+0000 7f4fdc99b700 1 -- 172.21.15.156:0/4174472591 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd80d24f0 con 0x7f4fd815c9b0 2023-11-28T13:08:20.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.489+0000 7f4fd77fe700 1 -- 172.21.15.156:0/4174472591 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286767805 0 0) 0x7f4fd80d1950 con 0x7f4fd8161340 2023-11-28T13:08:20.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.489+0000 7f4fdc99b700 1 --2- 172.21.15.156:0/4174472591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fd815c9b0 0x7f4fd80b4bf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4fd806c170 tx=0x7f4fcc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5883dcab100aabd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:20.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.489+0000 7f4fd77fe700 1 -- 172.21.15.156:0/4174472591 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc011830 con 0x7f4fd815c9b0 2023-11-28T13:08:20.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.489+0000 7f4fd77fe700 1 -- 172.21.15.156:0/4174472591 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc011990 con 0x7f4fd815c9b0 2023-11-28T13:08:20.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.489+0000 7f4fd77fe700 1 -- 172.21.15.156:0/4174472591 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc011d20 con 0x7f4fd815c9b0 2023-11-28T13:08:20.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.490+0000 7f4fdebff700 1 -- 172.21.15.156:0/4174472591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fd815c9b0 msgr2=0x7f4fd80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:20.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.490+0000 7f4fdebff700 1 --2- 172.21.15.156:0/4174472591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fd815c9b0 0x7f4fd80b4bf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4fd806c170 tx=0x7f4fcc009b20 comp rx=0 tx=0).stop 2023-11-28T13:08:20.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.490+0000 7f4fdebff700 1 -- 172.21.15.156:0/4174472591 shutdown_connections 2023-11-28T13:08:20.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.490+0000 7f4fdebff700 1 --2- 172.21.15.156:0/4174472591 >> v2:172.21.15.186:3300/0 conn(0x7f4fd80cd7c0 0x7f4fd80cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:20.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.490+0000 7f4fdebff700 1 --2- 172.21.15.156:0/4174472591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fd815c9b0 0x7f4fd80b4bf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:20.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.490+0000 7f4fdebff700 1 -- 172.21.15.156:0/4174472591 >> 172.21.15.156:0/4174472591 conn(0x7f4fd8072600 msgr2=0x7f4fd80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:20.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.490+0000 7f4fdebff700 1 -- 172.21.15.156:0/4174472591 shutdown_connections 2023-11-28T13:08:20.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.490+0000 7f4fdebff700 1 -- 172.21.15.156:0/4174472591 wait complete. 2023-11-28T13:08:20.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.492+0000 7f4fdebff700 1 Processor -- start 2023-11-28T13:08:20.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.492+0000 7f4fdebff700 1 -- start start 2023-11-28T13:08:20.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.493+0000 7f4fdebff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fd80cd7c0 0x7f4fd8154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:20.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.493+0000 7f4fdebff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4fd815c9b0 0x7f4fd81f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:20.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.493+0000 7f4fdebff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fd8161340 0x7f4fd815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:20.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.493+0000 7f4fdebff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd80cc130 con 0x7f4fd80cd7c0 2023-11-28T13:08:20.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.493+0000 7f4fdebff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd81547a0 con 0x7f4fd8161340 2023-11-28T13:08:20.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.493+0000 7f4fdebff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd80cb800 con 0x7f4fd815c9b0 2023-11-28T13:08:20.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.493+0000 7f4fdc99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fd80cd7c0 0x7f4fd8154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:20.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.493+0000 7f4fdc99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fd80cd7c0 0x7f4fd8154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46752/0 (socket says 172.21.15.156:46752) 2023-11-28T13:08:20.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.493+0000 7f4fdc99b700 1 -- 172.21.15.156:0/58422316 learned_addr learned my addr 172.21.15.156:0/58422316 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:20.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.493+0000 7f4fdd19c700 1 --2- 172.21.15.156:0/58422316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fd8161340 0x7f4fd815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:20.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.493+0000 7f4fdc99b700 1 -- 172.21.15.156:0/58422316 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4fd815c9b0 msgr2=0x7f4fd81f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:20.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.493+0000 7f4fd7fff700 1 --2- 172.21.15.156:0/58422316 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4fd815c9b0 0x7f4fd81f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:20.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.493+0000 7f4fdc99b700 1 --2- 172.21.15.156:0/58422316 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4fd815c9b0 0x7f4fd81f24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:20.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.493+0000 7f4fdc99b700 1 -- 172.21.15.156:0/58422316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fd8161340 msgr2=0x7f4fd815e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:20.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.493+0000 7f4fdc99b700 1 --2- 172.21.15.156:0/58422316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fd8161340 0x7f4fd815e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:20.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.493+0000 7f4fdc99b700 1 -- 172.21.15.156:0/58422316 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc0094a0 con 0x7f4fd80cd7c0 2023-11-28T13:08:20.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.493+0000 7f4fdd19c700 1 --2- 172.21.15.156:0/58422316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fd8161340 0x7f4fd815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:20.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.494+0000 7f4fd7fff700 1 --2- 172.21.15.156:0/58422316 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4fd815c9b0 0x7f4fd81f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:20.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.494+0000 7f4fdc99b700 1 --2- 172.21.15.156:0/58422316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fd80cd7c0 0x7f4fd8154260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc014040 tx=0x7f4fcc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:20.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.494+0000 7f4fd57fa700 1 -- 172.21.15.156:0/58422316 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc0038a0 con 0x7f4fd80cd7c0 2023-11-28T13:08:20.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.494+0000 7f4fd57fa700 1 -- 172.21.15.156:0/58422316 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc01e070 con 0x7f4fd80cd7c0 2023-11-28T13:08:20.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.494+0000 7f4fd57fa700 1 -- 172.21.15.156:0/58422316 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc003c20 con 0x7f4fd80cd7c0 2023-11-28T13:08:20.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.494+0000 7f4fdebff700 1 -- 172.21.15.156:0/58422316 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd80d1300 con 0x7f4fd80cd7c0 2023-11-28T13:08:20.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.494+0000 7f4fdebff700 1 -- 172.21.15.156:0/58422316 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd80b5520 con 0x7f4fd80cd7c0 2023-11-28T13:08:20.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.495+0000 7f4fdebff700 1 -- 172.21.15.156:0/58422316 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd80d15e0 con 0x7f4fd80cd7c0 2023-11-28T13:08:20.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.496+0000 7f4fd57fa700 1 -- 172.21.15.156:0/58422316 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4fcc00f050 con 0x7f4fd80cd7c0 2023-11-28T13:08:20.503 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.498+0000 7f4fd57fa700 1 --2- 172.21.15.156:0/58422316 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4fbc0682f0 0x7f4fbc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:20.503 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.498+0000 7f4fd57fa700 1 -- 172.21.15.156:0/58422316 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4fcc086d50 con 0x7f4fd80cd7c0 2023-11-28T13:08:20.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.498+0000 7f4fd7fff700 1 --2- 172.21.15.156:0/58422316 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4fbc0682f0 0x7f4fbc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:20.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.499+0000 7f4fd7fff700 1 --2- 172.21.15.156:0/58422316 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4fbc0682f0 0x7f4fbc06a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4007fd0 tx=0x7f4fc4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:20.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.502+0000 7f4fd57fa700 1 -- 172.21.15.156:0/58422316 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4fcc0534c0 con 0x7f4fd80cd7c0 2023-11-28T13:08:20.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.737+0000 7f4fdebff700 1 -- 172.21.15.156:0/58422316 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4fd80745c0 con 0x7f4fd80cd7c0 2023-11-28T13:08:20.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.738+0000 7f4fd57fa700 1 -- 172.21.15.156:0/58422316 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4fcc015070 con 0x7f4fd80cd7c0 2023-11-28T13:08:20.740 INFO:teuthology.orchestra.run.smithi156.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T13:08:20.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.740+0000 7f4fdebff700 1 -- 172.21.15.156:0/58422316 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4fbc0682f0 msgr2=0x7f4fbc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:20.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.740+0000 7f4fdebff700 1 --2- 172.21.15.156:0/58422316 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4fbc0682f0 0x7f4fbc06a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4007fd0 tx=0x7f4fc4009040 comp rx=0 tx=0).stop 2023-11-28T13:08:20.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.740+0000 7f4fdebff700 1 -- 172.21.15.156:0/58422316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fd80cd7c0 msgr2=0x7f4fd8154260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:20.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.740+0000 7f4fdebff700 1 --2- 172.21.15.156:0/58422316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fd80cd7c0 0x7f4fd8154260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc014040 tx=0x7f4fcc004750 comp rx=0 tx=0).stop 2023-11-28T13:08:20.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.740+0000 7f4fdebff700 1 -- 172.21.15.156:0/58422316 shutdown_connections 2023-11-28T13:08:20.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.740+0000 7f4fdebff700 1 --2- 172.21.15.156:0/58422316 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4fbc0682f0 0x7f4fbc06a7a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:20.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.740+0000 7f4fdebff700 1 --2- 172.21.15.156:0/58422316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fd8161340 0x7f4fd815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:20.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.740+0000 7f4fdebff700 1 --2- 172.21.15.156:0/58422316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fd80cd7c0 0x7f4fd8154260 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:20.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.740+0000 7f4fdebff700 1 --2- 172.21.15.156:0/58422316 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4fd815c9b0 0x7f4fd81f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:20.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.740+0000 7f4fdebff700 1 -- 172.21.15.156:0/58422316 >> 172.21.15.156:0/58422316 conn(0x7f4fd8072600 msgr2=0x7f4fd80b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:20.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.740+0000 7f4fdebff700 1 -- 172.21.15.156:0/58422316 shutdown_connections 2023-11-28T13:08:20.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:20.740+0000 7f4fdebff700 1 -- 172.21.15.156:0/58422316 wait complete. 2023-11-28T13:08:20.760 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T13:08:20.760 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:08:20.760 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-11-28T13:08:20.760 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-28T13:08:20.760 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T13:08:20.760 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T13:08:20.760 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T13:08:20.760 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:08:20.760 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-28T13:08:20.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:20.761 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-11-28T13:08:20.805 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:08:20.805 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:08:20.806 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:08:20.806 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:20.806 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-28T13:08:20.806 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:20.806 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:08:20.806 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:20.806 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f0 2023-11-28T13:08:20.806 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:08:20.806 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:20.806 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f1 2023-11-28T13:08:20.806 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:08:20.807 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:20.807 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:08:20.807 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:20.807 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T13:08:20.807 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42163sec preferred_lft 42163sec 2023-11-28T13:08:20.807 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::d27c:fc8f:2674:dd68/64 scope link noprefixroute 2023-11-28T13:08:20.807 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:20.807 INFO:teuthology.orchestra.run.smithi156.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:08:20.807 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 96:cd:43:a7:29:10 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:20.807 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T13:08:20.807 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:20.807 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::4497:b0ff:fe27:d641/64 scope link 2023-11-28T13:08:20.808 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:20.808 INFO:teuthology.orchestra.run.smithi156.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:08:20.808 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 96:cd:43:a7:29:10 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:08:20.808 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::94cd:43ff:fea7:2910/64 scope link 2023-11-28T13:08:20.808 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:20.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:20.808 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:08:20.911 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-28T13:08:20.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:20.915 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-11-28T13:08:20.960 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-11-28T13:08:20.961 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:08:20.961 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T13:08:20.961 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T13:08:20.961 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:08:20.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:20.962 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T13:08:20.962 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T13:08:20.962 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T13:08:20.962 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T13:08:20.962 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T13:08:20.962 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T13:08:21.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:21.174 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T13:08:21.218 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T13:08:21.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:21.219 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list-id 2023-11-28T13:08:21.301 INFO:teuthology.orchestra.run.smithi156.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-28T13:08:21.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:21.302 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T13:08:21.302 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T13:08:21.303 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T13:08:21.303 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-28T13:08:21.303 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T13:08:21.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:21.486 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T13:08:21.554 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:08:21.554 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:08:21.554 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:08:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-28T13:08:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:21.555 INFO:teuthology.orchestra.run.smithi156.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:08:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 86:89:c4:f0:10:44 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T13:08:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T13:08:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::8489:c4ff:fef0:1044/64 scope link 2023-11-28T13:08:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:21.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:21.557 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T13:08:21.628 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:08:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:08:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:08:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-28T13:08:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:21.628 INFO:teuthology.orchestra.run.smithi156.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:08:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 86:89:c4:f0:10:44 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T13:08:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T13:08:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:21.629 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::8489:c4ff:fef0:1044/64 scope link 2023-11-28T13:08:21.629 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:21.631 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-28T13:08:21.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:21.632 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T13:08:21.632 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T13:08:21.632 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-28T13:08:21.632 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T13:08:21.632 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-28T13:08:21.632 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-28T13:08:21.632 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-28T13:08:21.632 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T13:08:21.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:21.884 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T13:08:21.884 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T13:08:21.884 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.1 up 2023-11-28T13:08:21.885 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T13:08:21.885 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T13:08:22.058 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:08:22.059 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-28T13:08:22.103 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-28T13:08:22.103 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:08:22.103 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-28T13:08:22.186 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T13:08:22.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:22.186 DEBUG:teuthology.orchestra.run.smithi156:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T13:08:22.331 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T13:08:22.331 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T13:08:22.331 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T13:08:22.349 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T13:08:22.350 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T13:08:22.350 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T13:08:22.350 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T13:08:22.350 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:08:22.372 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T13:08:22.372 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T13:08:22.373 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T13:08:22.373 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T13:08:22.373 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:08:22.373 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790" 2023-11-28T13:08:22.373 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2023-11-28T13:08:22.373 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=12d9e849-dc60-46f0-8e8e-9e291633f61b" will pass to kernel 2023-11-28T13:08:22.378 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T13:08:22.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:22.378 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-28T13:08:22.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:22.447 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:08:22.533 INFO:teuthology.orchestra.run.smithi156.stdout:client4573 2023-11-28T13:08:22.533 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:08:22.533 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:08:22.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:22.577 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:08:22.704 INFO:teuthology.orchestra.run.smithi156.stdout:client4573 2023-11-28T13:08:22.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:22.704 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:08:22.790 INFO:teuthology.orchestra.run.smithi156.stdout:client4573 2023-11-28T13:08:22.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:22.791 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/12d9e849-dc60-46f0-8e8e-9e291633f61b.client4573/status') 2023-11-28T13:08:22.899 INFO:teuthology.orchestra.run.smithi156.stdout:instance: client.4573 (3)172.21.15.156:0/1102988249 2023-11-28T13:08:22.899 INFO:teuthology.orchestra.run.smithi156.stdout:blocklisted: false 2023-11-28T13:08:22.899 INFO:teuthology.orchestra.run.smithi156.stderr:0+1 records in 2023-11-28T13:08:22.899 INFO:teuthology.orchestra.run.smithi156.stderr:0+1 records out 2023-11-28T13:08:22.899 INFO:teuthology.orchestra.run.smithi156.stderr:71 bytes copied, 5.4712e-05 s, 1.3 MB/s 2023-11-28T13:08:22.902 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-11-28T13:08:22.902 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T13:08:22.902 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T13:08:22.902 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:08:23.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.098+0000 7f31f03ff700 1 Processor -- start 2023-11-28T13:08:23.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.099+0000 7f31f03ff700 1 -- start start 2023-11-28T13:08:23.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.100+0000 7f31f03ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31e815c9b0 0x7f31e80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:23.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.100+0000 7f31f03ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f31e8161340 0x7f31e80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:23.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.100+0000 7f31f03ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e80d1950 con 0x7f31e80d2320 2023-11-28T13:08:23.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.100+0000 7f31f03ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e80b5800 con 0x7f31e8161340 2023-11-28T13:08:23.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.100+0000 7f31f03ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e80b5520 con 0x7f31e815c9b0 2023-11-28T13:08:23.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.100+0000 7f31ed99a700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f31e8161340 0x7f31e80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:23.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.100+0000 7f31ee19b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31e815c9b0 0x7f31e80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:23.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.100+0000 7f31ee99c700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f31e80d2320 0x7f31e80cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:55166/0 (socket says 172.21.15.156:55166) 2023-11-28T13:08:23.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.100+0000 7f31ee99c700 1 -- 172.21.15.156:0/3814007363 learned_addr learned my addr 172.21.15.156:0/3814007363 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:23.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.101+0000 7f31ed99a700 1 -- 172.21.15.156:0/3814007363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31e815c9b0 msgr2=0x7f31e80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:23.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.101+0000 7f31ed99a700 1 --2- 172.21.15.156:0/3814007363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31e815c9b0 0x7f31e80b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:23.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.101+0000 7f31ed99a700 1 -- 172.21.15.156:0/3814007363 >> v1:172.21.15.186:6789/0 conn(0x7f31e80d2320 legacy=0x7f31e80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:23.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.101+0000 7f31ed99a700 1 -- 172.21.15.156:0/3814007363 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e80d1300 con 0x7f31e8161340 2023-11-28T13:08:23.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.101+0000 7f31ed199700 1 -- 172.21.15.156:0/3814007363 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2684160227 0 0) 0x7f31e80d1950 con 0x7f31e80d2320 2023-11-28T13:08:23.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.101+0000 7f31ed99a700 1 --2- 172.21.15.156:0/3814007363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e8161340 0x7f31e80d1de0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009840 tx=0x7f31d8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f78c52d45cf67efc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:23.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.101+0000 7f31ed199700 1 -- 172.21.15.156:0/3814007363 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8011830 con 0x7f31e8161340 2023-11-28T13:08:23.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.101+0000 7f31ed199700 1 -- 172.21.15.156:0/3814007363 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d8011990 con 0x7f31e8161340 2023-11-28T13:08:23.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.101+0000 7f31ed199700 1 -- 172.21.15.156:0/3814007363 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8011d20 con 0x7f31e8161340 2023-11-28T13:08:23.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.102+0000 7f31f03ff700 1 -- 172.21.15.156:0/3814007363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e8161340 msgr2=0x7f31e80d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:23.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.102+0000 7f31f03ff700 1 --2- 172.21.15.156:0/3814007363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e8161340 0x7f31e80d1de0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009840 tx=0x7f31d8009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.102+0000 7f31f03ff700 1 -- 172.21.15.156:0/3814007363 shutdown_connections 2023-11-28T13:08:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.102+0000 7f31f03ff700 1 --2- 172.21.15.156:0/3814007363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31e815c9b0 0x7f31e80b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.102+0000 7f31f03ff700 1 --2- 172.21.15.156:0/3814007363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e8161340 0x7f31e80d1de0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.102+0000 7f31f03ff700 1 -- 172.21.15.156:0/3814007363 >> 172.21.15.156:0/3814007363 conn(0x7f31e8072600 msgr2=0x7f31e80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.102+0000 7f31f03ff700 1 -- 172.21.15.156:0/3814007363 shutdown_connections 2023-11-28T13:08:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.102+0000 7f31f03ff700 1 -- 172.21.15.156:0/3814007363 wait complete. 2023-11-28T13:08:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.104+0000 7f31f03ff700 1 Processor -- start 2023-11-28T13:08:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.104+0000 7f31f03ff700 1 -- start start 2023-11-28T13:08:23.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.105+0000 7f31f03ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31e80d2320 0x7f31e8150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:23.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.105+0000 7f31f03ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e815c9b0 0x7f31e8152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:23.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.105+0000 7f31f03ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31e8161340 0x7f31e81f2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:23.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.105+0000 7f31f03ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e81f3590 con 0x7f31e8161340 2023-11-28T13:08:23.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.105+0000 7f31f03ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e8150eb0 con 0x7f31e815c9b0 2023-11-28T13:08:23.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.105+0000 7f31f03ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e81f2fd0 con 0x7f31e80d2320 2023-11-28T13:08:23.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.105+0000 7f31ee99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31e8161340 0x7f31e81f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:23.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.105+0000 7f31ee99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31e8161340 0x7f31e81f2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46758/0 (socket says 172.21.15.156:46758) 2023-11-28T13:08:23.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.105+0000 7f31ee99c700 1 -- 172.21.15.156:0/1569610138 learned_addr learned my addr 172.21.15.156:0/1569610138 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:23.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.105+0000 7f31ed99a700 1 --2- 172.21.15.156:0/1569610138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e815c9b0 0x7f31e8152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:23.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.105+0000 7f31ee19b700 1 --2- 172.21.15.156:0/1569610138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31e80d2320 0x7f31e8150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:23.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.106+0000 7f31ee99c700 1 -- 172.21.15.156:0/1569610138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31e80d2320 msgr2=0x7f31e8150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:23.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.106+0000 7f31ee99c700 1 --2- 172.21.15.156:0/1569610138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31e80d2320 0x7f31e8150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:23.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.106+0000 7f31ee99c700 1 -- 172.21.15.156:0/1569610138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e815c9b0 msgr2=0x7f31e8152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:23.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.106+0000 7f31ee99c700 1 --2- 172.21.15.156:0/1569610138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e815c9b0 0x7f31e8152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:23.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.106+0000 7f31ee99c700 1 -- 172.21.15.156:0/1569610138 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d80094a0 con 0x7f31e8161340 2023-11-28T13:08:23.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.106+0000 7f31ed99a700 1 --2- 172.21.15.156:0/1569610138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e815c9b0 0x7f31e8152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:23.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.106+0000 7f31ee99c700 1 --2- 172.21.15.156:0/1569610138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31e8161340 0x7f31e81f2550 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f31e000cba0 tx=0x7f31e000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:23.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.106+0000 7f31deffd700 1 -- 172.21.15.156:0/1569610138 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e000aaa0 con 0x7f31e8161340 2023-11-28T13:08:23.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.106+0000 7f31deffd700 1 -- 172.21.15.156:0/1569610138 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e000ac20 con 0x7f31e8161340 2023-11-28T13:08:23.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.106+0000 7f31deffd700 1 -- 172.21.15.156:0/1569610138 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e00070d0 con 0x7f31e8161340 2023-11-28T13:08:23.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.106+0000 7f31f03ff700 1 -- 172.21.15.156:0/1569610138 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e80b5130 con 0x7f31e8161340 2023-11-28T13:08:23.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.106+0000 7f31f03ff700 1 -- 172.21.15.156:0/1569610138 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e80d15e0 con 0x7f31e8161340 2023-11-28T13:08:23.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.107+0000 7f31f03ff700 1 -- 172.21.15.156:0/1569610138 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e80b52c0 con 0x7f31e8161340 2023-11-28T13:08:23.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.108+0000 7f31deffd700 1 -- 172.21.15.156:0/1569610138 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f31e0010070 con 0x7f31e8161340 2023-11-28T13:08:23.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.109+0000 7f31deffd700 1 --2- 172.21.15.156:0/1569610138 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f31cc0682f0 0x7f31cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:23.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.109+0000 7f31deffd700 1 -- 172.21.15.156:0/1569610138 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f31e00856d0 con 0x7f31e8161340 2023-11-28T13:08:23.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.109+0000 7f31ee19b700 1 --2- 172.21.15.156:0/1569610138 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f31cc0682f0 0x7f31cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:23.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.110+0000 7f31ee19b700 1 --2- 172.21.15.156:0/1569610138 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f31cc0682f0 0x7f31cc06a7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009be0 tx=0x7f31e4009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:23.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.113+0000 7f31deffd700 1 -- 172.21.15.156:0/1569610138 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f31e0051e40 con 0x7f31e8161340 2023-11-28T13:08:23.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.349+0000 7f31f03ff700 1 -- 172.21.15.156:0/1569610138 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f31e80745c0 con 0x7f31e8161340 2023-11-28T13:08:23.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.349+0000 7f31deffd700 1 -- 172.21.15.156:0/1569610138 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f31e0018070 con 0x7f31e8161340 2023-11-28T13:08:23.351 INFO:teuthology.orchestra.run.smithi156.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T13:08:23.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.352+0000 7f31f03ff700 1 -- 172.21.15.156:0/1569610138 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f31cc0682f0 msgr2=0x7f31cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:23.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.352+0000 7f31f03ff700 1 --2- 172.21.15.156:0/1569610138 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f31cc0682f0 0x7f31cc06a7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009be0 tx=0x7f31e4009210 comp rx=0 tx=0).stop 2023-11-28T13:08:23.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.352+0000 7f31f03ff700 1 -- 172.21.15.156:0/1569610138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31e8161340 msgr2=0x7f31e81f2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:23.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.352+0000 7f31f03ff700 1 --2- 172.21.15.156:0/1569610138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31e8161340 0x7f31e81f2550 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f31e000cba0 tx=0x7f31e000ceb0 comp rx=0 tx=0).stop 2023-11-28T13:08:23.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.352+0000 7f31f03ff700 1 -- 172.21.15.156:0/1569610138 shutdown_connections 2023-11-28T13:08:23.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.352+0000 7f31f03ff700 1 --2- 172.21.15.156:0/1569610138 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f31cc0682f0 0x7f31cc06a7a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:23.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.352+0000 7f31f03ff700 1 --2- 172.21.15.156:0/1569610138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31e8161340 0x7f31e81f2550 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:23.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.352+0000 7f31f03ff700 1 --2- 172.21.15.156:0/1569610138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f31e80d2320 0x7f31e8150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:23.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.352+0000 7f31f03ff700 1 --2- 172.21.15.156:0/1569610138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e815c9b0 0x7f31e8152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:23.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.352+0000 7f31f03ff700 1 -- 172.21.15.156:0/1569610138 >> 172.21.15.156:0/1569610138 conn(0x7f31e8072600 msgr2=0x7f31e80b8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:23.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.352+0000 7f31f03ff700 1 -- 172.21.15.156:0/1569610138 shutdown_connections 2023-11-28T13:08:23.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:23.352+0000 7f31f03ff700 1 -- 172.21.15.156:0/1569610138 wait complete. 2023-11-28T13:08:23.371 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T13:08:23.372 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:08:23.372 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi186.front.sepia.ceph.com 2023-11-28T13:08:23.372 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-28T13:08:23.372 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T13:08:23.372 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T13:08:23.372 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T13:08:23.372 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:08:23.372 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-28T13:08:23.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:23.372 DEBUG:teuthology.orchestra.run.smithi186:> ip addr 2023-11-28T13:08:23.421 INFO:teuthology.orchestra.run.smithi186.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:08:23.422 INFO:teuthology.orchestra.run.smithi186.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:08:23.422 INFO:teuthology.orchestra.run.smithi186.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:08:23.422 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:23.422 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 ::1/128 scope host 2023-11-28T13:08:23.422 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:23.422 INFO:teuthology.orchestra.run.smithi186.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:08:23.423 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2c brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:23.423 INFO:teuthology.orchestra.run.smithi186.stdout: altname enp7s0f0 2023-11-28T13:08:23.423 INFO:teuthology.orchestra.run.smithi186.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:08:23.423 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2d brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:23.423 INFO:teuthology.orchestra.run.smithi186.stdout: altname enp7s0f1 2023-11-28T13:08:23.424 INFO:teuthology.orchestra.run.smithi186.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:08:23.424 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f4 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:23.424 INFO:teuthology.orchestra.run.smithi186.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:08:23.424 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f5 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:23.424 INFO:teuthology.orchestra.run.smithi186.stdout: inet 172.21.15.186/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T13:08:23.424 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft 42155sec preferred_lft 42155sec 2023-11-28T13:08:23.424 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::6239:84b8:dc67:ce5e/64 scope link noprefixroute 2023-11-28T13:08:23.425 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:23.425 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T13:08:23.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:23.425 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T13:08:23.425 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T13:08:23.425 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link add name ceph-brx type bridge 2023-11-28T13:08:23.426 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip addr flush dev ceph-brx 2023-11-28T13:08:23.426 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set ceph-brx up 2023-11-28T13:08:23.426 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T13:08:23.426 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T13:08:23.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:23.864 DEBUG:teuthology.orchestra.run.smithi186:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:08:23.933 INFO:teuthology.orchestra.run.smithi186.stdout:1 2023-11-28T13:08:23.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:23.937 DEBUG:teuthology.orchestra.run.smithi186:> route 2023-11-28T13:08:23.995 INFO:teuthology.orchestra.run.smithi186.stdout:Kernel IP routing table 2023-11-28T13:08:23.995 INFO:teuthology.orchestra.run.smithi186.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:08:23.995 INFO:teuthology.orchestra.run.smithi186.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T13:08:23.995 INFO:teuthology.orchestra.run.smithi186.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T13:08:23.996 INFO:teuthology.orchestra.run.smithi186.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:08:23.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:23.996 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T13:08:23.996 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T13:08:23.996 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T13:08:23.996 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T13:08:23.996 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T13:08:23.997 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T13:08:24.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:24.695 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2023-11-28T13:08:24.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:24.740 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list-id 2023-11-28T13:08:24.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:24.824 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T13:08:24.824 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T13:08:24.824 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T13:08:24.824 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-28T13:08:24.824 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T13:08:25.006 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-28T13:08:25.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:25.007 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T13:08:25.007 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T13:08:25.008 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-28T13:08:25.008 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T13:08:25.008 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-28T13:08:25.009 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-28T13:08:25.009 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-28T13:08:25.009 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T13:08:25.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:25.317 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T13:08:25.317 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T13:08:25.317 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set brx.0 up 2023-11-28T13:08:25.317 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T13:08:25.317 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T13:08:25.490 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:08:25.490 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-28T13:08:25.534 INFO:teuthology.orchestra.run.smithi186.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-28T13:08:25.534 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:08:25.535 DEBUG:teuthology.orchestra.run.smithi186:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-28T13:08:25.618 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T13:08:25.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:25.618 DEBUG:teuthology.orchestra.run.smithi186:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T13:08:25.795 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T13:08:25.795 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T13:08:25.795 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2023-11-28T13:08:26.206 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T13:08:26.207 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T13:08:26.207 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2023-11-28T13:08:26.207 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T13:08:26.207 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:08:26.231 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T13:08:26.231 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T13:08:26.231 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2023-11-28T13:08:26.232 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T13:08:26.232 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:08:26.232 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790" 2023-11-28T13:08:26.232 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2023-11-28T13:08:26.232 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=12d9e849-dc60-46f0-8e8e-9e291633f61b" will pass to kernel 2023-11-28T13:08:26.239 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T13:08:26.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:26.239 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-28T13:08:26.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:26.310 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:08:26.408 INFO:teuthology.orchestra.run.smithi186.stdout:client4460 2023-11-28T13:08:26.408 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:08:26.408 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:08:26.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:26.453 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:08:26.578 INFO:teuthology.orchestra.run.smithi186.stdout:client4460 2023-11-28T13:08:26.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:26.579 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:08:26.664 INFO:teuthology.orchestra.run.smithi186.stdout:client4460 2023-11-28T13:08:26.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:26.665 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/12d9e849-dc60-46f0-8e8e-9e291633f61b.client4460/status') 2023-11-28T13:08:26.776 INFO:teuthology.orchestra.run.smithi186.stdout:instance: client.4460 (3)192.168.0.1:0/3205441272 2023-11-28T13:08:26.776 INFO:teuthology.orchestra.run.smithi186.stdout:blocklisted: false 2023-11-28T13:08:26.777 INFO:teuthology.orchestra.run.smithi186.stderr:0+1 records in 2023-11-28T13:08:26.777 INFO:teuthology.orchestra.run.smithi186.stderr:0+1 records out 2023-11-28T13:08:26.777 INFO:teuthology.orchestra.run.smithi186.stderr:69 bytes copied, 5.2312e-05 s, 1.3 MB/s 2023-11-28T13:08:26.777 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-11-28T13:08:26.777 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T13:08:26.777 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T13:08:26.777 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:08:26.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.984+0000 7f7088bff700 1 Processor -- start 2023-11-28T13:08:26.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.984+0000 7f7088bff700 1 -- start start 2023-11-28T13:08:26.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.986+0000 7f7088bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7084161340 0x7f70840d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:26.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.986+0000 7f7088bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f70840d2320 0x7f70840cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:26.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.986+0000 7f7088bff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70840d1950 con 0x7f708415c9b0 2023-11-28T13:08:26.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.986+0000 7f7088bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70840b5800 con 0x7f70840d2320 2023-11-28T13:08:26.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.986+0000 7f7088bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70840b5300 con 0x7f7084161340 2023-11-28T13:08:26.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.986+0000 7f7082d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f70840d2320 0x7f70840cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:26.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.986+0000 7f7082d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f70840d2320 0x7f70840cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46392/0 (socket says 172.21.15.156:46392) 2023-11-28T13:08:26.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.986+0000 7f7082d9d700 1 -- 172.21.15.156:0/53846248 learned_addr learned my addr 172.21.15.156:0/53846248 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:26.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.986+0000 7f708259c700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f708415c9b0 0x7f70840b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:45692/0 (socket says 172.21.15.156:45692) 2023-11-28T13:08:26.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.986+0000 7f707bfff700 1 --2- 172.21.15.156:0/53846248 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7084161340 0x7f70840d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:26.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.986+0000 7f7082d9d700 1 -- 172.21.15.156:0/53846248 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7084161340 msgr2=0x7f70840d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:26.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.986+0000 7f7082d9d700 1 --2- 172.21.15.156:0/53846248 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7084161340 0x7f70840d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:26.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.986+0000 7f7082d9d700 1 -- 172.21.15.156:0/53846248 >> v1:172.21.15.186:6789/0 conn(0x7f708415c9b0 legacy=0x7f70840b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:26.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.986+0000 7f7082d9d700 1 -- 172.21.15.156:0/53846248 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70840b5520 con 0x7f70840d2320 2023-11-28T13:08:26.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.987+0000 7f7082d9d700 1 --2- 172.21.15.156:0/53846248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70840d2320 0x7f70840cd7c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f707c009e80 tx=0x7f707c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a829053b9b0880 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:26.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.987+0000 7f7081d9b700 1 -- 172.21.15.156:0/53846248 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c00e070 con 0x7f70840d2320 2023-11-28T13:08:26.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.987+0000 7f7081d9b700 1 -- 172.21.15.156:0/53846248 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c00b8d0 con 0x7f70840d2320 2023-11-28T13:08:26.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.987+0000 7f7081d9b700 1 -- 172.21.15.156:0/53846248 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c00bc80 con 0x7f70840d2320 2023-11-28T13:08:26.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.987+0000 7f7088bff700 1 -- 172.21.15.156:0/53846248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70840d2320 msgr2=0x7f70840cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:26.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.987+0000 7f7088bff700 1 --2- 172.21.15.156:0/53846248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70840d2320 0x7f70840cd7c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f707c009e80 tx=0x7f707c00b330 comp rx=0 tx=0).stop 2023-11-28T13:08:26.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.987+0000 7f7088bff700 1 -- 172.21.15.156:0/53846248 shutdown_connections 2023-11-28T13:08:26.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.987+0000 7f7088bff700 1 --2- 172.21.15.156:0/53846248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70840d2320 0x7f70840cd7c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:26.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.987+0000 7f7088bff700 1 --2- 172.21.15.156:0/53846248 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7084161340 0x7f70840d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:26.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.987+0000 7f7088bff700 1 -- 172.21.15.156:0/53846248 >> 172.21.15.156:0/53846248 conn(0x7f7084072600 msgr2=0x7f70840cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:26.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.988+0000 7f7088bff700 1 -- 172.21.15.156:0/53846248 shutdown_connections 2023-11-28T13:08:26.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.988+0000 7f7088bff700 1 -- 172.21.15.156:0/53846248 wait complete. 2023-11-28T13:08:26.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.990+0000 7f7088bff700 1 Processor -- start 2023-11-28T13:08:26.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.990+0000 7f7088bff700 1 -- start start 2023-11-28T13:08:26.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.991+0000 7f7088bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70840d2320 0x7f7084150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:26.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.991+0000 7f7088bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f708415c9b0 0x7f708414fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:26.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.991+0000 7f7088bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7084161340 0x7f70841e7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:26.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.991+0000 7f7088bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70841e7e80 con 0x7f70840d2320 2023-11-28T13:08:26.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.991+0000 7f7088bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70840c85b0 con 0x7f7084161340 2023-11-28T13:08:26.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.991+0000 7f7088bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70840c8890 con 0x7f708415c9b0 2023-11-28T13:08:26.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.991+0000 7f708259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70840d2320 0x7f7084150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:26.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.992+0000 7f708259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70840d2320 0x7f7084150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46400/0 (socket says 172.21.15.156:46400) 2023-11-28T13:08:26.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.992+0000 7f708259c700 1 -- 172.21.15.156:0/4274693240 learned_addr learned my addr 172.21.15.156:0/4274693240 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:26.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.992+0000 7f707bfff700 1 --2- 172.21.15.156:0/4274693240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f708415c9b0 0x7f708414fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:26.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.992+0000 7f7082d9d700 1 --2- 172.21.15.156:0/4274693240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7084161340 0x7f70841e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:26.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.992+0000 7f708259c700 1 -- 172.21.15.156:0/4274693240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f708415c9b0 msgr2=0x7f708414fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:26.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.992+0000 7f708259c700 1 --2- 172.21.15.156:0/4274693240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f708415c9b0 0x7f708414fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:26.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.992+0000 7f708259c700 1 -- 172.21.15.156:0/4274693240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7084161340 msgr2=0x7f70841e7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:26.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.992+0000 7f708259c700 1 --2- 172.21.15.156:0/4274693240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7084161340 0x7f70841e7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:26.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.992+0000 7f708259c700 1 -- 172.21.15.156:0/4274693240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c009b30 con 0x7f70840d2320 2023-11-28T13:08:26.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.992+0000 7f708259c700 1 --2- 172.21.15.156:0/4274693240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70840d2320 0x7f7084150690 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f707400c500 tx=0x7f707400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:26.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.992+0000 7f707affd700 1 -- 172.21.15.156:0/4274693240 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074005cb0 con 0x7f70840d2320 2023-11-28T13:08:26.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.992+0000 7f7088bff700 1 -- 172.21.15.156:0/4274693240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70840d15e0 con 0x7f70840d2320 2023-11-28T13:08:26.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.992+0000 7f707affd700 1 -- 172.21.15.156:0/4274693240 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7074005e10 con 0x7f70840d2320 2023-11-28T13:08:26.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.992+0000 7f707affd700 1 -- 172.21.15.156:0/4274693240 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70740072d0 con 0x7f70840d2320 2023-11-28T13:08:26.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.993+0000 7f7088bff700 1 -- 172.21.15.156:0/4274693240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70840d1300 con 0x7f70840d2320 2023-11-28T13:08:26.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.993+0000 7f7088bff700 1 -- 172.21.15.156:0/4274693240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708415cd80 con 0x7f70840d2320 2023-11-28T13:08:26.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.994+0000 7f707affd700 1 -- 172.21.15.156:0/4274693240 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7074019030 con 0x7f70840d2320 2023-11-28T13:08:26.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.996+0000 7f707affd700 1 --2- 172.21.15.156:0/4274693240 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f70640682f0 0x7f706406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:26.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.996+0000 7f707affd700 1 -- 172.21.15.156:0/4274693240 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7074084e90 con 0x7f70840d2320 2023-11-28T13:08:26.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.996+0000 7f707bfff700 1 --2- 172.21.15.156:0/4274693240 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f70640682f0 0x7f706406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:26.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:26.997+0000 7f707bfff700 1 --2- 172.21.15.156:0/4274693240 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f70640682f0 0x7f706406a7a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f70841f5ed0 tx=0x7f706c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:27.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:27.000+0000 7f707affd700 1 -- 172.21.15.156:0/4274693240 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f707404b5e0 con 0x7f70840d2320 2023-11-28T13:08:27.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:27.240+0000 7f7088bff700 1 -- 172.21.15.156:0/4274693240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f70840745c0 con 0x7f70840d2320 2023-11-28T13:08:27.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:27.241+0000 7f707affd700 1 -- 172.21.15.156:0/4274693240 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7074017020 con 0x7f70840d2320 2023-11-28T13:08:27.242 INFO:teuthology.orchestra.run.smithi156.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T13:08:27.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:27.243+0000 7f7088bff700 1 -- 172.21.15.156:0/4274693240 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f70640682f0 msgr2=0x7f706406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:27.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:27.243+0000 7f7088bff700 1 --2- 172.21.15.156:0/4274693240 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f70640682f0 0x7f706406a7a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f70841f5ed0 tx=0x7f706c009210 comp rx=0 tx=0).stop 2023-11-28T13:08:27.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:27.243+0000 7f7088bff700 1 -- 172.21.15.156:0/4274693240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70840d2320 msgr2=0x7f7084150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:27.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:27.243+0000 7f7088bff700 1 --2- 172.21.15.156:0/4274693240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70840d2320 0x7f7084150690 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f707400c500 tx=0x7f707400c810 comp rx=0 tx=0).stop 2023-11-28T13:08:27.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:27.243+0000 7f7088bff700 1 -- 172.21.15.156:0/4274693240 shutdown_connections 2023-11-28T13:08:27.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:27.243+0000 7f7088bff700 1 --2- 172.21.15.156:0/4274693240 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f70640682f0 0x7f706406a7a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:27.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:27.243+0000 7f7088bff700 1 --2- 172.21.15.156:0/4274693240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7084161340 0x7f70841e7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:27.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:27.243+0000 7f7088bff700 1 --2- 172.21.15.156:0/4274693240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70840d2320 0x7f7084150690 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:27.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:27.243+0000 7f7088bff700 1 --2- 172.21.15.156:0/4274693240 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f708415c9b0 0x7f708414fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:27.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:27.243+0000 7f7088bff700 1 -- 172.21.15.156:0/4274693240 >> 172.21.15.156:0/4274693240 conn(0x7f7084072600 msgr2=0x7f70840b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:27.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:27.243+0000 7f7088bff700 1 -- 172.21.15.156:0/4274693240 shutdown_connections 2023-11-28T13:08:27.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:27.243+0000 7f7088bff700 1 -- 172.21.15.156:0/4274693240 wait complete. 2023-11-28T13:08:27.263 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T13:08:27.263 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:08:27.264 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi186.front.sepia.ceph.com 2023-11-28T13:08:27.264 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-28T13:08:27.264 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T13:08:27.264 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T13:08:27.264 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T13:08:27.264 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:08:27.264 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-28T13:08:27.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:27.265 DEBUG:teuthology.orchestra.run.smithi186:> ip addr 2023-11-28T13:08:27.311 INFO:teuthology.orchestra.run.smithi186.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:08:27.312 INFO:teuthology.orchestra.run.smithi186.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:08:27.312 INFO:teuthology.orchestra.run.smithi186.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:08:27.312 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:27.312 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 ::1/128 scope host 2023-11-28T13:08:27.312 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:27.312 INFO:teuthology.orchestra.run.smithi186.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:08:27.312 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2c brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:27.313 INFO:teuthology.orchestra.run.smithi186.stdout: altname enp7s0f0 2023-11-28T13:08:27.313 INFO:teuthology.orchestra.run.smithi186.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:08:27.313 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2d brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:27.313 INFO:teuthology.orchestra.run.smithi186.stdout: altname enp7s0f1 2023-11-28T13:08:27.313 INFO:teuthology.orchestra.run.smithi186.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:08:27.313 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f4 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:27.313 INFO:teuthology.orchestra.run.smithi186.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:08:27.313 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f5 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:27.313 INFO:teuthology.orchestra.run.smithi186.stdout: inet 172.21.15.186/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T13:08:27.313 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft 42151sec preferred_lft 42151sec 2023-11-28T13:08:27.314 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::6239:84b8:dc67:ce5e/64 scope link noprefixroute 2023-11-28T13:08:27.314 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:27.314 INFO:teuthology.orchestra.run.smithi186.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:08:27.314 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0a:0c:72:e9:72:e4 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:08:27.314 INFO:teuthology.orchestra.run.smithi186.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T13:08:27.314 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:27.314 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::4cf9:32ff:fe01:f9c5/64 scope link 2023-11-28T13:08:27.314 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:27.314 INFO:teuthology.orchestra.run.smithi186.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:08:27.314 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0a:0c:72:e9:72:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T13:08:27.314 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::80c:72ff:fee9:72e4/64 scope link 2023-11-28T13:08:27.315 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:27.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:27.315 DEBUG:teuthology.orchestra.run.smithi186:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:08:27.419 INFO:teuthology.orchestra.run.smithi186.stdout:1 2023-11-28T13:08:27.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:27.422 DEBUG:teuthology.orchestra.run.smithi186:> route 2023-11-28T13:08:27.468 INFO:teuthology.orchestra.run.smithi186.stdout:Kernel IP routing table 2023-11-28T13:08:27.469 INFO:teuthology.orchestra.run.smithi186.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:08:27.469 INFO:teuthology.orchestra.run.smithi186.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T13:08:27.469 INFO:teuthology.orchestra.run.smithi186.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T13:08:27.469 INFO:teuthology.orchestra.run.smithi186.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:08:27.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:27.470 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T13:08:27.470 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T13:08:27.470 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T13:08:27.471 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T13:08:27.471 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T13:08:27.471 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T13:08:27.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:27.641 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2023-11-28T13:08:27.685 INFO:teuthology.orchestra.run.smithi186.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T13:08:27.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:27.686 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list-id 2023-11-28T13:08:27.768 INFO:teuthology.orchestra.run.smithi186.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-28T13:08:27.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:27.770 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T13:08:27.770 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T13:08:27.770 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T13:08:27.770 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-28T13:08:27.770 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T13:08:27.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:27.952 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T13:08:28.021 INFO:teuthology.orchestra.run.smithi186.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:08:28.021 INFO:teuthology.orchestra.run.smithi186.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:08:28.022 INFO:teuthology.orchestra.run.smithi186.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:08:28.022 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:28.022 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 ::1/128 scope host 2023-11-28T13:08:28.022 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:28.022 INFO:teuthology.orchestra.run.smithi186.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:08:28.022 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 6a:e8:cf:00:f1:9f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T13:08:28.022 INFO:teuthology.orchestra.run.smithi186.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T13:08:28.022 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:28.022 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::68e8:cfff:fe00:f19f/64 scope link 2023-11-28T13:08:28.022 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:28.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:28.025 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T13:08:28.094 INFO:teuthology.orchestra.run.smithi186.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:08:28.094 INFO:teuthology.orchestra.run.smithi186.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:08:28.094 INFO:teuthology.orchestra.run.smithi186.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:08:28.094 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:28.094 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 ::1/128 scope host 2023-11-28T13:08:28.094 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:28.094 INFO:teuthology.orchestra.run.smithi186.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:08:28.094 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 6a:e8:cf:00:f1:9f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T13:08:28.094 INFO:teuthology.orchestra.run.smithi186.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T13:08:28.094 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:28.095 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::68e8:cfff:fe00:f19f/64 scope link 2023-11-28T13:08:28.095 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:08:28.097 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-28T13:08:28.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:28.097 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T13:08:28.097 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T13:08:28.097 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-28T13:08:28.097 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T13:08:28.097 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-28T13:08:28.097 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-28T13:08:28.097 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-28T13:08:28.098 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T13:08:28.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:28.348 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-11-28T13:08:28.348 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-11-28T13:08:28.348 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set brx.1 up 2023-11-28T13:08:28.348 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T13:08:28.348 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-11-28T13:08:28.522 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:08:28.522 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-28T13:08:28.565 INFO:teuthology.orchestra.run.smithi186.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-28T13:08:28.565 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:08:28.565 DEBUG:teuthology.orchestra.run.smithi186:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-28T13:08:28.647 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T13:08:28.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:28.648 DEBUG:teuthology.orchestra.run.smithi186:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T13:08:28.791 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T13:08:28.792 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T13:08:28.792 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2023-11-28T13:08:28.811 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T13:08:28.811 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T13:08:28.811 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2023-11-28T13:08:28.811 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T13:08:28.811 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:08:28.832 INFO:teuthology.orchestra.run.smithi186.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T13:08:28.832 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T13:08:28.833 INFO:teuthology.orchestra.run.smithi186.stdout:invalid new device string format 2023-11-28T13:08:28.833 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T13:08:28.833 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:08:28.833 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: resolved to: "172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790" 2023-11-28T13:08:28.833 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2023-11-28T13:08:28.833 INFO:teuthology.orchestra.run.smithi186.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=12d9e849-dc60-46f0-8e8e-9e291633f61b" will pass to kernel 2023-11-28T13:08:28.838 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T13:08:28.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:28.838 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-28T13:08:28.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:28.907 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:08:28.992 INFO:teuthology.orchestra.run.smithi186.stdout:client4599 2023-11-28T13:08:28.993 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:08:28.993 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:08:29.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:29.038 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:08:29.161 INFO:teuthology.orchestra.run.smithi186.stdout:client4599 2023-11-28T13:08:29.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:29.162 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:08:29.246 INFO:teuthology.orchestra.run.smithi186.stdout:client4599 2023-11-28T13:08:29.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:29.247 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/12d9e849-dc60-46f0-8e8e-9e291633f61b.client4599/status') 2023-11-28T13:08:29.354 INFO:teuthology.orchestra.run.smithi186.stdout:instance: client.4599 (3)172.21.15.186:0/2427761258 2023-11-28T13:08:29.354 INFO:teuthology.orchestra.run.smithi186.stdout:blocklisted: false 2023-11-28T13:08:29.354 INFO:teuthology.orchestra.run.smithi186.stderr:0+1 records in 2023-11-28T13:08:29.354 INFO:teuthology.orchestra.run.smithi186.stderr:0+1 records out 2023-11-28T13:08:29.355 INFO:teuthology.orchestra.run.smithi186.stderr:71 bytes copied, 6.8218e-05 s, 1.0 MB/s 2023-11-28T13:08:29.357 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-28T13:08:29.376 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:08:29.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.582+0000 7fd19bdff700 1 Processor -- start 2023-11-28T13:08:29.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.582+0000 7fd19bdff700 1 -- start start 2023-11-28T13:08:29.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.583+0000 7fd19bdff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd19415c9b0 0x7fd1940b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:29.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.583+0000 7fd19bdff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1940d1950 con 0x7fd194161340 2023-11-28T13:08:29.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.583+0000 7fd19bdff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1940d24f0 con 0x7fd1940cd7c0 2023-11-28T13:08:29.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.583+0000 7fd19bdff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1940b5800 con 0x7fd19415c9b0 2023-11-28T13:08:29.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.583+0000 7fd19939a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd194161340 0x7fd1940d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50970/0 (socket says 172.21.15.156:50970) 2023-11-28T13:08:29.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.583+0000 7fd19939a700 1 -- 172.21.15.156:0/3786902392 learned_addr learned my addr 172.21.15.156:0/3786902392 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:29.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.584+0000 7fd199b9b700 1 --2- 172.21.15.156:0/3786902392 >> v2:172.21.15.186:3300/0 conn(0x7fd19415c9b0 0x7fd1940b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:29.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.584+0000 7fd198b99700 1 -- 172.21.15.156:0/3786902392 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201342459 0 0) 0x7fd1940d1950 con 0x7fd194161340 2023-11-28T13:08:29.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.584+0000 7fd198b99700 1 -- 172.21.15.156:0/3786902392 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1800035b0 con 0x7fd194161340 2023-11-28T13:08:29.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.584+0000 7fd198b99700 1 -- 172.21.15.156:0/3786902392 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 711225706 0 0) 0x7fd1800035b0 con 0x7fd194161340 2023-11-28T13:08:29.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.584+0000 7fd198b99700 1 -- 172.21.15.156:0/3786902392 >> v2:172.21.15.186:3300/0 conn(0x7fd19415c9b0 msgr2=0x7fd1940b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:29.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.584+0000 7fd198b99700 1 --2- 172.21.15.156:0/3786902392 >> v2:172.21.15.186:3300/0 conn(0x7fd19415c9b0 0x7fd1940b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:29.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.584+0000 7fd198b99700 1 -- 172.21.15.156:0/3786902392 >> v1:172.21.15.186:6789/0 conn(0x7fd1940cd7c0 legacy=0x7fd1940b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:29.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.584+0000 7fd198b99700 1 -- 172.21.15.156:0/3786902392 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1940b5520 con 0x7fd194161340 2023-11-28T13:08:29.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.584+0000 7fd198b99700 1 -- 172.21.15.156:0/3786902392 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7fd184001670 con 0x7fd194161340 2023-11-28T13:08:29.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.584+0000 7fd198b99700 1 -- 172.21.15.156:0/3786902392 >> v1:172.21.15.156:6789/0 conn(0x7fd194161340 legacy=0x7fd1940d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:29.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.584+0000 7fd198b99700 1 --2- 172.21.15.156:0/3786902392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd180003a00 0x7fd180003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:29.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.584+0000 7fd198b99700 1 --2- 172.21.15.156:0/3786902392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd180004850 0x7fd180004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:29.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.584+0000 7fd198b99700 1 --2- 172.21.15.156:0/3786902392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd180005200 0x7fd18000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:29.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.584+0000 7fd198b99700 1 -- 172.21.15.156:0/3786902392 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1940b5800 con 0x7fd180003a00 2023-11-28T13:08:29.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.584+0000 7fd198b99700 1 -- 172.21.15.156:0/3786902392 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18000c110 con 0x7fd180004850 2023-11-28T13:08:29.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.584+0000 7fd198b99700 1 -- 172.21.15.156:0/3786902392 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18000c2d0 con 0x7fd180005200 2023-11-28T13:08:29.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.585+0000 7fd199b9b700 1 --2- 172.21.15.156:0/3786902392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd180003a00 0x7fd180003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:29.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.585+0000 7fd19939a700 1 --2- 172.21.15.156:0/3786902392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd180004850 0x7fd180004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:29.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.585+0000 7fd19a39c700 1 --2- 172.21.15.156:0/3786902392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd180005200 0x7fd18000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:29.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.585+0000 7fd199b9b700 1 -- 172.21.15.156:0/3786902392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd180005200 msgr2=0x7fd18000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:29.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.585+0000 7fd199b9b700 1 --2- 172.21.15.156:0/3786902392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd180005200 0x7fd18000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:29.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.585+0000 7fd199b9b700 1 -- 172.21.15.156:0/3786902392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd180004850 msgr2=0x7fd180004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:29.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.585+0000 7fd199b9b700 1 --2- 172.21.15.156:0/3786902392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd180004850 0x7fd180004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:29.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.585+0000 7fd199b9b700 1 -- 172.21.15.156:0/3786902392 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1800035b0 con 0x7fd180003a00 2023-11-28T13:08:29.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.585+0000 7fd19a39c700 1 -- 172.21.15.156:0/3786902392 reap_dead start 2023-11-28T13:08:29.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.585+0000 7fd199b9b700 1 --2- 172.21.15.156:0/3786902392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd180003a00 0x7fd180003e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd190009b60 tx=0x7fd180005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8af90b2f9cedb7f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:29.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.585+0000 7fd198b99700 1 -- 172.21.15.156:0/3786902392 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19000e6c0 con 0x7fd180003a00 2023-11-28T13:08:29.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.585+0000 7fd198b99700 1 -- 172.21.15.156:0/3786902392 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19000e820 con 0x7fd180003a00 2023-11-28T13:08:29.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.585+0000 7fd198b99700 1 -- 172.21.15.156:0/3786902392 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19000eb90 con 0x7fd180003a00 2023-11-28T13:08:29.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.586+0000 7fd19bdff700 1 -- 172.21.15.156:0/3786902392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd180003a00 msgr2=0x7fd180003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:29.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.586+0000 7fd19bdff700 1 --2- 172.21.15.156:0/3786902392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd180003a00 0x7fd180003e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd190009b60 tx=0x7fd180005880 comp rx=0 tx=0).stop 2023-11-28T13:08:29.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.586+0000 7fd19bdff700 1 -- 172.21.15.156:0/3786902392 shutdown_connections 2023-11-28T13:08:29.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.586+0000 7fd19bdff700 1 --2- 172.21.15.156:0/3786902392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd180003a00 0x7fd180003e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:29.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.586+0000 7fd19bdff700 1 -- 172.21.15.156:0/3786902392 >> 172.21.15.156:0/3786902392 conn(0x7fd194072600 msgr2=0x7fd1940cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:29.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.586+0000 7fd19bdff700 1 -- 172.21.15.156:0/3786902392 shutdown_connections 2023-11-28T13:08:29.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.586+0000 7fd19bdff700 1 -- 172.21.15.156:0/3786902392 wait complete. 2023-11-28T13:08:29.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.588+0000 7fd19bdff700 1 Processor -- start 2023-11-28T13:08:29.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.588+0000 7fd19bdff700 1 -- start start 2023-11-28T13:08:29.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.589+0000 7fd19bdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd180003a00 0x7fd1941573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:29.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.589+0000 7fd19bdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1940b6ff0 0x7fd1940b7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:29.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.589+0000 7fd19bdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1941e7230 0x7fd1941e7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:29.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.589+0000 7fd19bdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1940b79a0 con 0x7fd1940b6ff0 2023-11-28T13:08:29.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.589+0000 7fd19bdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1940ca2e0 con 0x7fd1941e7230 2023-11-28T13:08:29.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.589+0000 7fd19bdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1940ca550 con 0x7fd180003a00 2023-11-28T13:08:29.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.589+0000 7fd19939a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1940b6ff0 0x7fd1940b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:29.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.589+0000 7fd199b9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd180003a00 0x7fd1941573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:29.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.589+0000 7fd19939a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1940b6ff0 0x7fd1940b7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46420/0 (socket says 172.21.15.156:46420) 2023-11-28T13:08:29.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.589+0000 7fd19939a700 1 -- 172.21.15.156:0/3445222121 learned_addr learned my addr 172.21.15.156:0/3445222121 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:29.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.589+0000 7fd19a39c700 1 --2- 172.21.15.156:0/3445222121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1941e7230 0x7fd1941e7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:29.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.590+0000 7fd19939a700 1 -- 172.21.15.156:0/3445222121 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd180003a00 msgr2=0x7fd1941573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:29.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.590+0000 7fd19939a700 1 --2- 172.21.15.156:0/3445222121 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd180003a00 0x7fd1941573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:29.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.590+0000 7fd19939a700 1 -- 172.21.15.156:0/3445222121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1941e7230 msgr2=0x7fd1941e7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:29.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.590+0000 7fd19939a700 1 --2- 172.21.15.156:0/3445222121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1941e7230 0x7fd1941e7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:29.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.590+0000 7fd19939a700 1 -- 172.21.15.156:0/3445222121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1940d15e0 con 0x7fd1940b6ff0 2023-11-28T13:08:29.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.590+0000 7fd199b9b700 1 --2- 172.21.15.156:0/3445222121 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd180003a00 0x7fd1941573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:29.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.590+0000 7fd19939a700 1 --2- 172.21.15.156:0/3445222121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1940b6ff0 0x7fd1940b7460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd184004f50 tx=0x7fd184007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:29.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.590+0000 7fd18a7fc700 1 -- 172.21.15.156:0/3445222121 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184012620 con 0x7fd1940b6ff0 2023-11-28T13:08:29.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.590+0000 7fd18a7fc700 1 -- 172.21.15.156:0/3445222121 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1840127c0 con 0x7fd1940b6ff0 2023-11-28T13:08:29.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.590+0000 7fd19bdff700 1 -- 172.21.15.156:0/3445222121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1940d1300 con 0x7fd1940b6ff0 2023-11-28T13:08:29.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.590+0000 7fd18a7fc700 1 -- 172.21.15.156:0/3445222121 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184012b70 con 0x7fd1940b6ff0 2023-11-28T13:08:29.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.590+0000 7fd19bdff700 1 -- 172.21.15.156:0/3445222121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19415cd80 con 0x7fd1940b6ff0 2023-11-28T13:08:29.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.591+0000 7fd19bdff700 1 -- 172.21.15.156:0/3445222121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1940745c0 con 0x7fd1940b6ff0 2023-11-28T13:08:29.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.592+0000 7fd18a7fc700 1 -- 172.21.15.156:0/3445222121 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd184012cd0 con 0x7fd1940b6ff0 2023-11-28T13:08:29.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.593+0000 7fd18a7fc700 1 --2- 172.21.15.156:0/3445222121 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd1780682f0 0x7fd17806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:29.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.593+0000 7fd18a7fc700 1 -- 172.21.15.156:0/3445222121 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd184085df0 con 0x7fd1940b6ff0 2023-11-28T13:08:29.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.594+0000 7fd199b9b700 1 --2- 172.21.15.156:0/3445222121 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd1780682f0 0x7fd17806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:29.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.594+0000 7fd199b9b700 1 --2- 172.21.15.156:0/3445222121 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd1780682f0 0x7fd17806a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd1900090b0 tx=0x7fd19001f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:29.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.597+0000 7fd18a7fc700 1 -- 172.21.15.156:0/3445222121 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd184052560 con 0x7fd1940b6ff0 2023-11-28T13:08:29.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.790+0000 7fd19bdff700 1 -- 172.21.15.156:0/3445222121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1940cd4a0 con 0x7fd1940b6ff0 2023-11-28T13:08:29.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.791+0000 7fd18a7fc700 1 -- 172.21.15.156:0/3445222121 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17897 (secure 0 0 0) 0x7fd184007af0 con 0x7fd1940b6ff0 2023-11-28T13:08:29.793 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:29.793 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":21,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:08:04.060053+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:08:00.903485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:08:02.037854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:08:29.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.793+0000 7fd19bdff700 1 -- 172.21.15.156:0/3445222121 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd1780682f0 msgr2=0x7fd17806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:29.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.793+0000 7fd19bdff700 1 --2- 172.21.15.156:0/3445222121 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd1780682f0 0x7fd17806a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd1900090b0 tx=0x7fd19001f040 comp rx=0 tx=0).stop 2023-11-28T13:08:29.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.793+0000 7fd19bdff700 1 -- 172.21.15.156:0/3445222121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1940b6ff0 msgr2=0x7fd1940b7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:29.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.793+0000 7fd19bdff700 1 --2- 172.21.15.156:0/3445222121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1940b6ff0 0x7fd1940b7460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd184004f50 tx=0x7fd184007340 comp rx=0 tx=0).stop 2023-11-28T13:08:29.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.793+0000 7fd19bdff700 1 -- 172.21.15.156:0/3445222121 shutdown_connections 2023-11-28T13:08:29.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.793+0000 7fd19bdff700 1 --2- 172.21.15.156:0/3445222121 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd1780682f0 0x7fd17806a7a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:29.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.793+0000 7fd19bdff700 1 --2- 172.21.15.156:0/3445222121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1941e7230 0x7fd1941e7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:29.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.793+0000 7fd19bdff700 1 --2- 172.21.15.156:0/3445222121 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd180003a00 0x7fd1941573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:29.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.793+0000 7fd19bdff700 1 --2- 172.21.15.156:0/3445222121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1940b6ff0 0x7fd1940b7460 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:29.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.794+0000 7fd19bdff700 1 -- 172.21.15.156:0/3445222121 >> 172.21.15.156:0/3445222121 conn(0x7fd194072600 msgr2=0x7fd19415bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:29.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.794+0000 7fd19bdff700 1 -- 172.21.15.156:0/3445222121 shutdown_connections 2023-11-28T13:08:29.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:29.794+0000 7fd19bdff700 1 -- 172.21.15.156:0/3445222121 wait complete. 2023-11-28T13:08:29.815 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:08:00.903485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:08:02.037854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:08:29.815 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:08:30.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.013+0000 7f4d8b7ff700 1 Processor -- start 2023-11-28T13:08:30.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.014+0000 7f4d8b7ff700 1 -- start start 2023-11-28T13:08:30.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.015+0000 7f4d8b7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d8415c9b0 0x7f4d840b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.015+0000 7f4d8b7ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d840d1950 con 0x7f4d84161340 2023-11-28T13:08:30.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.015+0000 7f4d8b7ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d840d24f0 con 0x7f4d840cd7c0 2023-11-28T13:08:30.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.015+0000 7f4d8b7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840b5800 con 0x7f4d8415c9b0 2023-11-28T13:08:30.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.015+0000 7f4d88d9a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4d84161340 0x7f4d840d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50982/0 (socket says 172.21.15.156:50982) 2023-11-28T13:08:30.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.015+0000 7f4d88d9a700 1 -- 172.21.15.156:0/3793884381 learned_addr learned my addr 172.21.15.156:0/3793884381 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:30.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.015+0000 7f4d8959b700 1 --2- 172.21.15.156:0/3793884381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d8415c9b0 0x7f4d840b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.015+0000 7f4d7bfff700 1 -- 172.21.15.156:0/3793884381 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2869585346 0 0) 0x7f4d840d1950 con 0x7f4d84161340 2023-11-28T13:08:30.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.015+0000 7f4d7bfff700 1 -- 172.21.15.156:0/3793884381 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d700035b0 con 0x7f4d84161340 2023-11-28T13:08:30.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d7bfff700 1 -- 172.21.15.156:0/3793884381 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4103574972 0 0) 0x7f4d700035b0 con 0x7f4d84161340 2023-11-28T13:08:30.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d7bfff700 1 -- 172.21.15.156:0/3793884381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d8415c9b0 msgr2=0x7f4d840b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:30.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d7bfff700 1 --2- 172.21.15.156:0/3793884381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d8415c9b0 0x7f4d840b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d7bfff700 1 -- 172.21.15.156:0/3793884381 >> v1:172.21.15.186:6789/0 conn(0x7f4d840cd7c0 legacy=0x7f4d840b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:30.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d7bfff700 1 -- 172.21.15.156:0/3793884381 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d840b5520 con 0x7f4d84161340 2023-11-28T13:08:30.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d7bfff700 1 -- 172.21.15.156:0/3793884381 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f4d74001670 con 0x7f4d84161340 2023-11-28T13:08:30.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d7bfff700 1 -- 172.21.15.156:0/3793884381 >> v1:172.21.15.156:6789/0 conn(0x7f4d84161340 legacy=0x7f4d840d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:30.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d7bfff700 1 --2- 172.21.15.156:0/3793884381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d70003a30 0x7f4d70003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d7bfff700 1 --2- 172.21.15.156:0/3793884381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d70004850 0x7f4d70004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d7bfff700 1 --2- 172.21.15.156:0/3793884381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d8415c9b0 0x7f4d70005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d7bfff700 1 -- 172.21.15.156:0/3793884381 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840b5800 con 0x7f4d70003a30 2023-11-28T13:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d7bfff700 1 -- 172.21.15.156:0/3793884381 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7000bb10 con 0x7f4d70004850 2023-11-28T13:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d7bfff700 1 -- 172.21.15.156:0/3793884381 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7000bcd0 con 0x7f4d8415c9b0 2023-11-28T13:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d8959b700 1 --2- 172.21.15.156:0/3793884381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d70003a30 0x7f4d70003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d89d9c700 1 --2- 172.21.15.156:0/3793884381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d8415c9b0 0x7f4d70005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d88d9a700 1 --2- 172.21.15.156:0/3793884381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d70004850 0x7f4d70004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d8959b700 1 -- 172.21.15.156:0/3793884381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d8415c9b0 msgr2=0x7f4d70005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d8959b700 1 --2- 172.21.15.156:0/3793884381 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d8415c9b0 0x7f4d70005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d8959b700 1 -- 172.21.15.156:0/3793884381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d70004850 msgr2=0x7f4d70004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d8959b700 1 --2- 172.21.15.156:0/3793884381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d70004850 0x7f4d70004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d8959b700 1 -- 172.21.15.156:0/3793884381 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d700035b0 con 0x7f4d70003a30 2023-11-28T13:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.016+0000 7f4d88d9a700 1 --2- 172.21.15.156:0/3793884381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d70004850 0x7f4d70004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.017+0000 7f4d8959b700 1 --2- 172.21.15.156:0/3793884381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d70003a30 0x7f4d70003e40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009b60 tx=0x7f4d70005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebcd1cb41f08afa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:30.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.017+0000 7f4d7bfff700 1 -- 172.21.15.156:0/3793884381 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8000e6c0 con 0x7f4d70003a30 2023-11-28T13:08:30.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.017+0000 7f4d7bfff700 1 -- 172.21.15.156:0/3793884381 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8000e820 con 0x7f4d70003a30 2023-11-28T13:08:30.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.017+0000 7f4d7bfff700 1 -- 172.21.15.156:0/3793884381 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8000eb90 con 0x7f4d70003a30 2023-11-28T13:08:30.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.017+0000 7f4d8b7ff700 1 -- 172.21.15.156:0/3793884381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d70003a30 msgr2=0x7f4d70003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:30.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.017+0000 7f4d8b7ff700 1 --2- 172.21.15.156:0/3793884381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d70003a30 0x7f4d70003e40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009b60 tx=0x7f4d70005880 comp rx=0 tx=0).stop 2023-11-28T13:08:30.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.017+0000 7f4d89d9c700 1 -- 172.21.15.156:0/3793884381 reap_dead start 2023-11-28T13:08:30.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.018+0000 7f4d8b7ff700 1 -- 172.21.15.156:0/3793884381 shutdown_connections 2023-11-28T13:08:30.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.018+0000 7f4d8b7ff700 1 -- 172.21.15.156:0/3793884381 >> 172.21.15.156:0/3793884381 conn(0x7f4d84072600 msgr2=0x7f4d840cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:30.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.018+0000 7f4d8b7ff700 1 -- 172.21.15.156:0/3793884381 shutdown_connections 2023-11-28T13:08:30.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.018+0000 7f4d8b7ff700 1 -- 172.21.15.156:0/3793884381 wait complete. 2023-11-28T13:08:30.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.020+0000 7f4d8b7ff700 1 Processor -- start 2023-11-28T13:08:30.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.020+0000 7f4d8b7ff700 1 -- start start 2023-11-28T13:08:30.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.021+0000 7f4d8b7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d70003a30 0x7f4d8414a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.021+0000 7f4d8b7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d840cabd0 0x7f4d840cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.021+0000 7f4d8b7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d840cb040 0x7f4d840ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.021+0000 7f4d8b7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840b8ab0 con 0x7f4d840cabd0 2023-11-28T13:08:30.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.021+0000 7f4d8b7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840b7ea0 con 0x7f4d840cb040 2023-11-28T13:08:30.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.021+0000 7f4d8b7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840b8180 con 0x7f4d70003a30 2023-11-28T13:08:30.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.022+0000 7f4d88d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d840cabd0 0x7f4d840cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.022+0000 7f4d8959b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d70003a30 0x7f4d8414a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.022+0000 7f4d88d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d840cabd0 0x7f4d840cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46436/0 (socket says 172.21.15.156:46436) 2023-11-28T13:08:30.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.022+0000 7f4d88d9a700 1 -- 172.21.15.156:0/2166693607 learned_addr learned my addr 172.21.15.156:0/2166693607 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:30.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.022+0000 7f4d89d9c700 1 --2- 172.21.15.156:0/2166693607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d840cb040 0x7f4d840ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.022+0000 7f4d88d9a700 1 -- 172.21.15.156:0/2166693607 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d70003a30 msgr2=0x7f4d8414a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:30.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.022+0000 7f4d88d9a700 1 --2- 172.21.15.156:0/2166693607 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d70003a30 0x7f4d8414a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.022+0000 7f4d88d9a700 1 -- 172.21.15.156:0/2166693607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d840cb040 msgr2=0x7f4d840ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:30.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.022+0000 7f4d88d9a700 1 --2- 172.21.15.156:0/2166693607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d840cb040 0x7f4d840ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.022+0000 7f4d88d9a700 1 -- 172.21.15.156:0/2166693607 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d840d15e0 con 0x7f4d840cabd0 2023-11-28T13:08:30.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.022+0000 7f4d88d9a700 1 --2- 172.21.15.156:0/2166693607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d840cabd0 0x7f4d840cc890 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400e780 tx=0x7f4d7400ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:30.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.023+0000 7f4d79ffb700 1 -- 172.21.15.156:0/2166693607 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d74010510 con 0x7f4d840cabd0 2023-11-28T13:08:30.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.023+0000 7f4d79ffb700 1 -- 172.21.15.156:0/2166693607 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d74010670 con 0x7f4d840cabd0 2023-11-28T13:08:30.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.023+0000 7f4d8b7ff700 1 -- 172.21.15.156:0/2166693607 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d840d1300 con 0x7f4d840cabd0 2023-11-28T13:08:30.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.023+0000 7f4d79ffb700 1 -- 172.21.15.156:0/2166693607 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d740109e0 con 0x7f4d840cabd0 2023-11-28T13:08:30.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.023+0000 7f4d8b7ff700 1 -- 172.21.15.156:0/2166693607 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d8415cd80 con 0x7f4d840cabd0 2023-11-28T13:08:30.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.024+0000 7f4d8b7ff700 1 -- 172.21.15.156:0/2166693607 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d84066d60 con 0x7f4d840cabd0 2023-11-28T13:08:30.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.025+0000 7f4d79ffb700 1 -- 172.21.15.156:0/2166693607 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4d7401b030 con 0x7f4d840cabd0 2023-11-28T13:08:30.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.025+0000 7f4d79ffb700 1 --2- 172.21.15.156:0/2166693607 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d680682f0 0x7f4d6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.026+0000 7f4d79ffb700 1 -- 172.21.15.156:0/2166693607 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d74085230 con 0x7f4d840cabd0 2023-11-28T13:08:30.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.026+0000 7f4d8959b700 1 --2- 172.21.15.156:0/2166693607 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d680682f0 0x7f4d6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.026+0000 7f4d8959b700 1 --2- 172.21.15.156:0/2166693607 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d680682f0 0x7f4d6806a7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4d800090b0 tx=0x7f4d8001f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:30.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.029+0000 7f4d79ffb700 1 -- 172.21.15.156:0/2166693607 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4d7404d5e0 con 0x7f4d840cabd0 2023-11-28T13:08:30.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.220+0000 7f4d8b7ff700 1 -- 172.21.15.156:0/2166693607 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4d840745c0 con 0x7f4d840cabd0 2023-11-28T13:08:30.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.221+0000 7f4d79ffb700 1 -- 172.21.15.156:0/2166693607 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4d740190a0 con 0x7f4d840cabd0 2023-11-28T13:08:30.223 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:08:30.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.224+0000 7f4d8b7ff700 1 -- 172.21.15.156:0/2166693607 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d680682f0 msgr2=0x7f4d6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:30.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.224+0000 7f4d8b7ff700 1 --2- 172.21.15.156:0/2166693607 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d680682f0 0x7f4d6806a7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4d800090b0 tx=0x7f4d8001f040 comp rx=0 tx=0).stop 2023-11-28T13:08:30.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.224+0000 7f4d8b7ff700 1 -- 172.21.15.156:0/2166693607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d840cabd0 msgr2=0x7f4d840cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:30.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.224+0000 7f4d8b7ff700 1 --2- 172.21.15.156:0/2166693607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d840cabd0 0x7f4d840cc890 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400e780 tx=0x7f4d7400ea90 comp rx=0 tx=0).stop 2023-11-28T13:08:30.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.224+0000 7f4d8b7ff700 1 -- 172.21.15.156:0/2166693607 shutdown_connections 2023-11-28T13:08:30.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.224+0000 7f4d8b7ff700 1 --2- 172.21.15.156:0/2166693607 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d680682f0 0x7f4d6806a7a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.224+0000 7f4d8b7ff700 1 --2- 172.21.15.156:0/2166693607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d840cb040 0x7f4d840ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.224+0000 7f4d8b7ff700 1 --2- 172.21.15.156:0/2166693607 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d70003a30 0x7f4d8414a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.224+0000 7f4d8b7ff700 1 --2- 172.21.15.156:0/2166693607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d840cabd0 0x7f4d840cc890 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.224+0000 7f4d8b7ff700 1 -- 172.21.15.156:0/2166693607 >> 172.21.15.156:0/2166693607 conn(0x7f4d84072600 msgr2=0x7f4d8415bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:30.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.224+0000 7f4d8b7ff700 1 -- 172.21.15.156:0/2166693607 shutdown_connections 2023-11-28T13:08:30.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.224+0000 7f4d8b7ff700 1 -- 172.21.15.156:0/2166693607 wait complete. 2023-11-28T13:08:30.244 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:08:30.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.441+0000 7f260fdff700 1 Processor -- start 2023-11-28T13:08:30.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.442+0000 7f260fdff700 1 -- start start 2023-11-28T13:08:30.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.443+0000 7f260fdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2608161340 0x7f26080d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.443+0000 7f260fdff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f26080d2320 0x7f26080cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.443+0000 7f260fdff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26080d1950 con 0x7f260815c9b0 2023-11-28T13:08:30.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.443+0000 7f260fdff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26080b5800 con 0x7f26080d2320 2023-11-28T13:08:30.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.443+0000 7f260fdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26080b5300 con 0x7f2608161340 2023-11-28T13:08:30.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.443+0000 7f260db9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f260815c9b0 0x7f26080b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50988/0 (socket says 172.21.15.156:50988) 2023-11-28T13:08:30.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.443+0000 7f260db9b700 1 -- 172.21.15.156:0/3591926038 learned_addr learned my addr 172.21.15.156:0/3591926038 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:30.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.443+0000 7f260d39a700 1 --2- 172.21.15.156:0/3591926038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2608161340 0x7f26080d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.443+0000 7f260e39c700 1 --2- 172.21.15.156:0/3591926038 >> v2:172.21.15.186:3300/0 conn(0x7f26080d2320 0x7f26080cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.443+0000 7f260cb99700 1 -- 172.21.15.156:0/3591926038 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2664128713 0 0) 0x7f26080d1950 con 0x7f260815c9b0 2023-11-28T13:08:30.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.444+0000 7f260cb99700 1 -- 172.21.15.156:0/3591926038 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25f40035b0 con 0x7f260815c9b0 2023-11-28T13:08:30.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.444+0000 7f260d39a700 1 -- 172.21.15.156:0/3591926038 >> v2:172.21.15.186:3300/0 conn(0x7f26080d2320 msgr2=0x7f26080cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:30.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.444+0000 7f260d39a700 1 --2- 172.21.15.156:0/3591926038 >> v2:172.21.15.186:3300/0 conn(0x7f26080d2320 0x7f26080cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.444+0000 7f260d39a700 1 -- 172.21.15.156:0/3591926038 >> v1:172.21.15.156:6789/0 conn(0x7f260815c9b0 legacy=0x7f26080b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:30.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.444+0000 7f260d39a700 1 -- 172.21.15.156:0/3591926038 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26080b5520 con 0x7f2608161340 2023-11-28T13:08:30.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.444+0000 7f260d39a700 1 --2- 172.21.15.156:0/3591926038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2608161340 0x7f26080d1de0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009840 tx=0x7f25f8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abe57e3f1ff378b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:30.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.444+0000 7f260cb99700 1 -- 172.21.15.156:0/3591926038 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8011830 con 0x7f2608161340 2023-11-28T13:08:30.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.444+0000 7f260cb99700 1 -- 172.21.15.156:0/3591926038 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f8011990 con 0x7f2608161340 2023-11-28T13:08:30.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.444+0000 7f260cb99700 1 -- 172.21.15.156:0/3591926038 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8011d00 con 0x7f2608161340 2023-11-28T13:08:30.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.445+0000 7f260fdff700 1 -- 172.21.15.156:0/3591926038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2608161340 msgr2=0x7f26080d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:30.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.445+0000 7f260fdff700 1 --2- 172.21.15.156:0/3591926038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2608161340 0x7f26080d1de0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009840 tx=0x7f25f8009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:30.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.445+0000 7f260fdff700 1 -- 172.21.15.156:0/3591926038 shutdown_connections 2023-11-28T13:08:30.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.445+0000 7f260fdff700 1 --2- 172.21.15.156:0/3591926038 >> v2:172.21.15.186:3300/0 conn(0x7f26080d2320 0x7f26080cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.445+0000 7f260fdff700 1 --2- 172.21.15.156:0/3591926038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2608161340 0x7f26080d1de0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.445+0000 7f260fdff700 1 -- 172.21.15.156:0/3591926038 >> 172.21.15.156:0/3591926038 conn(0x7f2608072600 msgr2=0x7f26080cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:30.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.445+0000 7f260fdff700 1 -- 172.21.15.156:0/3591926038 shutdown_connections 2023-11-28T13:08:30.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.445+0000 7f260fdff700 1 -- 172.21.15.156:0/3591926038 wait complete. 2023-11-28T13:08:30.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.447+0000 7f260fdff700 1 Processor -- start 2023-11-28T13:08:30.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.447+0000 7f260fdff700 1 -- start start 2023-11-28T13:08:30.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.448+0000 7f260fdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26080d2320 0x7f2608156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.448+0000 7f260fdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f260815c9b0 0x7f26081e6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.448+0000 7f260fdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2608161340 0x7f26080b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.448+0000 7f260fdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26081e7300 con 0x7f26080d2320 2023-11-28T13:08:30.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.448+0000 7f260fdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26081f3060 con 0x7f260815c9b0 2023-11-28T13:08:30.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.448+0000 7f260fdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26081f2a70 con 0x7f2608161340 2023-11-28T13:08:30.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.448+0000 7f260db9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26080d2320 0x7f2608156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.448+0000 7f260db9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26080d2320 0x7f2608156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46452/0 (socket says 172.21.15.156:46452) 2023-11-28T13:08:30.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.448+0000 7f260db9b700 1 -- 172.21.15.156:0/1953435593 learned_addr learned my addr 172.21.15.156:0/1953435593 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:30.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.448+0000 7f260e39c700 1 --2- 172.21.15.156:0/1953435593 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2608161340 0x7f26080b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.448+0000 7f260d39a700 1 --2- 172.21.15.156:0/1953435593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f260815c9b0 0x7f26081e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.449+0000 7f260db9b700 1 -- 172.21.15.156:0/1953435593 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2608161340 msgr2=0x7f26080b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:30.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.449+0000 7f260db9b700 1 --2- 172.21.15.156:0/1953435593 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2608161340 0x7f26080b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.449+0000 7f260db9b700 1 -- 172.21.15.156:0/1953435593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f260815c9b0 msgr2=0x7f26081e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:30.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.449+0000 7f260db9b700 1 --2- 172.21.15.156:0/1953435593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f260815c9b0 0x7f26081e6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.449+0000 7f260db9b700 1 -- 172.21.15.156:0/1953435593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f80094a0 con 0x7f26080d2320 2023-11-28T13:08:30.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.449+0000 7f260db9b700 1 --2- 172.21.15.156:0/1953435593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26080d2320 0x7f2608156960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f260400c420 tx=0x7f260400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:30.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.449+0000 7f25fe7fc700 1 -- 172.21.15.156:0/1953435593 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2604005d70 con 0x7f26080d2320 2023-11-28T13:08:30.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.449+0000 7f25fe7fc700 1 -- 172.21.15.156:0/1953435593 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260400ce30 con 0x7f26080d2320 2023-11-28T13:08:30.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.449+0000 7f260fdff700 1 -- 172.21.15.156:0/1953435593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26080d15e0 con 0x7f26080d2320 2023-11-28T13:08:30.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.449+0000 7f25fe7fc700 1 -- 172.21.15.156:0/1953435593 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26040072d0 con 0x7f26080d2320 2023-11-28T13:08:30.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.449+0000 7f260fdff700 1 -- 172.21.15.156:0/1953435593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26080d1300 con 0x7f26080d2320 2023-11-28T13:08:30.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.450+0000 7f260fdff700 1 -- 172.21.15.156:0/1953435593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260815cd80 con 0x7f26080d2320 2023-11-28T13:08:30.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.451+0000 7f25fe7fc700 1 -- 172.21.15.156:0/1953435593 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2604019030 con 0x7f26080d2320 2023-11-28T13:08:30.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.453+0000 7f25fe7fc700 1 --2- 172.21.15.156:0/1953435593 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25ec0682f0 0x7f25ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.453+0000 7f25fe7fc700 1 -- 172.21.15.156:0/1953435593 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2604084df0 con 0x7f26080d2320 2023-11-28T13:08:30.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.453+0000 7f260d39a700 1 --2- 172.21.15.156:0/1953435593 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25ec0682f0 0x7f25ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.454+0000 7f260d39a700 1 --2- 172.21.15.156:0/1953435593 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25ec0682f0 0x7f25ec06a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009b90 tx=0x7f25f8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:30.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.457+0000 7f25fe7fc700 1 -- 172.21.15.156:0/1953435593 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f260404b5e0 con 0x7f26080d2320 2023-11-28T13:08:30.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.649+0000 7f260fdff700 1 -- 172.21.15.156:0/1953435593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f26080745c0 con 0x7f26080d2320 2023-11-28T13:08:30.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.650+0000 7f25fe7fc700 1 -- 172.21.15.156:0/1953435593 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2604017020 con 0x7f26080d2320 2023-11-28T13:08:30.651 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:08:30.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.652+0000 7f260fdff700 1 -- 172.21.15.156:0/1953435593 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25ec0682f0 msgr2=0x7f25ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:30.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.652+0000 7f260fdff700 1 --2- 172.21.15.156:0/1953435593 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25ec0682f0 0x7f25ec06a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009b90 tx=0x7f25f8008030 comp rx=0 tx=0).stop 2023-11-28T13:08:30.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.652+0000 7f260fdff700 1 -- 172.21.15.156:0/1953435593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26080d2320 msgr2=0x7f2608156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:30.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.652+0000 7f260fdff700 1 --2- 172.21.15.156:0/1953435593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26080d2320 0x7f2608156960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f260400c420 tx=0x7f260400c730 comp rx=0 tx=0).stop 2023-11-28T13:08:30.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.652+0000 7f260fdff700 1 -- 172.21.15.156:0/1953435593 shutdown_connections 2023-11-28T13:08:30.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.652+0000 7f260fdff700 1 --2- 172.21.15.156:0/1953435593 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25ec0682f0 0x7f25ec06a7a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.652+0000 7f260fdff700 1 --2- 172.21.15.156:0/1953435593 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2608161340 0x7f26080b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.652+0000 7f260fdff700 1 --2- 172.21.15.156:0/1953435593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26080d2320 0x7f2608156960 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.652+0000 7f260fdff700 1 --2- 172.21.15.156:0/1953435593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f260815c9b0 0x7f26081e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.652+0000 7f260fdff700 1 -- 172.21.15.156:0/1953435593 >> 172.21.15.156:0/1953435593 conn(0x7f2608072600 msgr2=0x7f260806b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:30.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.652+0000 7f260fdff700 1 -- 172.21.15.156:0/1953435593 shutdown_connections 2023-11-28T13:08:30.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.652+0000 7f260fdff700 1 -- 172.21.15.156:0/1953435593 wait complete. 2023-11-28T13:08:30.672 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:08:30.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.872+0000 7f0261fff700 1 Processor -- start 2023-11-28T13:08:30.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.872+0000 7f0261fff700 1 -- start start 2023-11-28T13:08:30.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.873+0000 7f0261fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f025c0d1330 0x7f025c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.873+0000 7f0261fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f025c0b20b0 0x7f025c0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.873+0000 7f0261fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025c0b0230 con 0x7f025c0d01b0 2023-11-28T13:08:30.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.873+0000 7f0261fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025c15cec0 con 0x7f025c0d1330 2023-11-28T13:08:30.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.873+0000 7f0261fff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025c0b4570 con 0x7f025c0b20b0 2023-11-28T13:08:30.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.873+0000 7f025affd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f025c0d1330 0x7f025c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.873+0000 7f025affd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f025c0d1330 0x7f025c0b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46454/0 (socket says 172.21.15.156:46454) 2023-11-28T13:08:30.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.873+0000 7f025affd700 1 -- 172.21.15.156:0/436462327 learned_addr learned my addr 172.21.15.156:0/436462327 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:30.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.874+0000 7f025bfff700 1 --2- 172.21.15.156:0/436462327 >> v2:172.21.15.186:3300/0 conn(0x7f025c0b20b0 0x7f025c0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.874+0000 7f025affd700 1 -- 172.21.15.156:0/436462327 >> v2:172.21.15.186:3300/0 conn(0x7f025c0b20b0 msgr2=0x7f025c0b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:30.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.874+0000 7f025affd700 1 --2- 172.21.15.156:0/436462327 >> v2:172.21.15.186:3300/0 conn(0x7f025c0b20b0 0x7f025c0b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.874+0000 7f025affd700 1 -- 172.21.15.156:0/436462327 >> v1:172.21.15.186:6789/0 conn(0x7f025c0d01b0 legacy=0x7f025c0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:30.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.874+0000 7f025affd700 1 -- 172.21.15.156:0/436462327 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c0b4850 con 0x7f025c0d1330 2023-11-28T13:08:30.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.874+0000 7f025bfff700 1 --2- 172.21.15.156:0/436462327 >> v2:172.21.15.186:3300/0 conn(0x7f025c0b20b0 0x7f025c0b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:30.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.874+0000 7f025affd700 1 --2- 172.21.15.156:0/436462327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c0d1330 0x7f025c0b06c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f025c066e80 tx=0x7f0250009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0804c64cc30d87f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:30.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.874+0000 7f025a7fc700 1 -- 172.21.15.156:0/436462327 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250004230 con 0x7f025c0d1330 2023-11-28T13:08:30.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.874+0000 7f025a7fc700 1 -- 172.21.15.156:0/436462327 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0250004390 con 0x7f025c0d1330 2023-11-28T13:08:30.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.874+0000 7f025a7fc700 1 -- 172.21.15.156:0/436462327 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02500036a0 con 0x7f025c0d1330 2023-11-28T13:08:30.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.875+0000 7f0261fff700 1 -- 172.21.15.156:0/436462327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c0d1330 msgr2=0x7f025c0b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:30.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.875+0000 7f0261fff700 1 --2- 172.21.15.156:0/436462327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c0d1330 0x7f025c0b06c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f025c066e80 tx=0x7f0250009bd0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.875+0000 7f0261fff700 1 -- 172.21.15.156:0/436462327 shutdown_connections 2023-11-28T13:08:30.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.875+0000 7f0261fff700 1 --2- 172.21.15.156:0/436462327 >> v2:172.21.15.186:3300/0 conn(0x7f025c0b20b0 0x7f025c0b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.875+0000 7f0261fff700 1 --2- 172.21.15.156:0/436462327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c0d1330 0x7f025c0b06c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.875+0000 7f0261fff700 1 -- 172.21.15.156:0/436462327 >> 172.21.15.156:0/436462327 conn(0x7f025c09c3f0 msgr2=0x7f025c0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:30.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.875+0000 7f0261fff700 1 -- 172.21.15.156:0/436462327 shutdown_connections 2023-11-28T13:08:30.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.875+0000 7f0261fff700 1 -- 172.21.15.156:0/436462327 wait complete. 2023-11-28T13:08:30.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.877+0000 7f0261fff700 1 Processor -- start 2023-11-28T13:08:30.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.878+0000 7f0261fff700 1 -- start start 2023-11-28T13:08:30.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.879+0000 7f0261fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c0b20b0 0x7f025c156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.879+0000 7f0261fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f025c0d01b0 0x7f025c1578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.879+0000 7f0261fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f025c0d1330 0x7f025c15e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.879+0000 7f0261fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c15dc10 con 0x7f025c0b20b0 2023-11-28T13:08:30.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.879+0000 7f0261fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c156c80 con 0x7f025c0d01b0 2023-11-28T13:08:30.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.879+0000 7f0261fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025c15d000 con 0x7f025c0d1330 2023-11-28T13:08:30.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.879+0000 7f025b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c0b20b0 0x7f025c156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.879+0000 7f025b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c0b20b0 0x7f025c156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46468/0 (socket says 172.21.15.156:46468) 2023-11-28T13:08:30.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.879+0000 7f025b7fe700 1 -- 172.21.15.156:0/898466470 learned_addr learned my addr 172.21.15.156:0/898466470 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:30.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.879+0000 7f025affd700 1 --2- 172.21.15.156:0/898466470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f025c0d01b0 0x7f025c1578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.879+0000 7f025bfff700 1 --2- 172.21.15.156:0/898466470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f025c0d1330 0x7f025c15e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.879+0000 7f025b7fe700 1 -- 172.21.15.156:0/898466470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f025c0d1330 msgr2=0x7f025c15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:30.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.879+0000 7f025b7fe700 1 --2- 172.21.15.156:0/898466470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f025c0d1330 0x7f025c15e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.879+0000 7f025b7fe700 1 -- 172.21.15.156:0/898466470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f025c0d01b0 msgr2=0x7f025c1578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:30.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.879+0000 7f025b7fe700 1 --2- 172.21.15.156:0/898466470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f025c0d01b0 0x7f025c1578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:30.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.879+0000 7f025b7fe700 1 -- 172.21.15.156:0/898466470 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0250009550 con 0x7f025c0b20b0 2023-11-28T13:08:30.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.879+0000 7f025affd700 1 --2- 172.21.15.156:0/898466470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f025c0d01b0 0x7f025c1578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:30.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.880+0000 7f025b7fe700 1 --2- 172.21.15.156:0/898466470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c0b20b0 0x7f025c156740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c4a0 tx=0x7f024c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:30.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.880+0000 7f0243fff700 1 -- 172.21.15.156:0/898466470 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c0054e0 con 0x7f025c0b20b0 2023-11-28T13:08:30.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.880+0000 7f0243fff700 1 -- 172.21.15.156:0/898466470 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c003680 con 0x7f025c0b20b0 2023-11-28T13:08:30.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.880+0000 7f0261fff700 1 -- 172.21.15.156:0/898466470 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025c0afec0 con 0x7f025c0b20b0 2023-11-28T13:08:30.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.880+0000 7f0243fff700 1 -- 172.21.15.156:0/898466470 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c003a30 con 0x7f025c0b20b0 2023-11-28T13:08:30.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.880+0000 7f0261fff700 1 -- 172.21.15.156:0/898466470 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c0afbe0 con 0x7f025c0b20b0 2023-11-28T13:08:30.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.881+0000 7f0261fff700 1 -- 172.21.15.156:0/898466470 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025c0b2600 con 0x7f025c0b20b0 2023-11-28T13:08:30.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.882+0000 7f0243fff700 1 -- 172.21.15.156:0/898466470 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f024c010070 con 0x7f025c0b20b0 2023-11-28T13:08:30.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.883+0000 7f0243fff700 1 --2- 172.21.15.156:0/898466470 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02480682f0 0x7f024806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:30.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.883+0000 7f0243fff700 1 -- 172.21.15.156:0/898466470 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f024c084d20 con 0x7f025c0b20b0 2023-11-28T13:08:30.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.883+0000 7f025affd700 1 --2- 172.21.15.156:0/898466470 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02480682f0 0x7f024806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:30.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.884+0000 7f025affd700 1 --2- 172.21.15.156:0/898466470 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02480682f0 0x7f024806a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f025c0d1720 tx=0x7f0250008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:30.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:30.887+0000 7f0243fff700 1 -- 172.21.15.156:0/898466470 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f024c051490 con 0x7f025c0b20b0 2023-11-28T13:08:31.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.077+0000 7f0261fff700 1 -- 172.21.15.156:0/898466470 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f025c15def0 con 0x7f025c0b20b0 2023-11-28T13:08:31.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.078+0000 7f0243fff700 1 -- 172.21.15.156:0/898466470 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f024c054ab0 con 0x7f025c0b20b0 2023-11-28T13:08:31.079 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T13:08:31.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.080+0000 7f0261fff700 1 -- 172.21.15.156:0/898466470 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02480682f0 msgr2=0x7f024806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:31.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.080+0000 7f0261fff700 1 --2- 172.21.15.156:0/898466470 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02480682f0 0x7f024806a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f025c0d1720 tx=0x7f0250008030 comp rx=0 tx=0).stop 2023-11-28T13:08:31.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.080+0000 7f0261fff700 1 -- 172.21.15.156:0/898466470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c0b20b0 msgr2=0x7f025c156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:31.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.080+0000 7f0261fff700 1 --2- 172.21.15.156:0/898466470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c0b20b0 0x7f025c156740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c4a0 tx=0x7f024c00c860 comp rx=0 tx=0).stop 2023-11-28T13:08:31.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.080+0000 7f0261fff700 1 -- 172.21.15.156:0/898466470 shutdown_connections 2023-11-28T13:08:31.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.080+0000 7f0261fff700 1 --2- 172.21.15.156:0/898466470 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02480682f0 0x7f024806a7a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.080+0000 7f0261fff700 1 --2- 172.21.15.156:0/898466470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f025c0d1330 0x7f025c15e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.080+0000 7f0261fff700 1 --2- 172.21.15.156:0/898466470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f025c0b20b0 0x7f025c156740 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.080+0000 7f0261fff700 1 --2- 172.21.15.156:0/898466470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f025c0d01b0 0x7f025c1578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.080+0000 7f0261fff700 1 -- 172.21.15.156:0/898466470 >> 172.21.15.156:0/898466470 conn(0x7f025c09c3f0 msgr2=0x7f025c15c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:31.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.080+0000 7f0261fff700 1 -- 172.21.15.156:0/898466470 shutdown_connections 2023-11-28T13:08:31.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.080+0000 7f0261fff700 1 -- 172.21.15.156:0/898466470 wait complete. 2023-11-28T13:08:31.100 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:08:31.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.297+0000 7f2eb31ff700 1 Processor -- start 2023-11-28T13:08:31.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.297+0000 7f2eb31ff700 1 -- start start 2023-11-28T13:08:31.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.298+0000 7f2eb31ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2eac159860 0x7f2eac0c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:31.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.299+0000 7f2eb31ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eac0ccc00 con 0x7f2eac159490 2023-11-28T13:08:31.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.299+0000 7f2eb31ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eac0ccec0 con 0x7f2eac0b0d80 2023-11-28T13:08:31.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.299+0000 7f2eb31ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eac0cd1a0 con 0x7f2eac159860 2023-11-28T13:08:31.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.299+0000 7f2eb0f9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f2eac159490 0x7f2eac0b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51002/0 (socket says 172.21.15.156:51002) 2023-11-28T13:08:31.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.299+0000 7f2eb0f9b700 1 -- 172.21.15.156:0/2119866959 learned_addr learned my addr 172.21.15.156:0/2119866959 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:31.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.299+0000 7f2eabfff700 1 --2- 172.21.15.156:0/2119866959 >> v2:172.21.15.186:3300/0 conn(0x7f2eac159860 0x7f2eac0c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:31.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.299+0000 7f2eab7fe700 1 -- 172.21.15.156:0/2119866959 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660588796 0 0) 0x7f2eac0ccc00 con 0x7f2eac159490 2023-11-28T13:08:31.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.299+0000 7f2eab7fe700 1 -- 172.21.15.156:0/2119866959 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e9c0035b0 con 0x7f2eac159490 2023-11-28T13:08:31.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.299+0000 7f2eab7fe700 1 -- 172.21.15.156:0/2119866959 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4189441644 0 0) 0x7f2e9c0035b0 con 0x7f2eac159490 2023-11-28T13:08:31.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eab7fe700 1 -- 172.21.15.156:0/2119866959 >> v2:172.21.15.186:3300/0 conn(0x7f2eac159860 msgr2=0x7f2eac0c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:31.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eab7fe700 1 --2- 172.21.15.156:0/2119866959 >> v2:172.21.15.186:3300/0 conn(0x7f2eac159860 0x7f2eac0c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eab7fe700 1 -- 172.21.15.156:0/2119866959 >> v1:172.21.15.186:6789/0 conn(0x7f2eac0b0d80 legacy=0x7f2eac0c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:31.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eab7fe700 1 -- 172.21.15.156:0/2119866959 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac0cd480 con 0x7f2eac159490 2023-11-28T13:08:31.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eab7fe700 1 -- 172.21.15.156:0/2119866959 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f2ea0001670 con 0x7f2eac159490 2023-11-28T13:08:31.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eab7fe700 1 -- 172.21.15.156:0/2119866959 >> v1:172.21.15.156:6789/0 conn(0x7f2eac159490 legacy=0x7f2eac0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:31.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eab7fe700 1 --2- 172.21.15.156:0/2119866959 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9c003a00 0x7f2e9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:31.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eab7fe700 1 --2- 172.21.15.156:0/2119866959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e9c0043f0 0x7f2e9c0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:31.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eab7fe700 1 --2- 172.21.15.156:0/2119866959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e9c0050b0 0x7f2e9c00dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:31.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eab7fe700 1 -- 172.21.15.156:0/2119866959 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac0cd1a0 con 0x7f2e9c0043f0 2023-11-28T13:08:31.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eab7fe700 1 -- 172.21.15.156:0/2119866959 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c00e210 con 0x7f2e9c0050b0 2023-11-28T13:08:31.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eab7fe700 1 -- 172.21.15.156:0/2119866959 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c00e3d0 con 0x7f2e9c003a00 2023-11-28T13:08:31.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eabfff700 1 --2- 172.21.15.156:0/2119866959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e9c0043f0 0x7f2e9c0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:31.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eb179c700 1 --2- 172.21.15.156:0/2119866959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e9c0050b0 0x7f2e9c00dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:31.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eb0f9b700 1 --2- 172.21.15.156:0/2119866959 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9c003a00 0x7f2e9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:31.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eabfff700 1 -- 172.21.15.156:0/2119866959 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9c003a00 msgr2=0x7f2e9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:31.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eabfff700 1 --2- 172.21.15.156:0/2119866959 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9c003a00 0x7f2e9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eabfff700 1 -- 172.21.15.156:0/2119866959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e9c0050b0 msgr2=0x7f2e9c00dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:31.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eabfff700 1 --2- 172.21.15.156:0/2119866959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e9c0050b0 0x7f2e9c00dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eabfff700 1 -- 172.21.15.156:0/2119866959 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0035b0 con 0x7f2e9c0043f0 2023-11-28T13:08:31.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eb179c700 1 --2- 172.21.15.156:0/2119866959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e9c0050b0 0x7f2e9c00dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:31.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.300+0000 7f2eb179c700 1 -- 172.21.15.156:0/2119866959 reap_dead start 2023-11-28T13:08:31.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.301+0000 7f2eabfff700 1 --2- 172.21.15.156:0/2119866959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e9c0043f0 0x7f2e9c0048b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009b60 tx=0x7f2e9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7bfc784c0c7c88 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:31.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.301+0000 7f2eab7fe700 1 -- 172.21.15.156:0/2119866959 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9800e6c0 con 0x7f2e9c0043f0 2023-11-28T13:08:31.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.301+0000 7f2eab7fe700 1 -- 172.21.15.156:0/2119866959 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9800e820 con 0x7f2e9c0043f0 2023-11-28T13:08:31.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.301+0000 7f2eab7fe700 1 -- 172.21.15.156:0/2119866959 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9800eb90 con 0x7f2e9c0043f0 2023-11-28T13:08:31.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.301+0000 7f2eb31ff700 1 -- 172.21.15.156:0/2119866959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e9c0043f0 msgr2=0x7f2e9c0048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:31.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.301+0000 7f2eb31ff700 1 --2- 172.21.15.156:0/2119866959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e9c0043f0 0x7f2e9c0048b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009b60 tx=0x7f2e9c005880 comp rx=0 tx=0).stop 2023-11-28T13:08:31.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.301+0000 7f2eb31ff700 1 -- 172.21.15.156:0/2119866959 shutdown_connections 2023-11-28T13:08:31.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.301+0000 7f2eb31ff700 1 --2- 172.21.15.156:0/2119866959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e9c0043f0 0x7f2e9c0048b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.301+0000 7f2eb31ff700 1 -- 172.21.15.156:0/2119866959 >> 172.21.15.156:0/2119866959 conn(0x7f2eac0cb670 msgr2=0x7f2eac0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:31.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.302+0000 7f2eb31ff700 1 -- 172.21.15.156:0/2119866959 shutdown_connections 2023-11-28T13:08:31.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.302+0000 7f2eb31ff700 1 -- 172.21.15.156:0/2119866959 wait complete. 2023-11-28T13:08:31.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.303+0000 7f2eb31ff700 1 Processor -- start 2023-11-28T13:08:31.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.303+0000 7f2eb31ff700 1 -- start start 2023-11-28T13:08:31.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.304+0000 7f2eb31ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e9c0043f0 0x7f2eac1f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:31.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.304+0000 7f2eb31ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2eac0c4db0 0x7f2eac0b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:31.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.304+0000 7f2eb31ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eac159fa0 0x7f2eac15a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:31.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.304+0000 7f2eb31ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac1f2830 con 0x7f2eac159fa0 2023-11-28T13:08:31.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.304+0000 7f2eb31ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac0b3960 con 0x7f2e9c0043f0 2023-11-28T13:08:31.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.304+0000 7f2eb31ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac0b2a70 con 0x7f2eac0c4db0 2023-11-28T13:08:31.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.305+0000 7f2eb179c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eac159fa0 0x7f2eac15a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:31.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.305+0000 7f2eb179c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eac159fa0 0x7f2eac15a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46482/0 (socket says 172.21.15.156:46482) 2023-11-28T13:08:31.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.305+0000 7f2eb179c700 1 -- 172.21.15.156:0/4267749628 learned_addr learned my addr 172.21.15.156:0/4267749628 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:31.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.305+0000 7f2eb0f9b700 1 --2- 172.21.15.156:0/4267749628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e9c0043f0 0x7f2eac1f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:31.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.305+0000 7f2eabfff700 1 --2- 172.21.15.156:0/4267749628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2eac0c4db0 0x7f2eac0b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:31.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.305+0000 7f2eb179c700 1 -- 172.21.15.156:0/4267749628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2eac0c4db0 msgr2=0x7f2eac0b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:31.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.305+0000 7f2eb179c700 1 --2- 172.21.15.156:0/4267749628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2eac0c4db0 0x7f2eac0b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.305+0000 7f2eb179c700 1 -- 172.21.15.156:0/4267749628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e9c0043f0 msgr2=0x7f2eac1f22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:31.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.305+0000 7f2eb179c700 1 --2- 172.21.15.156:0/4267749628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e9c0043f0 0x7f2eac1f22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.305+0000 7f2eb179c700 1 -- 172.21.15.156:0/4267749628 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac0cdc80 con 0x7f2eac159fa0 2023-11-28T13:08:31.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.305+0000 7f2eb0f9b700 1 --2- 172.21.15.156:0/4267749628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e9c0043f0 0x7f2eac1f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:31.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.305+0000 7f2eabfff700 1 --2- 172.21.15.156:0/4267749628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2eac0c4db0 0x7f2eac0b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:31.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.305+0000 7f2eb179c700 1 --2- 172.21.15.156:0/4267749628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eac159fa0 0x7f2eac15a440 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0b17c0 tx=0x7f2ea400c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:31.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.306+0000 7f2ea97fa700 1 -- 172.21.15.156:0/4267749628 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4003c10 con 0x7f2eac159fa0 2023-11-28T13:08:31.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.306+0000 7f2ea97fa700 1 -- 172.21.15.156:0/4267749628 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea4003d70 con 0x7f2eac159fa0 2023-11-28T13:08:31.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.306+0000 7f2ea97fa700 1 -- 172.21.15.156:0/4267749628 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea400c920 con 0x7f2eac159fa0 2023-11-28T13:08:31.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.306+0000 7f2eb31ff700 1 -- 172.21.15.156:0/4267749628 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eac0cd760 con 0x7f2eac159fa0 2023-11-28T13:08:31.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.306+0000 7f2eb31ff700 1 -- 172.21.15.156:0/4267749628 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eac0cde10 con 0x7f2eac159fa0 2023-11-28T13:08:31.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.307+0000 7f2eb31ff700 1 -- 172.21.15.156:0/4267749628 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eac0745c0 con 0x7f2eac159fa0 2023-11-28T13:08:31.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.308+0000 7f2ea97fa700 1 -- 172.21.15.156:0/4267749628 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2ea400ca80 con 0x7f2eac159fa0 2023-11-28T13:08:31.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.309+0000 7f2ea97fa700 1 --2- 172.21.15.156:0/4267749628 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2e900682f0 0x7f2e9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:31.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.309+0000 7f2ea97fa700 1 -- 172.21.15.156:0/4267749628 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2ea4085f60 con 0x7f2eac159fa0 2023-11-28T13:08:31.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.309+0000 7f2eb0f9b700 1 --2- 172.21.15.156:0/4267749628 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2e900682f0 0x7f2e9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:31.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.310+0000 7f2eb0f9b700 1 --2- 172.21.15.156:0/4267749628 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2e900682f0 0x7f2e9006a7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00017a0 tx=0x7f2ea0001d60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:31.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.313+0000 7f2ea97fa700 1 -- 172.21.15.156:0/4267749628 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2ea40527c0 con 0x7f2eac159fa0 2023-11-28T13:08:31.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.503+0000 7f2eb31ff700 1 -- 172.21.15.156:0/4267749628 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2eac153540 con 0x7f2eac159fa0 2023-11-28T13:08:31.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.504+0000 7f2ea97fa700 1 -- 172.21.15.156:0/4267749628 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2ea4055de0 con 0x7f2eac159fa0 2023-11-28T13:08:31.506 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T13:08:31.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.506+0000 7f2eb31ff700 1 -- 172.21.15.156:0/4267749628 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2e900682f0 msgr2=0x7f2e9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:31.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.506+0000 7f2eb31ff700 1 --2- 172.21.15.156:0/4267749628 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2e900682f0 0x7f2e9006a7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00017a0 tx=0x7f2ea0001d60 comp rx=0 tx=0).stop 2023-11-28T13:08:31.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.506+0000 7f2eb31ff700 1 -- 172.21.15.156:0/4267749628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eac159fa0 msgr2=0x7f2eac15a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:31.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.506+0000 7f2eb31ff700 1 --2- 172.21.15.156:0/4267749628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eac159fa0 0x7f2eac15a440 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0b17c0 tx=0x7f2ea400c0e0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.506+0000 7f2eb31ff700 1 -- 172.21.15.156:0/4267749628 shutdown_connections 2023-11-28T13:08:31.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.506+0000 7f2eb31ff700 1 --2- 172.21.15.156:0/4267749628 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2e900682f0 0x7f2e9006a7a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.506+0000 7f2eb31ff700 1 --2- 172.21.15.156:0/4267749628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eac159fa0 0x7f2eac15a440 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.506+0000 7f2eb31ff700 1 --2- 172.21.15.156:0/4267749628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e9c0043f0 0x7f2eac1f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.509 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.506+0000 7f2eb31ff700 1 --2- 172.21.15.156:0/4267749628 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2eac0c4db0 0x7f2eac0b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.509 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.506+0000 7f2eb31ff700 1 -- 172.21.15.156:0/4267749628 >> 172.21.15.156:0/4267749628 conn(0x7f2eac0cb670 msgr2=0x7f2eac06bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:31.509 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.506+0000 7f2eb31ff700 1 -- 172.21.15.156:0/4267749628 shutdown_connections 2023-11-28T13:08:31.509 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.506+0000 7f2eb31ff700 1 -- 172.21.15.156:0/4267749628 wait complete. 2023-11-28T13:08:31.526 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:08:31.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.722+0000 7f7d20dff700 1 Processor -- start 2023-11-28T13:08:31.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.722+0000 7f7d20dff700 1 -- start start 2023-11-28T13:08:31.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.723+0000 7f7d20dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d1c161340 0x7f7d1c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:31.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.724+0000 7f7d20dff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7d1c0d2320 0x7f7d1c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:31.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.724+0000 7f7d20dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1c0d1950 con 0x7f7d1c15c9b0 2023-11-28T13:08:31.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.724+0000 7f7d20dff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1c0b5800 con 0x7f7d1c0d2320 2023-11-28T13:08:31.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.724+0000 7f7d20dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c0b5300 con 0x7f7d1c161340 2023-11-28T13:08:31.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.724+0000 7f7d13fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d1c161340 0x7f7d1c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:31.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.724+0000 7f7d1ad9d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7d1c0d2320 0x7f7d1c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:31.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.724+0000 7f7d1a59c700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f7d1c15c9b0 0x7f7d1c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:45744/0 (socket says 172.21.15.156:45744) 2023-11-28T13:08:31.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.724+0000 7f7d1a59c700 1 -- 172.21.15.156:0/3047024068 learned_addr learned my addr 172.21.15.156:0/3047024068 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:31.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.724+0000 7f7d1ad9d700 1 --2- 172.21.15.156:0/3047024068 >> v2:172.21.15.186:3300/0 conn(0x7f7d1c0d2320 0x7f7d1c0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:57000/0 (socket says 172.21.15.156:57000) 2023-11-28T13:08:31.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.724+0000 7f7d13fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d1c161340 0x7f7d1c0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:54354/0 (socket says 172.21.15.156:54354) 2023-11-28T13:08:31.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.724+0000 7f7d19d9b700 1 -- 172.21.15.156:0/3047024068 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663000717 0 0) 0x7f7d1c0d1950 con 0x7f7d1c15c9b0 2023-11-28T13:08:31.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.724+0000 7f7d19d9b700 1 -- 172.21.15.156:0/3047024068 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d080035b0 con 0x7f7d1c15c9b0 2023-11-28T13:08:31.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.725+0000 7f7d1ad9d700 1 -- 172.21.15.156:0/3047024068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d1c161340 msgr2=0x7f7d1c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:31.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.725+0000 7f7d1ad9d700 1 --2- 172.21.15.156:0/3047024068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d1c161340 0x7f7d1c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.725+0000 7f7d1ad9d700 1 -- 172.21.15.156:0/3047024068 >> v1:172.21.15.186:6789/0 conn(0x7f7d1c15c9b0 legacy=0x7f7d1c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:31.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.725+0000 7f7d1ad9d700 1 -- 172.21.15.156:0/3047024068 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c0b5520 con 0x7f7d1c0d2320 2023-11-28T13:08:31.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.725+0000 7f7d1ad9d700 1 --2- 172.21.15.156:0/3047024068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d1c0d2320 0x7f7d1c0cd7c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009dc0 tx=0x7f7d1400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f25af4f299a6b06 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:31.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.725+0000 7f7d19d9b700 1 -- 172.21.15.156:0/3047024068 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1400e070 con 0x7f7d1c0d2320 2023-11-28T13:08:31.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.725+0000 7f7d19d9b700 1 -- 172.21.15.156:0/3047024068 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1400bae0 con 0x7f7d1c0d2320 2023-11-28T13:08:31.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.725+0000 7f7d19d9b700 1 -- 172.21.15.156:0/3047024068 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1400be90 con 0x7f7d1c0d2320 2023-11-28T13:08:31.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.726+0000 7f7d20dff700 1 -- 172.21.15.156:0/3047024068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d1c0d2320 msgr2=0x7f7d1c0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:31.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.726+0000 7f7d20dff700 1 --2- 172.21.15.156:0/3047024068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d1c0d2320 0x7f7d1c0cd7c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009dc0 tx=0x7f7d1400b330 comp rx=0 tx=0).stop 2023-11-28T13:08:31.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.726+0000 7f7d20dff700 1 -- 172.21.15.156:0/3047024068 shutdown_connections 2023-11-28T13:08:31.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.726+0000 7f7d20dff700 1 --2- 172.21.15.156:0/3047024068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d1c0d2320 0x7f7d1c0cd7c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.726+0000 7f7d20dff700 1 --2- 172.21.15.156:0/3047024068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d1c161340 0x7f7d1c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.726+0000 7f7d20dff700 1 -- 172.21.15.156:0/3047024068 >> 172.21.15.156:0/3047024068 conn(0x7f7d1c072600 msgr2=0x7f7d1c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:31.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.726+0000 7f7d20dff700 1 -- 172.21.15.156:0/3047024068 shutdown_connections 2023-11-28T13:08:31.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.726+0000 7f7d20dff700 1 -- 172.21.15.156:0/3047024068 wait complete. 2023-11-28T13:08:31.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.727+0000 7f7d20dff700 1 Processor -- start 2023-11-28T13:08:31.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.728+0000 7f7d20dff700 1 -- start start 2023-11-28T13:08:31.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.728+0000 7f7d20dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d1c0d2320 0x7f7d1c153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:31.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.728+0000 7f7d20dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d1c15c9b0 0x7f7d1c0c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:31.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.729+0000 7f7d20dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d1c161340 0x7f7d1c1593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:31.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.729+0000 7f7d20dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c152580 con 0x7f7d1c0d2320 2023-11-28T13:08:31.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.729+0000 7f7d20dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c153900 con 0x7f7d1c15c9b0 2023-11-28T13:08:31.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.729+0000 7f7d20dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c1522a0 con 0x7f7d1c161340 2023-11-28T13:08:31.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.729+0000 7f7d1a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d1c0d2320 0x7f7d1c153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:31.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.729+0000 7f7d1a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d1c0d2320 0x7f7d1c153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46490/0 (socket says 172.21.15.156:46490) 2023-11-28T13:08:31.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.729+0000 7f7d1a59c700 1 -- 172.21.15.156:0/174177270 learned_addr learned my addr 172.21.15.156:0/174177270 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:31.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.729+0000 7f7d13fff700 1 --2- 172.21.15.156:0/174177270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d1c15c9b0 0x7f7d1c0c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:31.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.729+0000 7f7d1ad9d700 1 --2- 172.21.15.156:0/174177270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d1c161340 0x7f7d1c1593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:31.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.729+0000 7f7d1a59c700 1 -- 172.21.15.156:0/174177270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d1c161340 msgr2=0x7f7d1c1593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:31.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.729+0000 7f7d1a59c700 1 --2- 172.21.15.156:0/174177270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d1c161340 0x7f7d1c1593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.729+0000 7f7d1a59c700 1 -- 172.21.15.156:0/174177270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d1c15c9b0 msgr2=0x7f7d1c0c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:31.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.729+0000 7f7d1a59c700 1 --2- 172.21.15.156:0/174177270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d1c15c9b0 0x7f7d1c0c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.729+0000 7f7d1a59c700 1 -- 172.21.15.156:0/174177270 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d14009a70 con 0x7f7d1c0d2320 2023-11-28T13:08:31.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.730+0000 7f7d1a59c700 1 --2- 172.21.15.156:0/174177270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d1c0d2320 0x7f7d1c153d20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c001eb0 tx=0x7f7d0c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:31.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.730+0000 7f7d12ffd700 1 -- 172.21.15.156:0/174177270 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c005d70 con 0x7f7d1c0d2320 2023-11-28T13:08:31.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.730+0000 7f7d12ffd700 1 -- 172.21.15.156:0/174177270 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c0070d0 con 0x7f7d1c0d2320 2023-11-28T13:08:31.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.730+0000 7f7d20dff700 1 -- 172.21.15.156:0/174177270 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1c0d15e0 con 0x7f7d1c0d2320 2023-11-28T13:08:31.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.730+0000 7f7d12ffd700 1 -- 172.21.15.156:0/174177270 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c007460 con 0x7f7d1c0d2320 2023-11-28T13:08:31.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.730+0000 7f7d20dff700 1 -- 172.21.15.156:0/174177270 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1c0d1300 con 0x7f7d1c0d2320 2023-11-28T13:08:31.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.731+0000 7f7d20dff700 1 -- 172.21.15.156:0/174177270 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1c15cd80 con 0x7f7d1c0d2320 2023-11-28T13:08:31.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.732+0000 7f7d12ffd700 1 -- 172.21.15.156:0/174177270 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7d0c019030 con 0x7f7d1c0d2320 2023-11-28T13:08:31.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.732+0000 7f7d12ffd700 1 --2- 172.21.15.156:0/174177270 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7cfc0682f0 0x7f7cfc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:31.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.733+0000 7f7d12ffd700 1 -- 172.21.15.156:0/174177270 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d0c085080 con 0x7f7d1c0d2320 2023-11-28T13:08:31.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.733+0000 7f7d13fff700 1 --2- 172.21.15.156:0/174177270 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7cfc0682f0 0x7f7cfc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:31.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.734+0000 7f7d13fff700 1 --2- 172.21.15.156:0/174177270 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7cfc0682f0 0x7f7cfc06a7a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c146f50 tx=0x7f7d04009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:31.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.736+0000 7f7d12ffd700 1 -- 172.21.15.156:0/174177270 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7d0c051850 con 0x7f7d1c0d2320 2023-11-28T13:08:31.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.932+0000 7f7d20dff700 1 -- 172.21.15.156:0/174177270 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d1c1530a0 con 0x7f7d1c0d2320 2023-11-28T13:08:31.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.933+0000 7f7d12ffd700 1 -- 172.21.15.156:0/174177270 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17897 (secure 0 0 0) 0x7f7d1c1530a0 con 0x7f7d1c0d2320 2023-11-28T13:08:31.935 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:31.935 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":21,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:08:04.060053+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:08:00.903485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:08:02.037854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:08:31.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.935+0000 7f7d20dff700 1 -- 172.21.15.156:0/174177270 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7cfc0682f0 msgr2=0x7f7cfc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:31.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.936+0000 7f7d20dff700 1 --2- 172.21.15.156:0/174177270 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7cfc0682f0 0x7f7cfc06a7a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c146f50 tx=0x7f7d04009210 comp rx=0 tx=0).stop 2023-11-28T13:08:31.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.936+0000 7f7d20dff700 1 -- 172.21.15.156:0/174177270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d1c0d2320 msgr2=0x7f7d1c153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:31.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.936+0000 7f7d20dff700 1 --2- 172.21.15.156:0/174177270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d1c0d2320 0x7f7d1c153d20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c001eb0 tx=0x7f7d0c00c7b0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.936+0000 7f7d20dff700 1 -- 172.21.15.156:0/174177270 shutdown_connections 2023-11-28T13:08:31.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.936+0000 7f7d20dff700 1 --2- 172.21.15.156:0/174177270 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7cfc0682f0 0x7f7cfc06a7a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.936+0000 7f7d20dff700 1 --2- 172.21.15.156:0/174177270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d1c161340 0x7f7d1c1593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.936+0000 7f7d20dff700 1 --2- 172.21.15.156:0/174177270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d1c0d2320 0x7f7d1c153d20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.936+0000 7f7d20dff700 1 --2- 172.21.15.156:0/174177270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d1c15c9b0 0x7f7d1c0c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:31.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.936+0000 7f7d20dff700 1 -- 172.21.15.156:0/174177270 >> 172.21.15.156:0/174177270 conn(0x7f7d1c072600 msgr2=0x7f7d1c15fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:31.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.936+0000 7f7d20dff700 1 -- 172.21.15.156:0/174177270 shutdown_connections 2023-11-28T13:08:31.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:31.936+0000 7f7d20dff700 1 -- 172.21.15.156:0/174177270 wait complete. 2023-11-28T13:08:31.956 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:08:00.903485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:08:02.037854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:08:31.958 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:08:32.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.155+0000 7f5968dff700 1 Processor -- start 2023-11-28T13:08:32.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.155+0000 7f5968dff700 1 -- start start 2023-11-28T13:08:32.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.156+0000 7f5968dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59640d01b0 0x7f59640d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:32.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.156+0000 7f5968dff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f59640b20b0 0x7f59640d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:32.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.157+0000 7f5968dff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f59640b29b0 0x7f59640b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:32.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.157+0000 7f5968dff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596415cec0 con 0x7f59640b20b0 2023-11-28T13:08:32.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.157+0000 7f5968dff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59640b0230 con 0x7f59640b29b0 2023-11-28T13:08:32.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.157+0000 7f5968dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59640b4570 con 0x7f59640d01b0 2023-11-28T13:08:32.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.157+0000 7f595bfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f59640b20b0 0x7f59640d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:32.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.157+0000 7f595bfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f59640b20b0 0x7f59640d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46496/0 (socket says 172.21.15.156:46496) 2023-11-28T13:08:32.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.157+0000 7f595bfff700 1 -- 172.21.15.156:0/3672372419 learned_addr learned my addr 172.21.15.156:0/3672372419 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:32.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.157+0000 7f5962d9d700 1 --2- 172.21.15.156:0/3672372419 >> v2:172.21.15.186:3300/0 conn(0x7f59640b29b0 0x7f59640b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:32.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.157+0000 7f596259c700 1 --2- 172.21.15.156:0/3672372419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59640d01b0 0x7f59640d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:32.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.157+0000 7f595bfff700 1 -- 172.21.15.156:0/3672372419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59640d01b0 msgr2=0x7f59640d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:32.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.157+0000 7f595bfff700 1 --2- 172.21.15.156:0/3672372419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59640d01b0 0x7f59640d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.157+0000 7f595bfff700 1 -- 172.21.15.156:0/3672372419 >> v2:172.21.15.186:3300/0 conn(0x7f59640b29b0 msgr2=0x7f59640b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:32.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.157+0000 7f595bfff700 1 --2- 172.21.15.156:0/3672372419 >> v2:172.21.15.186:3300/0 conn(0x7f59640b29b0 0x7f59640b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.157+0000 7f595bfff700 1 -- 172.21.15.156:0/3672372419 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59640b4850 con 0x7f59640b20b0 2023-11-28T13:08:32.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.157+0000 7f5962d9d700 1 --2- 172.21.15.156:0/3672372419 >> v2:172.21.15.186:3300/0 conn(0x7f59640b29b0 0x7f59640b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:32.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.157+0000 7f596259c700 1 --2- 172.21.15.156:0/3672372419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59640d01b0 0x7f59640d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:32.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.158+0000 7f595bfff700 1 --2- 172.21.15.156:0/3672372419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59640b20b0 0x7f59640d1160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5964066e80 tx=0x7f594c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd2ee7ae5049f05 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:32.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.158+0000 7f5961d9b700 1 -- 172.21.15.156:0/3672372419 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c00b870 con 0x7f59640b20b0 2023-11-28T13:08:32.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.158+0000 7f5961d9b700 1 -- 172.21.15.156:0/3672372419 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c00b9d0 con 0x7f59640b20b0 2023-11-28T13:08:32.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.158+0000 7f5961d9b700 1 -- 172.21.15.156:0/3672372419 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c00bd60 con 0x7f59640b20b0 2023-11-28T13:08:32.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.158+0000 7f5968dff700 1 -- 172.21.15.156:0/3672372419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59640b20b0 msgr2=0x7f59640d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:32.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.158+0000 7f5968dff700 1 --2- 172.21.15.156:0/3672372419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59640b20b0 0x7f59640d1160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5964066e80 tx=0x7f594c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.158+0000 7f5968dff700 1 -- 172.21.15.156:0/3672372419 shutdown_connections 2023-11-28T13:08:32.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.158+0000 7f5968dff700 1 --2- 172.21.15.156:0/3672372419 >> v2:172.21.15.186:3300/0 conn(0x7f59640b29b0 0x7f59640b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.158+0000 7f5968dff700 1 --2- 172.21.15.156:0/3672372419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59640d01b0 0x7f59640d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.158+0000 7f5968dff700 1 --2- 172.21.15.156:0/3672372419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59640b20b0 0x7f59640d1160 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.158+0000 7f5968dff700 1 -- 172.21.15.156:0/3672372419 >> 172.21.15.156:0/3672372419 conn(0x7f596409c3f0 msgr2=0x7f59640d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:32.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.159+0000 7f5968dff700 1 -- 172.21.15.156:0/3672372419 shutdown_connections 2023-11-28T13:08:32.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.159+0000 7f5968dff700 1 -- 172.21.15.156:0/3672372419 wait complete. 2023-11-28T13:08:32.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.160+0000 7f5968dff700 1 Processor -- start 2023-11-28T13:08:32.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.160+0000 7f5968dff700 1 -- start start 2023-11-28T13:08:32.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.161+0000 7f5968dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59640b20b0 0x7f59640b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.161+0000 7f5968dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59640b29b0 0x7f59640cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.161+0000 7f5968dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59640d01b0 0x7f59640cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.161+0000 7f5968dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59640b6000 con 0x7f59640b20b0 2023-11-28T13:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.161+0000 7f5968dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59640b62e0 con 0x7f59640b29b0 2023-11-28T13:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.161+0000 7f5968dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59640b65c0 con 0x7f59640d01b0 2023-11-28T13:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.162+0000 7f596259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59640b20b0 0x7f59640b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.162+0000 7f596259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59640b20b0 0x7f59640b70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46508/0 (socket says 172.21.15.156:46508) 2023-11-28T13:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.162+0000 7f596259c700 1 -- 172.21.15.156:0/1313900495 learned_addr learned my addr 172.21.15.156:0/1313900495 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.162+0000 7f5962d9d700 1 --2- 172.21.15.156:0/1313900495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59640d01b0 0x7f59640cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:32.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.162+0000 7f595bfff700 1 --2- 172.21.15.156:0/1313900495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59640b29b0 0x7f59640cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:32.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.162+0000 7f596259c700 1 -- 172.21.15.156:0/1313900495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59640d01b0 msgr2=0x7f59640cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:32.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.162+0000 7f596259c700 1 --2- 172.21.15.156:0/1313900495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59640d01b0 0x7f59640cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.162+0000 7f596259c700 1 -- 172.21.15.156:0/1313900495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59640b29b0 msgr2=0x7f59640cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:32.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.162+0000 7f596259c700 1 --2- 172.21.15.156:0/1313900495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59640b29b0 0x7f59640cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.162+0000 7f596259c700 1 -- 172.21.15.156:0/1313900495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c009550 con 0x7f59640b20b0 2023-11-28T13:08:32.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.162+0000 7f595bfff700 1 --2- 172.21.15.156:0/1313900495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59640b29b0 0x7f59640cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:32.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.162+0000 7f596259c700 1 --2- 172.21.15.156:0/1313900495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59640b20b0 0x7f59640b70a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f595400c7f0 tx=0x7f595400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:32.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.162+0000 7f595affd700 1 -- 172.21.15.156:0/1313900495 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954011800 con 0x7f59640b20b0 2023-11-28T13:08:32.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.162+0000 7f595affd700 1 -- 172.21.15.156:0/1313900495 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5954011960 con 0x7f59640b20b0 2023-11-28T13:08:32.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.162+0000 7f595affd700 1 -- 172.21.15.156:0/1313900495 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954011d10 con 0x7f59640b20b0 2023-11-28T13:08:32.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.163+0000 7f5968dff700 1 -- 172.21.15.156:0/1313900495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59640afec0 con 0x7f59640b20b0 2023-11-28T13:08:32.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.163+0000 7f5968dff700 1 -- 172.21.15.156:0/1313900495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59640afbe0 con 0x7f59640b20b0 2023-11-28T13:08:32.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.164+0000 7f5968dff700 1 -- 172.21.15.156:0/1313900495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59640b2600 con 0x7f59640b20b0 2023-11-28T13:08:32.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.164+0000 7f595affd700 1 -- 172.21.15.156:0/1313900495 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5954021070 con 0x7f59640b20b0 2023-11-28T13:08:32.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.165+0000 7f595affd700 1 --2- 172.21.15.156:0/1313900495 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f59440682f0 0x7f594406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:32.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.165+0000 7f595affd700 1 -- 172.21.15.156:0/1313900495 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5954085b10 con 0x7f59640b20b0 2023-11-28T13:08:32.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.166+0000 7f595bfff700 1 --2- 172.21.15.156:0/1313900495 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f59440682f0 0x7f594406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:32.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.166+0000 7f595bfff700 1 --2- 172.21.15.156:0/1313900495 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f59440682f0 0x7f594406a7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f59640a0f30 tx=0x7f594c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:32.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.169+0000 7f595affd700 1 -- 172.21.15.156:0/1313900495 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5954052370 con 0x7f59640b20b0 2023-11-28T13:08:32.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.358+0000 7f5968dff700 1 -- 172.21.15.156:0/1313900495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5964068690 con 0x7f59640b20b0 2023-11-28T13:08:32.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.359+0000 7f595affd700 1 -- 172.21.15.156:0/1313900495 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5954055990 con 0x7f59640b20b0 2023-11-28T13:08:32.361 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:08:32.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.361+0000 7f5968dff700 1 -- 172.21.15.156:0/1313900495 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f59440682f0 msgr2=0x7f594406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:32.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.361+0000 7f5968dff700 1 --2- 172.21.15.156:0/1313900495 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f59440682f0 0x7f594406a7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f59640a0f30 tx=0x7f594c008030 comp rx=0 tx=0).stop 2023-11-28T13:08:32.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.361+0000 7f5968dff700 1 -- 172.21.15.156:0/1313900495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59640b20b0 msgr2=0x7f59640b70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:32.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.361+0000 7f5968dff700 1 --2- 172.21.15.156:0/1313900495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59640b20b0 0x7f59640b70a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f595400c7f0 tx=0x7f595400cbb0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.361+0000 7f5968dff700 1 -- 172.21.15.156:0/1313900495 shutdown_connections 2023-11-28T13:08:32.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.361+0000 7f5968dff700 1 --2- 172.21.15.156:0/1313900495 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f59440682f0 0x7f594406a7a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.361+0000 7f5968dff700 1 --2- 172.21.15.156:0/1313900495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59640d01b0 0x7f59640cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.361+0000 7f5968dff700 1 --2- 172.21.15.156:0/1313900495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59640b20b0 0x7f59640b70a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.361+0000 7f5968dff700 1 --2- 172.21.15.156:0/1313900495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59640b29b0 0x7f59640cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.361+0000 7f5968dff700 1 -- 172.21.15.156:0/1313900495 >> 172.21.15.156:0/1313900495 conn(0x7f596409c3f0 msgr2=0x7f59640cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:32.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.361+0000 7f5968dff700 1 -- 172.21.15.156:0/1313900495 shutdown_connections 2023-11-28T13:08:32.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.361+0000 7f5968dff700 1 -- 172.21.15.156:0/1313900495 wait complete. 2023-11-28T13:08:32.381 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:08:32.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.581+0000 7f586dfff700 1 Processor -- start 2023-11-28T13:08:32.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.582+0000 7f586dfff700 1 -- start start 2023-11-28T13:08:32.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.583+0000 7f586dfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f58680b20b0 0x7f58680b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:32.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.583+0000 7f586dfff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58680b0230 con 0x7f58680d1330 2023-11-28T13:08:32.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.583+0000 7f586dfff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58680b4570 con 0x7f58680d01b0 2023-11-28T13:08:32.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.583+0000 7f586dfff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f586815cec0 con 0x7f58680b20b0 2023-11-28T13:08:32.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.583+0000 7f5867fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f58680b20b0 0x7f58680b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:32.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.583+0000 7f5867fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f58680b20b0 0x7f58680b0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46520/0 (socket says 172.21.15.156:46520) 2023-11-28T13:08:32.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.583+0000 7f5867fff700 1 -- 172.21.15.156:0/4007544631 learned_addr learned my addr 172.21.15.156:0/4007544631 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:32.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.583+0000 7f585ffff700 1 --1- 172.21.15.156:0/4007544631 >> v1:172.21.15.156:6789/0 conn(0x7f58680d1330 0x7f58680b06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51004/0 (socket says 172.21.15.156:51004) 2023-11-28T13:08:32.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.584+0000 7f5867fff700 1 -- 172.21.15.156:0/4007544631 >> v1:172.21.15.186:6789/0 conn(0x7f58680d01b0 legacy=0x7f58680d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:32.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.584+0000 7f5867fff700 1 -- 172.21.15.156:0/4007544631 >> v1:172.21.15.156:6789/0 conn(0x7f58680d1330 legacy=0x7f58680b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:32.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.584+0000 7f5867fff700 1 -- 172.21.15.156:0/4007544631 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58680b4850 con 0x7f58680b20b0 2023-11-28T13:08:32.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.584+0000 7f5866ffd700 1 -- 172.21.15.156:0/4007544631 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 475024651 0 0) 0x7f58680b0230 con 0x7f58680d1330 2023-11-28T13:08:32.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.584+0000 7f5867fff700 1 --2- 172.21.15.156:0/4007544631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58680b20b0 0x7f58680b0dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5860009e50 tx=0x7f586000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e46ae784bf26a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:32.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.584+0000 7f5866ffd700 1 -- 172.21.15.156:0/4007544631 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586000e070 con 0x7f58680b20b0 2023-11-28T13:08:32.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.584+0000 7f5866ffd700 1 -- 172.21.15.156:0/4007544631 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586000b9c0 con 0x7f58680b20b0 2023-11-28T13:08:32.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.584+0000 7f5866ffd700 1 -- 172.21.15.156:0/4007544631 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586000bd70 con 0x7f58680b20b0 2023-11-28T13:08:32.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.585+0000 7f586dfff700 1 -- 172.21.15.156:0/4007544631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58680b20b0 msgr2=0x7f58680b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:32.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.585+0000 7f586dfff700 1 --2- 172.21.15.156:0/4007544631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58680b20b0 0x7f58680b0dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5860009e50 tx=0x7f586000b330 comp rx=0 tx=0).stop 2023-11-28T13:08:32.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.585+0000 7f586dfff700 1 -- 172.21.15.156:0/4007544631 shutdown_connections 2023-11-28T13:08:32.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.585+0000 7f586dfff700 1 --2- 172.21.15.156:0/4007544631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58680b20b0 0x7f58680b0dd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.585+0000 7f586dfff700 1 -- 172.21.15.156:0/4007544631 >> 172.21.15.156:0/4007544631 conn(0x7f586809c3f0 msgr2=0x7f58680d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:32.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.585+0000 7f586dfff700 1 -- 172.21.15.156:0/4007544631 shutdown_connections 2023-11-28T13:08:32.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.585+0000 7f586dfff700 1 -- 172.21.15.156:0/4007544631 wait complete. 2023-11-28T13:08:32.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.586+0000 7f586dfff700 1 Processor -- start 2023-11-28T13:08:32.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.587+0000 7f586dfff700 1 -- start start 2023-11-28T13:08:32.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.587+0000 7f586dfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58680b20b0 0x7f58680b8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:32.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.588+0000 7f586dfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58680d01b0 0x7f58680ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:32.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.588+0000 7f586dfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58680d1330 0x7f58680cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:32.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.588+0000 7f586dfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58680b7b80 con 0x7f58680d01b0 2023-11-28T13:08:32.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.588+0000 7f586dfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58680b6f70 con 0x7f58680b20b0 2023-11-28T13:08:32.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.588+0000 7f586dfff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58680b7250 con 0x7f58680d1330 2023-11-28T13:08:32.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.588+0000 7f585ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58680d01b0 0x7f58680ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:32.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.588+0000 7f585ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58680d01b0 0x7f58680ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46528/0 (socket says 172.21.15.156:46528) 2023-11-28T13:08:32.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.588+0000 7f585ffff700 1 -- 172.21.15.156:0/729210924 learned_addr learned my addr 172.21.15.156:0/729210924 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:32.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.588+0000 7f58677fe700 1 --2- 172.21.15.156:0/729210924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58680b20b0 0x7f58680b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:32.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.588+0000 7f585ffff700 1 -- 172.21.15.156:0/729210924 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58680d1330 msgr2=0x7f58680cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:32.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.588+0000 7f585ffff700 1 --2- 172.21.15.156:0/729210924 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58680d1330 0x7f58680cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.588+0000 7f585ffff700 1 -- 172.21.15.156:0/729210924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58680b20b0 msgr2=0x7f58680b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:32.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.588+0000 7f585ffff700 1 --2- 172.21.15.156:0/729210924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58680b20b0 0x7f58680b8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.588+0000 7f585ffff700 1 -- 172.21.15.156:0/729210924 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5860009b30 con 0x7f58680d01b0 2023-11-28T13:08:32.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.588+0000 7f58677fe700 1 --2- 172.21.15.156:0/729210924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58680b20b0 0x7f58680b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:32.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.588+0000 7f585ffff700 1 --2- 172.21.15.156:0/729210924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58680d01b0 0x7f58680ccb40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f585000c450 tx=0x7f585000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:32.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.589+0000 7f5864ff9700 1 -- 172.21.15.156:0/729210924 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850004c80 con 0x7f58680d01b0 2023-11-28T13:08:32.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.589+0000 7f5864ff9700 1 -- 172.21.15.156:0/729210924 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5850004e00 con 0x7f58680d01b0 2023-11-28T13:08:32.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.589+0000 7f586dfff700 1 -- 172.21.15.156:0/729210924 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58680afec0 con 0x7f58680d01b0 2023-11-28T13:08:32.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.589+0000 7f5864ff9700 1 -- 172.21.15.156:0/729210924 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850007310 con 0x7f58680d01b0 2023-11-28T13:08:32.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.589+0000 7f586dfff700 1 -- 172.21.15.156:0/729210924 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58680afbe0 con 0x7f58680d01b0 2023-11-28T13:08:32.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.590+0000 7f586dfff700 1 -- 172.21.15.156:0/729210924 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58680b2600 con 0x7f58680d01b0 2023-11-28T13:08:32.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.591+0000 7f5864ff9700 1 -- 172.21.15.156:0/729210924 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f585001a020 con 0x7f58680d01b0 2023-11-28T13:08:32.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.591+0000 7f5864ff9700 1 --2- 172.21.15.156:0/729210924 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f58480682f0 0x7f584806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:32.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.592+0000 7f5864ff9700 1 -- 172.21.15.156:0/729210924 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5850085910 con 0x7f58680d01b0 2023-11-28T13:08:32.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.592+0000 7f58677fe700 1 --2- 172.21.15.156:0/729210924 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f58480682f0 0x7f584806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:32.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.593+0000 7f58677fe700 1 --2- 172.21.15.156:0/729210924 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f58480682f0 0x7f584806a7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5858000f60 tx=0x7f5858009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:32.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.596+0000 7f5864ff9700 1 -- 172.21.15.156:0/729210924 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5850052020 con 0x7f58680d01b0 2023-11-28T13:08:32.755 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:08:32.761 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:08:32.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.794+0000 7f586dfff700 1 -- 172.21.15.156:0/729210924 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5868068690 con 0x7f58680d01b0 2023-11-28T13:08:32.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.795+0000 7f5864ff9700 1 -- 172.21.15.156:0/729210924 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5850055640 con 0x7f58680d01b0 2023-11-28T13:08:32.797 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:08:32.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.797+0000 7f586dfff700 1 -- 172.21.15.156:0/729210924 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f58480682f0 msgr2=0x7f584806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:32.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.797+0000 7f586dfff700 1 --2- 172.21.15.156:0/729210924 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f58480682f0 0x7f584806a7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5858000f60 tx=0x7f5858009040 comp rx=0 tx=0).stop 2023-11-28T13:08:32.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.797+0000 7f586dfff700 1 -- 172.21.15.156:0/729210924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58680d01b0 msgr2=0x7f58680ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:32.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.797+0000 7f586dfff700 1 --2- 172.21.15.156:0/729210924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58680d01b0 0x7f58680ccb40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f585000c450 tx=0x7f585000c810 comp rx=0 tx=0).stop 2023-11-28T13:08:32.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.797+0000 7f586dfff700 1 -- 172.21.15.156:0/729210924 shutdown_connections 2023-11-28T13:08:32.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.797+0000 7f586dfff700 1 --2- 172.21.15.156:0/729210924 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f58480682f0 0x7f584806a7a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.797+0000 7f586dfff700 1 --2- 172.21.15.156:0/729210924 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58680d1330 0x7f58680cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.797+0000 7f586dfff700 1 --2- 172.21.15.156:0/729210924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58680b20b0 0x7f58680b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.797+0000 7f586dfff700 1 --2- 172.21.15.156:0/729210924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58680d01b0 0x7f58680ccb40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:32.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.797+0000 7f586dfff700 1 -- 172.21.15.156:0/729210924 >> 172.21.15.156:0/729210924 conn(0x7f586809c3f0 msgr2=0x7f58680b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:32.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.797+0000 7f586dfff700 1 -- 172.21.15.156:0/729210924 shutdown_connections 2023-11-28T13:08:32.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:32.798+0000 7f586dfff700 1 -- 172.21.15.156:0/729210924 wait complete. 2023-11-28T13:08:32.819 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:08:33.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.058+0000 7f9a509ff700 1 Processor -- start 2023-11-28T13:08:33.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.059+0000 7f9a509ff700 1 -- start start 2023-11-28T13:08:33.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.060+0000 7f9a509ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9a4c15c9b0 0x7f9a4c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.060+0000 7f9a509ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a4c0cd7c0 0x7f9a4c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.060+0000 7f9a509ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4c0d1950 con 0x7f9a4c161340 2023-11-28T13:08:33.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.060+0000 7f9a509ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a4c0b5800 con 0x7f9a4c15c9b0 2023-11-28T13:08:33.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.060+0000 7f9a509ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0ce140 con 0x7f9a4c0cd7c0 2023-11-28T13:08:33.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.060+0000 7f9a4a59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9a4c15c9b0 0x7f9a4c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.060+0000 7f9a4a59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9a4c15c9b0 0x7f9a4c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46534/0 (socket says 172.21.15.156:46534) 2023-11-28T13:08:33.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.060+0000 7f9a4a59c700 1 -- 172.21.15.156:0/1421432805 learned_addr learned my addr 172.21.15.156:0/1421432805 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:33.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.060+0000 7f9a4ad9d700 1 --2- 172.21.15.156:0/1421432805 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a4c0cd7c0 0x7f9a4c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.060+0000 7f9a4a59c700 1 -- 172.21.15.156:0/1421432805 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a4c0cd7c0 msgr2=0x7f9a4c0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:33.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.060+0000 7f9a4a59c700 1 --2- 172.21.15.156:0/1421432805 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a4c0cd7c0 0x7f9a4c0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.060+0000 7f9a4a59c700 1 -- 172.21.15.156:0/1421432805 >> v1:172.21.15.186:6789/0 conn(0x7f9a4c161340 legacy=0x7f9a4c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:33.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.060+0000 7f9a4a59c700 1 -- 172.21.15.156:0/1421432805 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4c0d24f0 con 0x7f9a4c15c9b0 2023-11-28T13:08:33.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.061+0000 7f9a4a59c700 1 --2- 172.21.15.156:0/1421432805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c15c9b0 0x7f9a4c0b4bf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c0098f0 tx=0x7f9a3c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd43a09ef0f57e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:33.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.061+0000 7f9a49d9b700 1 -- 172.21.15.156:0/1421432805 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c00b8a0 con 0x7f9a4c15c9b0 2023-11-28T13:08:33.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.061+0000 7f9a49d9b700 1 -- 172.21.15.156:0/1421432805 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c00ba00 con 0x7f9a4c15c9b0 2023-11-28T13:08:33.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.061+0000 7f9a49d9b700 1 -- 172.21.15.156:0/1421432805 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c00bd90 con 0x7f9a4c15c9b0 2023-11-28T13:08:33.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.061+0000 7f9a509ff700 1 -- 172.21.15.156:0/1421432805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c15c9b0 msgr2=0x7f9a4c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:33.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.061+0000 7f9a509ff700 1 --2- 172.21.15.156:0/1421432805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c15c9b0 0x7f9a4c0b4bf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c0098f0 tx=0x7f9a3c009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:33.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.062+0000 7f9a509ff700 1 -- 172.21.15.156:0/1421432805 shutdown_connections 2023-11-28T13:08:33.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.062+0000 7f9a509ff700 1 --2- 172.21.15.156:0/1421432805 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a4c0cd7c0 0x7f9a4c0cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.062+0000 7f9a509ff700 1 --2- 172.21.15.156:0/1421432805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c15c9b0 0x7f9a4c0b4bf0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.062+0000 7f9a509ff700 1 -- 172.21.15.156:0/1421432805 >> 172.21.15.156:0/1421432805 conn(0x7f9a4c072600 msgr2=0x7f9a4c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:33.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.062+0000 7f9a509ff700 1 -- 172.21.15.156:0/1421432805 shutdown_connections 2023-11-28T13:08:33.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.062+0000 7f9a509ff700 1 -- 172.21.15.156:0/1421432805 wait complete. 2023-11-28T13:08:33.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.063+0000 7f9a509ff700 1 Processor -- start 2023-11-28T13:08:33.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.063+0000 7f9a509ff700 1 -- start start 2023-11-28T13:08:33.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.064+0000 7f9a509ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c0cd7c0 0x7f9a4c1e6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.064+0000 7f9a509ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a4c15c9b0 0x7f9a4c15ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.064+0000 7f9a509ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a4c161340 0x7f9a4c154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.064+0000 7f9a509ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c1e7770 con 0x7f9a4c0cd7c0 2023-11-28T13:08:33.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.064+0000 7f9a509ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c1e7a50 con 0x7f9a4c161340 2023-11-28T13:08:33.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.064+0000 7f9a509ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c1e7d30 con 0x7f9a4c15c9b0 2023-11-28T13:08:33.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.065+0000 7f9a4a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c0cd7c0 0x7f9a4c1e6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.065+0000 7f9a4a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c0cd7c0 0x7f9a4c1e6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46542/0 (socket says 172.21.15.156:46542) 2023-11-28T13:08:33.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.065+0000 7f9a4a59c700 1 -- 172.21.15.156:0/572379577 learned_addr learned my addr 172.21.15.156:0/572379577 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:33.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.065+0000 7f9a43fff700 1 --2- 172.21.15.156:0/572379577 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a4c15c9b0 0x7f9a4c15ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.065+0000 7f9a4ad9d700 1 --2- 172.21.15.156:0/572379577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a4c161340 0x7f9a4c154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.065+0000 7f9a4a59c700 1 -- 172.21.15.156:0/572379577 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a4c15c9b0 msgr2=0x7f9a4c15ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:33.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.065+0000 7f9a4a59c700 1 --2- 172.21.15.156:0/572379577 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a4c15c9b0 0x7f9a4c15ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.065+0000 7f9a4a59c700 1 -- 172.21.15.156:0/572379577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a4c161340 msgr2=0x7f9a4c154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:33.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.065+0000 7f9a4a59c700 1 --2- 172.21.15.156:0/572379577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a4c161340 0x7f9a4c154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.065+0000 7f9a4a59c700 1 -- 172.21.15.156:0/572379577 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c009550 con 0x7f9a4c0cd7c0 2023-11-28T13:08:33.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.065+0000 7f9a4a59c700 1 --2- 172.21.15.156:0/572379577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c0cd7c0 0x7f9a4c1e6e60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c009c40 tx=0x7f9a3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:33.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.066+0000 7f9a42ffd700 1 -- 172.21.15.156:0/572379577 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c004510 con 0x7f9a4c0cd7c0 2023-11-28T13:08:33.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.066+0000 7f9a42ffd700 1 -- 172.21.15.156:0/572379577 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c01e070 con 0x7f9a4c0cd7c0 2023-11-28T13:08:33.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.066+0000 7f9a42ffd700 1 -- 172.21.15.156:0/572379577 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c0038e0 con 0x7f9a4c0cd7c0 2023-11-28T13:08:33.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.066+0000 7f9a509ff700 1 -- 172.21.15.156:0/572379577 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4c0d1300 con 0x7f9a4c0cd7c0 2023-11-28T13:08:33.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.066+0000 7f9a509ff700 1 -- 172.21.15.156:0/572379577 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4c0b5520 con 0x7f9a4c0cd7c0 2023-11-28T13:08:33.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.067+0000 7f9a509ff700 1 -- 172.21.15.156:0/572379577 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4c0d15e0 con 0x7f9a4c0cd7c0 2023-11-28T13:08:33.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.068+0000 7f9a42ffd700 1 -- 172.21.15.156:0/572379577 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9a3c00f050 con 0x7f9a4c0cd7c0 2023-11-28T13:08:33.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.069+0000 7f9a42ffd700 1 --2- 172.21.15.156:0/572379577 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9a2c0682f0 0x7f9a2c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.069+0000 7f9a42ffd700 1 -- 172.21.15.156:0/572379577 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9a3c020070 con 0x7f9a4c0cd7c0 2023-11-28T13:08:33.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.069+0000 7f9a43fff700 1 --2- 172.21.15.156:0/572379577 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9a2c0682f0 0x7f9a2c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.070+0000 7f9a43fff700 1 --2- 172.21.15.156:0/572379577 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9a2c0682f0 0x7f9a2c06a7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c154d40 tx=0x7f9a34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:33.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.073+0000 7f9a42ffd700 1 -- 172.21.15.156:0/572379577 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9a3c053870 con 0x7f9a4c0cd7c0 2023-11-28T13:08:33.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.263+0000 7f9a509ff700 1 -- 172.21.15.156:0/572379577 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9a4c0745c0 con 0x7f9a4c0cd7c0 2023-11-28T13:08:33.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.263+0000 7f9a42ffd700 1 -- 172.21.15.156:0/572379577 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9a3c056e90 con 0x7f9a4c0cd7c0 2023-11-28T13:08:33.266 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T13:08:33.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.265+0000 7f9a509ff700 1 -- 172.21.15.156:0/572379577 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9a2c0682f0 msgr2=0x7f9a2c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:33.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.265+0000 7f9a509ff700 1 --2- 172.21.15.156:0/572379577 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9a2c0682f0 0x7f9a2c06a7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c154d40 tx=0x7f9a34009040 comp rx=0 tx=0).stop 2023-11-28T13:08:33.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.265+0000 7f9a509ff700 1 -- 172.21.15.156:0/572379577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c0cd7c0 msgr2=0x7f9a4c1e6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:33.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.265+0000 7f9a509ff700 1 --2- 172.21.15.156:0/572379577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c0cd7c0 0x7f9a4c1e6e60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c009c40 tx=0x7f9a3c004750 comp rx=0 tx=0).stop 2023-11-28T13:08:33.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.266+0000 7f9a509ff700 1 -- 172.21.15.156:0/572379577 shutdown_connections 2023-11-28T13:08:33.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.266+0000 7f9a509ff700 1 --2- 172.21.15.156:0/572379577 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9a2c0682f0 0x7f9a2c06a7a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.266+0000 7f9a509ff700 1 --2- 172.21.15.156:0/572379577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a4c161340 0x7f9a4c154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.266+0000 7f9a509ff700 1 --2- 172.21.15.156:0/572379577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c0cd7c0 0x7f9a4c1e6e60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.266+0000 7f9a509ff700 1 --2- 172.21.15.156:0/572379577 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9a4c15c9b0 0x7f9a4c15ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.266+0000 7f9a509ff700 1 -- 172.21.15.156:0/572379577 >> 172.21.15.156:0/572379577 conn(0x7f9a4c072600 msgr2=0x7f9a4c06a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:33.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.266+0000 7f9a509ff700 1 -- 172.21.15.156:0/572379577 shutdown_connections 2023-11-28T13:08:33.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.266+0000 7f9a509ff700 1 -- 172.21.15.156:0/572379577 wait complete. 2023-11-28T13:08:33.285 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:08:33.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.486+0000 7f1c66fff700 1 Processor -- start 2023-11-28T13:08:33.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.486+0000 7f1c66fff700 1 -- start start 2023-11-28T13:08:33.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.487+0000 7f1c66fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1c6015c9b0 0x7f1c600b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.487+0000 7f1c66fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c600cd7c0 0x7f1c600cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.487+0000 7f1c66fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c600d1950 con 0x7f1c60161340 2023-11-28T13:08:33.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.487+0000 7f1c66fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c600b5800 con 0x7f1c6015c9b0 2023-11-28T13:08:33.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.487+0000 7f1c66fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c600ce140 con 0x7f1c600cd7c0 2023-11-28T13:08:33.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.487+0000 7f1c64d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1c6015c9b0 0x7f1c600b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.487+0000 7f1c64d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1c6015c9b0 0x7f1c600b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46554/0 (socket says 172.21.15.156:46554) 2023-11-28T13:08:33.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.487+0000 7f1c64d9b700 1 -- 172.21.15.156:0/1344078517 learned_addr learned my addr 172.21.15.156:0/1344078517 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:33.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.487+0000 7f1c6559c700 1 --2- 172.21.15.156:0/1344078517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c600cd7c0 0x7f1c600cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.488+0000 7f1c64d9b700 1 -- 172.21.15.156:0/1344078517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c600cd7c0 msgr2=0x7f1c600cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:33.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.488+0000 7f1c64d9b700 1 --2- 172.21.15.156:0/1344078517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c600cd7c0 0x7f1c600cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.488+0000 7f1c64d9b700 1 -- 172.21.15.156:0/1344078517 >> v1:172.21.15.186:6789/0 conn(0x7f1c60161340 legacy=0x7f1c600d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:33.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.488+0000 7f1c64d9b700 1 -- 172.21.15.156:0/1344078517 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c600d24f0 con 0x7f1c6015c9b0 2023-11-28T13:08:33.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.488+0000 7f1c64d9b700 1 --2- 172.21.15.156:0/1344078517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c6015c9b0 0x7f1c600b4bf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1c540098f0 tx=0x7f1c54009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5299bf408d50a578 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:33.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.488+0000 7f1c5f7fe700 1 -- 172.21.15.156:0/1344078517 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5400b8a0 con 0x7f1c6015c9b0 2023-11-28T13:08:33.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.488+0000 7f1c5f7fe700 1 -- 172.21.15.156:0/1344078517 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5400ba00 con 0x7f1c6015c9b0 2023-11-28T13:08:33.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.488+0000 7f1c5f7fe700 1 -- 172.21.15.156:0/1344078517 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5400bd90 con 0x7f1c6015c9b0 2023-11-28T13:08:33.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.489+0000 7f1c66fff700 1 -- 172.21.15.156:0/1344078517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c6015c9b0 msgr2=0x7f1c600b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:33.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.489+0000 7f1c66fff700 1 --2- 172.21.15.156:0/1344078517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c6015c9b0 0x7f1c600b4bf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1c540098f0 tx=0x7f1c54009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:33.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.489+0000 7f1c66fff700 1 -- 172.21.15.156:0/1344078517 shutdown_connections 2023-11-28T13:08:33.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.489+0000 7f1c66fff700 1 --2- 172.21.15.156:0/1344078517 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c600cd7c0 0x7f1c600cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.489+0000 7f1c66fff700 1 --2- 172.21.15.156:0/1344078517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c6015c9b0 0x7f1c600b4bf0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.489+0000 7f1c66fff700 1 -- 172.21.15.156:0/1344078517 >> 172.21.15.156:0/1344078517 conn(0x7f1c60072600 msgr2=0x7f1c600cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:33.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.489+0000 7f1c66fff700 1 -- 172.21.15.156:0/1344078517 shutdown_connections 2023-11-28T13:08:33.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.489+0000 7f1c66fff700 1 -- 172.21.15.156:0/1344078517 wait complete. 2023-11-28T13:08:33.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.491+0000 7f1c66fff700 1 Processor -- start 2023-11-28T13:08:33.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.491+0000 7f1c66fff700 1 -- start start 2023-11-28T13:08:33.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.492+0000 7f1c66fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c600cd7c0 0x7f1c601e6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.492+0000 7f1c66fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c6015c9b0 0x7f1c6015ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.492+0000 7f1c66fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c60161340 0x7f1c60154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.492+0000 7f1c66fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c601e7770 con 0x7f1c600cd7c0 2023-11-28T13:08:33.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.492+0000 7f1c66fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c601e7a50 con 0x7f1c6015c9b0 2023-11-28T13:08:33.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.492+0000 7f1c66fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c601e7d30 con 0x7f1c60161340 2023-11-28T13:08:33.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.492+0000 7f1c64d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c600cd7c0 0x7f1c601e6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.492+0000 7f1c64d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c600cd7c0 0x7f1c601e6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46556/0 (socket says 172.21.15.156:46556) 2023-11-28T13:08:33.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.492+0000 7f1c64d9b700 1 -- 172.21.15.156:0/3492649110 learned_addr learned my addr 172.21.15.156:0/3492649110 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:33.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.492+0000 7f1c5ffff700 1 --2- 172.21.15.156:0/3492649110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c6015c9b0 0x7f1c6015ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.492+0000 7f1c6559c700 1 --2- 172.21.15.156:0/3492649110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c60161340 0x7f1c60154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.492+0000 7f1c64d9b700 1 -- 172.21.15.156:0/3492649110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c60161340 msgr2=0x7f1c60154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:33.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.492+0000 7f1c64d9b700 1 --2- 172.21.15.156:0/3492649110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c60161340 0x7f1c60154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.492+0000 7f1c64d9b700 1 -- 172.21.15.156:0/3492649110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c6015c9b0 msgr2=0x7f1c6015ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:33.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.492+0000 7f1c64d9b700 1 --2- 172.21.15.156:0/3492649110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c6015c9b0 0x7f1c6015ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.492+0000 7f1c64d9b700 1 -- 172.21.15.156:0/3492649110 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c54009550 con 0x7f1c600cd7c0 2023-11-28T13:08:33.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.492+0000 7f1c5ffff700 1 --2- 172.21.15.156:0/3492649110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c6015c9b0 0x7f1c6015ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:33.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.493+0000 7f1c64d9b700 1 --2- 172.21.15.156:0/3492649110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c600cd7c0 0x7f1c601e6e60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009c40 tx=0x7f1c54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:33.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.493+0000 7f1c5d7fa700 1 -- 172.21.15.156:0/3492649110 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c540036a0 con 0x7f1c600cd7c0 2023-11-28T13:08:33.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.493+0000 7f1c5d7fa700 1 -- 172.21.15.156:0/3492649110 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5401e070 con 0x7f1c600cd7c0 2023-11-28T13:08:33.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.493+0000 7f1c5d7fa700 1 -- 172.21.15.156:0/3492649110 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c54003a60 con 0x7f1c600cd7c0 2023-11-28T13:08:33.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.493+0000 7f1c66fff700 1 -- 172.21.15.156:0/3492649110 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c600d1300 con 0x7f1c600cd7c0 2023-11-28T13:08:33.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.493+0000 7f1c66fff700 1 -- 172.21.15.156:0/3492649110 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c600b5520 con 0x7f1c600cd7c0 2023-11-28T13:08:33.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.494+0000 7f1c66fff700 1 -- 172.21.15.156:0/3492649110 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c600d15e0 con 0x7f1c600cd7c0 2023-11-28T13:08:33.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.495+0000 7f1c5d7fa700 1 -- 172.21.15.156:0/3492649110 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1c5400f050 con 0x7f1c600cd7c0 2023-11-28T13:08:33.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.496+0000 7f1c5d7fa700 1 --2- 172.21.15.156:0/3492649110 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1c440682f0 0x7f1c4406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.496+0000 7f1c5d7fa700 1 -- 172.21.15.156:0/3492649110 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1c54020070 con 0x7f1c600cd7c0 2023-11-28T13:08:33.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.496+0000 7f1c5ffff700 1 --2- 172.21.15.156:0/3492649110 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1c440682f0 0x7f1c4406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.497+0000 7f1c5ffff700 1 --2- 172.21.15.156:0/3492649110 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1c440682f0 0x7f1c4406a7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1c60154d40 tx=0x7f1c4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:33.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.500+0000 7f1c5d7fa700 1 -- 172.21.15.156:0/3492649110 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1c540539e0 con 0x7f1c600cd7c0 2023-11-28T13:08:33.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.689+0000 7f1c66fff700 1 -- 172.21.15.156:0/3492649110 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1c600745c0 con 0x7f1c600cd7c0 2023-11-28T13:08:33.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.689+0000 7f1c5d7fa700 1 -- 172.21.15.156:0/3492649110 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1c54057000 con 0x7f1c600cd7c0 2023-11-28T13:08:33.691 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T13:08:33.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.691+0000 7f1c66fff700 1 -- 172.21.15.156:0/3492649110 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1c440682f0 msgr2=0x7f1c4406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:33.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.691+0000 7f1c66fff700 1 --2- 172.21.15.156:0/3492649110 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1c440682f0 0x7f1c4406a7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1c60154d40 tx=0x7f1c4c009040 comp rx=0 tx=0).stop 2023-11-28T13:08:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.691+0000 7f1c66fff700 1 -- 172.21.15.156:0/3492649110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c600cd7c0 msgr2=0x7f1c601e6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.691+0000 7f1c66fff700 1 --2- 172.21.15.156:0/3492649110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c600cd7c0 0x7f1c601e6e60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009c40 tx=0x7f1c54004750 comp rx=0 tx=0).stop 2023-11-28T13:08:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.692+0000 7f1c66fff700 1 -- 172.21.15.156:0/3492649110 shutdown_connections 2023-11-28T13:08:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.692+0000 7f1c66fff700 1 --2- 172.21.15.156:0/3492649110 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1c440682f0 0x7f1c4406a7a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.692+0000 7f1c66fff700 1 --2- 172.21.15.156:0/3492649110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c60161340 0x7f1c60154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.692+0000 7f1c66fff700 1 --2- 172.21.15.156:0/3492649110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c600cd7c0 0x7f1c601e6e60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.692+0000 7f1c66fff700 1 --2- 172.21.15.156:0/3492649110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c6015c9b0 0x7f1c6015ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.692+0000 7f1c66fff700 1 -- 172.21.15.156:0/3492649110 >> 172.21.15.156:0/3492649110 conn(0x7f1c60072600 msgr2=0x7f1c6006a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.692+0000 7f1c66fff700 1 -- 172.21.15.156:0/3492649110 shutdown_connections 2023-11-28T13:08:33.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.692+0000 7f1c66fff700 1 -- 172.21.15.156:0/3492649110 wait complete. 2023-11-28T13:08:33.712 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:08:33.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.906+0000 7f5579fff700 1 Processor -- start 2023-11-28T13:08:33.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.906+0000 7f5579fff700 1 -- start start 2023-11-28T13:08:33.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.907+0000 7f5579fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f55740d01b0 0x7f55740d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.907+0000 7f5579fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55740b16d0 0x7f55740d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.907+0000 7f5579fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55740b0230 con 0x7f55740b20b0 2023-11-28T13:08:33.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.907+0000 7f5579fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f557415cec0 con 0x7f55740d01b0 2023-11-28T13:08:33.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.907+0000 7f5579fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55740b4570 con 0x7f55740b16d0 2023-11-28T13:08:33.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.908+0000 7f55737fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f55740d01b0 0x7f55740d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.908+0000 7f55737fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f55740d01b0 0x7f55740d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46568/0 (socket says 172.21.15.156:46568) 2023-11-28T13:08:33.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.908+0000 7f55737fe700 1 -- 172.21.15.156:0/2189606531 learned_addr learned my addr 172.21.15.156:0/2189606531 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:33.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.908+0000 7f5573fff700 1 --2- 172.21.15.156:0/2189606531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55740b16d0 0x7f55740d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.908+0000 7f55737fe700 1 -- 172.21.15.156:0/2189606531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55740b16d0 msgr2=0x7f55740d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:33.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.908+0000 7f55737fe700 1 --2- 172.21.15.156:0/2189606531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55740b16d0 0x7f55740d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.908+0000 7f55737fe700 1 -- 172.21.15.156:0/2189606531 >> v1:172.21.15.186:6789/0 conn(0x7f55740b20b0 legacy=0x7f55740b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:33.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.908+0000 7f55737fe700 1 -- 172.21.15.156:0/2189606531 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55740b4850 con 0x7f55740d01b0 2023-11-28T13:08:33.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.908+0000 7f5572ffd700 1 -- 172.21.15.156:0/2189606531 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2684008072 0 0) 0x7f55740b0230 con 0x7f55740b20b0 2023-11-28T13:08:33.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.909+0000 7f55737fe700 1 --2- 172.21.15.156:0/2189606531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55740d01b0 0x7f55740d0c20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f55640098f0 tx=0x7f5564009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6939df630ae99c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:33.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.909+0000 7f5572ffd700 1 -- 172.21.15.156:0/2189606531 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564011830 con 0x7f55740d01b0 2023-11-28T13:08:33.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.909+0000 7f5572ffd700 1 -- 172.21.15.156:0/2189606531 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5564011990 con 0x7f55740d01b0 2023-11-28T13:08:33.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.909+0000 7f5572ffd700 1 -- 172.21.15.156:0/2189606531 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564011d20 con 0x7f55740d01b0 2023-11-28T13:08:33.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.909+0000 7f5579fff700 1 -- 172.21.15.156:0/2189606531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55740d01b0 msgr2=0x7f55740d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:33.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.909+0000 7f5579fff700 1 --2- 172.21.15.156:0/2189606531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55740d01b0 0x7f55740d0c20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f55640098f0 tx=0x7f5564009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:33.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.910+0000 7f5579fff700 1 -- 172.21.15.156:0/2189606531 shutdown_connections 2023-11-28T13:08:33.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.910+0000 7f5579fff700 1 --2- 172.21.15.156:0/2189606531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55740b16d0 0x7f55740d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.910+0000 7f5579fff700 1 --2- 172.21.15.156:0/2189606531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55740d01b0 0x7f55740d0c20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.910+0000 7f5579fff700 1 -- 172.21.15.156:0/2189606531 >> 172.21.15.156:0/2189606531 conn(0x7f557409c3f0 msgr2=0x7f55740d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:33.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.910+0000 7f5579fff700 1 -- 172.21.15.156:0/2189606531 shutdown_connections 2023-11-28T13:08:33.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.910+0000 7f5579fff700 1 -- 172.21.15.156:0/2189606531 wait complete. 2023-11-28T13:08:33.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.911+0000 7f5579fff700 1 Processor -- start 2023-11-28T13:08:33.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.911+0000 7f5579fff700 1 -- start start 2023-11-28T13:08:33.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.912+0000 7f5579fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55740b16d0 0x7f55740c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.912+0000 7f5579fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55740b20b0 0x7f557414cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.912+0000 7f5579fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55740d01b0 0x7f557414d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.912+0000 7f5579fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55740c9ee0 con 0x7f55740d01b0 2023-11-28T13:08:33.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.912+0000 7f5579fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55740cc6f0 con 0x7f55740b16d0 2023-11-28T13:08:33.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.912+0000 7f5579fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55740cc9d0 con 0x7f55740b20b0 2023-11-28T13:08:33.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.913+0000 7f5573fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55740d01b0 0x7f557414d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.913+0000 7f556bfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55740b20b0 0x7f557414cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.913+0000 7f55737fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55740b16d0 0x7f55740c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.913+0000 7f5573fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55740d01b0 0x7f557414d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46574/0 (socket says 172.21.15.156:46574) 2023-11-28T13:08:33.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.913+0000 7f5573fff700 1 -- 172.21.15.156:0/3920002406 learned_addr learned my addr 172.21.15.156:0/3920002406 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:33.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.913+0000 7f5573fff700 1 -- 172.21.15.156:0/3920002406 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55740b20b0 msgr2=0x7f557414cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:33.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.913+0000 7f5573fff700 1 --2- 172.21.15.156:0/3920002406 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55740b20b0 0x7f557414cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.913+0000 7f5573fff700 1 -- 172.21.15.156:0/3920002406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55740b16d0 msgr2=0x7f55740c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:33.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.913+0000 7f5573fff700 1 --2- 172.21.15.156:0/3920002406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55740b16d0 0x7f55740c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:33.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.913+0000 7f5573fff700 1 -- 172.21.15.156:0/3920002406 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5564009550 con 0x7f55740d01b0 2023-11-28T13:08:33.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.913+0000 7f5573fff700 1 --2- 172.21.15.156:0/3920002406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55740d01b0 0x7f557414d480 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f556c00efc0 tx=0x7f556c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:33.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.913+0000 7f5570ff9700 1 -- 172.21.15.156:0/3920002406 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c009ea0 con 0x7f55740d01b0 2023-11-28T13:08:33.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.914+0000 7f5570ff9700 1 -- 172.21.15.156:0/3920002406 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556c00f040 con 0x7f55740d01b0 2023-11-28T13:08:33.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.914+0000 7f5579fff700 1 -- 172.21.15.156:0/3920002406 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55740afec0 con 0x7f55740d01b0 2023-11-28T13:08:33.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.914+0000 7f5570ff9700 1 -- 172.21.15.156:0/3920002406 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c00ced0 con 0x7f55740d01b0 2023-11-28T13:08:33.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.914+0000 7f5579fff700 1 -- 172.21.15.156:0/3920002406 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55740afbe0 con 0x7f55740d01b0 2023-11-28T13:08:33.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.915+0000 7f5579fff700 1 -- 172.21.15.156:0/3920002406 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55740b2600 con 0x7f55740d01b0 2023-11-28T13:08:33.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.916+0000 7f5570ff9700 1 -- 172.21.15.156:0/3920002406 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f556c014480 con 0x7f55740d01b0 2023-11-28T13:08:33.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.916+0000 7f5570ff9700 1 --2- 172.21.15.156:0/3920002406 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f55540682f0 0x7f555406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:33.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.916+0000 7f5570ff9700 1 -- 172.21.15.156:0/3920002406 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f556c0865b0 con 0x7f55740d01b0 2023-11-28T13:08:33.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.917+0000 7f55737fe700 1 --2- 172.21.15.156:0/3920002406 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f55540682f0 0x7f555406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:33.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.917+0000 7f55737fe700 1 --2- 172.21.15.156:0/3920002406 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f55540682f0 0x7f555406a7a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5564014040 tx=0x7f5564008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:33.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:33.920+0000 7f5570ff9700 1 -- 172.21.15.156:0/3920002406 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f556c052db0 con 0x7f55740d01b0 2023-11-28T13:08:34.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.115+0000 7f5579fff700 1 -- 172.21.15.156:0/3920002406 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5574068690 con 0x7f55740d01b0 2023-11-28T13:08:34.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.116+0000 7f5570ff9700 1 -- 172.21.15.156:0/3920002406 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17897 (secure 0 0 0) 0x7f556c0563d0 con 0x7f55740d01b0 2023-11-28T13:08:34.119 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:34.119 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":21,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:08:04.060053+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:08:00.903485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:08:02.037854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:08:34.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.118+0000 7f5579fff700 1 -- 172.21.15.156:0/3920002406 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f55540682f0 msgr2=0x7f555406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:34.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.118+0000 7f5579fff700 1 --2- 172.21.15.156:0/3920002406 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f55540682f0 0x7f555406a7a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5564014040 tx=0x7f5564008030 comp rx=0 tx=0).stop 2023-11-28T13:08:34.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.118+0000 7f5579fff700 1 -- 172.21.15.156:0/3920002406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55740d01b0 msgr2=0x7f557414d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:34.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.118+0000 7f5579fff700 1 --2- 172.21.15.156:0/3920002406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55740d01b0 0x7f557414d480 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f556c00efc0 tx=0x7f556c00c3a0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.118+0000 7f5579fff700 1 -- 172.21.15.156:0/3920002406 shutdown_connections 2023-11-28T13:08:34.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.118+0000 7f5579fff700 1 --2- 172.21.15.156:0/3920002406 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f55540682f0 0x7f555406a7a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.118+0000 7f5579fff700 1 --2- 172.21.15.156:0/3920002406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55740d01b0 0x7f557414d480 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.118+0000 7f5579fff700 1 --2- 172.21.15.156:0/3920002406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55740b16d0 0x7f55740c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.118+0000 7f5579fff700 1 --2- 172.21.15.156:0/3920002406 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55740b20b0 0x7f557414cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.118+0000 7f5579fff700 1 -- 172.21.15.156:0/3920002406 >> 172.21.15.156:0/3920002406 conn(0x7f557409c3f0 msgr2=0x7f55740cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:34.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.119+0000 7f5579fff700 1 -- 172.21.15.156:0/3920002406 shutdown_connections 2023-11-28T13:08:34.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.119+0000 7f5579fff700 1 -- 172.21.15.156:0/3920002406 wait complete. 2023-11-28T13:08:34.139 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:08:00.903485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:08:02.037854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:08:34.139 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:08:34.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.337+0000 7face2fff700 1 Processor -- start 2023-11-28T13:08:34.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.337+0000 7face2fff700 1 -- start start 2023-11-28T13:08:34.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.338+0000 7face2fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7facdc15c9b0 0x7facdc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:34.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.338+0000 7face2fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7facdc0cd7c0 0x7facdc0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:34.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.338+0000 7face2fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facdc0d1950 con 0x7facdc161340 2023-11-28T13:08:34.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.338+0000 7face2fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facdc0b5800 con 0x7facdc0cd7c0 2023-11-28T13:08:34.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.338+0000 7face2fff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facdc0ce140 con 0x7facdc15c9b0 2023-11-28T13:08:34.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.339+0000 7face159c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7facdc0cd7c0 0x7facdc0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.339+0000 7face159c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7facdc0cd7c0 0x7facdc0cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46582/0 (socket says 172.21.15.156:46582) 2023-11-28T13:08:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.339+0000 7face159c700 1 -- 172.21.15.156:0/2821789326 learned_addr learned my addr 172.21.15.156:0/2821789326 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.339+0000 7face0d9b700 1 --2- 172.21.15.156:0/2821789326 >> v2:172.21.15.186:3300/0 conn(0x7facdc15c9b0 0x7facdc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:34.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.339+0000 7face159c700 1 -- 172.21.15.156:0/2821789326 >> v2:172.21.15.186:3300/0 conn(0x7facdc15c9b0 msgr2=0x7facdc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:34.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.339+0000 7face159c700 1 --2- 172.21.15.156:0/2821789326 >> v2:172.21.15.186:3300/0 conn(0x7facdc15c9b0 0x7facdc0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.339+0000 7face159c700 1 -- 172.21.15.156:0/2821789326 >> v1:172.21.15.186:6789/0 conn(0x7facdc161340 legacy=0x7facdc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:34.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.339+0000 7face159c700 1 -- 172.21.15.156:0/2821789326 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc0d24f0 con 0x7facdc0cd7c0 2023-11-28T13:08:34.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.339+0000 7face159c700 1 --2- 172.21.15.156:0/2821789326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facdc0cd7c0 0x7facdc0cdc00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7facdc067460 tx=0x7facd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61eb1a029c4678e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:34.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.340+0000 7facdb7fe700 1 -- 172.21.15.156:0/2821789326 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd400e070 con 0x7facdc0cd7c0 2023-11-28T13:08:34.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.340+0000 7facdb7fe700 1 -- 172.21.15.156:0/2821789326 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd4003a20 con 0x7facdc0cd7c0 2023-11-28T13:08:34.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.340+0000 7facdb7fe700 1 -- 172.21.15.156:0/2821789326 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd4003dd0 con 0x7facdc0cd7c0 2023-11-28T13:08:34.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.340+0000 7face2fff700 1 -- 172.21.15.156:0/2821789326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facdc0cd7c0 msgr2=0x7facdc0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:34.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.340+0000 7face2fff700 1 --2- 172.21.15.156:0/2821789326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facdc0cd7c0 0x7facdc0cdc00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7facdc067460 tx=0x7facd400b330 comp rx=0 tx=0).stop 2023-11-28T13:08:34.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.340+0000 7face2fff700 1 -- 172.21.15.156:0/2821789326 shutdown_connections 2023-11-28T13:08:34.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.340+0000 7face2fff700 1 --2- 172.21.15.156:0/2821789326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facdc0cd7c0 0x7facdc0cdc00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.340+0000 7face2fff700 1 --2- 172.21.15.156:0/2821789326 >> v2:172.21.15.186:3300/0 conn(0x7facdc15c9b0 0x7facdc0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.340+0000 7face2fff700 1 -- 172.21.15.156:0/2821789326 >> 172.21.15.156:0/2821789326 conn(0x7facdc072600 msgr2=0x7facdc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:34.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.340+0000 7face2fff700 1 -- 172.21.15.156:0/2821789326 shutdown_connections 2023-11-28T13:08:34.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.340+0000 7face2fff700 1 -- 172.21.15.156:0/2821789326 wait complete. 2023-11-28T13:08:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.342+0000 7face2fff700 1 Processor -- start 2023-11-28T13:08:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.342+0000 7face2fff700 1 -- start start 2023-11-28T13:08:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.343+0000 7face2fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7facdc0cd7c0 0x7facdc1e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.343+0000 7face2fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facdc15c9b0 0x7facdc0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.343+0000 7face2fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facdc161340 0x7facdc0c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.343+0000 7face2fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc1580d0 con 0x7facdc161340 2023-11-28T13:08:34.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.343+0000 7face2fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0c95b0 con 0x7facdc15c9b0 2023-11-28T13:08:34.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.343+0000 7face2fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0c8be0 con 0x7facdc0cd7c0 2023-11-28T13:08:34.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.343+0000 7face159c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facdc161340 0x7facdc0c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:34.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.343+0000 7face159c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facdc161340 0x7facdc0c77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46594/0 (socket says 172.21.15.156:46594) 2023-11-28T13:08:34.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.343+0000 7face159c700 1 -- 172.21.15.156:0/558906544 learned_addr learned my addr 172.21.15.156:0/558906544 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:34.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.343+0000 7face0d9b700 1 --2- 172.21.15.156:0/558906544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7facdc0cd7c0 0x7facdc1e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:34.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.343+0000 7facdbfff700 1 --2- 172.21.15.156:0/558906544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facdc15c9b0 0x7facdc0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:34.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.344+0000 7face159c700 1 -- 172.21.15.156:0/558906544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7facdc0cd7c0 msgr2=0x7facdc1e70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:34.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.344+0000 7face159c700 1 --2- 172.21.15.156:0/558906544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7facdc0cd7c0 0x7facdc1e70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.344+0000 7face159c700 1 -- 172.21.15.156:0/558906544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facdc15c9b0 msgr2=0x7facdc0c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:34.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.344+0000 7face159c700 1 --2- 172.21.15.156:0/558906544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facdc15c9b0 0x7facdc0c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.344+0000 7face159c700 1 -- 172.21.15.156:0/558906544 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd4009b30 con 0x7facdc161340 2023-11-28T13:08:34.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.344+0000 7face159c700 1 --2- 172.21.15.156:0/558906544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facdc161340 0x7facdc0c77a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7facd400b370 tx=0x7facd4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:34.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.344+0000 7facd97fa700 1 -- 172.21.15.156:0/558906544 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd4004440 con 0x7facdc161340 2023-11-28T13:08:34.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.344+0000 7facd97fa700 1 -- 172.21.15.156:0/558906544 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd40092e0 con 0x7facdc161340 2023-11-28T13:08:34.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.344+0000 7face2fff700 1 -- 172.21.15.156:0/558906544 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facdc0d1300 con 0x7facdc161340 2023-11-28T13:08:34.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.344+0000 7facd97fa700 1 -- 172.21.15.156:0/558906544 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd4004800 con 0x7facdc161340 2023-11-28T13:08:34.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.345+0000 7face2fff700 1 -- 172.21.15.156:0/558906544 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facdc0b5520 con 0x7facdc161340 2023-11-28T13:08:34.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.346+0000 7face2fff700 1 -- 172.21.15.156:0/558906544 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facdc0d15e0 con 0x7facdc161340 2023-11-28T13:08:34.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.346+0000 7facd97fa700 1 -- 172.21.15.156:0/558906544 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7facd402c070 con 0x7facdc161340 2023-11-28T13:08:34.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.348+0000 7facd97fa700 1 --2- 172.21.15.156:0/558906544 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7facc00682f0 0x7facc006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:34.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.348+0000 7facd97fa700 1 -- 172.21.15.156:0/558906544 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7facd4087130 con 0x7facdc161340 2023-11-28T13:08:34.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.348+0000 7face0d9b700 1 --2- 172.21.15.156:0/558906544 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7facc00682f0 0x7facc006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:34.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.349+0000 7face0d9b700 1 --2- 172.21.15.156:0/558906544 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7facc00682f0 0x7facc006a7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7facd0006b20 tx=0x7facd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:34.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.351+0000 7facd97fa700 1 -- 172.21.15.156:0/558906544 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7facd4053990 con 0x7facdc161340 2023-11-28T13:08:34.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.544+0000 7face2fff700 1 -- 172.21.15.156:0/558906544 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7facdc0c8d50 con 0x7facdc161340 2023-11-28T13:08:34.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.545+0000 7facd97fa700 1 -- 172.21.15.156:0/558906544 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7facd4056fb0 con 0x7facdc161340 2023-11-28T13:08:34.547 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:08:34.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.547+0000 7face2fff700 1 -- 172.21.15.156:0/558906544 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7facc00682f0 msgr2=0x7facc006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:34.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.547+0000 7face2fff700 1 --2- 172.21.15.156:0/558906544 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7facc00682f0 0x7facc006a7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7facd0006b20 tx=0x7facd0008040 comp rx=0 tx=0).stop 2023-11-28T13:08:34.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.547+0000 7face2fff700 1 -- 172.21.15.156:0/558906544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facdc161340 msgr2=0x7facdc0c77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:34.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.547+0000 7face2fff700 1 --2- 172.21.15.156:0/558906544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facdc161340 0x7facdc0c77a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7facd400b370 tx=0x7facd4004ad0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.547+0000 7face2fff700 1 -- 172.21.15.156:0/558906544 shutdown_connections 2023-11-28T13:08:34.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.547+0000 7face2fff700 1 --2- 172.21.15.156:0/558906544 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7facc00682f0 0x7facc006a7a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.547+0000 7face2fff700 1 --2- 172.21.15.156:0/558906544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facdc161340 0x7facdc0c77a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.547+0000 7face2fff700 1 --2- 172.21.15.156:0/558906544 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7facdc0cd7c0 0x7facdc1e70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.547+0000 7face2fff700 1 --2- 172.21.15.156:0/558906544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facdc15c9b0 0x7facdc0c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.547+0000 7face2fff700 1 -- 172.21.15.156:0/558906544 >> 172.21.15.156:0/558906544 conn(0x7facdc072600 msgr2=0x7facdc0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:34.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.547+0000 7face2fff700 1 -- 172.21.15.156:0/558906544 shutdown_connections 2023-11-28T13:08:34.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.547+0000 7face2fff700 1 -- 172.21.15.156:0/558906544 wait complete. 2023-11-28T13:08:34.567 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:08:34.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.764+0000 7f1d379ff700 1 Processor -- start 2023-11-28T13:08:34.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.765+0000 7f1d379ff700 1 -- start start 2023-11-28T13:08:34.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.766+0000 7f1d379ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1d3015c9b0 0x7f1d300b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:34.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.766+0000 7f1d379ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d300cd7c0 0x7f1d300cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:34.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.766+0000 7f1d379ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d300d1950 con 0x7f1d30161340 2023-11-28T13:08:34.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.766+0000 7f1d379ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d300b5800 con 0x7f1d3015c9b0 2023-11-28T13:08:34.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.766+0000 7f1d379ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d300ce140 con 0x7f1d300cd7c0 2023-11-28T13:08:34.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.766+0000 7f1d3579b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1d3015c9b0 0x7f1d300b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:34.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.766+0000 7f1d3579b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1d3015c9b0 0x7f1d300b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46610/0 (socket says 172.21.15.156:46610) 2023-11-28T13:08:34.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.766+0000 7f1d3579b700 1 -- 172.21.15.156:0/1615876897 learned_addr learned my addr 172.21.15.156:0/1615876897 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:34.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.766+0000 7f1d35f9c700 1 --2- 172.21.15.156:0/1615876897 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d300cd7c0 0x7f1d300cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:34.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.767+0000 7f1d3579b700 1 -- 172.21.15.156:0/1615876897 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d300cd7c0 msgr2=0x7f1d300cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:34.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.767+0000 7f1d3579b700 1 --2- 172.21.15.156:0/1615876897 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d300cd7c0 0x7f1d300cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.767+0000 7f1d3579b700 1 -- 172.21.15.156:0/1615876897 >> v1:172.21.15.186:6789/0 conn(0x7f1d30161340 legacy=0x7f1d300d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:34.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.767+0000 7f1d3579b700 1 -- 172.21.15.156:0/1615876897 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d300d24f0 con 0x7f1d3015c9b0 2023-11-28T13:08:34.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.767+0000 7f1d3579b700 1 --2- 172.21.15.156:0/1615876897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d3015c9b0 0x7f1d300b4bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0098f0 tx=0x7f1d2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1979877e8d7741d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:34.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.767+0000 7f1d27fff700 1 -- 172.21.15.156:0/1615876897 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c00b8a0 con 0x7f1d3015c9b0 2023-11-28T13:08:34.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.767+0000 7f1d27fff700 1 -- 172.21.15.156:0/1615876897 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2c00ba00 con 0x7f1d3015c9b0 2023-11-28T13:08:34.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.767+0000 7f1d27fff700 1 -- 172.21.15.156:0/1615876897 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c00bd90 con 0x7f1d3015c9b0 2023-11-28T13:08:34.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.768+0000 7f1d379ff700 1 -- 172.21.15.156:0/1615876897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d3015c9b0 msgr2=0x7f1d300b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:34.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.768+0000 7f1d379ff700 1 --2- 172.21.15.156:0/1615876897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d3015c9b0 0x7f1d300b4bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0098f0 tx=0x7f1d2c009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:34.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.768+0000 7f1d379ff700 1 -- 172.21.15.156:0/1615876897 shutdown_connections 2023-11-28T13:08:34.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.768+0000 7f1d379ff700 1 --2- 172.21.15.156:0/1615876897 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d300cd7c0 0x7f1d300cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.768+0000 7f1d379ff700 1 --2- 172.21.15.156:0/1615876897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d3015c9b0 0x7f1d300b4bf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.768+0000 7f1d379ff700 1 -- 172.21.15.156:0/1615876897 >> 172.21.15.156:0/1615876897 conn(0x7f1d30072600 msgr2=0x7f1d300cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:34.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.768+0000 7f1d379ff700 1 -- 172.21.15.156:0/1615876897 shutdown_connections 2023-11-28T13:08:34.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.768+0000 7f1d379ff700 1 -- 172.21.15.156:0/1615876897 wait complete. 2023-11-28T13:08:34.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.770+0000 7f1d379ff700 1 Processor -- start 2023-11-28T13:08:34.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.770+0000 7f1d379ff700 1 -- start start 2023-11-28T13:08:34.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.771+0000 7f1d379ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d300cd7c0 0x7f1d301e6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:34.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.771+0000 7f1d379ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d3015c9b0 0x7f1d3015ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:34.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.771+0000 7f1d379ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d30161340 0x7f1d30154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:34.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.771+0000 7f1d379ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d301e7770 con 0x7f1d300cd7c0 2023-11-28T13:08:34.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.771+0000 7f1d379ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d301e7a50 con 0x7f1d3015c9b0 2023-11-28T13:08:34.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.771+0000 7f1d379ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d301e7d30 con 0x7f1d30161340 2023-11-28T13:08:34.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.771+0000 7f1d3579b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d300cd7c0 0x7f1d301e6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:34.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.771+0000 7f1d3579b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d300cd7c0 0x7f1d301e6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46614/0 (socket says 172.21.15.156:46614) 2023-11-28T13:08:34.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.771+0000 7f1d3579b700 1 -- 172.21.15.156:0/1828718884 learned_addr learned my addr 172.21.15.156:0/1828718884 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:34.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.771+0000 7f1d34f9a700 1 --2- 172.21.15.156:0/1828718884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d3015c9b0 0x7f1d3015ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:34.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.771+0000 7f1d35f9c700 1 --2- 172.21.15.156:0/1828718884 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d30161340 0x7f1d30154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:34.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.771+0000 7f1d3579b700 1 -- 172.21.15.156:0/1828718884 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d30161340 msgr2=0x7f1d30154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:34.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.771+0000 7f1d3579b700 1 --2- 172.21.15.156:0/1828718884 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d30161340 0x7f1d30154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.771+0000 7f1d3579b700 1 -- 172.21.15.156:0/1828718884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d3015c9b0 msgr2=0x7f1d3015ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:34.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.771+0000 7f1d3579b700 1 --2- 172.21.15.156:0/1828718884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d3015c9b0 0x7f1d3015ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.771+0000 7f1d3579b700 1 -- 172.21.15.156:0/1828718884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c009550 con 0x7f1d300cd7c0 2023-11-28T13:08:34.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.771+0000 7f1d34f9a700 1 --2- 172.21.15.156:0/1828718884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d3015c9b0 0x7f1d3015ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:34.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.772+0000 7f1d3579b700 1 --2- 172.21.15.156:0/1828718884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d300cd7c0 0x7f1d301e6e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009c40 tx=0x7f1d2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:34.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.772+0000 7f1d25ffb700 1 -- 172.21.15.156:0/1828718884 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c0036a0 con 0x7f1d300cd7c0 2023-11-28T13:08:34.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.772+0000 7f1d25ffb700 1 -- 172.21.15.156:0/1828718884 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2c01e070 con 0x7f1d300cd7c0 2023-11-28T13:08:34.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.772+0000 7f1d379ff700 1 -- 172.21.15.156:0/1828718884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d300d1300 con 0x7f1d300cd7c0 2023-11-28T13:08:34.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.772+0000 7f1d25ffb700 1 -- 172.21.15.156:0/1828718884 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c003a60 con 0x7f1d300cd7c0 2023-11-28T13:08:34.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.772+0000 7f1d379ff700 1 -- 172.21.15.156:0/1828718884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d300b5520 con 0x7f1d300cd7c0 2023-11-28T13:08:34.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.773+0000 7f1d379ff700 1 -- 172.21.15.156:0/1828718884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d300d15e0 con 0x7f1d300cd7c0 2023-11-28T13:08:34.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.774+0000 7f1d25ffb700 1 -- 172.21.15.156:0/1828718884 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1d2c00f050 con 0x7f1d300cd7c0 2023-11-28T13:08:34.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.775+0000 7f1d25ffb700 1 --2- 172.21.15.156:0/1828718884 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1d140682f0 0x7f1d1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:34.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.775+0000 7f1d25ffb700 1 -- 172.21.15.156:0/1828718884 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1d2c020070 con 0x7f1d300cd7c0 2023-11-28T13:08:34.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.775+0000 7f1d34f9a700 1 --2- 172.21.15.156:0/1828718884 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1d140682f0 0x7f1d1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:34.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.776+0000 7f1d34f9a700 1 --2- 172.21.15.156:0/1828718884 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1d140682f0 0x7f1d1406a7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1d30154d40 tx=0x7f1d20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:34.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.779+0000 7f1d25ffb700 1 -- 172.21.15.156:0/1828718884 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1d2c053a40 con 0x7f1d300cd7c0 2023-11-28T13:08:34.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.969+0000 7f1d379ff700 1 -- 172.21.15.156:0/1828718884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d300745c0 con 0x7f1d300cd7c0 2023-11-28T13:08:34.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.970+0000 7f1d25ffb700 1 -- 172.21.15.156:0/1828718884 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d2c057060 con 0x7f1d300cd7c0 2023-11-28T13:08:34.972 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:08:34.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.972+0000 7f1d379ff700 1 -- 172.21.15.156:0/1828718884 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1d140682f0 msgr2=0x7f1d1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:34.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.972+0000 7f1d379ff700 1 --2- 172.21.15.156:0/1828718884 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1d140682f0 0x7f1d1406a7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1d30154d40 tx=0x7f1d20009040 comp rx=0 tx=0).stop 2023-11-28T13:08:34.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.972+0000 7f1d379ff700 1 -- 172.21.15.156:0/1828718884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d300cd7c0 msgr2=0x7f1d301e6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:34.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.972+0000 7f1d379ff700 1 --2- 172.21.15.156:0/1828718884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d300cd7c0 0x7f1d301e6e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009c40 tx=0x7f1d2c004750 comp rx=0 tx=0).stop 2023-11-28T13:08:34.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.972+0000 7f1d379ff700 1 -- 172.21.15.156:0/1828718884 shutdown_connections 2023-11-28T13:08:34.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.972+0000 7f1d379ff700 1 --2- 172.21.15.156:0/1828718884 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1d140682f0 0x7f1d1406a7a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.972+0000 7f1d379ff700 1 --2- 172.21.15.156:0/1828718884 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d30161340 0x7f1d30154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.972+0000 7f1d379ff700 1 --2- 172.21.15.156:0/1828718884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d300cd7c0 0x7f1d301e6e60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.972+0000 7f1d379ff700 1 --2- 172.21.15.156:0/1828718884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d3015c9b0 0x7f1d3015ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:34.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.972+0000 7f1d379ff700 1 -- 172.21.15.156:0/1828718884 >> 172.21.15.156:0/1828718884 conn(0x7f1d30072600 msgr2=0x7f1d3006a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:34.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.972+0000 7f1d379ff700 1 -- 172.21.15.156:0/1828718884 shutdown_connections 2023-11-28T13:08:34.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:34.972+0000 7f1d379ff700 1 -- 172.21.15.156:0/1828718884 wait complete. 2023-11-28T13:08:34.992 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:08:35.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.193+0000 7f56b33ff700 1 Processor -- start 2023-11-28T13:08:35.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.193+0000 7f56b33ff700 1 -- start start 2023-11-28T13:08:35.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.194+0000 7f56b33ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f56ac0d01b0 0x7f56ac0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:35.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.195+0000 7f56b33ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f56ac0b16d0 0x7f56ac0d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:35.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.195+0000 7f56b33ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ac0b0230 con 0x7f56ac0b20b0 2023-11-28T13:08:35.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.195+0000 7f56b33ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ac15cec0 con 0x7f56ac0d01b0 2023-11-28T13:08:35.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.195+0000 7f56b33ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ac0b4570 con 0x7f56ac0b16d0 2023-11-28T13:08:35.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.195+0000 7f56b119b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f56ac0d01b0 0x7f56ac0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:35.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.195+0000 7f56b099a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f56ac0b20b0 0x7f56ac0b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51016/0 (socket says 172.21.15.156:51016) 2023-11-28T13:08:35.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.195+0000 7f56b099a700 1 -- 172.21.15.156:0/2029335926 learned_addr learned my addr 172.21.15.156:0/2029335926 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:35.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.195+0000 7f56b119b700 1 --2- 172.21.15.156:0/2029335926 >> v2:172.21.15.156:3300/0 conn(0x7f56ac0d01b0 0x7f56ac0d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46618/0 (socket says 172.21.15.156:46618) 2023-11-28T13:08:35.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.195+0000 7f56b199c700 1 --2- 172.21.15.156:0/2029335926 >> v2:172.21.15.186:3300/0 conn(0x7f56ac0b16d0 0x7f56ac0d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:35.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.195+0000 7f56b119b700 1 -- 172.21.15.156:0/2029335926 >> v2:172.21.15.186:3300/0 conn(0x7f56ac0b16d0 msgr2=0x7f56ac0d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:35.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.195+0000 7f56b119b700 1 --2- 172.21.15.156:0/2029335926 >> v2:172.21.15.186:3300/0 conn(0x7f56ac0b16d0 0x7f56ac0d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.195+0000 7f56b119b700 1 -- 172.21.15.156:0/2029335926 >> v1:172.21.15.156:6789/0 conn(0x7f56ac0b20b0 legacy=0x7f56ac0b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:35.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.195+0000 7f56b119b700 1 -- 172.21.15.156:0/2029335926 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac0b4850 con 0x7f56ac0d01b0 2023-11-28T13:08:35.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.195+0000 7f56a3fff700 1 -- 172.21.15.156:0/2029335926 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2923766995 0 0) 0x7f56ac0b0230 con 0x7f56ac0b20b0 2023-11-28T13:08:35.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.195+0000 7f56b199c700 1 --2- 172.21.15.156:0/2029335926 >> v2:172.21.15.186:3300/0 conn(0x7f56ac0b16d0 0x7f56ac0d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:35.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.196+0000 7f56b119b700 1 --2- 172.21.15.156:0/2029335926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0d01b0 0x7f56ac0d0c20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0cd630 tx=0x7f56a8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa923c19c30a80ad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:35.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.196+0000 7f56a3fff700 1 -- 172.21.15.156:0/2029335926 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8011830 con 0x7f56ac0d01b0 2023-11-28T13:08:35.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.196+0000 7f56a3fff700 1 -- 172.21.15.156:0/2029335926 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a8011990 con 0x7f56ac0d01b0 2023-11-28T13:08:35.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.196+0000 7f56a3fff700 1 -- 172.21.15.156:0/2029335926 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8011d20 con 0x7f56ac0d01b0 2023-11-28T13:08:35.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.196+0000 7f56b33ff700 1 -- 172.21.15.156:0/2029335926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0d01b0 msgr2=0x7f56ac0d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:35.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.196+0000 7f56b33ff700 1 --2- 172.21.15.156:0/2029335926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0d01b0 0x7f56ac0d0c20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0cd630 tx=0x7f56a8009b20 comp rx=0 tx=0).stop 2023-11-28T13:08:35.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.196+0000 7f56b33ff700 1 -- 172.21.15.156:0/2029335926 shutdown_connections 2023-11-28T13:08:35.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.196+0000 7f56b33ff700 1 --2- 172.21.15.156:0/2029335926 >> v2:172.21.15.186:3300/0 conn(0x7f56ac0b16d0 0x7f56ac0d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.196+0000 7f56b33ff700 1 --2- 172.21.15.156:0/2029335926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0d01b0 0x7f56ac0d0c20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.197+0000 7f56b33ff700 1 -- 172.21.15.156:0/2029335926 >> 172.21.15.156:0/2029335926 conn(0x7f56ac09c3f0 msgr2=0x7f56ac0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:35.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.197+0000 7f56b33ff700 1 -- 172.21.15.156:0/2029335926 shutdown_connections 2023-11-28T13:08:35.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.197+0000 7f56b33ff700 1 -- 172.21.15.156:0/2029335926 wait complete. 2023-11-28T13:08:35.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.198+0000 7f56b33ff700 1 Processor -- start 2023-11-28T13:08:35.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.198+0000 7f56b33ff700 1 -- start start 2023-11-28T13:08:35.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.199+0000 7f56b33ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56ac0b16d0 0x7f56ac15c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:35.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.199+0000 7f56b33ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0b20b0 0x7f56ac0b8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:35.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.199+0000 7f56b33ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56ac0d01b0 0x7f56ac145f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:35.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.199+0000 7f56b33ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac1e3ff0 con 0x7f56ac0b20b0 2023-11-28T13:08:35.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.199+0000 7f56b33ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac0b7df0 con 0x7f56ac0b16d0 2023-11-28T13:08:35.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.199+0000 7f56b33ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac15c760 con 0x7f56ac0d01b0 2023-11-28T13:08:35.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.200+0000 7f56b099a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0b20b0 0x7f56ac0b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:35.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.200+0000 7f56b099a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0b20b0 0x7f56ac0b8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46630/0 (socket says 172.21.15.156:46630) 2023-11-28T13:08:35.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.200+0000 7f56b099a700 1 -- 172.21.15.156:0/1923130807 learned_addr learned my addr 172.21.15.156:0/1923130807 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:35.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.200+0000 7f56b119b700 1 --2- 172.21.15.156:0/1923130807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56ac0b16d0 0x7f56ac15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:35.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.200+0000 7f56b199c700 1 --2- 172.21.15.156:0/1923130807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56ac0d01b0 0x7f56ac145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:35.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.200+0000 7f56b099a700 1 -- 172.21.15.156:0/1923130807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56ac0d01b0 msgr2=0x7f56ac145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:35.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.200+0000 7f56b099a700 1 --2- 172.21.15.156:0/1923130807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56ac0d01b0 0x7f56ac145f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.200+0000 7f56b099a700 1 -- 172.21.15.156:0/1923130807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56ac0b16d0 msgr2=0x7f56ac15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:35.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.200+0000 7f56b099a700 1 --2- 172.21.15.156:0/1923130807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56ac0b16d0 0x7f56ac15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.200+0000 7f56b099a700 1 -- 172.21.15.156:0/1923130807 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a80094a0 con 0x7f56ac0b20b0 2023-11-28T13:08:35.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.200+0000 7f56b099a700 1 --2- 172.21.15.156:0/1923130807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0b20b0 0x7f56ac0b8280 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f569c00c750 tx=0x7f569c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:35.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.201+0000 7f56a1ffb700 1 -- 172.21.15.156:0/1923130807 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c004c80 con 0x7f56ac0b20b0 2023-11-28T13:08:35.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.201+0000 7f56a1ffb700 1 -- 172.21.15.156:0/1923130807 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569c004e00 con 0x7f56ac0b20b0 2023-11-28T13:08:35.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.201+0000 7f56a1ffb700 1 -- 172.21.15.156:0/1923130807 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c007310 con 0x7f56ac0b20b0 2023-11-28T13:08:35.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.201+0000 7f56b33ff700 1 -- 172.21.15.156:0/1923130807 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ac0afec0 con 0x7f56ac0b20b0 2023-11-28T13:08:35.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.201+0000 7f56b33ff700 1 -- 172.21.15.156:0/1923130807 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ac0afbe0 con 0x7f56ac0b20b0 2023-11-28T13:08:35.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.202+0000 7f56b33ff700 1 -- 172.21.15.156:0/1923130807 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ac0b2600 con 0x7f56ac0b20b0 2023-11-28T13:08:35.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.203+0000 7f56a1ffb700 1 -- 172.21.15.156:0/1923130807 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f569c01a020 con 0x7f56ac0b20b0 2023-11-28T13:08:35.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.203+0000 7f56a1ffb700 1 --2- 172.21.15.156:0/1923130807 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56900682f0 0x7f569006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:35.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.204+0000 7f56a1ffb700 1 -- 172.21.15.156:0/1923130807 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f569c085870 con 0x7f56ac0b20b0 2023-11-28T13:08:35.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.204+0000 7f56b119b700 1 --2- 172.21.15.156:0/1923130807 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56900682f0 0x7f569006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:35.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.205+0000 7f56b119b700 1 --2- 172.21.15.156:0/1923130807 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56900682f0 0x7f569006a7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f56a8000c00 tx=0x7f56a801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:35.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.208+0000 7f56a1ffb700 1 -- 172.21.15.156:0/1923130807 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f569c052120 con 0x7f56ac0b20b0 2023-11-28T13:08:35.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.398+0000 7f56b33ff700 1 -- 172.21.15.156:0/1923130807 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f56ac068690 con 0x7f56ac0b20b0 2023-11-28T13:08:35.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.399+0000 7f56a1ffb700 1 -- 172.21.15.156:0/1923130807 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f569c055740 con 0x7f56ac0b20b0 2023-11-28T13:08:35.400 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T13:08:35.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.401+0000 7f56b33ff700 1 -- 172.21.15.156:0/1923130807 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56900682f0 msgr2=0x7f569006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:35.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.401+0000 7f56b33ff700 1 --2- 172.21.15.156:0/1923130807 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56900682f0 0x7f569006a7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f56a8000c00 tx=0x7f56a801c040 comp rx=0 tx=0).stop 2023-11-28T13:08:35.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.401+0000 7f56b33ff700 1 -- 172.21.15.156:0/1923130807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0b20b0 msgr2=0x7f56ac0b8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:35.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.401+0000 7f56b33ff700 1 --2- 172.21.15.156:0/1923130807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0b20b0 0x7f56ac0b8280 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f569c00c750 tx=0x7f569c00ca60 comp rx=0 tx=0).stop 2023-11-28T13:08:35.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.401+0000 7f56b33ff700 1 -- 172.21.15.156:0/1923130807 shutdown_connections 2023-11-28T13:08:35.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.401+0000 7f56b33ff700 1 --2- 172.21.15.156:0/1923130807 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56900682f0 0x7f569006a7a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.401+0000 7f56b33ff700 1 --2- 172.21.15.156:0/1923130807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56ac0d01b0 0x7f56ac145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.401+0000 7f56b33ff700 1 --2- 172.21.15.156:0/1923130807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56ac0b16d0 0x7f56ac15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.401+0000 7f56b33ff700 1 --2- 172.21.15.156:0/1923130807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0b20b0 0x7f56ac0b8280 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.401+0000 7f56b33ff700 1 -- 172.21.15.156:0/1923130807 >> 172.21.15.156:0/1923130807 conn(0x7f56ac09c3f0 msgr2=0x7f56ac0cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:35.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.401+0000 7f56b33ff700 1 -- 172.21.15.156:0/1923130807 shutdown_connections 2023-11-28T13:08:35.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.401+0000 7f56b33ff700 1 -- 172.21.15.156:0/1923130807 wait complete. 2023-11-28T13:08:35.422 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:08:35.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.620+0000 7f51597ff700 1 Processor -- start 2023-11-28T13:08:35.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.621+0000 7f51597ff700 1 -- start start 2023-11-28T13:08:35.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.622+0000 7f51597ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f51540d0ff0 0x7f51540cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:35.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.622+0000 7f51597ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f51540b4750 0x7f51540b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:35.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.622+0000 7f51597ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51540d13c0 0x7f51540b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:35.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.622+0000 7f51597ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51540b3b20 con 0x7f51540b4750 2023-11-28T13:08:35.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.622+0000 7f51597ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51540cf220 con 0x7f51540d0ff0 2023-11-28T13:08:35.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.622+0000 7f51597ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51540b3e50 con 0x7f51540d13c0 2023-11-28T13:08:35.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.622+0000 7f514bfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f51540b4750 0x7f51540b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:35.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.622+0000 7f514bfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f51540b4750 0x7f51540b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46634/0 (socket says 172.21.15.156:46634) 2023-11-28T13:08:35.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.622+0000 7f514bfff700 1 -- 172.21.15.156:0/933368707 learned_addr learned my addr 172.21.15.156:0/933368707 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:35.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.622+0000 7f5152ffd700 1 --2- 172.21.15.156:0/933368707 >> v2:172.21.15.186:3300/0 conn(0x7f51540d0ff0 0x7f51540cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:35.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.622+0000 7f51537fe700 1 --2- 172.21.15.156:0/933368707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51540d13c0 0x7f51540b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:35.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.623+0000 7f514bfff700 1 -- 172.21.15.156:0/933368707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51540d13c0 msgr2=0x7f51540b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:35.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.623+0000 7f514bfff700 1 --2- 172.21.15.156:0/933368707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51540d13c0 0x7f51540b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.623+0000 7f514bfff700 1 -- 172.21.15.156:0/933368707 >> v2:172.21.15.186:3300/0 conn(0x7f51540d0ff0 msgr2=0x7f51540cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:35.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.623+0000 7f514bfff700 1 --2- 172.21.15.156:0/933368707 >> v2:172.21.15.186:3300/0 conn(0x7f51540d0ff0 0x7f51540cff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.623+0000 7f514bfff700 1 -- 172.21.15.156:0/933368707 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51540cf470 con 0x7f51540b4750 2023-11-28T13:08:35.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.623+0000 7f514bfff700 1 --2- 172.21.15.156:0/933368707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540b4750 0x7f51540b30a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f51540668d0 tx=0x7f513c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760432bdc32fa639 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:35.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.623+0000 7f51527fc700 1 -- 172.21.15.156:0/933368707 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c011830 con 0x7f51540b4750 2023-11-28T13:08:35.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.623+0000 7f51527fc700 1 -- 172.21.15.156:0/933368707 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513c011990 con 0x7f51540b4750 2023-11-28T13:08:35.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.623+0000 7f51527fc700 1 -- 172.21.15.156:0/933368707 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c011d20 con 0x7f51540b4750 2023-11-28T13:08:35.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.624+0000 7f51597ff700 1 -- 172.21.15.156:0/933368707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540b4750 msgr2=0x7f51540b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:35.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.624+0000 7f51597ff700 1 --2- 172.21.15.156:0/933368707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540b4750 0x7f51540b30a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f51540668d0 tx=0x7f513c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.624+0000 7f51597ff700 1 -- 172.21.15.156:0/933368707 shutdown_connections 2023-11-28T13:08:35.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.624+0000 7f51597ff700 1 --2- 172.21.15.156:0/933368707 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51540d13c0 0x7f51540b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.624+0000 7f51597ff700 1 --2- 172.21.15.156:0/933368707 >> v2:172.21.15.186:3300/0 conn(0x7f51540d0ff0 0x7f51540cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.624+0000 7f51597ff700 1 --2- 172.21.15.156:0/933368707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540b4750 0x7f51540b30a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.624+0000 7f51597ff700 1 -- 172.21.15.156:0/933368707 >> 172.21.15.156:0/933368707 conn(0x7f515409c3c0 msgr2=0x7f51540d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:35.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.624+0000 7f51597ff700 1 -- 172.21.15.156:0/933368707 shutdown_connections 2023-11-28T13:08:35.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.624+0000 7f51597ff700 1 -- 172.21.15.156:0/933368707 wait complete. 2023-11-28T13:08:35.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.626+0000 7f51597ff700 1 Processor -- start 2023-11-28T13:08:35.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.626+0000 7f51597ff700 1 -- start start 2023-11-28T13:08:35.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f51597ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51540b4750 0x7f515415c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:35.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f51597ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51540d0ff0 0x7f515415b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:35.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f51597ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540d13c0 0x7f51540c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:35.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f51597ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515415cd00 con 0x7f51540d13c0 2023-11-28T13:08:35.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f51597ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51540c88d0 con 0x7f51540b4750 2023-11-28T13:08:35.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f51597ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f515415ca20 con 0x7f51540d0ff0 2023-11-28T13:08:35.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f51537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540d13c0 0x7f51540c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:35.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f51537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540d13c0 0x7f51540c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46650/0 (socket says 172.21.15.156:46650) 2023-11-28T13:08:35.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f51537fe700 1 -- 172.21.15.156:0/2674512069 learned_addr learned my addr 172.21.15.156:0/2674512069 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:35.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f5152ffd700 1 --2- 172.21.15.156:0/2674512069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51540b4750 0x7f515415c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:35.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f514bfff700 1 --2- 172.21.15.156:0/2674512069 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51540d0ff0 0x7f515415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:35.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f51537fe700 1 -- 172.21.15.156:0/2674512069 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51540d0ff0 msgr2=0x7f515415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:35.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f51537fe700 1 --2- 172.21.15.156:0/2674512069 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51540d0ff0 0x7f515415b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f51537fe700 1 -- 172.21.15.156:0/2674512069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51540b4750 msgr2=0x7f515415c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:35.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f51537fe700 1 --2- 172.21.15.156:0/2674512069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51540b4750 0x7f515415c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f51537fe700 1 -- 172.21.15.156:0/2674512069 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513c009550 con 0x7f51540d13c0 2023-11-28T13:08:35.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f5152ffd700 1 --2- 172.21.15.156:0/2674512069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51540b4750 0x7f515415c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:35.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.627+0000 7f514bfff700 1 --2- 172.21.15.156:0/2674512069 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51540d0ff0 0x7f515415b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:35.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.628+0000 7f51537fe700 1 --2- 172.21.15.156:0/2674512069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540d13c0 0x7f51540c8390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f514c00efc0 tx=0x7f514c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:35.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.628+0000 7f514b7fe700 1 -- 172.21.15.156:0/2674512069 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c00cd40 con 0x7f51540d13c0 2023-11-28T13:08:35.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.628+0000 7f514b7fe700 1 -- 172.21.15.156:0/2674512069 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c00f040 con 0x7f51540d13c0 2023-11-28T13:08:35.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.628+0000 7f514b7fe700 1 -- 172.21.15.156:0/2674512069 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c0146c0 con 0x7f51540d13c0 2023-11-28T13:08:35.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.628+0000 7f51597ff700 1 -- 172.21.15.156:0/2674512069 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51540cfa30 con 0x7f51540d13c0 2023-11-28T13:08:35.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.628+0000 7f51597ff700 1 -- 172.21.15.156:0/2674512069 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51540cf750 con 0x7f51540d13c0 2023-11-28T13:08:35.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.629+0000 7f51597ff700 1 -- 172.21.15.156:0/2674512069 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51540d0a50 con 0x7f51540d13c0 2023-11-28T13:08:35.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.630+0000 7f514b7fe700 1 -- 172.21.15.156:0/2674512069 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f514c014820 con 0x7f51540d13c0 2023-11-28T13:08:35.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.631+0000 7f514b7fe700 1 --2- 172.21.15.156:0/2674512069 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f51340682f0 0x7f513406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:35.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.631+0000 7f514b7fe700 1 -- 172.21.15.156:0/2674512069 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f514c0867f0 con 0x7f51540d13c0 2023-11-28T13:08:35.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.631+0000 7f5152ffd700 1 --2- 172.21.15.156:0/2674512069 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f51340682f0 0x7f513406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:35.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.632+0000 7f5152ffd700 1 --2- 172.21.15.156:0/2674512069 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f51340682f0 0x7f513406a7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f51540c91e0 tx=0x7f5144008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:35.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.635+0000 7f514b7fe700 1 -- 172.21.15.156:0/2674512069 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f514c052f60 con 0x7f51540d13c0 2023-11-28T13:08:35.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.823+0000 7f51597ff700 1 -- 172.21.15.156:0/2674512069 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f51541e4380 con 0x7f51540d13c0 2023-11-28T13:08:35.825 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.823+0000 7f514b7fe700 1 -- 172.21.15.156:0/2674512069 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f51541e4380 con 0x7f51540d13c0 2023-11-28T13:08:35.825 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T13:08:35.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.826+0000 7f51597ff700 1 -- 172.21.15.156:0/2674512069 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f51340682f0 msgr2=0x7f513406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:35.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.826+0000 7f51597ff700 1 --2- 172.21.15.156:0/2674512069 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f51340682f0 0x7f513406a7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f51540c91e0 tx=0x7f5144008040 comp rx=0 tx=0).stop 2023-11-28T13:08:35.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.826+0000 7f51597ff700 1 -- 172.21.15.156:0/2674512069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540d13c0 msgr2=0x7f51540c8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:35.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.826+0000 7f51597ff700 1 --2- 172.21.15.156:0/2674512069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540d13c0 0x7f51540c8390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f514c00efc0 tx=0x7f514c00c3a0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.826+0000 7f51597ff700 1 -- 172.21.15.156:0/2674512069 shutdown_connections 2023-11-28T13:08:35.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.826+0000 7f51597ff700 1 --2- 172.21.15.156:0/2674512069 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f51340682f0 0x7f513406a7a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.826+0000 7f51597ff700 1 --2- 172.21.15.156:0/2674512069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540d13c0 0x7f51540c8390 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.826+0000 7f51597ff700 1 --2- 172.21.15.156:0/2674512069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51540b4750 0x7f515415c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.826+0000 7f51597ff700 1 --2- 172.21.15.156:0/2674512069 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51540d0ff0 0x7f515415b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:35.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.826+0000 7f51597ff700 1 -- 172.21.15.156:0/2674512069 >> 172.21.15.156:0/2674512069 conn(0x7f515409c3c0 msgr2=0x7f51540b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:35.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.826+0000 7f51597ff700 1 -- 172.21.15.156:0/2674512069 shutdown_connections 2023-11-28T13:08:35.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:35.826+0000 7f51597ff700 1 -- 172.21.15.156:0/2674512069 wait complete. 2023-11-28T13:08:35.846 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:08:36.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.040+0000 7fe9415ff700 1 Processor -- start 2023-11-28T13:08:36.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.041+0000 7fe9415ff700 1 -- start start 2023-11-28T13:08:36.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.042+0000 7fe9415ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe93c161340 0x7fe93c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.042+0000 7fe9415ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe93c0d2320 0x7fe93c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.042+0000 7fe9415ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe93c0d1950 con 0x7fe93c15c9b0 2023-11-28T13:08:36.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.042+0000 7fe9415ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe93c0b5800 con 0x7fe93c161340 2023-11-28T13:08:36.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.042+0000 7fe9415ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe93c0b5300 con 0x7fe93c0d2320 2023-11-28T13:08:36.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.042+0000 7fe93affd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fe93c15c9b0 0x7fe93c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51952/0 (socket says 172.21.15.156:51952) 2023-11-28T13:08:36.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.042+0000 7fe93affd700 1 -- 172.21.15.156:0/709548696 learned_addr learned my addr 172.21.15.156:0/709548696 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:36.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.042+0000 7fe933fff700 1 --2- 172.21.15.156:0/709548696 >> v2:172.21.15.156:3300/0 conn(0x7fe93c161340 0x7fe93c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.042+0000 7fe93b7fe700 1 --2- 172.21.15.156:0/709548696 >> v2:172.21.15.186:3300/0 conn(0x7fe93c0d2320 0x7fe93c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.042+0000 7fe933fff700 1 -- 172.21.15.156:0/709548696 >> v2:172.21.15.186:3300/0 conn(0x7fe93c0d2320 msgr2=0x7fe93c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:36.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.042+0000 7fe933fff700 1 --2- 172.21.15.156:0/709548696 >> v2:172.21.15.186:3300/0 conn(0x7fe93c0d2320 0x7fe93c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.042+0000 7fe933fff700 1 -- 172.21.15.156:0/709548696 >> v1:172.21.15.156:6789/0 conn(0x7fe93c15c9b0 legacy=0x7fe93c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:36.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.043+0000 7fe933fff700 1 -- 172.21.15.156:0/709548696 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe93c0b5520 con 0x7fe93c161340 2023-11-28T13:08:36.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.043+0000 7fe93a7fc700 1 -- 172.21.15.156:0/709548696 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370614751 0 0) 0x7fe93c0d1950 con 0x7fe93c15c9b0 2023-11-28T13:08:36.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.043+0000 7fe93b7fe700 1 --2- 172.21.15.156:0/709548696 >> v2:172.21.15.186:3300/0 conn(0x7fe93c0d2320 0x7fe93c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:36.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.043+0000 7fe933fff700 1 --2- 172.21.15.156:0/709548696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe93c161340 0x7fe93c0d1de0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe93c06c170 tx=0x7fe924009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bae55f6d662ee81 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:36.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.043+0000 7fe93a7fc700 1 -- 172.21.15.156:0/709548696 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924011830 con 0x7fe93c161340 2023-11-28T13:08:36.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.043+0000 7fe93a7fc700 1 -- 172.21.15.156:0/709548696 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe924011990 con 0x7fe93c161340 2023-11-28T13:08:36.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.043+0000 7fe93a7fc700 1 -- 172.21.15.156:0/709548696 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924011d20 con 0x7fe93c161340 2023-11-28T13:08:36.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.044+0000 7fe9415ff700 1 -- 172.21.15.156:0/709548696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe93c161340 msgr2=0x7fe93c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.044+0000 7fe9415ff700 1 --2- 172.21.15.156:0/709548696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe93c161340 0x7fe93c0d1de0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe93c06c170 tx=0x7fe924009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.044+0000 7fe9415ff700 1 -- 172.21.15.156:0/709548696 shutdown_connections 2023-11-28T13:08:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.044+0000 7fe9415ff700 1 --2- 172.21.15.156:0/709548696 >> v2:172.21.15.186:3300/0 conn(0x7fe93c0d2320 0x7fe93c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.044+0000 7fe9415ff700 1 --2- 172.21.15.156:0/709548696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe93c161340 0x7fe93c0d1de0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.044+0000 7fe9415ff700 1 -- 172.21.15.156:0/709548696 >> 172.21.15.156:0/709548696 conn(0x7fe93c072600 msgr2=0x7fe93c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.044+0000 7fe9415ff700 1 -- 172.21.15.156:0/709548696 shutdown_connections 2023-11-28T13:08:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.044+0000 7fe9415ff700 1 -- 172.21.15.156:0/709548696 wait complete. 2023-11-28T13:08:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.045+0000 7fe9415ff700 1 Processor -- start 2023-11-28T13:08:36.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.046+0000 7fe9415ff700 1 -- start start 2023-11-28T13:08:36.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.046+0000 7fe9415ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe93c0d2320 0x7fe93c150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.046+0000 7fe9415ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe93c15c9b0 0x7fe93c1592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.046+0000 7fe9415ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe93c161340 0x7fe93c159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.046+0000 7fe9415ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c1e8780 con 0x7fe93c15c9b0 2023-11-28T13:08:36.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.046+0000 7fe9415ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c153700 con 0x7fe93c161340 2023-11-28T13:08:36.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.046+0000 7fe9415ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c154af0 con 0x7fe93c0d2320 2023-11-28T13:08:36.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.047+0000 7fe933fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe93c15c9b0 0x7fe93c1592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.047+0000 7fe933fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe93c15c9b0 0x7fe93c1592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41400/0 (socket says 172.21.15.156:41400) 2023-11-28T13:08:36.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.047+0000 7fe933fff700 1 -- 172.21.15.156:0/592961196 learned_addr learned my addr 172.21.15.156:0/592961196 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:36.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.047+0000 7fe93b7fe700 1 --2- 172.21.15.156:0/592961196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe93c161340 0x7fe93c159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.047+0000 7fe93affd700 1 --2- 172.21.15.156:0/592961196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe93c0d2320 0x7fe93c150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.047+0000 7fe933fff700 1 -- 172.21.15.156:0/592961196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe93c0d2320 msgr2=0x7fe93c150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:36.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.047+0000 7fe933fff700 1 --2- 172.21.15.156:0/592961196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe93c0d2320 0x7fe93c150770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.047+0000 7fe933fff700 1 -- 172.21.15.156:0/592961196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe93c161340 msgr2=0x7fe93c159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:36.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.047+0000 7fe933fff700 1 --2- 172.21.15.156:0/592961196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe93c161340 0x7fe93c159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.047+0000 7fe933fff700 1 -- 172.21.15.156:0/592961196 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9240094a0 con 0x7fe93c15c9b0 2023-11-28T13:08:36.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.047+0000 7fe93affd700 1 --2- 172.21.15.156:0/592961196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe93c0d2320 0x7fe93c150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:36.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.047+0000 7fe933fff700 1 --2- 172.21.15.156:0/592961196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe93c15c9b0 0x7fe93c1592f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe9240097c0 tx=0x7fe924004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:36.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.048+0000 7fe9337fe700 1 -- 172.21.15.156:0/592961196 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9240045f0 con 0x7fe93c15c9b0 2023-11-28T13:08:36.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.048+0000 7fe9337fe700 1 -- 172.21.15.156:0/592961196 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92401f070 con 0x7fe93c15c9b0 2023-11-28T13:08:36.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.048+0000 7fe9337fe700 1 -- 172.21.15.156:0/592961196 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924003ae0 con 0x7fe93c15c9b0 2023-11-28T13:08:36.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.048+0000 7fe9415ff700 1 -- 172.21.15.156:0/592961196 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe93c0d15e0 con 0x7fe93c15c9b0 2023-11-28T13:08:36.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.048+0000 7fe9415ff700 1 -- 172.21.15.156:0/592961196 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93c0d1300 con 0x7fe93c15c9b0 2023-11-28T13:08:36.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.049+0000 7fe9415ff700 1 -- 172.21.15.156:0/592961196 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe93c15cd80 con 0x7fe93c15c9b0 2023-11-28T13:08:36.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.050+0000 7fe9337fe700 1 -- 172.21.15.156:0/592961196 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe92400f050 con 0x7fe93c15c9b0 2023-11-28T13:08:36.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.051+0000 7fe9337fe700 1 --2- 172.21.15.156:0/592961196 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe91c0682f0 0x7fe91c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.051+0000 7fe9337fe700 1 -- 172.21.15.156:0/592961196 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe924021070 con 0x7fe93c15c9b0 2023-11-28T13:08:36.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.051+0000 7fe93affd700 1 --2- 172.21.15.156:0/592961196 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe91c0682f0 0x7fe91c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.052+0000 7fe93affd700 1 --2- 172.21.15.156:0/592961196 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe91c0682f0 0x7fe91c06a7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe92c007fd0 tx=0x7fe92c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:36.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.055+0000 7fe9337fe700 1 -- 172.21.15.156:0/592961196 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe924054010 con 0x7fe93c15c9b0 2023-11-28T13:08:36.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.246+0000 7fe9415ff700 1 -- 172.21.15.156:0/592961196 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe93c0745c0 con 0x7fe93c15c9b0 2023-11-28T13:08:36.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.247+0000 7fe9337fe700 1 -- 172.21.15.156:0/592961196 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17897 (secure 0 0 0) 0x7fe924015090 con 0x7fe93c15c9b0 2023-11-28T13:08:36.249 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:36.249 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":21,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:08:04.060053+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:08:00.903485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:08:02.037854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:08:36.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.249+0000 7fe9415ff700 1 -- 172.21.15.156:0/592961196 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe91c0682f0 msgr2=0x7fe91c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:36.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.249+0000 7fe9415ff700 1 --2- 172.21.15.156:0/592961196 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe91c0682f0 0x7fe91c06a7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe92c007fd0 tx=0x7fe92c009040 comp rx=0 tx=0).stop 2023-11-28T13:08:36.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.249+0000 7fe9415ff700 1 -- 172.21.15.156:0/592961196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe93c15c9b0 msgr2=0x7fe93c1592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:36.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.249+0000 7fe9415ff700 1 --2- 172.21.15.156:0/592961196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe93c15c9b0 0x7fe93c1592f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe9240097c0 tx=0x7fe924004750 comp rx=0 tx=0).stop 2023-11-28T13:08:36.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.249+0000 7fe9415ff700 1 -- 172.21.15.156:0/592961196 shutdown_connections 2023-11-28T13:08:36.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.249+0000 7fe9415ff700 1 --2- 172.21.15.156:0/592961196 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe91c0682f0 0x7fe91c06a7a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.249+0000 7fe9415ff700 1 --2- 172.21.15.156:0/592961196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe93c161340 0x7fe93c159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.249+0000 7fe9415ff700 1 --2- 172.21.15.156:0/592961196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe93c0d2320 0x7fe93c150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.249+0000 7fe9415ff700 1 --2- 172.21.15.156:0/592961196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe93c15c9b0 0x7fe93c1592f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.249+0000 7fe9415ff700 1 -- 172.21.15.156:0/592961196 >> 172.21.15.156:0/592961196 conn(0x7fe93c072600 msgr2=0x7fe93c160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:36.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.250+0000 7fe9415ff700 1 -- 172.21.15.156:0/592961196 shutdown_connections 2023-11-28T13:08:36.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.250+0000 7fe9415ff700 1 -- 172.21.15.156:0/592961196 wait complete. 2023-11-28T13:08:36.270 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:08:00.903485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:08:02.037854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:08:36.270 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:08:36.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.467+0000 7ff2cbfff700 1 Processor -- start 2023-11-28T13:08:36.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.467+0000 7ff2cbfff700 1 -- start start 2023-11-28T13:08:36.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.468+0000 7ff2cbfff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff2c4161340 0x7ff2c40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.468+0000 7ff2cbfff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c40d1950 con 0x7ff2c40d2320 2023-11-28T13:08:36.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.468+0000 7ff2cbfff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c40b5520 con 0x7ff2c415c9b0 2023-11-28T13:08:36.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.468+0000 7ff2cbfff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c40b5800 con 0x7ff2c4161340 2023-11-28T13:08:36.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.469+0000 7ff2ca59c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff2c40d2320 0x7ff2c40cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51960/0 (socket says 172.21.15.156:51960) 2023-11-28T13:08:36.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.469+0000 7ff2ca59c700 1 -- 172.21.15.156:0/2477644388 learned_addr learned my addr 172.21.15.156:0/2477644388 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:36.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.469+0000 7ff2c959a700 1 --2- 172.21.15.156:0/2477644388 >> v2:172.21.15.186:3300/0 conn(0x7ff2c4161340 0x7ff2c40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.469+0000 7ff2c8d99700 1 -- 172.21.15.156:0/2477644388 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335487160 0 0) 0x7ff2c40d1950 con 0x7ff2c40d2320 2023-11-28T13:08:36.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.469+0000 7ff2c8d99700 1 -- 172.21.15.156:0/2477644388 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2b00035b0 con 0x7ff2c40d2320 2023-11-28T13:08:36.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.469+0000 7ff2c8d99700 1 -- 172.21.15.156:0/2477644388 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140404428 0 0) 0x7ff2c40b5520 con 0x7ff2c415c9b0 2023-11-28T13:08:36.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.469+0000 7ff2c8d99700 1 -- 172.21.15.156:0/2477644388 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c40d1950 con 0x7ff2c415c9b0 2023-11-28T13:08:36.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.469+0000 7ff2c8d99700 1 -- 172.21.15.156:0/2477644388 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3209352214 0 0) 0x7ff2b00035b0 con 0x7ff2c40d2320 2023-11-28T13:08:36.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.469+0000 7ff2c8d99700 1 -- 172.21.15.156:0/2477644388 >> v2:172.21.15.186:3300/0 conn(0x7ff2c4161340 msgr2=0x7ff2c40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:36.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.469+0000 7ff2c8d99700 1 --2- 172.21.15.156:0/2477644388 >> v2:172.21.15.186:3300/0 conn(0x7ff2c4161340 0x7ff2c40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.469+0000 7ff2c8d99700 1 -- 172.21.15.156:0/2477644388 >> v1:172.21.15.186:6789/0 conn(0x7ff2c415c9b0 legacy=0x7ff2c40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:36.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.469+0000 7ff2c8d99700 1 -- 172.21.15.156:0/2477644388 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c40d1300 con 0x7ff2c40d2320 2023-11-28T13:08:36.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.469+0000 7ff2c8d99700 1 -- 172.21.15.156:0/2477644388 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7ff2bc0019f0 con 0x7ff2c40d2320 2023-11-28T13:08:36.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.469+0000 7ff2c8d99700 1 -- 172.21.15.156:0/2477644388 >> v1:172.21.15.156:6789/0 conn(0x7ff2c40d2320 legacy=0x7ff2c40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:36.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.470+0000 7ff2c8d99700 1 --2- 172.21.15.156:0/2477644388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2b0003b90 0x7ff2b0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.470+0000 7ff2c8d99700 1 --2- 172.21.15.156:0/2477644388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2b0004850 0x7ff2b0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.470+0000 7ff2c8d99700 1 --2- 172.21.15.156:0/2477644388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2b0005200 0x7ff2b000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.470+0000 7ff2c8d99700 1 -- 172.21.15.156:0/2477644388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c40b5800 con 0x7ff2b0004850 2023-11-28T13:08:36.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.470+0000 7ff2c8d99700 1 -- 172.21.15.156:0/2477644388 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b000c110 con 0x7ff2b0005200 2023-11-28T13:08:36.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.470+0000 7ff2c8d99700 1 -- 172.21.15.156:0/2477644388 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b000c2d0 con 0x7ff2b0003b90 2023-11-28T13:08:36.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.470+0000 7ff2c959a700 1 --2- 172.21.15.156:0/2477644388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2b0004850 0x7ff2b0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.470+0000 7ff2ca59c700 1 --2- 172.21.15.156:0/2477644388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2b0005200 0x7ff2b000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.470+0000 7ff2c9d9b700 1 --2- 172.21.15.156:0/2477644388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2b0003b90 0x7ff2b0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.470+0000 7ff2c959a700 1 -- 172.21.15.156:0/2477644388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2b0003b90 msgr2=0x7ff2b0003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:36.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.470+0000 7ff2c959a700 1 --2- 172.21.15.156:0/2477644388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2b0003b90 0x7ff2b0003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.470+0000 7ff2c959a700 1 -- 172.21.15.156:0/2477644388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2b0005200 msgr2=0x7ff2b000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:36.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.470+0000 7ff2c959a700 1 --2- 172.21.15.156:0/2477644388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2b0005200 0x7ff2b000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.470+0000 7ff2c959a700 1 -- 172.21.15.156:0/2477644388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b00035b0 con 0x7ff2b0004850 2023-11-28T13:08:36.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.470+0000 7ff2ca59c700 1 --2- 172.21.15.156:0/2477644388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2b0005200 0x7ff2b000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:36.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.470+0000 7ff2ca59c700 1 -- 172.21.15.156:0/2477644388 reap_dead start 2023-11-28T13:08:36.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.470+0000 7ff2c959a700 1 --2- 172.21.15.156:0/2477644388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2b0004850 0x7ff2b0004cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009b30 tx=0x7ff2b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e470b66db6a479 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:36.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.471+0000 7ff2c8d99700 1 -- 172.21.15.156:0/2477644388 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b400e6c0 con 0x7ff2b0004850 2023-11-28T13:08:36.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.471+0000 7ff2c8d99700 1 -- 172.21.15.156:0/2477644388 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b400e820 con 0x7ff2b0004850 2023-11-28T13:08:36.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.471+0000 7ff2c8d99700 1 -- 172.21.15.156:0/2477644388 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b400eb90 con 0x7ff2b0004850 2023-11-28T13:08:36.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.471+0000 7ff2cbfff700 1 -- 172.21.15.156:0/2477644388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2b0004850 msgr2=0x7ff2b0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:36.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.471+0000 7ff2cbfff700 1 --2- 172.21.15.156:0/2477644388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2b0004850 0x7ff2b0004cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009b30 tx=0x7ff2b0005880 comp rx=0 tx=0).stop 2023-11-28T13:08:36.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.471+0000 7ff2cbfff700 1 -- 172.21.15.156:0/2477644388 shutdown_connections 2023-11-28T13:08:36.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.471+0000 7ff2cbfff700 1 --2- 172.21.15.156:0/2477644388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2b0004850 0x7ff2b0004cc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.471+0000 7ff2cbfff700 1 -- 172.21.15.156:0/2477644388 >> 172.21.15.156:0/2477644388 conn(0x7ff2c4072600 msgr2=0x7ff2c40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:36.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.471+0000 7ff2cbfff700 1 -- 172.21.15.156:0/2477644388 shutdown_connections 2023-11-28T13:08:36.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.471+0000 7ff2cbfff700 1 -- 172.21.15.156:0/2477644388 wait complete. 2023-11-28T13:08:36.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.473+0000 7ff2cbfff700 1 Processor -- start 2023-11-28T13:08:36.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.473+0000 7ff2cbfff700 1 -- start start 2023-11-28T13:08:36.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.474+0000 7ff2cbfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2b0004850 0x7ff2c41e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.474+0000 7ff2cbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c4159720 0x7ff2c41e6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.474+0000 7ff2cbfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2c4159b90 0x7ff2c41e7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.474+0000 7ff2cbfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c41e8ac0 con 0x7ff2c4159720 2023-11-28T13:08:36.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.474+0000 7ff2cbfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c4155010 con 0x7ff2c4159b90 2023-11-28T13:08:36.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.474+0000 7ff2cbfff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c41e9420 con 0x7ff2b0004850 2023-11-28T13:08:36.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.474+0000 7ff2c959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c4159720 0x7ff2c41e6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.474+0000 7ff2c959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c4159720 0x7ff2c41e6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41418/0 (socket says 172.21.15.156:41418) 2023-11-28T13:08:36.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.474+0000 7ff2c959a700 1 -- 172.21.15.156:0/1696981668 learned_addr learned my addr 172.21.15.156:0/1696981668 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:36.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.475+0000 7ff2ca59c700 1 --2- 172.21.15.156:0/1696981668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2c4159b90 0x7ff2c41e7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.475+0000 7ff2c9d9b700 1 --2- 172.21.15.156:0/1696981668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2b0004850 0x7ff2c41e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.475+0000 7ff2c959a700 1 -- 172.21.15.156:0/1696981668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2b0004850 msgr2=0x7ff2c41e8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:36.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.475+0000 7ff2c959a700 1 --2- 172.21.15.156:0/1696981668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2b0004850 0x7ff2c41e8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.475+0000 7ff2c959a700 1 -- 172.21.15.156:0/1696981668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2c4159b90 msgr2=0x7ff2c41e7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:36.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.475+0000 7ff2c959a700 1 --2- 172.21.15.156:0/1696981668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2c4159b90 0x7ff2c41e7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.475+0000 7ff2c959a700 1 -- 172.21.15.156:0/1696981668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c415cd80 con 0x7ff2c4159720 2023-11-28T13:08:36.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.475+0000 7ff2c9d9b700 1 --2- 172.21.15.156:0/1696981668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2b0004850 0x7ff2c41e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.475+0000 7ff2c959a700 1 --2- 172.21.15.156:0/1696981668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c4159720 0x7ff2c41e6ed0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400e4b0 tx=0x7ff2b40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.475+0000 7ff2ba7fc700 1 -- 172.21.15.156:0/1696981668 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b400e6c0 con 0x7ff2c4159720 2023-11-28T13:08:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.475+0000 7ff2ba7fc700 1 -- 172.21.15.156:0/1696981668 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b4003ab0 con 0x7ff2c4159720 2023-11-28T13:08:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.475+0000 7ff2cbfff700 1 -- 172.21.15.156:0/1696981668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c40d15e0 con 0x7ff2c4159720 2023-11-28T13:08:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.475+0000 7ff2ba7fc700 1 -- 172.21.15.156:0/1696981668 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4003e60 con 0x7ff2c4159720 2023-11-28T13:08:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.476+0000 7ff2cbfff700 1 -- 172.21.15.156:0/1696981668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c415cf10 con 0x7ff2c4159720 2023-11-28T13:08:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.477+0000 7ff2cbfff700 1 -- 172.21.15.156:0/1696981668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c41e8bd0 con 0x7ff2c4159720 2023-11-28T13:08:36.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.477+0000 7ff2ba7fc700 1 -- 172.21.15.156:0/1696981668 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff2b4025070 con 0x7ff2c4159720 2023-11-28T13:08:36.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.478+0000 7ff2ba7fc700 1 --2- 172.21.15.156:0/1696981668 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2a8068320 0x7ff2a806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.478+0000 7ff2ba7fc700 1 -- 172.21.15.156:0/1696981668 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff2b40874d0 con 0x7ff2c4159720 2023-11-28T13:08:36.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.479+0000 7ff2c9d9b700 1 --2- 172.21.15.156:0/1696981668 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2a8068320 0x7ff2a806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.479+0000 7ff2c9d9b700 1 --2- 172.21.15.156:0/1696981668 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2a8068320 0x7ff2a806a7d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4063f40 tx=0x7ff2c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:36.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.482+0000 7ff2ba7fc700 1 -- 172.21.15.156:0/1696981668 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff2b4053cd0 con 0x7ff2c4159720 2023-11-28T13:08:36.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.675+0000 7ff2cbfff700 1 -- 172.21.15.156:0/1696981668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff2c40745c0 con 0x7ff2c4159720 2023-11-28T13:08:36.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.675+0000 7ff2ba7fc700 1 -- 172.21.15.156:0/1696981668 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff2b40572f0 con 0x7ff2c4159720 2023-11-28T13:08:36.677 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:08:36.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.678+0000 7ff2cbfff700 1 -- 172.21.15.156:0/1696981668 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2a8068320 msgr2=0x7ff2a806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:36.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.678+0000 7ff2cbfff700 1 --2- 172.21.15.156:0/1696981668 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2a8068320 0x7ff2a806a7d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4063f40 tx=0x7ff2c0009040 comp rx=0 tx=0).stop 2023-11-28T13:08:36.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.678+0000 7ff2cbfff700 1 -- 172.21.15.156:0/1696981668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c4159720 msgr2=0x7ff2c41e6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:36.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.678+0000 7ff2cbfff700 1 --2- 172.21.15.156:0/1696981668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c4159720 0x7ff2c41e6ed0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400e4b0 tx=0x7ff2b40047c0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.678+0000 7ff2cbfff700 1 -- 172.21.15.156:0/1696981668 shutdown_connections 2023-11-28T13:08:36.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.678+0000 7ff2cbfff700 1 --2- 172.21.15.156:0/1696981668 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2a8068320 0x7ff2a806a7d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.678+0000 7ff2cbfff700 1 --2- 172.21.15.156:0/1696981668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2c4159b90 0x7ff2c41e7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.678+0000 7ff2cbfff700 1 --2- 172.21.15.156:0/1696981668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2b0004850 0x7ff2c41e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.678+0000 7ff2cbfff700 1 --2- 172.21.15.156:0/1696981668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c4159720 0x7ff2c41e6ed0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.678+0000 7ff2cbfff700 1 -- 172.21.15.156:0/1696981668 >> 172.21.15.156:0/1696981668 conn(0x7ff2c4072600 msgr2=0x7ff2c415a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:36.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.678+0000 7ff2cbfff700 1 -- 172.21.15.156:0/1696981668 shutdown_connections 2023-11-28T13:08:36.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.678+0000 7ff2cbfff700 1 -- 172.21.15.156:0/1696981668 wait complete. 2023-11-28T13:08:36.698 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:08:36.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.897+0000 7f994a39d700 1 Processor -- start 2023-11-28T13:08:36.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.897+0000 7f994a39d700 1 -- start start 2023-11-28T13:08:36.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.898+0000 7f994a39d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f99440cffa0 0x7f9944160af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.898+0000 7f994a39d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f99440cf170 0x7f99440d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.898+0000 7f994a39d700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99440d0d20 con 0x7f994415c450 2023-11-28T13:08:36.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.898+0000 7f994a39d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99440cdd80 con 0x7f99440cffa0 2023-11-28T13:08:36.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.898+0000 7f994a39d700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99440d0a80 con 0x7f99440cf170 2023-11-28T13:08:36.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.899+0000 7f9948b9a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f99440cffa0 0x7f9944160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.899+0000 7f9948b9a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f99440cffa0 0x7f9944160af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41428/0 (socket says 172.21.15.156:41428) 2023-11-28T13:08:36.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.899+0000 7f9948b9a700 1 -- 172.21.15.156:0/1603809817 learned_addr learned my addr 172.21.15.156:0/1603809817 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:36.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.899+0000 7f9949b9c700 1 --2- 172.21.15.156:0/1603809817 >> v2:172.21.15.186:3300/0 conn(0x7f99440cf170 0x7f99440d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.899+0000 7f9948b9a700 1 -- 172.21.15.156:0/1603809817 >> v2:172.21.15.186:3300/0 conn(0x7f99440cf170 msgr2=0x7f99440d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:36.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.899+0000 7f9948b9a700 1 --2- 172.21.15.156:0/1603809817 >> v2:172.21.15.186:3300/0 conn(0x7f99440cf170 0x7f99440d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.899+0000 7f9948b9a700 1 -- 172.21.15.156:0/1603809817 >> v1:172.21.15.186:6789/0 conn(0x7f994415c450 legacy=0x7f99440cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:36.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.899+0000 7f9948b9a700 1 -- 172.21.15.156:0/1603809817 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99440cd500 con 0x7f99440cffa0 2023-11-28T13:08:36.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.899+0000 7f9948b9a700 1 --2- 172.21.15.156:0/1603809817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99440cffa0 0x7f9944160af0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f99440d0c90 tx=0x7f9934009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22de48c5d3a6803 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.900+0000 7f993bfff700 1 -- 172.21.15.156:0/1603809817 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9934011830 con 0x7f99440cffa0 2023-11-28T13:08:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.900+0000 7f993bfff700 1 -- 172.21.15.156:0/1603809817 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9934011990 con 0x7f99440cffa0 2023-11-28T13:08:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.900+0000 7f993bfff700 1 -- 172.21.15.156:0/1603809817 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9934011d20 con 0x7f99440cffa0 2023-11-28T13:08:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.900+0000 7f994a39d700 1 -- 172.21.15.156:0/1603809817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99440cffa0 msgr2=0x7f9944160af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.900+0000 7f994a39d700 1 --2- 172.21.15.156:0/1603809817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99440cffa0 0x7f9944160af0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f99440d0c90 tx=0x7f9934009bd0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.900+0000 7f994a39d700 1 -- 172.21.15.156:0/1603809817 shutdown_connections 2023-11-28T13:08:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.900+0000 7f994a39d700 1 --2- 172.21.15.156:0/1603809817 >> v2:172.21.15.186:3300/0 conn(0x7f99440cf170 0x7f99440d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.900+0000 7f994a39d700 1 --2- 172.21.15.156:0/1603809817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99440cffa0 0x7f9944160af0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.900+0000 7f994a39d700 1 -- 172.21.15.156:0/1603809817 >> 172.21.15.156:0/1603809817 conn(0x7f9944072630 msgr2=0x7f994406a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.900+0000 7f994a39d700 1 -- 172.21.15.156:0/1603809817 shutdown_connections 2023-11-28T13:08:36.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.900+0000 7f994a39d700 1 -- 172.21.15.156:0/1603809817 wait complete. 2023-11-28T13:08:36.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.902+0000 7f994a39d700 1 Processor -- start 2023-11-28T13:08:36.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.902+0000 7f994a39d700 1 -- start start 2023-11-28T13:08:36.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.903+0000 7f994a39d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99440cf170 0x7f9944154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.903+0000 7f994a39d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99440cffa0 0x7f99440cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.903+0000 7f994a39d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f994415c450 0x7f99441e7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.903+0000 7f994a39d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99440cca60 con 0x7f99440cffa0 2023-11-28T13:08:36.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.903+0000 7f994a39d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994415e390 con 0x7f99440cf170 2023-11-28T13:08:36.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.903+0000 7f994a39d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99440cb1a0 con 0x7f994415c450 2023-11-28T13:08:36.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.903+0000 7f9948b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99440cffa0 0x7f99440cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.903+0000 7f9948b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99440cffa0 0x7f99440cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41444/0 (socket says 172.21.15.156:41444) 2023-11-28T13:08:36.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.903+0000 7f9948b9a700 1 -- 172.21.15.156:0/1323109746 learned_addr learned my addr 172.21.15.156:0/1323109746 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:36.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.903+0000 7f994939b700 1 --2- 172.21.15.156:0/1323109746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99440cf170 0x7f9944154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.904+0000 7f9949b9c700 1 --2- 172.21.15.156:0/1323109746 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f994415c450 0x7f99441e7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.904+0000 7f9948b9a700 1 -- 172.21.15.156:0/1323109746 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f994415c450 msgr2=0x7f99441e7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:36.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.904+0000 7f9948b9a700 1 --2- 172.21.15.156:0/1323109746 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f994415c450 0x7f99441e7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.904+0000 7f9948b9a700 1 -- 172.21.15.156:0/1323109746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99440cf170 msgr2=0x7f9944154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:36.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.904+0000 7f9948b9a700 1 --2- 172.21.15.156:0/1323109746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99440cf170 0x7f9944154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:36.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.904+0000 7f9948b9a700 1 -- 172.21.15.156:0/1323109746 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9934009550 con 0x7f99440cffa0 2023-11-28T13:08:36.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.904+0000 7f9948b9a700 1 --2- 172.21.15.156:0/1323109746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99440cffa0 0x7f99440cc520 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9934000c00 tx=0x7f9934004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:36.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.904+0000 7f9939ffb700 1 -- 172.21.15.156:0/1323109746 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9934004400 con 0x7f99440cffa0 2023-11-28T13:08:36.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.904+0000 7f9939ffb700 1 -- 172.21.15.156:0/1323109746 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993401f070 con 0x7f99440cffa0 2023-11-28T13:08:36.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.904+0000 7f9939ffb700 1 -- 172.21.15.156:0/1323109746 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99340038c0 con 0x7f99440cffa0 2023-11-28T13:08:36.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.904+0000 7f994a39d700 1 -- 172.21.15.156:0/1323109746 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99440cdaa0 con 0x7f99440cffa0 2023-11-28T13:08:36.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.904+0000 7f994a39d700 1 -- 172.21.15.156:0/1323109746 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99440cd7c0 con 0x7f99440cffa0 2023-11-28T13:08:36.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.905+0000 7f994a39d700 1 -- 172.21.15.156:0/1323109746 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99440cf5f0 con 0x7f99440cffa0 2023-11-28T13:08:36.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.906+0000 7f9939ffb700 1 -- 172.21.15.156:0/1323109746 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f993402a020 con 0x7f99440cffa0 2023-11-28T13:08:36.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.907+0000 7f9939ffb700 1 --2- 172.21.15.156:0/1323109746 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f99280682f0 0x7f992806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:36.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.907+0000 7f9939ffb700 1 -- 172.21.15.156:0/1323109746 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f99340872e0 con 0x7f99440cffa0 2023-11-28T13:08:36.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.907+0000 7f994939b700 1 --2- 172.21.15.156:0/1323109746 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f99280682f0 0x7f992806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:36.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.908+0000 7f994939b700 1 --2- 172.21.15.156:0/1323109746 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f99280682f0 0x7f992806a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9940007fd0 tx=0x7f9940009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:36.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:36.911+0000 7f9939ffb700 1 -- 172.21.15.156:0/1323109746 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9934053a50 con 0x7f99440cffa0 2023-11-28T13:08:37.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.103+0000 7f994a39d700 1 -- 172.21.15.156:0/1323109746 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9944061e80 con 0x7f99440cffa0 2023-11-28T13:08:37.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.104+0000 7f9939ffb700 1 -- 172.21.15.156:0/1323109746 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9934057070 con 0x7f99440cffa0 2023-11-28T13:08:37.105 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:08:37.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.106+0000 7f994a39d700 1 -- 172.21.15.156:0/1323109746 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f99280682f0 msgr2=0x7f992806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:37.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.106+0000 7f994a39d700 1 --2- 172.21.15.156:0/1323109746 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f99280682f0 0x7f992806a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9940007fd0 tx=0x7f9940009040 comp rx=0 tx=0).stop 2023-11-28T13:08:37.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.106+0000 7f994a39d700 1 -- 172.21.15.156:0/1323109746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99440cffa0 msgr2=0x7f99440cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:37.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.106+0000 7f994a39d700 1 --2- 172.21.15.156:0/1323109746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99440cffa0 0x7f99440cc520 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9934000c00 tx=0x7f9934004750 comp rx=0 tx=0).stop 2023-11-28T13:08:37.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.106+0000 7f994a39d700 1 -- 172.21.15.156:0/1323109746 shutdown_connections 2023-11-28T13:08:37.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.106+0000 7f994a39d700 1 --2- 172.21.15.156:0/1323109746 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f99280682f0 0x7f992806a7a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.106+0000 7f994a39d700 1 --2- 172.21.15.156:0/1323109746 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f994415c450 0x7f99441e7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.106+0000 7f994a39d700 1 --2- 172.21.15.156:0/1323109746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99440cf170 0x7f9944154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.106+0000 7f994a39d700 1 --2- 172.21.15.156:0/1323109746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99440cffa0 0x7f99440cc520 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.106+0000 7f994a39d700 1 -- 172.21.15.156:0/1323109746 >> 172.21.15.156:0/1323109746 conn(0x7f9944072630 msgr2=0x7f99440b8160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:37.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.106+0000 7f994a39d700 1 -- 172.21.15.156:0/1323109746 shutdown_connections 2023-11-28T13:08:37.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.106+0000 7f994a39d700 1 -- 172.21.15.156:0/1323109746 wait complete. 2023-11-28T13:08:37.126 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:08:37.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.325+0000 7fd6b9fff700 1 Processor -- start 2023-11-28T13:08:37.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.325+0000 7fd6b9fff700 1 -- start start 2023-11-28T13:08:37.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.326+0000 7fd6b9fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd6b40cd7c0 0x7fd6b40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:37.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.326+0000 7fd6b9fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b40d1950 con 0x7fd6b415c9b0 2023-11-28T13:08:37.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.326+0000 7fd6b9fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b40ce140 con 0x7fd6b4161340 2023-11-28T13:08:37.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.326+0000 7fd6b9fff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b40b5800 con 0x7fd6b40cd7c0 2023-11-28T13:08:37.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.327+0000 7fd6b37fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd6b415c9b0 0x7fd6b40b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51972/0 (socket says 172.21.15.156:51972) 2023-11-28T13:08:37.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.327+0000 7fd6b37fe700 1 -- 172.21.15.156:0/1433955693 learned_addr learned my addr 172.21.15.156:0/1433955693 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:37.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.327+0000 7fd6b3fff700 1 --2- 172.21.15.156:0/1433955693 >> v2:172.21.15.186:3300/0 conn(0x7fd6b40cd7c0 0x7fd6b40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:37.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.327+0000 7fd6b27fc700 1 -- 172.21.15.156:0/1433955693 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1209208536 0 0) 0x7fd6b40d1950 con 0x7fd6b415c9b0 2023-11-28T13:08:37.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.327+0000 7fd6b27fc700 1 -- 172.21.15.156:0/1433955693 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd69c0035b0 con 0x7fd6b415c9b0 2023-11-28T13:08:37.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.327+0000 7fd6b27fc700 1 -- 172.21.15.156:0/1433955693 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537833525 0 0) 0x7fd6b40ce140 con 0x7fd6b4161340 2023-11-28T13:08:37.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.327+0000 7fd6b27fc700 1 -- 172.21.15.156:0/1433955693 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6b40d1950 con 0x7fd6b4161340 2023-11-28T13:08:37.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.327+0000 7fd6b27fc700 1 -- 172.21.15.156:0/1433955693 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1380873416 0 0) 0x7fd69c0035b0 con 0x7fd6b415c9b0 2023-11-28T13:08:37.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.327+0000 7fd6b27fc700 1 -- 172.21.15.156:0/1433955693 >> v2:172.21.15.186:3300/0 conn(0x7fd6b40cd7c0 msgr2=0x7fd6b40cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:37.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.327+0000 7fd6b27fc700 1 --2- 172.21.15.156:0/1433955693 >> v2:172.21.15.186:3300/0 conn(0x7fd6b40cd7c0 0x7fd6b40cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.327+0000 7fd6b27fc700 1 -- 172.21.15.156:0/1433955693 >> v1:172.21.15.186:6789/0 conn(0x7fd6b4161340 legacy=0x7fd6b40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:37.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.327+0000 7fd6b27fc700 1 -- 172.21.15.156:0/1433955693 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b40d24f0 con 0x7fd6b415c9b0 2023-11-28T13:08:37.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.327+0000 7fd6b27fc700 1 -- 172.21.15.156:0/1433955693 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7fd6a4001670 con 0x7fd6b415c9b0 2023-11-28T13:08:37.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.327+0000 7fd6b27fc700 1 -- 172.21.15.156:0/1433955693 >> v1:172.21.15.156:6789/0 conn(0x7fd6b415c9b0 legacy=0x7fd6b40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:37.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.327+0000 7fd6b27fc700 1 --2- 172.21.15.156:0/1433955693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd69c003be0 0x7fd69c003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:37.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.328+0000 7fd6b27fc700 1 --2- 172.21.15.156:0/1433955693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c004850 0x7fd69c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:37.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.328+0000 7fd6b27fc700 1 --2- 172.21.15.156:0/1433955693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd69c005250 0x7fd69c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:37.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.328+0000 7fd6b27fc700 1 -- 172.21.15.156:0/1433955693 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b40b5800 con 0x7fd69c004850 2023-11-28T13:08:37.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.328+0000 7fd6b27fc700 1 -- 172.21.15.156:0/1433955693 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c00c160 con 0x7fd69c005250 2023-11-28T13:08:37.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.328+0000 7fd6b27fc700 1 -- 172.21.15.156:0/1433955693 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c00c320 con 0x7fd69c003be0 2023-11-28T13:08:37.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.328+0000 7fd6b2ffd700 1 --2- 172.21.15.156:0/1433955693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c004850 0x7fd69c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:37.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.328+0000 7fd6b3fff700 1 --2- 172.21.15.156:0/1433955693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd69c005250 0x7fd69c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:37.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.328+0000 7fd6b37fe700 1 --2- 172.21.15.156:0/1433955693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd69c003be0 0x7fd69c003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:37.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.328+0000 7fd6b2ffd700 1 -- 172.21.15.156:0/1433955693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd69c003be0 msgr2=0x7fd69c003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:37.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.328+0000 7fd6b2ffd700 1 --2- 172.21.15.156:0/1433955693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd69c003be0 0x7fd69c003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.328+0000 7fd6b2ffd700 1 -- 172.21.15.156:0/1433955693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd69c005250 msgr2=0x7fd69c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:37.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.328+0000 7fd6b2ffd700 1 --2- 172.21.15.156:0/1433955693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd69c005250 0x7fd69c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.328+0000 7fd6b2ffd700 1 -- 172.21.15.156:0/1433955693 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c0035b0 con 0x7fd69c004850 2023-11-28T13:08:37.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.328+0000 7fd6b3fff700 1 -- 172.21.15.156:0/1433955693 reap_dead start 2023-11-28T13:08:37.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.328+0000 7fd6b2ffd700 1 --2- 172.21.15.156:0/1433955693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c004850 0x7fd69c004cc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8007fd0 tx=0x7fd69c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d3ffe7353de616 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:37.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.329+0000 7fd6b27fc700 1 -- 172.21.15.156:0/1433955693 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8001430 con 0x7fd69c004850 2023-11-28T13:08:37.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.329+0000 7fd6b27fc700 1 -- 172.21.15.156:0/1433955693 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a80052f0 con 0x7fd69c004850 2023-11-28T13:08:37.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.329+0000 7fd6b27fc700 1 -- 172.21.15.156:0/1433955693 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8001d40 con 0x7fd69c004850 2023-11-28T13:08:37.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.329+0000 7fd6b9fff700 1 -- 172.21.15.156:0/1433955693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c004850 msgr2=0x7fd69c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:37.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.329+0000 7fd6b9fff700 1 --2- 172.21.15.156:0/1433955693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c004850 0x7fd69c004cc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8007fd0 tx=0x7fd69c005880 comp rx=0 tx=0).stop 2023-11-28T13:08:37.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.329+0000 7fd6b9fff700 1 -- 172.21.15.156:0/1433955693 shutdown_connections 2023-11-28T13:08:37.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.329+0000 7fd6b9fff700 1 --2- 172.21.15.156:0/1433955693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c004850 0x7fd69c004cc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.329+0000 7fd6b9fff700 1 -- 172.21.15.156:0/1433955693 >> 172.21.15.156:0/1433955693 conn(0x7fd6b4072600 msgr2=0x7fd6b40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:37.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.329+0000 7fd6b9fff700 1 -- 172.21.15.156:0/1433955693 shutdown_connections 2023-11-28T13:08:37.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.329+0000 7fd6b9fff700 1 -- 172.21.15.156:0/1433955693 wait complete. 2023-11-28T13:08:37.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.332+0000 7fd6b9fff700 1 Processor -- start 2023-11-28T13:08:37.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.332+0000 7fd6b9fff700 1 -- start start 2023-11-28T13:08:37.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.333+0000 7fd6b9fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c004850 0x7fd6b4157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:37.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.333+0000 7fd6b9fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd6b4156b90 0x7fd6b40c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:37.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.333+0000 7fd6b9fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6b4156fd0 0x7fd6b40b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:37.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.333+0000 7fd6b9fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b40ca040 con 0x7fd69c004850 2023-11-28T13:08:37.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.333+0000 7fd6b9fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b41e7220 con 0x7fd6b4156fd0 2023-11-28T13:08:37.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.333+0000 7fd6b9fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b41e7470 con 0x7fd6b4156b90 2023-11-28T13:08:37.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.333+0000 7fd6b37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c004850 0x7fd6b4157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:37.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.333+0000 7fd6b37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c004850 0x7fd6b4157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41468/0 (socket says 172.21.15.156:41468) 2023-11-28T13:08:37.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.333+0000 7fd6b37fe700 1 -- 172.21.15.156:0/1144095388 learned_addr learned my addr 172.21.15.156:0/1144095388 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:37.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.333+0000 7fd6b2ffd700 1 --2- 172.21.15.156:0/1144095388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd6b4156b90 0x7fd6b40c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:37.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.334+0000 7fd6b3fff700 1 --2- 172.21.15.156:0/1144095388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6b4156fd0 0x7fd6b40b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:37.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.334+0000 7fd6b37fe700 1 -- 172.21.15.156:0/1144095388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd6b4156b90 msgr2=0x7fd6b40c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:37.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.334+0000 7fd6b37fe700 1 --2- 172.21.15.156:0/1144095388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd6b4156b90 0x7fd6b40c6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.334+0000 7fd6b37fe700 1 -- 172.21.15.156:0/1144095388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6b4156fd0 msgr2=0x7fd6b40b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:37.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.334+0000 7fd6b37fe700 1 --2- 172.21.15.156:0/1144095388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6b4156fd0 0x7fd6b40b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.334+0000 7fd6b37fe700 1 -- 172.21.15.156:0/1144095388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b40d1300 con 0x7fd69c004850 2023-11-28T13:08:37.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.334+0000 7fd6b37fe700 1 --2- 172.21.15.156:0/1144095388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c004850 0x7fd6b4157500 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400e550 tx=0x7fd6a400e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:37.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.334+0000 7fd69bfff700 1 -- 172.21.15.156:0/1144095388 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4014480 con 0x7fd69c004850 2023-11-28T13:08:37.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.334+0000 7fd69bfff700 1 -- 172.21.15.156:0/1144095388 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a40145e0 con 0x7fd69c004850 2023-11-28T13:08:37.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.334+0000 7fd69bfff700 1 -- 172.21.15.156:0/1144095388 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4014950 con 0x7fd69c004850 2023-11-28T13:08:37.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.334+0000 7fd6b9fff700 1 -- 172.21.15.156:0/1144095388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b40b5520 con 0x7fd69c004850 2023-11-28T13:08:37.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.334+0000 7fd6b9fff700 1 -- 172.21.15.156:0/1144095388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b40d15e0 con 0x7fd69c004850 2023-11-28T13:08:37.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.336+0000 7fd6b9fff700 1 -- 172.21.15.156:0/1144095388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b4066d60 con 0x7fd69c004850 2023-11-28T13:08:37.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.336+0000 7fd69bfff700 1 -- 172.21.15.156:0/1144095388 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd6a401b030 con 0x7fd69c004850 2023-11-28T13:08:37.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.338+0000 7fd69bfff700 1 --2- 172.21.15.156:0/1144095388 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd6a00682c0 0x7fd6a006a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:37.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.338+0000 7fd69bfff700 1 -- 172.21.15.156:0/1144095388 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd6a4085230 con 0x7fd69c004850 2023-11-28T13:08:37.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.338+0000 7fd6b2ffd700 1 --2- 172.21.15.156:0/1144095388 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd6a00682c0 0x7fd6a006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:37.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.339+0000 7fd6b2ffd700 1 --2- 172.21.15.156:0/1144095388 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd6a00682c0 0x7fd6a006a770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8005040 tx=0x7fd6a800a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:37.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.342+0000 7fd69bfff700 1 -- 172.21.15.156:0/1144095388 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd6a404d5e0 con 0x7fd69c004850 2023-11-28T13:08:37.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.534+0000 7fd6b9fff700 1 -- 172.21.15.156:0/1144095388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd6b40745c0 con 0x7fd69c004850 2023-11-28T13:08:37.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.534+0000 7fd69bfff700 1 -- 172.21.15.156:0/1144095388 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd6a4019020 con 0x7fd69c004850 2023-11-28T13:08:37.536 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T13:08:37.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.537+0000 7fd6b9fff700 1 -- 172.21.15.156:0/1144095388 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd6a00682c0 msgr2=0x7fd6a006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:37.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.537+0000 7fd6b9fff700 1 --2- 172.21.15.156:0/1144095388 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd6a00682c0 0x7fd6a006a770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8005040 tx=0x7fd6a800a000 comp rx=0 tx=0).stop 2023-11-28T13:08:37.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.537+0000 7fd6b9fff700 1 -- 172.21.15.156:0/1144095388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c004850 msgr2=0x7fd6b4157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:37.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.537+0000 7fd6b9fff700 1 --2- 172.21.15.156:0/1144095388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c004850 0x7fd6b4157500 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400e550 tx=0x7fd6a400e860 comp rx=0 tx=0).stop 2023-11-28T13:08:37.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.537+0000 7fd6b9fff700 1 -- 172.21.15.156:0/1144095388 shutdown_connections 2023-11-28T13:08:37.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.537+0000 7fd6b9fff700 1 --2- 172.21.15.156:0/1144095388 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd6a00682c0 0x7fd6a006a770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.537+0000 7fd6b9fff700 1 --2- 172.21.15.156:0/1144095388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6b4156fd0 0x7fd6b40b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.537+0000 7fd6b9fff700 1 --2- 172.21.15.156:0/1144095388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd69c004850 0x7fd6b4157500 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.537+0000 7fd6b9fff700 1 --2- 172.21.15.156:0/1144095388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd6b4156b90 0x7fd6b40c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.537+0000 7fd6b9fff700 1 -- 172.21.15.156:0/1144095388 >> 172.21.15.156:0/1144095388 conn(0x7fd6b4072600 msgr2=0x7fd6b40b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:37.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.537+0000 7fd6b9fff700 1 -- 172.21.15.156:0/1144095388 shutdown_connections 2023-11-28T13:08:37.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.537+0000 7fd6b9fff700 1 -- 172.21.15.156:0/1144095388 wait complete. 2023-11-28T13:08:37.557 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:08:37.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.755+0000 7fe6f57ff700 1 Processor -- start 2023-11-28T13:08:37.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.755+0000 7fe6f57ff700 1 -- start start 2023-11-28T13:08:37.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.757+0000 7fe6f57ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe6f015c9b0 0x7fe6f00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:37.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.757+0000 7fe6f57ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe6f0161340 0x7fe6f00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:37.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.757+0000 7fe6f57ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f00d1950 con 0x7fe6f00d2320 2023-11-28T13:08:37.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.757+0000 7fe6f57ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f00b5800 con 0x7fe6f0161340 2023-11-28T13:08:37.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.757+0000 7fe6f57ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f00b5520 con 0x7fe6f015c9b0 2023-11-28T13:08:37.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.757+0000 7fe6e7fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe6f0161340 0x7fe6f00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:37.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.757+0000 7fe6e7fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe6f0161340 0x7fe6f00d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41480/0 (socket says 172.21.15.156:41480) 2023-11-28T13:08:37.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.757+0000 7fe6e7fff700 1 -- 172.21.15.156:0/229728130 learned_addr learned my addr 172.21.15.156:0/229728130 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:37.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.757+0000 7fe6eeffd700 1 --2- 172.21.15.156:0/229728130 >> v2:172.21.15.186:3300/0 conn(0x7fe6f015c9b0 0x7fe6f00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:37.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.757+0000 7fe6e7fff700 1 -- 172.21.15.156:0/229728130 >> v2:172.21.15.186:3300/0 conn(0x7fe6f015c9b0 msgr2=0x7fe6f00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:37.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.757+0000 7fe6e7fff700 1 --2- 172.21.15.156:0/229728130 >> v2:172.21.15.186:3300/0 conn(0x7fe6f015c9b0 0x7fe6f00b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.757+0000 7fe6e7fff700 1 -- 172.21.15.156:0/229728130 >> v1:172.21.15.186:6789/0 conn(0x7fe6f00d2320 legacy=0x7fe6f00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:37.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.757+0000 7fe6e7fff700 1 -- 172.21.15.156:0/229728130 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f00d1300 con 0x7fe6f0161340 2023-11-28T13:08:37.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.758+0000 7fe6e7fff700 1 --2- 172.21.15.156:0/229728130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6f0161340 0x7fe6f00d1de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0067460 tx=0x7fe6d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161ada12215a9787 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:37.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.758+0000 7fe6ee7fc700 1 -- 172.21.15.156:0/229728130 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d800b870 con 0x7fe6f0161340 2023-11-28T13:08:37.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.758+0000 7fe6ee7fc700 1 -- 172.21.15.156:0/229728130 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d800b9d0 con 0x7fe6f0161340 2023-11-28T13:08:37.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.758+0000 7fe6ee7fc700 1 -- 172.21.15.156:0/229728130 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d800bd60 con 0x7fe6f0161340 2023-11-28T13:08:37.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.758+0000 7fe6f57ff700 1 -- 172.21.15.156:0/229728130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6f0161340 msgr2=0x7fe6f00d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:37.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.758+0000 7fe6f57ff700 1 --2- 172.21.15.156:0/229728130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6f0161340 0x7fe6f00d1de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0067460 tx=0x7fe6d8009bd0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.759+0000 7fe6f57ff700 1 -- 172.21.15.156:0/229728130 shutdown_connections 2023-11-28T13:08:37.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.759+0000 7fe6f57ff700 1 --2- 172.21.15.156:0/229728130 >> v2:172.21.15.186:3300/0 conn(0x7fe6f015c9b0 0x7fe6f00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.759+0000 7fe6f57ff700 1 --2- 172.21.15.156:0/229728130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6f0161340 0x7fe6f00d1de0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.759+0000 7fe6f57ff700 1 -- 172.21.15.156:0/229728130 >> 172.21.15.156:0/229728130 conn(0x7fe6f0072600 msgr2=0x7fe6f00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:37.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.759+0000 7fe6f57ff700 1 -- 172.21.15.156:0/229728130 shutdown_connections 2023-11-28T13:08:37.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.759+0000 7fe6f57ff700 1 -- 172.21.15.156:0/229728130 wait complete. 2023-11-28T13:08:37.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.761+0000 7fe6f57ff700 1 Processor -- start 2023-11-28T13:08:37.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.761+0000 7fe6f57ff700 1 -- start start 2023-11-28T13:08:37.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.762+0000 7fe6f57ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6f00d2320 0x7fe6f01468d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:37.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.762+0000 7fe6f57ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6f015c9b0 0x7fe6f00c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:37.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.762+0000 7fe6f57ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6f0161340 0x7fe6f0159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:37.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.763+0000 7fe6f57ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f00c8c80 con 0x7fe6f00d2320 2023-11-28T13:08:37.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.763+0000 7fe6f57ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f00c8f60 con 0x7fe6f015c9b0 2023-11-28T13:08:37.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.763+0000 7fe6f57ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f00c9240 con 0x7fe6f0161340 2023-11-28T13:08:37.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.763+0000 7fe6eeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6f00d2320 0x7fe6f01468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:37.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.763+0000 7fe6eeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6f00d2320 0x7fe6f01468d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41490/0 (socket says 172.21.15.156:41490) 2023-11-28T13:08:37.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.763+0000 7fe6eeffd700 1 -- 172.21.15.156:0/2426354288 learned_addr learned my addr 172.21.15.156:0/2426354288 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:37.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.763+0000 7fe6e7fff700 1 --2- 172.21.15.156:0/2426354288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6f015c9b0 0x7fe6f00c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:37.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.763+0000 7fe6ef7fe700 1 --2- 172.21.15.156:0/2426354288 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6f0161340 0x7fe6f0159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:37.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.763+0000 7fe6eeffd700 1 -- 172.21.15.156:0/2426354288 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6f0161340 msgr2=0x7fe6f0159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:37.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.763+0000 7fe6eeffd700 1 --2- 172.21.15.156:0/2426354288 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6f0161340 0x7fe6f0159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.763+0000 7fe6eeffd700 1 -- 172.21.15.156:0/2426354288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6f015c9b0 msgr2=0x7fe6f00c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:37.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.763+0000 7fe6eeffd700 1 --2- 172.21.15.156:0/2426354288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6f015c9b0 0x7fe6f00c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.763+0000 7fe6eeffd700 1 -- 172.21.15.156:0/2426354288 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d8009550 con 0x7fe6f00d2320 2023-11-28T13:08:37.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.763+0000 7fe6eeffd700 1 --2- 172.21.15.156:0/2426354288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6f00d2320 0x7fe6f01468d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000e810 tx=0x7fe6e000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:37.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.763+0000 7fe6e77fe700 1 -- 172.21.15.156:0/2426354288 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0013610 con 0x7fe6f00d2320 2023-11-28T13:08:37.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.764+0000 7fe6e77fe700 1 -- 172.21.15.156:0/2426354288 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e0013770 con 0x7fe6f00d2320 2023-11-28T13:08:37.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.764+0000 7fe6e77fe700 1 -- 172.21.15.156:0/2426354288 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0013b20 con 0x7fe6f00d2320 2023-11-28T13:08:37.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.764+0000 7fe6f57ff700 1 -- 172.21.15.156:0/2426354288 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f00b5130 con 0x7fe6f00d2320 2023-11-28T13:08:37.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.764+0000 7fe6f57ff700 1 -- 172.21.15.156:0/2426354288 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f00d15e0 con 0x7fe6f00d2320 2023-11-28T13:08:37.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.765+0000 7fe6f57ff700 1 -- 172.21.15.156:0/2426354288 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f00b52c0 con 0x7fe6f00d2320 2023-11-28T13:08:37.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.765+0000 7fe6e77fe700 1 -- 172.21.15.156:0/2426354288 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe6e0023070 con 0x7fe6f00d2320 2023-11-28T13:08:37.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.766+0000 7fe6e77fe700 1 --2- 172.21.15.156:0/2426354288 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6d00682f0 0x7fe6d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:37.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.766+0000 7fe6e77fe700 1 -- 172.21.15.156:0/2426354288 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe6e0085b00 con 0x7fe6f00d2320 2023-11-28T13:08:37.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.767+0000 7fe6e7fff700 1 --2- 172.21.15.156:0/2426354288 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6d00682f0 0x7fe6d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:37.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.768+0000 7fe6e7fff700 1 --2- 172.21.15.156:0/2426354288 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6d00682f0 0x7fe6d006a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8000c00 tx=0x7fe6d8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:37.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.771+0000 7fe6e77fe700 1 -- 172.21.15.156:0/2426354288 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe6e0087e60 con 0x7fe6f00d2320 2023-11-28T13:08:37.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.962+0000 7fe6f57ff700 1 -- 172.21.15.156:0/2426354288 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe6f00745c0 con 0x7fe6f00d2320 2023-11-28T13:08:37.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.963+0000 7fe6e77fe700 1 -- 172.21.15.156:0/2426354288 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe6e0055c20 con 0x7fe6f00d2320 2023-11-28T13:08:37.965 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T13:08:37.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.965+0000 7fe6f57ff700 1 -- 172.21.15.156:0/2426354288 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6d00682f0 msgr2=0x7fe6d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:37.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.965+0000 7fe6f57ff700 1 --2- 172.21.15.156:0/2426354288 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6d00682f0 0x7fe6d006a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8000c00 tx=0x7fe6d8008030 comp rx=0 tx=0).stop 2023-11-28T13:08:37.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.965+0000 7fe6f57ff700 1 -- 172.21.15.156:0/2426354288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6f00d2320 msgr2=0x7fe6f01468d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:37.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.965+0000 7fe6f57ff700 1 --2- 172.21.15.156:0/2426354288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6f00d2320 0x7fe6f01468d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000e810 tx=0x7fe6e000ebd0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.965+0000 7fe6f57ff700 1 -- 172.21.15.156:0/2426354288 shutdown_connections 2023-11-28T13:08:37.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.965+0000 7fe6f57ff700 1 --2- 172.21.15.156:0/2426354288 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6d00682f0 0x7fe6d006a7a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.965+0000 7fe6f57ff700 1 --2- 172.21.15.156:0/2426354288 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6f0161340 0x7fe6f0159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.965+0000 7fe6f57ff700 1 --2- 172.21.15.156:0/2426354288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6f00d2320 0x7fe6f01468d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.965+0000 7fe6f57ff700 1 --2- 172.21.15.156:0/2426354288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6f015c9b0 0x7fe6f00c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:37.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.965+0000 7fe6f57ff700 1 -- 172.21.15.156:0/2426354288 >> 172.21.15.156:0/2426354288 conn(0x7fe6f0072600 msgr2=0x7fe6f015d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:37.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.965+0000 7fe6f57ff700 1 -- 172.21.15.156:0/2426354288 shutdown_connections 2023-11-28T13:08:37.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:37.965+0000 7fe6f57ff700 1 -- 172.21.15.156:0/2426354288 wait complete. 2023-11-28T13:08:37.992 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:08:37.993 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T13:08:38.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.188+0000 7fd54f9ff700 1 Processor -- start 2023-11-28T13:08:38.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.188+0000 7fd54f9ff700 1 -- start start 2023-11-28T13:08:38.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.189+0000 7fd54f9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd54815c9b0 0x7fd5480b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:38.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.189+0000 7fd54f9ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd548161340 0x7fd5480d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:38.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.189+0000 7fd54f9ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5480d1950 con 0x7fd5480d2320 2023-11-28T13:08:38.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.189+0000 7fd54f9ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5480b5800 con 0x7fd548161340 2023-11-28T13:08:38.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.189+0000 7fd54f9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5480b5520 con 0x7fd54815c9b0 2023-11-28T13:08:38.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.190+0000 7fd54df9c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd5480d2320 0x7fd5480cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51982/0 (socket says 172.21.15.156:51982) 2023-11-28T13:08:38.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.190+0000 7fd54df9c700 1 -- 172.21.15.156:0/3589922077 learned_addr learned my addr 172.21.15.156:0/3589922077 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:38.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.190+0000 7fd54cf9a700 1 --2- 172.21.15.156:0/3589922077 >> v2:172.21.15.186:3300/0 conn(0x7fd548161340 0x7fd5480d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:38.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.190+0000 7fd54d79b700 1 --2- 172.21.15.156:0/3589922077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd54815c9b0 0x7fd5480b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:38.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.190+0000 7fd53ffff700 1 -- 172.21.15.156:0/3589922077 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1062131573 0 0) 0x7fd5480d1950 con 0x7fd5480d2320 2023-11-28T13:08:38.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.190+0000 7fd53ffff700 1 -- 172.21.15.156:0/3589922077 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5340035b0 con 0x7fd5480d2320 2023-11-28T13:08:38.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.190+0000 7fd54cf9a700 1 -- 172.21.15.156:0/3589922077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd54815c9b0 msgr2=0x7fd5480b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:38.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.190+0000 7fd53ffff700 1 -- 172.21.15.156:0/3589922077 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1237403167 0 0) 0x7fd5340035b0 con 0x7fd5480d2320 2023-11-28T13:08:38.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.190+0000 7fd54cf9a700 1 --2- 172.21.15.156:0/3589922077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd54815c9b0 0x7fd5480b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.190+0000 7fd54cf9a700 1 -- 172.21.15.156:0/3589922077 >> v1:172.21.15.156:6789/0 conn(0x7fd5480d2320 legacy=0x7fd5480cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:38.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.190+0000 7fd54cf9a700 1 -- 172.21.15.156:0/3589922077 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5480d1300 con 0x7fd548161340 2023-11-28T13:08:38.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.190+0000 7fd54d79b700 1 --2- 172.21.15.156:0/3589922077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd54815c9b0 0x7fd5480b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:38.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.191+0000 7fd54cf9a700 1 --2- 172.21.15.156:0/3589922077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd548161340 0x7fd5480d1de0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd538009840 tx=0x7fd538009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2ccd583e07bbd49 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:38.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.191+0000 7fd53ffff700 1 -- 172.21.15.156:0/3589922077 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5400019f0 con 0x7fd548161340 2023-11-28T13:08:38.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.191+0000 7fd53ffff700 1 -- 172.21.15.156:0/3589922077 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd538004030 con 0x7fd548161340 2023-11-28T13:08:38.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.191+0000 7fd53ffff700 1 -- 172.21.15.156:0/3589922077 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5380043c0 con 0x7fd548161340 2023-11-28T13:08:38.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.192+0000 7fd54f9ff700 1 -- 172.21.15.156:0/3589922077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd548161340 msgr2=0x7fd5480d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:38.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.192+0000 7fd54f9ff700 1 --2- 172.21.15.156:0/3589922077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd548161340 0x7fd5480d1de0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd538009840 tx=0x7fd538009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:38.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.192+0000 7fd54f9ff700 1 -- 172.21.15.156:0/3589922077 shutdown_connections 2023-11-28T13:08:38.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.192+0000 7fd54f9ff700 1 --2- 172.21.15.156:0/3589922077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd54815c9b0 0x7fd5480b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.192+0000 7fd54f9ff700 1 --2- 172.21.15.156:0/3589922077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd548161340 0x7fd5480d1de0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.192+0000 7fd54f9ff700 1 -- 172.21.15.156:0/3589922077 >> 172.21.15.156:0/3589922077 conn(0x7fd548072600 msgr2=0x7fd5480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:38.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.192+0000 7fd54f9ff700 1 -- 172.21.15.156:0/3589922077 shutdown_connections 2023-11-28T13:08:38.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.192+0000 7fd54f9ff700 1 -- 172.21.15.156:0/3589922077 wait complete. 2023-11-28T13:08:38.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.194+0000 7fd54f9ff700 1 Processor -- start 2023-11-28T13:08:38.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.194+0000 7fd54f9ff700 1 -- start start 2023-11-28T13:08:38.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.195+0000 7fd54f9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5480d2320 0x7fd54806adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:38.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.195+0000 7fd54f9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd54815c9b0 0x7fd5480c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:38.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.195+0000 7fd54f9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd548161340 0x7fd5480c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:38.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.195+0000 7fd54f9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5480c9b30 con 0x7fd5480d2320 2023-11-28T13:08:38.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.195+0000 7fd54f9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5480c96a0 con 0x7fd548161340 2023-11-28T13:08:38.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.195+0000 7fd54f9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5480c8a90 con 0x7fd54815c9b0 2023-11-28T13:08:38.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.195+0000 7fd54d79b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5480d2320 0x7fd54806adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:38.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.195+0000 7fd54d79b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5480d2320 0x7fd54806adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41498/0 (socket says 172.21.15.156:41498) 2023-11-28T13:08:38.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.195+0000 7fd54d79b700 1 -- 172.21.15.156:0/2492314249 learned_addr learned my addr 172.21.15.156:0/2492314249 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:38.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.195+0000 7fd54d79b700 1 -- 172.21.15.156:0/2492314249 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd54815c9b0 msgr2=0x7fd5480c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:38.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.195+0000 7fd54d79b700 1 --2- 172.21.15.156:0/2492314249 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd54815c9b0 0x7fd5480c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.195+0000 7fd54d79b700 1 -- 172.21.15.156:0/2492314249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd548161340 msgr2=0x7fd5480c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:38.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.195+0000 7fd54d79b700 1 --2- 172.21.15.156:0/2492314249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd548161340 0x7fd5480c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.195+0000 7fd54d79b700 1 -- 172.21.15.156:0/2492314249 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5440094a0 con 0x7fd5480d2320 2023-11-28T13:08:38.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.195+0000 7fd54d79b700 1 --2- 172.21.15.156:0/2492314249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5480d2320 0x7fd54806adb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd54400ebe0 tx=0x7fd544009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:38.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.196+0000 7fd53dffb700 1 -- 172.21.15.156:0/2492314249 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54400c910 con 0x7fd5480d2320 2023-11-28T13:08:38.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.196+0000 7fd53dffb700 1 -- 172.21.15.156:0/2492314249 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54400ca70 con 0x7fd5480d2320 2023-11-28T13:08:38.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.196+0000 7fd54f9ff700 1 -- 172.21.15.156:0/2492314249 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5380094a0 con 0x7fd5480d2320 2023-11-28T13:08:38.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.196+0000 7fd53dffb700 1 -- 172.21.15.156:0/2492314249 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54400cde0 con 0x7fd5480d2320 2023-11-28T13:08:38.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.196+0000 7fd54f9ff700 1 -- 172.21.15.156:0/2492314249 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5480b5130 con 0x7fd5480d2320 2023-11-28T13:08:38.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.197+0000 7fd54f9ff700 1 -- 172.21.15.156:0/2492314249 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5480d15e0 con 0x7fd5480d2320 2023-11-28T13:08:38.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.198+0000 7fd53dffb700 1 -- 172.21.15.156:0/2492314249 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd544019030 con 0x7fd5480d2320 2023-11-28T13:08:38.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.198+0000 7fd53dffb700 1 --2- 172.21.15.156:0/2492314249 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd52c0682f0 0x7fd52c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:38.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.199+0000 7fd53dffb700 1 -- 172.21.15.156:0/2492314249 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd544085e40 con 0x7fd5480d2320 2023-11-28T13:08:38.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.199+0000 7fd54cf9a700 1 --2- 172.21.15.156:0/2492314249 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd52c0682f0 0x7fd52c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:38.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.200+0000 7fd54cf9a700 1 --2- 172.21.15.156:0/2492314249 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd52c0682f0 0x7fd52c06a7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd548151da0 tx=0x7fd538010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:38.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.202+0000 7fd53dffb700 1 -- 172.21.15.156:0/2492314249 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd5440525b0 con 0x7fd5480d2320 2023-11-28T13:08:38.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.467+0000 7fd54f9ff700 1 -- 172.21.15.156:0/2492314249 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fd5480b52c0 con 0x7fd5480d2320 2023-11-28T13:08:38.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.497+0000 7fd53dffb700 1 -- 172.21.15.156:0/2492314249 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v48) v1 ==== 145+0+0 (secure 0 0 0) 0x7fd5440170d0 con 0x7fd5480d2320 2023-11-28T13:08:38.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.500+0000 7fd54f9ff700 1 -- 172.21.15.156:0/2492314249 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd52c0682f0 msgr2=0x7fd52c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:38.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.500+0000 7fd54f9ff700 1 --2- 172.21.15.156:0/2492314249 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd52c0682f0 0x7fd52c06a7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd548151da0 tx=0x7fd538010000 comp rx=0 tx=0).stop 2023-11-28T13:08:38.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.500+0000 7fd54f9ff700 1 -- 172.21.15.156:0/2492314249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5480d2320 msgr2=0x7fd54806adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:38.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.500+0000 7fd54f9ff700 1 --2- 172.21.15.156:0/2492314249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5480d2320 0x7fd54806adb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd54400ebe0 tx=0x7fd544009ec0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.500+0000 7fd54f9ff700 1 -- 172.21.15.156:0/2492314249 shutdown_connections 2023-11-28T13:08:38.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.500+0000 7fd54f9ff700 1 --2- 172.21.15.156:0/2492314249 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd52c0682f0 0x7fd52c06a7a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.500+0000 7fd54f9ff700 1 --2- 172.21.15.156:0/2492314249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd548161340 0x7fd5480c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.500+0000 7fd54f9ff700 1 --2- 172.21.15.156:0/2492314249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5480d2320 0x7fd54806adb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.500+0000 7fd54f9ff700 1 --2- 172.21.15.156:0/2492314249 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd54815c9b0 0x7fd5480c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.500+0000 7fd54f9ff700 1 -- 172.21.15.156:0/2492314249 >> 172.21.15.156:0/2492314249 conn(0x7fd548072600 msgr2=0x7fd54815bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:38.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.500+0000 7fd54f9ff700 1 -- 172.21.15.156:0/2492314249 shutdown_connections 2023-11-28T13:08:38.503 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.500+0000 7fd54f9ff700 1 -- 172.21.15.156:0/2492314249 wait complete. 2023-11-28T13:08:38.520 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T13:08:38.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.718+0000 7fcd429ff700 1 Processor -- start 2023-11-28T13:08:38.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.718+0000 7fcd429ff700 1 -- start start 2023-11-28T13:08:38.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.719+0000 7fcd429ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fcd3c15c9b0 0x7fcd3c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:38.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.719+0000 7fcd429ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c161340 0x7fcd3c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:38.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.719+0000 7fcd429ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcd3c0d2320 0x7fcd3c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:38.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.719+0000 7fcd429ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3c0b5800 con 0x7fcd3c0d2320 2023-11-28T13:08:38.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.719+0000 7fcd429ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3c0d1950 con 0x7fcd3c15c9b0 2023-11-28T13:08:38.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.719+0000 7fcd429ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c0b5520 con 0x7fcd3c161340 2023-11-28T13:08:38.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.720+0000 7fcd40f9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcd3c0d2320 0x7fcd3c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:38.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.720+0000 7fcd40f9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcd3c0d2320 0x7fcd3c0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41510/0 (socket says 172.21.15.156:41510) 2023-11-28T13:08:38.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.720+0000 7fcd40f9c700 1 -- 172.21.15.156:0/777717028 learned_addr learned my addr 172.21.15.156:0/777717028 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:38.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.720+0000 7fcd3bfff700 1 --2- 172.21.15.156:0/777717028 >> v2:172.21.15.186:3300/0 conn(0x7fcd3c15c9b0 0x7fcd3c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:38.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.720+0000 7fcd40f9c700 1 -- 172.21.15.156:0/777717028 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c161340 msgr2=0x7fcd3c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:38.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.720+0000 7fcd3b7fe700 1 --2- 172.21.15.156:0/777717028 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c161340 0x7fcd3c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:38.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.720+0000 7fcd40f9c700 1 --2- 172.21.15.156:0/777717028 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c161340 0x7fcd3c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.720+0000 7fcd40f9c700 1 -- 172.21.15.156:0/777717028 >> v2:172.21.15.186:3300/0 conn(0x7fcd3c15c9b0 msgr2=0x7fcd3c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:38.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.720+0000 7fcd40f9c700 1 --2- 172.21.15.156:0/777717028 >> v2:172.21.15.186:3300/0 conn(0x7fcd3c15c9b0 0x7fcd3c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.720+0000 7fcd40f9c700 1 -- 172.21.15.156:0/777717028 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c0d1300 con 0x7fcd3c0d2320 2023-11-28T13:08:38.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.720+0000 7fcd3b7fe700 1 --2- 172.21.15.156:0/777717028 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c161340 0x7fcd3c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:38.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.720+0000 7fcd3bfff700 1 --2- 172.21.15.156:0/777717028 >> v2:172.21.15.186:3300/0 conn(0x7fcd3c15c9b0 0x7fcd3c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:38.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.720+0000 7fcd40f9c700 1 --2- 172.21.15.156:0/777717028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3c0d2320 0x7fcd3c0cd7c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c067460 tx=0x7fcd3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c85afbdc7b02ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:38.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.721+0000 7fcd3affd700 1 -- 172.21.15.156:0/777717028 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3400e070 con 0x7fcd3c0d2320 2023-11-28T13:08:38.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.721+0000 7fcd3affd700 1 -- 172.21.15.156:0/777717028 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3400b9c0 con 0x7fcd3c0d2320 2023-11-28T13:08:38.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.721+0000 7fcd3affd700 1 -- 172.21.15.156:0/777717028 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3400bd70 con 0x7fcd3c0d2320 2023-11-28T13:08:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.721+0000 7fcd429ff700 1 -- 172.21.15.156:0/777717028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3c0d2320 msgr2=0x7fcd3c0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.721+0000 7fcd429ff700 1 --2- 172.21.15.156:0/777717028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3c0d2320 0x7fcd3c0cd7c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c067460 tx=0x7fcd3400b330 comp rx=0 tx=0).stop 2023-11-28T13:08:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.721+0000 7fcd429ff700 1 -- 172.21.15.156:0/777717028 shutdown_connections 2023-11-28T13:08:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.721+0000 7fcd429ff700 1 --2- 172.21.15.156:0/777717028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3c0d2320 0x7fcd3c0cd7c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.721+0000 7fcd429ff700 1 --2- 172.21.15.156:0/777717028 >> v2:172.21.15.186:3300/0 conn(0x7fcd3c15c9b0 0x7fcd3c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.721+0000 7fcd429ff700 1 --2- 172.21.15.156:0/777717028 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c161340 0x7fcd3c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.721+0000 7fcd429ff700 1 -- 172.21.15.156:0/777717028 >> 172.21.15.156:0/777717028 conn(0x7fcd3c072600 msgr2=0x7fcd3c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.721+0000 7fcd429ff700 1 -- 172.21.15.156:0/777717028 shutdown_connections 2023-11-28T13:08:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.721+0000 7fcd429ff700 1 -- 172.21.15.156:0/777717028 wait complete. 2023-11-28T13:08:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.723+0000 7fcd429ff700 1 Processor -- start 2023-11-28T13:08:38.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.723+0000 7fcd429ff700 1 -- start start 2023-11-28T13:08:38.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.724+0000 7fcd429ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3c0d2320 0x7fcd3c155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:38.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.724+0000 7fcd429ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c15c9b0 0x7fcd3c1577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:38.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.724+0000 7fcd429ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3c161340 0x7fcd3c14fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:38.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.724+0000 7fcd429ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c150960 con 0x7fcd3c161340 2023-11-28T13:08:38.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.724+0000 7fcd429ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c1e7a20 con 0x7fcd3c0d2320 2023-11-28T13:08:38.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.724+0000 7fcd429ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c1e7d00 con 0x7fcd3c15c9b0 2023-11-28T13:08:38.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.724+0000 7fcd40f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3c161340 0x7fcd3c14fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:38.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.724+0000 7fcd40f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3c161340 0x7fcd3c14fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41524/0 (socket says 172.21.15.156:41524) 2023-11-28T13:08:38.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.724+0000 7fcd40f9c700 1 -- 172.21.15.156:0/361400632 learned_addr learned my addr 172.21.15.156:0/361400632 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:38.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.724+0000 7fcd3bfff700 1 --2- 172.21.15.156:0/361400632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3c0d2320 0x7fcd3c155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:38.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.724+0000 7fcd3b7fe700 1 --2- 172.21.15.156:0/361400632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c15c9b0 0x7fcd3c1577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:38.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.725+0000 7fcd40f9c700 1 -- 172.21.15.156:0/361400632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c15c9b0 msgr2=0x7fcd3c1577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:38.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.725+0000 7fcd40f9c700 1 --2- 172.21.15.156:0/361400632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c15c9b0 0x7fcd3c1577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.725+0000 7fcd40f9c700 1 -- 172.21.15.156:0/361400632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3c0d2320 msgr2=0x7fcd3c155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:38.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.725+0000 7fcd40f9c700 1 --2- 172.21.15.156:0/361400632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3c0d2320 0x7fcd3c155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.725+0000 7fcd40f9c700 1 -- 172.21.15.156:0/361400632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd34009b30 con 0x7fcd3c161340 2023-11-28T13:08:38.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.725+0000 7fcd40f9c700 1 --2- 172.21.15.156:0/361400632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3c161340 0x7fcd3c14fe40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400b370 tx=0x7fcd34004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:38.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.725+0000 7fcd38ff9700 1 -- 172.21.15.156:0/361400632 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34004250 con 0x7fcd3c161340 2023-11-28T13:08:38.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.725+0000 7fcd38ff9700 1 -- 172.21.15.156:0/361400632 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd340043f0 con 0x7fcd3c161340 2023-11-28T13:08:38.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.725+0000 7fcd429ff700 1 -- 172.21.15.156:0/361400632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3c0b5130 con 0x7fcd3c161340 2023-11-28T13:08:38.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.725+0000 7fcd38ff9700 1 -- 172.21.15.156:0/361400632 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd340047a0 con 0x7fcd3c161340 2023-11-28T13:08:38.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.725+0000 7fcd429ff700 1 -- 172.21.15.156:0/361400632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3c0d15e0 con 0x7fcd3c161340 2023-11-28T13:08:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.726+0000 7fcd429ff700 1 -- 172.21.15.156:0/361400632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3c0b52c0 con 0x7fcd3c161340 2023-11-28T13:08:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.727+0000 7fcd38ff9700 1 -- 172.21.15.156:0/361400632 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcd3401b430 con 0x7fcd3c161340 2023-11-28T13:08:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.728+0000 7fcd38ff9700 1 --2- 172.21.15.156:0/361400632 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcd280682f0 0x7fcd2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.728+0000 7fcd38ff9700 1 -- 172.21.15.156:0/361400632 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcd34087070 con 0x7fcd3c161340 2023-11-28T13:08:38.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.728+0000 7fcd3bfff700 1 --2- 172.21.15.156:0/361400632 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcd280682f0 0x7fcd2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:38.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.729+0000 7fcd3bfff700 1 --2- 172.21.15.156:0/361400632 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcd280682f0 0x7fcd2806a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c150bb0 tx=0x7fcd2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:38.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.732+0000 7fcd38ff9700 1 -- 172.21.15.156:0/361400632 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcd340b85e0 con 0x7fcd3c161340 2023-11-28T13:08:38.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.926+0000 7fcd429ff700 1 -- 172.21.15.156:0/361400632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcd3c0745c0 con 0x7fcd3c161340 2023-11-28T13:08:38.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.930+0000 7fcd38ff9700 1 -- 172.21.15.156:0/361400632 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcd34053780 con 0x7fcd3c161340 2023-11-28T13:08:38.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.932+0000 7fcd429ff700 1 -- 172.21.15.156:0/361400632 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcd280682f0 msgr2=0x7fcd2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:38.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.932+0000 7fcd429ff700 1 --2- 172.21.15.156:0/361400632 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcd280682f0 0x7fcd2806a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c150bb0 tx=0x7fcd2c008040 comp rx=0 tx=0).stop 2023-11-28T13:08:38.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.932+0000 7fcd429ff700 1 -- 172.21.15.156:0/361400632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3c161340 msgr2=0x7fcd3c14fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:38.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.932+0000 7fcd429ff700 1 --2- 172.21.15.156:0/361400632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3c161340 0x7fcd3c14fe40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400b370 tx=0x7fcd34004ad0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.932+0000 7fcd429ff700 1 -- 172.21.15.156:0/361400632 shutdown_connections 2023-11-28T13:08:38.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.932+0000 7fcd429ff700 1 --2- 172.21.15.156:0/361400632 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcd280682f0 0x7fcd2806a7a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.932+0000 7fcd429ff700 1 --2- 172.21.15.156:0/361400632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3c161340 0x7fcd3c14fe40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.932+0000 7fcd429ff700 1 --2- 172.21.15.156:0/361400632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3c0d2320 0x7fcd3c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.932+0000 7fcd429ff700 1 --2- 172.21.15.156:0/361400632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c15c9b0 0x7fcd3c1577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:38.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.932+0000 7fcd429ff700 1 -- 172.21.15.156:0/361400632 >> 172.21.15.156:0/361400632 conn(0x7fcd3c072600 msgr2=0x7fcd3c15ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:38.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.932+0000 7fcd429ff700 1 -- 172.21.15.156:0/361400632 shutdown_connections 2023-11-28T13:08:38.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:38.933+0000 7fcd429ff700 1 -- 172.21.15.156:0/361400632 wait complete. 2023-11-28T13:08:38.953 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:08:38.953 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:08:39.054 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:08:39.156 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:08:39.156 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:08:39.156 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:08:39.200 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:08:39.200 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:08:39.284 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:08:39.284 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:08:39.368 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:08:39.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.603+0000 7f76c5bff700 1 Processor -- start 2023-11-28T13:08:39.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.603+0000 7f76c5bff700 1 -- start start 2023-11-28T13:08:39.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.604+0000 7f76c5bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76c00cd7c0 0x7f76c00cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:39.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.604+0000 7f76c5bff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c00d1950 con 0x7f76c0161340 2023-11-28T13:08:39.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.605+0000 7f76c5bff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c00ce140 con 0x7f76c015c9b0 2023-11-28T13:08:39.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.605+0000 7f76c5bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c00b5800 con 0x7f76c00cd7c0 2023-11-28T13:08:39.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.605+0000 7f76b7fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f76c0161340 0x7f76c00d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:51988/0 (socket says 172.21.15.156:51988) 2023-11-28T13:08:39.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.605+0000 7f76b7fff700 1 -- 172.21.15.156:0/666801355 learned_addr learned my addr 172.21.15.156:0/666801355 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:39.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.605+0000 7f76bffff700 1 --2- 172.21.15.156:0/666801355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76c00cd7c0 0x7f76c00cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:39.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.605+0000 7f76beffd700 1 -- 172.21.15.156:0/666801355 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600436160 0 0) 0x7f76c00d1950 con 0x7f76c0161340 2023-11-28T13:08:39.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.605+0000 7f76beffd700 1 -- 172.21.15.156:0/666801355 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76ac0035b0 con 0x7f76c0161340 2023-11-28T13:08:39.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.605+0000 7f76beffd700 1 -- 172.21.15.156:0/666801355 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181058024 0 0) 0x7f76c00ce140 con 0x7f76c015c9b0 2023-11-28T13:08:39.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.605+0000 7f76beffd700 1 -- 172.21.15.156:0/666801355 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76c00d1950 con 0x7f76c015c9b0 2023-11-28T13:08:39.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.605+0000 7f76beffd700 1 -- 172.21.15.156:0/666801355 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3790228658 0 0) 0x7f76ac0035b0 con 0x7f76c0161340 2023-11-28T13:08:39.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76beffd700 1 -- 172.21.15.156:0/666801355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76c00cd7c0 msgr2=0x7f76c00cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:39.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76beffd700 1 --2- 172.21.15.156:0/666801355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76c00cd7c0 0x7f76c00cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:39.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76beffd700 1 -- 172.21.15.156:0/666801355 >> v1:172.21.15.186:6789/0 conn(0x7f76c015c9b0 legacy=0x7f76c00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:39.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76beffd700 1 -- 172.21.15.156:0/666801355 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c00d24f0 con 0x7f76c0161340 2023-11-28T13:08:39.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76beffd700 1 -- 172.21.15.156:0/666801355 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f76a8001670 con 0x7f76c0161340 2023-11-28T13:08:39.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76bffff700 1 --2- 172.21.15.156:0/666801355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76c00cd7c0 0x7f76c00cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:39.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76beffd700 1 -- 172.21.15.156:0/666801355 >> v1:172.21.15.156:6789/0 conn(0x7f76c0161340 legacy=0x7f76c00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:39.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76beffd700 1 --2- 172.21.15.156:0/666801355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76ac003bc0 0x7f76ac003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:39.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76beffd700 1 --2- 172.21.15.156:0/666801355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76ac004850 0x7f76ac004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:39.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76beffd700 1 --2- 172.21.15.156:0/666801355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76c00cd7c0 0x7f76ac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:39.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76beffd700 1 -- 172.21.15.156:0/666801355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c00b5800 con 0x7f76ac003bc0 2023-11-28T13:08:39.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76beffd700 1 -- 172.21.15.156:0/666801355 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac00baf0 con 0x7f76ac004850 2023-11-28T13:08:39.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76beffd700 1 -- 172.21.15.156:0/666801355 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac00bcb0 con 0x7f76c00cd7c0 2023-11-28T13:08:39.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76bf7fe700 1 --2- 172.21.15.156:0/666801355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76ac003bc0 0x7f76ac003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:39.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76b7fff700 1 --2- 172.21.15.156:0/666801355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76ac004850 0x7f76ac004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:39.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76bffff700 1 --2- 172.21.15.156:0/666801355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76c00cd7c0 0x7f76ac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:39.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76bf7fe700 1 -- 172.21.15.156:0/666801355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76c00cd7c0 msgr2=0x7f76ac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:39.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76bf7fe700 1 --2- 172.21.15.156:0/666801355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76c00cd7c0 0x7f76ac005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:39.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76bf7fe700 1 -- 172.21.15.156:0/666801355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76ac004850 msgr2=0x7f76ac004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:39.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76bf7fe700 1 --2- 172.21.15.156:0/666801355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76ac004850 0x7f76ac004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:39.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.606+0000 7f76bf7fe700 1 -- 172.21.15.156:0/666801355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b8009a70 con 0x7f76ac003bc0 2023-11-28T13:08:39.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.607+0000 7f76bf7fe700 1 --2- 172.21.15.156:0/666801355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76ac003bc0 0x7f76ac003fb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f76b0001940 tx=0x7f76ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe014ad995e2444 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:39.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.607+0000 7f76beffd700 1 -- 172.21.15.156:0/666801355 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b0001430 con 0x7f76ac003bc0 2023-11-28T13:08:39.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.607+0000 7f76beffd700 1 -- 172.21.15.156:0/666801355 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b0001d70 con 0x7f76ac003bc0 2023-11-28T13:08:39.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.607+0000 7f76beffd700 1 -- 172.21.15.156:0/666801355 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b0005510 con 0x7f76ac003bc0 2023-11-28T13:08:39.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.607+0000 7f76c5bff700 1 -- 172.21.15.156:0/666801355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76ac003bc0 msgr2=0x7f76ac003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:39.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.607+0000 7f76c5bff700 1 --2- 172.21.15.156:0/666801355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76ac003bc0 0x7f76ac003fb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f76b0001940 tx=0x7f76ac005880 comp rx=0 tx=0).stop 2023-11-28T13:08:39.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.607+0000 7f76bffff700 1 -- 172.21.15.156:0/666801355 reap_dead start 2023-11-28T13:08:39.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.607+0000 7f76c5bff700 1 -- 172.21.15.156:0/666801355 shutdown_connections 2023-11-28T13:08:39.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.607+0000 7f76c5bff700 1 -- 172.21.15.156:0/666801355 >> 172.21.15.156:0/666801355 conn(0x7f76c0072600 msgr2=0x7f76c00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:39.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.608+0000 7f76c5bff700 1 -- 172.21.15.156:0/666801355 shutdown_connections 2023-11-28T13:08:39.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.608+0000 7f76c5bff700 1 -- 172.21.15.156:0/666801355 wait complete. 2023-11-28T13:08:39.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.609+0000 7f76c5bff700 1 Processor -- start 2023-11-28T13:08:39.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.610+0000 7f76c5bff700 1 -- start start 2023-11-28T13:08:39.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.610+0000 7f76c5bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76ac003bc0 0x7f76c01592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:39.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.610+0000 7f76c5bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76c0156ed0 0x7f76c01562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:39.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.610+0000 7f76c5bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76c0157400 0x7f76c01567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:39.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.610+0000 7f76c5bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c0156d20 con 0x7f76c0156ed0 2023-11-28T13:08:39.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.610+0000 7f76c5bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c00c9b10 con 0x7f76ac003bc0 2023-11-28T13:08:39.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.610+0000 7f76c5bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c00b52f0 con 0x7f76c0157400 2023-11-28T13:08:39.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.611+0000 7f76b7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76c0156ed0 0x7f76c01562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:39.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.611+0000 7f76b7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76c0156ed0 0x7f76c01562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41538/0 (socket says 172.21.15.156:41538) 2023-11-28T13:08:39.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.611+0000 7f76b7fff700 1 -- 172.21.15.156:0/745820668 learned_addr learned my addr 172.21.15.156:0/745820668 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:39.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.611+0000 7f76bf7fe700 1 --2- 172.21.15.156:0/745820668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76ac003bc0 0x7f76c01592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:39.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.611+0000 7f76bffff700 1 --2- 172.21.15.156:0/745820668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76c0157400 0x7f76c01567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:39.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.611+0000 7f76b7fff700 1 -- 172.21.15.156:0/745820668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76c0157400 msgr2=0x7f76c01567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:39.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.611+0000 7f76b7fff700 1 --2- 172.21.15.156:0/745820668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76c0157400 0x7f76c01567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:39.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.611+0000 7f76b7fff700 1 -- 172.21.15.156:0/745820668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76ac003bc0 msgr2=0x7f76c01592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:39.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.611+0000 7f76b7fff700 1 --2- 172.21.15.156:0/745820668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76ac003bc0 0x7f76c01592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:39.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.611+0000 7f76b7fff700 1 -- 172.21.15.156:0/745820668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c00d1300 con 0x7f76c0156ed0 2023-11-28T13:08:39.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.611+0000 7f76bffff700 1 --2- 172.21.15.156:0/745820668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76c0157400 0x7f76c01567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:39.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.611+0000 7f76b7fff700 1 --2- 172.21.15.156:0/745820668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76c0156ed0 0x7f76c01562a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f76a800e730 tx=0x7f76a800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:39.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.612+0000 7f76bcff9700 1 -- 172.21.15.156:0/745820668 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8012590 con 0x7f76c0156ed0 2023-11-28T13:08:39.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.612+0000 7f76bcff9700 1 -- 172.21.15.156:0/745820668 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a80126f0 con 0x7f76c0156ed0 2023-11-28T13:08:39.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.612+0000 7f76bcff9700 1 -- 172.21.15.156:0/745820668 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8012aa0 con 0x7f76c0156ed0 2023-11-28T13:08:39.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.612+0000 7f76c5bff700 1 -- 172.21.15.156:0/745820668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c00b5520 con 0x7f76c0156ed0 2023-11-28T13:08:39.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.612+0000 7f76c5bff700 1 -- 172.21.15.156:0/745820668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c00d15e0 con 0x7f76c0156ed0 2023-11-28T13:08:39.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.613+0000 7f76c5bff700 1 -- 172.21.15.156:0/745820668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c00ca2d0 con 0x7f76c0156ed0 2023-11-28T13:08:39.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.614+0000 7f76bcff9700 1 -- 172.21.15.156:0/745820668 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f76a8012c00 con 0x7f76c0156ed0 2023-11-28T13:08:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.615+0000 7f76bcff9700 1 --2- 172.21.15.156:0/745820668 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f76a00682f0 0x7f76a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.615+0000 7f76bcff9700 1 -- 172.21.15.156:0/745820668 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f76a8085be0 con 0x7f76c0156ed0 2023-11-28T13:08:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.615+0000 7f76bf7fe700 1 --2- 172.21.15.156:0/745820668 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f76a00682f0 0x7f76a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.616+0000 7f76bf7fe700 1 --2- 172.21.15.156:0/745820668 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f76a00682f0 0x7f76a006a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f76c00b4cf0 tx=0x7f76b000a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:39.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.619+0000 7f76bcff9700 1 -- 172.21.15.156:0/745820668 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f76c00ca2d0 con 0x7f76c0156ed0 2023-11-28T13:08:39.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.812+0000 7f76c5bff700 1 -- 172.21.15.156:0/745820668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76c00ca2d0 con 0x7f76c0156ed0 2023-11-28T13:08:39.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.814+0000 7f76bcff9700 1 -- 172.21.15.156:0/745820668 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17897 (secure 0 0 0) 0x7f76c00ca2d0 con 0x7f76c0156ed0 2023-11-28T13:08:39.815 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:39.816 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":21,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:08:04.060053+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:08:00.903485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:08:02.037854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:08:39.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.816+0000 7f76c5bff700 1 -- 172.21.15.156:0/745820668 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f76a00682f0 msgr2=0x7f76a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:39.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.816+0000 7f76c5bff700 1 --2- 172.21.15.156:0/745820668 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f76a00682f0 0x7f76a006a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f76c00b4cf0 tx=0x7f76b000a000 comp rx=0 tx=0).stop 2023-11-28T13:08:39.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.816+0000 7f76c5bff700 1 -- 172.21.15.156:0/745820668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76c0156ed0 msgr2=0x7f76c01562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:39.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.816+0000 7f76c5bff700 1 --2- 172.21.15.156:0/745820668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76c0156ed0 0x7f76c01562a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f76a800e730 tx=0x7f76a800eaf0 comp rx=0 tx=0).stop 2023-11-28T13:08:39.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.816+0000 7f76c5bff700 1 -- 172.21.15.156:0/745820668 shutdown_connections 2023-11-28T13:08:39.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.816+0000 7f76c5bff700 1 --2- 172.21.15.156:0/745820668 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f76a00682f0 0x7f76a006a7a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:39.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.816+0000 7f76c5bff700 1 --2- 172.21.15.156:0/745820668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76c0157400 0x7f76c01567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:39.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.816+0000 7f76c5bff700 1 --2- 172.21.15.156:0/745820668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76ac003bc0 0x7f76c01592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:39.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.816+0000 7f76c5bff700 1 --2- 172.21.15.156:0/745820668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76c0156ed0 0x7f76c01562a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:39.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.816+0000 7f76c5bff700 1 -- 172.21.15.156:0/745820668 >> 172.21.15.156:0/745820668 conn(0x7f76c0072600 msgr2=0x7f76c0160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:39.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.816+0000 7f76c5bff700 1 -- 172.21.15.156:0/745820668 shutdown_connections 2023-11-28T13:08:39.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:39.816+0000 7f76c5bff700 1 -- 172.21.15.156:0/745820668 wait complete. 2023-11-28T13:08:39.836 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:08:00.903485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:08:02.037854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:08:39.837 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:08:40.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.034+0000 7f10ad5ff700 1 Processor -- start 2023-11-28T13:08:40.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.034+0000 7f10ad5ff700 1 -- start start 2023-11-28T13:08:40.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.036+0000 7f10ad5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10a8161340 0x7f10a80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.036+0000 7f10ad5ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f10a80d2320 0x7f10a80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.036+0000 7f10ad5ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a80d1950 con 0x7f10a815c9b0 2023-11-28T13:08:40.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.036+0000 7f10ad5ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a80b5800 con 0x7f10a80d2320 2023-11-28T13:08:40.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.036+0000 7f10ad5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a80b5300 con 0x7f10a8161340 2023-11-28T13:08:40.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.036+0000 7f10a6ffd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f10a815c9b0 0x7f10a80b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:52000/0 (socket says 172.21.15.156:52000) 2023-11-28T13:08:40.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.036+0000 7f10a6ffd700 1 -- 172.21.15.156:0/4030379172 learned_addr learned my addr 172.21.15.156:0/4030379172 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:40.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.036+0000 7f10a77fe700 1 --2- 172.21.15.156:0/4030379172 >> v2:172.21.15.156:3300/0 conn(0x7f10a80d2320 0x7f10a80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.036+0000 7f109ffff700 1 --2- 172.21.15.156:0/4030379172 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10a8161340 0x7f10a80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.036+0000 7f10a67fc700 1 -- 172.21.15.156:0/4030379172 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2312484460 0 0) 0x7f10a80d1950 con 0x7f10a815c9b0 2023-11-28T13:08:40.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.036+0000 7f10a77fe700 1 -- 172.21.15.156:0/4030379172 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10a8161340 msgr2=0x7f10a80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:40.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.036+0000 7f10a77fe700 1 --2- 172.21.15.156:0/4030379172 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10a8161340 0x7f10a80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.036+0000 7f10a77fe700 1 -- 172.21.15.156:0/4030379172 >> v1:172.21.15.156:6789/0 conn(0x7f10a815c9b0 legacy=0x7f10a80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:40.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.036+0000 7f10a77fe700 1 -- 172.21.15.156:0/4030379172 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a80b5520 con 0x7f10a80d2320 2023-11-28T13:08:40.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.036+0000 7f109ffff700 1 --2- 172.21.15.156:0/4030379172 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10a8161340 0x7f10a80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:40.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.037+0000 7f10a77fe700 1 --2- 172.21.15.156:0/4030379172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10a80d2320 0x7f10a80cd7c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009dc0 tx=0x7f10a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81a353a00d3d0fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:40.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.037+0000 7f10a67fc700 1 -- 172.21.15.156:0/4030379172 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a000e070 con 0x7f10a80d2320 2023-11-28T13:08:40.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.037+0000 7f10a67fc700 1 -- 172.21.15.156:0/4030379172 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a000bb90 con 0x7f10a80d2320 2023-11-28T13:08:40.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.037+0000 7f10a67fc700 1 -- 172.21.15.156:0/4030379172 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a0012430 con 0x7f10a80d2320 2023-11-28T13:08:40.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.037+0000 7f10ad5ff700 1 -- 172.21.15.156:0/4030379172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10a80d2320 msgr2=0x7f10a80cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:40.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.037+0000 7f10ad5ff700 1 --2- 172.21.15.156:0/4030379172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10a80d2320 0x7f10a80cd7c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009dc0 tx=0x7f10a000b330 comp rx=0 tx=0).stop 2023-11-28T13:08:40.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.037+0000 7f10ad5ff700 1 -- 172.21.15.156:0/4030379172 shutdown_connections 2023-11-28T13:08:40.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.037+0000 7f10ad5ff700 1 --2- 172.21.15.156:0/4030379172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10a80d2320 0x7f10a80cd7c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.037+0000 7f10ad5ff700 1 --2- 172.21.15.156:0/4030379172 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10a8161340 0x7f10a80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.037+0000 7f10ad5ff700 1 -- 172.21.15.156:0/4030379172 >> 172.21.15.156:0/4030379172 conn(0x7f10a8072600 msgr2=0x7f10a80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:40.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.038+0000 7f10ad5ff700 1 -- 172.21.15.156:0/4030379172 shutdown_connections 2023-11-28T13:08:40.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.038+0000 7f10ad5ff700 1 -- 172.21.15.156:0/4030379172 wait complete. 2023-11-28T13:08:40.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.039+0000 7f10ad5ff700 1 Processor -- start 2023-11-28T13:08:40.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.039+0000 7f10ad5ff700 1 -- start start 2023-11-28T13:08:40.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.040+0000 7f10ad5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10a80d2320 0x7f10a80c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.040+0000 7f10ad5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10a815c9b0 0x7f10a80b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.040+0000 7f10ad5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10a8161340 0x7f10a80b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.040+0000 7f10ad5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a8152ce0 con 0x7f10a80d2320 2023-11-28T13:08:40.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.040+0000 7f10ad5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a8150d60 con 0x7f10a8161340 2023-11-28T13:08:40.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.040+0000 7f10ad5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a81576d0 con 0x7f10a815c9b0 2023-11-28T13:08:40.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.040+0000 7f10a6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10a80d2320 0x7f10a80c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.040+0000 7f10a6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10a80d2320 0x7f10a80c6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41554/0 (socket says 172.21.15.156:41554) 2023-11-28T13:08:40.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.041+0000 7f10a6ffd700 1 -- 172.21.15.156:0/1155290346 learned_addr learned my addr 172.21.15.156:0/1155290346 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:40.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.041+0000 7f109ffff700 1 --2- 172.21.15.156:0/1155290346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10a815c9b0 0x7f10a80b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.041+0000 7f10a77fe700 1 --2- 172.21.15.156:0/1155290346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10a8161340 0x7f10a80b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.041+0000 7f10a6ffd700 1 -- 172.21.15.156:0/1155290346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10a815c9b0 msgr2=0x7f10a80b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:40.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.041+0000 7f10a6ffd700 1 --2- 172.21.15.156:0/1155290346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10a815c9b0 0x7f10a80b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.041+0000 7f10a6ffd700 1 -- 172.21.15.156:0/1155290346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10a8161340 msgr2=0x7f10a80b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:40.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.041+0000 7f10a6ffd700 1 --2- 172.21.15.156:0/1155290346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10a8161340 0x7f10a80b8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.041+0000 7f10a6ffd700 1 -- 172.21.15.156:0/1155290346 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a0009a70 con 0x7f10a80d2320 2023-11-28T13:08:40.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.041+0000 7f10a77fe700 1 --2- 172.21.15.156:0/1155290346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10a8161340 0x7f10a80b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:40.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.041+0000 7f10a6ffd700 1 --2- 172.21.15.156:0/1155290346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10a80d2320 0x7f10a80c6040 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f109800c550 tx=0x7f109800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:40.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.041+0000 7f109f7fe700 1 -- 172.21.15.156:0/1155290346 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098004c80 con 0x7f10a80d2320 2023-11-28T13:08:40.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.041+0000 7f109f7fe700 1 -- 172.21.15.156:0/1155290346 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1098004e00 con 0x7f10a80d2320 2023-11-28T13:08:40.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.041+0000 7f10ad5ff700 1 -- 172.21.15.156:0/1155290346 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a80d15e0 con 0x7f10a80d2320 2023-11-28T13:08:40.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.041+0000 7f109f7fe700 1 -- 172.21.15.156:0/1155290346 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098007310 con 0x7f10a80d2320 2023-11-28T13:08:40.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.042+0000 7f10ad5ff700 1 -- 172.21.15.156:0/1155290346 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a80d1300 con 0x7f10a80d2320 2023-11-28T13:08:40.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.042+0000 7f10ad5ff700 1 -- 172.21.15.156:0/1155290346 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a815cd80 con 0x7f10a80d2320 2023-11-28T13:08:40.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.043+0000 7f109f7fe700 1 -- 172.21.15.156:0/1155290346 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f109801a020 con 0x7f10a80d2320 2023-11-28T13:08:40.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.044+0000 7f109f7fe700 1 --2- 172.21.15.156:0/1155290346 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f10880682f0 0x7f108806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.044+0000 7f109f7fe700 1 -- 172.21.15.156:0/1155290346 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1098085810 con 0x7f10a80d2320 2023-11-28T13:08:40.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.044+0000 7f109ffff700 1 --2- 172.21.15.156:0/1155290346 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f10880682f0 0x7f108806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.045+0000 7f109ffff700 1 --2- 172.21.15.156:0/1155290346 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f10880682f0 0x7f108806a7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f10a806ba30 tx=0x7f1090008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:40.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.048+0000 7f109f7fe700 1 -- 172.21.15.156:0/1155290346 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1098052010 con 0x7f10a80d2320 2023-11-28T13:08:40.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.239+0000 7f10ad5ff700 1 -- 172.21.15.156:0/1155290346 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f10a80745c0 con 0x7f10a80d2320 2023-11-28T13:08:40.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.240+0000 7f109f7fe700 1 -- 172.21.15.156:0/1155290346 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1098055630 con 0x7f10a80d2320 2023-11-28T13:08:40.242 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:08:40.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.242+0000 7f10ad5ff700 1 -- 172.21.15.156:0/1155290346 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f10880682f0 msgr2=0x7f108806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:40.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.242+0000 7f10ad5ff700 1 --2- 172.21.15.156:0/1155290346 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f10880682f0 0x7f108806a7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f10a806ba30 tx=0x7f1090008040 comp rx=0 tx=0).stop 2023-11-28T13:08:40.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.242+0000 7f10ad5ff700 1 -- 172.21.15.156:0/1155290346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10a80d2320 msgr2=0x7f10a80c6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:40.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.242+0000 7f10ad5ff700 1 --2- 172.21.15.156:0/1155290346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10a80d2320 0x7f10a80c6040 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f109800c550 tx=0x7f109800c910 comp rx=0 tx=0).stop 2023-11-28T13:08:40.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.242+0000 7f10ad5ff700 1 -- 172.21.15.156:0/1155290346 shutdown_connections 2023-11-28T13:08:40.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.242+0000 7f10ad5ff700 1 --2- 172.21.15.156:0/1155290346 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f10880682f0 0x7f108806a7a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.242+0000 7f10ad5ff700 1 --2- 172.21.15.156:0/1155290346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10a8161340 0x7f10a80b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.242+0000 7f10ad5ff700 1 --2- 172.21.15.156:0/1155290346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10a80d2320 0x7f10a80c6040 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.242+0000 7f10ad5ff700 1 --2- 172.21.15.156:0/1155290346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10a815c9b0 0x7f10a80b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.242+0000 7f10ad5ff700 1 -- 172.21.15.156:0/1155290346 >> 172.21.15.156:0/1155290346 conn(0x7f10a8072600 msgr2=0x7f10a80b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:40.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.242+0000 7f10ad5ff700 1 -- 172.21.15.156:0/1155290346 shutdown_connections 2023-11-28T13:08:40.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.242+0000 7f10ad5ff700 1 -- 172.21.15.156:0/1155290346 wait complete. 2023-11-28T13:08:40.263 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:08:40.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.460+0000 7f35f49ff700 1 Processor -- start 2023-11-28T13:08:40.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.461+0000 7f35f49ff700 1 -- start start 2023-11-28T13:08:40.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.462+0000 7f35f49ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f35f0161340 0x7f35f00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.462+0000 7f35f49ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f35f00d2320 0x7f35f00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.462+0000 7f35f49ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f00d1950 con 0x7f35f015c9b0 2023-11-28T13:08:40.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.462+0000 7f35f49ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f00b5800 con 0x7f35f00d2320 2023-11-28T13:08:40.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.462+0000 7f35f49ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f00b5300 con 0x7f35f0161340 2023-11-28T13:08:40.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.462+0000 7f35eed9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f35f00d2320 0x7f35f00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.462+0000 7f35eed9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f35f00d2320 0x7f35f00cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41562/0 (socket says 172.21.15.156:41562) 2023-11-28T13:08:40.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.462+0000 7f35eed9d700 1 -- 172.21.15.156:0/2024976812 learned_addr learned my addr 172.21.15.156:0/2024976812 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:40.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.462+0000 7f35e7fff700 1 --2- 172.21.15.156:0/2024976812 >> v2:172.21.15.186:3300/0 conn(0x7f35f0161340 0x7f35f00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.462+0000 7f35eed9d700 1 -- 172.21.15.156:0/2024976812 >> v2:172.21.15.186:3300/0 conn(0x7f35f0161340 msgr2=0x7f35f00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:40.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.462+0000 7f35eed9d700 1 --2- 172.21.15.156:0/2024976812 >> v2:172.21.15.186:3300/0 conn(0x7f35f0161340 0x7f35f00d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.463+0000 7f35eed9d700 1 -- 172.21.15.156:0/2024976812 >> v1:172.21.15.186:6789/0 conn(0x7f35f015c9b0 legacy=0x7f35f00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:40.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.463+0000 7f35eed9d700 1 -- 172.21.15.156:0/2024976812 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f00b5520 con 0x7f35f00d2320 2023-11-28T13:08:40.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.463+0000 7f35e7fff700 1 --2- 172.21.15.156:0/2024976812 >> v2:172.21.15.186:3300/0 conn(0x7f35f0161340 0x7f35f00d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:40.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.463+0000 7f35eed9d700 1 --2- 172.21.15.156:0/2024976812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35f00d2320 0x7f35f00cd7c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f35f00685c0 tx=0x7f35e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c544af432b9db6c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:40.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.463+0000 7f35edd9b700 1 -- 172.21.15.156:0/2024976812 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e800e070 con 0x7f35f00d2320 2023-11-28T13:08:40.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.463+0000 7f35edd9b700 1 -- 172.21.15.156:0/2024976812 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e8003a20 con 0x7f35f00d2320 2023-11-28T13:08:40.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.463+0000 7f35edd9b700 1 -- 172.21.15.156:0/2024976812 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e8003dd0 con 0x7f35f00d2320 2023-11-28T13:08:40.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.464+0000 7f35f49ff700 1 -- 172.21.15.156:0/2024976812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35f00d2320 msgr2=0x7f35f00cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:40.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.464+0000 7f35f49ff700 1 --2- 172.21.15.156:0/2024976812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35f00d2320 0x7f35f00cd7c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f35f00685c0 tx=0x7f35e800b330 comp rx=0 tx=0).stop 2023-11-28T13:08:40.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.464+0000 7f35f49ff700 1 -- 172.21.15.156:0/2024976812 shutdown_connections 2023-11-28T13:08:40.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.464+0000 7f35f49ff700 1 --2- 172.21.15.156:0/2024976812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35f00d2320 0x7f35f00cd7c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.464+0000 7f35f49ff700 1 --2- 172.21.15.156:0/2024976812 >> v2:172.21.15.186:3300/0 conn(0x7f35f0161340 0x7f35f00d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.464+0000 7f35f49ff700 1 -- 172.21.15.156:0/2024976812 >> 172.21.15.156:0/2024976812 conn(0x7f35f0072600 msgr2=0x7f35f00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:40.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.464+0000 7f35f49ff700 1 -- 172.21.15.156:0/2024976812 shutdown_connections 2023-11-28T13:08:40.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.464+0000 7f35f49ff700 1 -- 172.21.15.156:0/2024976812 wait complete. 2023-11-28T13:08:40.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.465+0000 7f35f49ff700 1 Processor -- start 2023-11-28T13:08:40.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.466+0000 7f35f49ff700 1 -- start start 2023-11-28T13:08:40.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.466+0000 7f35f49ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35f00d2320 0x7f35f00c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.466+0000 7f35f49ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f015c9b0 0x7f35f00c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.466+0000 7f35f49ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35f0161340 0x7f35f01f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.466+0000 7f35f49ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f01f3120 con 0x7f35f0161340 2023-11-28T13:08:40.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.466+0000 7f35f49ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f01f2780 con 0x7f35f015c9b0 2023-11-28T13:08:40.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.466+0000 7f35f49ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f01f2a60 con 0x7f35f00d2320 2023-11-28T13:08:40.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.467+0000 7f35eed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35f0161340 0x7f35f01f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.467+0000 7f35eed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35f0161340 0x7f35f01f18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41572/0 (socket says 172.21.15.156:41572) 2023-11-28T13:08:40.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.467+0000 7f35eed9d700 1 -- 172.21.15.156:0/2484794672 learned_addr learned my addr 172.21.15.156:0/2484794672 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:40.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.467+0000 7f35e7fff700 1 --2- 172.21.15.156:0/2484794672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f015c9b0 0x7f35f00c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.467+0000 7f35ee59c700 1 --2- 172.21.15.156:0/2484794672 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35f00d2320 0x7f35f00c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.467+0000 7f35eed9d700 1 -- 172.21.15.156:0/2484794672 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35f00d2320 msgr2=0x7f35f00c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:40.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.467+0000 7f35eed9d700 1 --2- 172.21.15.156:0/2484794672 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35f00d2320 0x7f35f00c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.467+0000 7f35eed9d700 1 -- 172.21.15.156:0/2484794672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f015c9b0 msgr2=0x7f35f00c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:40.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.467+0000 7f35eed9d700 1 --2- 172.21.15.156:0/2484794672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f015c9b0 0x7f35f00c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.467+0000 7f35eed9d700 1 -- 172.21.15.156:0/2484794672 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e8009b30 con 0x7f35f0161340 2023-11-28T13:08:40.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.467+0000 7f35eed9d700 1 --2- 172.21.15.156:0/2484794672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35f0161340 0x7f35f01f18f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f35e8004000 tx=0x7f35e8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:40.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.468+0000 7f35e6ffd700 1 -- 172.21.15.156:0/2484794672 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e8004440 con 0x7f35f0161340 2023-11-28T13:08:40.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.468+0000 7f35e6ffd700 1 -- 172.21.15.156:0/2484794672 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e80092e0 con 0x7f35f0161340 2023-11-28T13:08:40.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.468+0000 7f35f49ff700 1 -- 172.21.15.156:0/2484794672 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f00d15e0 con 0x7f35f0161340 2023-11-28T13:08:40.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.468+0000 7f35e6ffd700 1 -- 172.21.15.156:0/2484794672 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e8004820 con 0x7f35f0161340 2023-11-28T13:08:40.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.468+0000 7f35f49ff700 1 -- 172.21.15.156:0/2484794672 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f00d1300 con 0x7f35f0161340 2023-11-28T13:08:40.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.469+0000 7f35f49ff700 1 -- 172.21.15.156:0/2484794672 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f015cd80 con 0x7f35f0161340 2023-11-28T13:08:40.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.470+0000 7f35e6ffd700 1 -- 172.21.15.156:0/2484794672 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f35e8022030 con 0x7f35f0161340 2023-11-28T13:08:40.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.470+0000 7f35e6ffd700 1 --2- 172.21.15.156:0/2484794672 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35d00682f0 0x7f35d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.471+0000 7f35e6ffd700 1 -- 172.21.15.156:0/2484794672 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f35e80871c0 con 0x7f35f0161340 2023-11-28T13:08:40.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.471+0000 7f35ee59c700 1 --2- 172.21.15.156:0/2484794672 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35d00682f0 0x7f35d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.472+0000 7f35ee59c700 1 --2- 172.21.15.156:0/2484794672 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35d00682f0 0x7f35d006a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009870 tx=0x7f35e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:40.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.475+0000 7f35e6ffd700 1 -- 172.21.15.156:0/2484794672 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f35e8053930 con 0x7f35f0161340 2023-11-28T13:08:40.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.668+0000 7f35f49ff700 1 -- 172.21.15.156:0/2484794672 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35f01f3260 con 0x7f35f0161340 2023-11-28T13:08:40.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.669+0000 7f35e6ffd700 1 -- 172.21.15.156:0/2484794672 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35e8007310 con 0x7f35f0161340 2023-11-28T13:08:40.670 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:08:40.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.671+0000 7f35f49ff700 1 -- 172.21.15.156:0/2484794672 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35d00682f0 msgr2=0x7f35d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:40.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.671+0000 7f35f49ff700 1 --2- 172.21.15.156:0/2484794672 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35d00682f0 0x7f35d006a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009870 tx=0x7f35e0008040 comp rx=0 tx=0).stop 2023-11-28T13:08:40.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.671+0000 7f35f49ff700 1 -- 172.21.15.156:0/2484794672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35f0161340 msgr2=0x7f35f01f18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:40.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.671+0000 7f35f49ff700 1 --2- 172.21.15.156:0/2484794672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35f0161340 0x7f35f01f18f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f35e8004000 tx=0x7f35e8004ad0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.671+0000 7f35f49ff700 1 -- 172.21.15.156:0/2484794672 shutdown_connections 2023-11-28T13:08:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.671+0000 7f35f49ff700 1 --2- 172.21.15.156:0/2484794672 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35d00682f0 0x7f35d006a7a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.671+0000 7f35f49ff700 1 --2- 172.21.15.156:0/2484794672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35f0161340 0x7f35f01f18f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.671+0000 7f35f49ff700 1 --2- 172.21.15.156:0/2484794672 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35f00d2320 0x7f35f00c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.671+0000 7f35f49ff700 1 --2- 172.21.15.156:0/2484794672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f015c9b0 0x7f35f00c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.671+0000 7f35f49ff700 1 -- 172.21.15.156:0/2484794672 >> 172.21.15.156:0/2484794672 conn(0x7f35f0072600 msgr2=0x7f35f00b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:40.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.671+0000 7f35f49ff700 1 -- 172.21.15.156:0/2484794672 shutdown_connections 2023-11-28T13:08:40.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.671+0000 7f35f49ff700 1 -- 172.21.15.156:0/2484794672 wait complete. 2023-11-28T13:08:40.691 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:08:40.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.889+0000 7f78979ff700 1 Processor -- start 2023-11-28T13:08:40.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.889+0000 7f78979ff700 1 -- start start 2023-11-28T13:08:40.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.890+0000 7f78979ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7890161340 0x7f78900d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.890+0000 7f78979ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f78900d2320 0x7f78900cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.890+0000 7f78979ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78900d1950 con 0x7f789015c9b0 2023-11-28T13:08:40.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.890+0000 7f78979ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78900b5800 con 0x7f78900d2320 2023-11-28T13:08:40.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.890+0000 7f78979ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78900b5300 con 0x7f7890161340 2023-11-28T13:08:40.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.891+0000 7f7894f9a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7890161340 0x7f78900d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.891+0000 7f7895f9c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f78900d2320 0x7f78900cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.891+0000 7f7894f9a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7890161340 0x7f78900d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:33354/0 (socket says 172.21.15.156:33354) 2023-11-28T13:08:40.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.891+0000 7f7894f9a700 1 -- 172.21.15.156:0/750084834 learned_addr learned my addr 172.21.15.156:0/750084834 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:40.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.891+0000 7f7894f9a700 1 -- 172.21.15.156:0/750084834 >> v2:172.21.15.186:3300/0 conn(0x7f78900d2320 msgr2=0x7f78900cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:40.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.891+0000 7f7894f9a700 1 --2- 172.21.15.156:0/750084834 >> v2:172.21.15.186:3300/0 conn(0x7f78900d2320 0x7f78900cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.891+0000 7f7894f9a700 1 -- 172.21.15.156:0/750084834 >> v1:172.21.15.186:6789/0 conn(0x7f789015c9b0 legacy=0x7f78900b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:40.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.891+0000 7f7894f9a700 1 -- 172.21.15.156:0/750084834 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78900b5520 con 0x7f7890161340 2023-11-28T13:08:40.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.892+0000 7f7894f9a700 1 --2- 172.21.15.156:0/750084834 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7890161340 0x7f78900d1de0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f78800098f0 tx=0x7f7880009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc177e9c64977cdd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:40.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.892+0000 7f7887fff700 1 -- 172.21.15.156:0/750084834 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880011830 con 0x7f7890161340 2023-11-28T13:08:40.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.892+0000 7f7887fff700 1 -- 172.21.15.156:0/750084834 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7880011990 con 0x7f7890161340 2023-11-28T13:08:40.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.892+0000 7f7887fff700 1 -- 172.21.15.156:0/750084834 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880011d20 con 0x7f7890161340 2023-11-28T13:08:40.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.893+0000 7f78979ff700 1 -- 172.21.15.156:0/750084834 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7890161340 msgr2=0x7f78900d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:40.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.893+0000 7f78979ff700 1 --2- 172.21.15.156:0/750084834 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7890161340 0x7f78900d1de0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f78800098f0 tx=0x7f7880009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:40.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.893+0000 7f78979ff700 1 -- 172.21.15.156:0/750084834 shutdown_connections 2023-11-28T13:08:40.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.893+0000 7f78979ff700 1 --2- 172.21.15.156:0/750084834 >> v2:172.21.15.186:3300/0 conn(0x7f78900d2320 0x7f78900cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.893+0000 7f78979ff700 1 --2- 172.21.15.156:0/750084834 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7890161340 0x7f78900d1de0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.893+0000 7f78979ff700 1 -- 172.21.15.156:0/750084834 >> 172.21.15.156:0/750084834 conn(0x7f7890072600 msgr2=0x7f78900cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:40.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.893+0000 7f78979ff700 1 -- 172.21.15.156:0/750084834 shutdown_connections 2023-11-28T13:08:40.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.893+0000 7f78979ff700 1 -- 172.21.15.156:0/750084834 wait complete. 2023-11-28T13:08:40.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.895+0000 7f78979ff700 1 Processor -- start 2023-11-28T13:08:40.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.895+0000 7f78979ff700 1 -- start start 2023-11-28T13:08:40.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.896+0000 7f78979ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78900d2320 0x7f789014a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.896+0000 7f78979ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f789015c9b0 0x7f78900c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.896+0000 7f78979ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7890161340 0x7f789006a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.896+0000 7f78979ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78901f3470 con 0x7f789015c9b0 2023-11-28T13:08:40.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.896+0000 7f78979ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78901f3750 con 0x7f78900d2320 2023-11-28T13:08:40.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.896+0000 7f78979ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78901f3a30 con 0x7f7890161340 2023-11-28T13:08:40.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.896+0000 7f7894f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f789015c9b0 0x7f78900c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.896+0000 7f7894f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f789015c9b0 0x7f78900c8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41576/0 (socket says 172.21.15.156:41576) 2023-11-28T13:08:40.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.896+0000 7f7894f9a700 1 -- 172.21.15.156:0/337084865 learned_addr learned my addr 172.21.15.156:0/337084865 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:40.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.896+0000 7f789579b700 1 --2- 172.21.15.156:0/337084865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78900d2320 0x7f789014a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.896+0000 7f7895f9c700 1 --2- 172.21.15.156:0/337084865 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7890161340 0x7f789006a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.896+0000 7f7894f9a700 1 -- 172.21.15.156:0/337084865 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7890161340 msgr2=0x7f789006a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:40.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.896+0000 7f7894f9a700 1 --2- 172.21.15.156:0/337084865 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7890161340 0x7f789006a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.896+0000 7f7894f9a700 1 -- 172.21.15.156:0/337084865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78900d2320 msgr2=0x7f789014a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:40.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.896+0000 7f7894f9a700 1 --2- 172.21.15.156:0/337084865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78900d2320 0x7f789014a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:40.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.896+0000 7f7894f9a700 1 -- 172.21.15.156:0/337084865 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7880009550 con 0x7f789015c9b0 2023-11-28T13:08:40.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.897+0000 7f7894f9a700 1 --2- 172.21.15.156:0/337084865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f789015c9b0 0x7f78900c8690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7880009c40 tx=0x7f7880004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:40.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.897+0000 7f7885ffb700 1 -- 172.21.15.156:0/337084865 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880004480 con 0x7f789015c9b0 2023-11-28T13:08:40.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.897+0000 7f7885ffb700 1 -- 172.21.15.156:0/337084865 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788001f070 con 0x7f789015c9b0 2023-11-28T13:08:40.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.897+0000 7f78979ff700 1 -- 172.21.15.156:0/337084865 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78900d15e0 con 0x7f789015c9b0 2023-11-28T13:08:40.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.897+0000 7f7885ffb700 1 -- 172.21.15.156:0/337084865 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78800038e0 con 0x7f789015c9b0 2023-11-28T13:08:40.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.897+0000 7f78979ff700 1 -- 172.21.15.156:0/337084865 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78900d1300 con 0x7f789015c9b0 2023-11-28T13:08:40.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.898+0000 7f78979ff700 1 -- 172.21.15.156:0/337084865 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789015cd80 con 0x7f789015c9b0 2023-11-28T13:08:40.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.899+0000 7f7885ffb700 1 -- 172.21.15.156:0/337084865 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f788002a020 con 0x7f789015c9b0 2023-11-28T13:08:40.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.900+0000 7f7885ffb700 1 --2- 172.21.15.156:0/337084865 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78740682f0 0x7f787406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:40.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.900+0000 7f7885ffb700 1 -- 172.21.15.156:0/337084865 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7880087380 con 0x7f789015c9b0 2023-11-28T13:08:40.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.900+0000 7f789579b700 1 --2- 172.21.15.156:0/337084865 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78740682f0 0x7f787406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:40.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.901+0000 7f789579b700 1 --2- 172.21.15.156:0/337084865 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78740682f0 0x7f787406a7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f789006b4c0 tx=0x7f788c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:40.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:40.904+0000 7f7885ffb700 1 -- 172.21.15.156:0/337084865 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7880053af0 con 0x7f789015c9b0 2023-11-28T13:08:41.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.094+0000 7f78979ff700 1 -- 172.21.15.156:0/337084865 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78900745c0 con 0x7f789015c9b0 2023-11-28T13:08:41.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.094+0000 7f7885ffb700 1 -- 172.21.15.156:0/337084865 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7880057110 con 0x7f789015c9b0 2023-11-28T13:08:41.096 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T13:08:41.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.097+0000 7f78979ff700 1 -- 172.21.15.156:0/337084865 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78740682f0 msgr2=0x7f787406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:41.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.097+0000 7f78979ff700 1 --2- 172.21.15.156:0/337084865 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78740682f0 0x7f787406a7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f789006b4c0 tx=0x7f788c009040 comp rx=0 tx=0).stop 2023-11-28T13:08:41.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.097+0000 7f78979ff700 1 -- 172.21.15.156:0/337084865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f789015c9b0 msgr2=0x7f78900c8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:41.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.097+0000 7f78979ff700 1 --2- 172.21.15.156:0/337084865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f789015c9b0 0x7f78900c8690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7880009c40 tx=0x7f7880004750 comp rx=0 tx=0).stop 2023-11-28T13:08:41.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.097+0000 7f78979ff700 1 -- 172.21.15.156:0/337084865 shutdown_connections 2023-11-28T13:08:41.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.097+0000 7f78979ff700 1 --2- 172.21.15.156:0/337084865 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78740682f0 0x7f787406a7a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.097+0000 7f78979ff700 1 --2- 172.21.15.156:0/337084865 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7890161340 0x7f789006a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.097+0000 7f78979ff700 1 --2- 172.21.15.156:0/337084865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78900d2320 0x7f789014a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.097+0000 7f78979ff700 1 --2- 172.21.15.156:0/337084865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f789015c9b0 0x7f78900c8690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.097+0000 7f78979ff700 1 -- 172.21.15.156:0/337084865 >> 172.21.15.156:0/337084865 conn(0x7f7890072600 msgr2=0x7f78900b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:41.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.097+0000 7f78979ff700 1 -- 172.21.15.156:0/337084865 shutdown_connections 2023-11-28T13:08:41.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.097+0000 7f78979ff700 1 -- 172.21.15.156:0/337084865 wait complete. 2023-11-28T13:08:41.117 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:08:41.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.314+0000 7f2dba9ff700 1 Processor -- start 2023-11-28T13:08:41.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.314+0000 7f2dba9ff700 1 -- start start 2023-11-28T13:08:41.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.315+0000 7f2dba9ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2db4161340 0x7f2db40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:41.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.315+0000 7f2dba9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db40d2320 0x7f2db40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:41.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.315+0000 7f2dba9ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db40d1950 con 0x7f2db415c9b0 2023-11-28T13:08:41.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.315+0000 7f2dba9ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db40b5800 con 0x7f2db4161340 2023-11-28T13:08:41.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.315+0000 7f2dba9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db40b5300 con 0x7f2db40d2320 2023-11-28T13:08:41.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.316+0000 7f2db37fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2db4161340 0x7f2db40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:41.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.316+0000 7f2db37fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2db4161340 0x7f2db40d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41586/0 (socket says 172.21.15.156:41586) 2023-11-28T13:08:41.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.316+0000 7f2db37fe700 1 -- 172.21.15.156:0/430851386 learned_addr learned my addr 172.21.15.156:0/430851386 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:41.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.316+0000 7f2db8f9c700 1 --2- 172.21.15.156:0/430851386 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db40d2320 0x7f2db40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:41.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.316+0000 7f2db37fe700 1 -- 172.21.15.156:0/430851386 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db40d2320 msgr2=0x7f2db40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:41.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.316+0000 7f2db37fe700 1 --2- 172.21.15.156:0/430851386 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db40d2320 0x7f2db40cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.316+0000 7f2db37fe700 1 -- 172.21.15.156:0/430851386 >> v1:172.21.15.186:6789/0 conn(0x7f2db415c9b0 legacy=0x7f2db40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:41.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.316+0000 7f2db37fe700 1 -- 172.21.15.156:0/430851386 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db40b5520 con 0x7f2db4161340 2023-11-28T13:08:41.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.316+0000 7f2db37fe700 1 --2- 172.21.15.156:0/430851386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2db4161340 0x7f2db40d1de0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2da80098f0 tx=0x7f2da8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac06630efe95cba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:41.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.317+0000 7f2db2ffd700 1 -- 172.21.15.156:0/430851386 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da8004230 con 0x7f2db4161340 2023-11-28T13:08:41.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.317+0000 7f2db2ffd700 1 -- 172.21.15.156:0/430851386 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da8004390 con 0x7f2db4161340 2023-11-28T13:08:41.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.317+0000 7f2db2ffd700 1 -- 172.21.15.156:0/430851386 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da80036a0 con 0x7f2db4161340 2023-11-28T13:08:41.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.317+0000 7f2dba9ff700 1 -- 172.21.15.156:0/430851386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2db4161340 msgr2=0x7f2db40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:41.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.317+0000 7f2dba9ff700 1 --2- 172.21.15.156:0/430851386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2db4161340 0x7f2db40d1de0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2da80098f0 tx=0x7f2da8009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:41.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.317+0000 7f2dba9ff700 1 -- 172.21.15.156:0/430851386 shutdown_connections 2023-11-28T13:08:41.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.317+0000 7f2dba9ff700 1 --2- 172.21.15.156:0/430851386 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db40d2320 0x7f2db40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.317+0000 7f2dba9ff700 1 --2- 172.21.15.156:0/430851386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2db4161340 0x7f2db40d1de0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.317+0000 7f2dba9ff700 1 -- 172.21.15.156:0/430851386 >> 172.21.15.156:0/430851386 conn(0x7f2db4072600 msgr2=0x7f2db40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:41.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.317+0000 7f2dba9ff700 1 -- 172.21.15.156:0/430851386 shutdown_connections 2023-11-28T13:08:41.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.317+0000 7f2dba9ff700 1 -- 172.21.15.156:0/430851386 wait complete. 2023-11-28T13:08:41.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.319+0000 7f2dba9ff700 1 Processor -- start 2023-11-28T13:08:41.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.319+0000 7f2dba9ff700 1 -- start start 2023-11-28T13:08:41.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.320+0000 7f2dba9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2db40d2320 0x7f2db40c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:41.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.320+0000 7f2dba9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2db415c9b0 0x7f2db4146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:41.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.320+0000 7f2dba9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db4161340 0x7f2db41579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:41.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.320+0000 7f2dba9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db4153c80 con 0x7f2db415c9b0 2023-11-28T13:08:41.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.320+0000 7f2dba9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db40c8140 con 0x7f2db40d2320 2023-11-28T13:08:41.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.320+0000 7f2dba9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db40c8420 con 0x7f2db4161340 2023-11-28T13:08:41.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.321+0000 7f2db37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2db415c9b0 0x7f2db4146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:41.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.321+0000 7f2db37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2db415c9b0 0x7f2db4146d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41600/0 (socket says 172.21.15.156:41600) 2023-11-28T13:08:41.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.321+0000 7f2db37fe700 1 -- 172.21.15.156:0/3775774731 learned_addr learned my addr 172.21.15.156:0/3775774731 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:41.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.321+0000 7f2db8f9c700 1 --2- 172.21.15.156:0/3775774731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db4161340 0x7f2db41579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:41.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.321+0000 7f2db3fff700 1 --2- 172.21.15.156:0/3775774731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2db40d2320 0x7f2db40c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:41.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.321+0000 7f2db37fe700 1 -- 172.21.15.156:0/3775774731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db4161340 msgr2=0x7f2db41579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:41.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.321+0000 7f2db37fe700 1 --2- 172.21.15.156:0/3775774731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db4161340 0x7f2db41579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.321+0000 7f2db37fe700 1 -- 172.21.15.156:0/3775774731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2db40d2320 msgr2=0x7f2db40c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:41.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.321+0000 7f2db37fe700 1 --2- 172.21.15.156:0/3775774731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2db40d2320 0x7f2db40c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.321+0000 7f2db37fe700 1 -- 172.21.15.156:0/3775774731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da8009550 con 0x7f2db415c9b0 2023-11-28T13:08:41.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.321+0000 7f2db3fff700 1 --2- 172.21.15.156:0/3775774731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2db40d2320 0x7f2db40c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:41.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.321+0000 7f2db37fe700 1 --2- 172.21.15.156:0/3775774731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2db415c9b0 0x7f2db4146d60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2db40675d0 tx=0x7f2da8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:41.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.321+0000 7f2db0ff9700 1 -- 172.21.15.156:0/3775774731 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da8003cc0 con 0x7f2db415c9b0 2023-11-28T13:08:41.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.322+0000 7f2db0ff9700 1 -- 172.21.15.156:0/3775774731 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da801f070 con 0x7f2db415c9b0 2023-11-28T13:08:41.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.322+0000 7f2dba9ff700 1 -- 172.21.15.156:0/3775774731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db40d15e0 con 0x7f2db415c9b0 2023-11-28T13:08:41.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.322+0000 7f2db0ff9700 1 -- 172.21.15.156:0/3775774731 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da800b8a0 con 0x7f2db415c9b0 2023-11-28T13:08:41.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.322+0000 7f2dba9ff700 1 -- 172.21.15.156:0/3775774731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db40d1300 con 0x7f2db415c9b0 2023-11-28T13:08:41.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.323+0000 7f2dba9ff700 1 -- 172.21.15.156:0/3775774731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db415cd80 con 0x7f2db415c9b0 2023-11-28T13:08:41.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.324+0000 7f2db0ff9700 1 -- 172.21.15.156:0/3775774731 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2da802a020 con 0x7f2db415c9b0 2023-11-28T13:08:41.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.324+0000 7f2db0ff9700 1 --2- 172.21.15.156:0/3775774731 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2da00682f0 0x7f2da006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:41.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.325+0000 7f2db0ff9700 1 -- 172.21.15.156:0/3775774731 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2da8087430 con 0x7f2db415c9b0 2023-11-28T13:08:41.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.325+0000 7f2db3fff700 1 --2- 172.21.15.156:0/3775774731 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2da00682f0 0x7f2da006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:41.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.325+0000 7f2db3fff700 1 --2- 172.21.15.156:0/3775774731 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2da00682f0 0x7f2da006a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2da4006fd0 tx=0x7f2da4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:41.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.329+0000 7f2db0ff9700 1 -- 172.21.15.156:0/3775774731 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2da8053c30 con 0x7f2db415c9b0 2023-11-28T13:08:41.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.519+0000 7f2dba9ff700 1 -- 172.21.15.156:0/3775774731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2db415cd80 con 0x7f2db415c9b0 2023-11-28T13:08:41.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.519+0000 7f2db0ff9700 1 -- 172.21.15.156:0/3775774731 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2da8010380 con 0x7f2db415c9b0 2023-11-28T13:08:41.521 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T13:08:41.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.522+0000 7f2dba9ff700 1 -- 172.21.15.156:0/3775774731 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2da00682f0 msgr2=0x7f2da006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:41.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.522+0000 7f2dba9ff700 1 --2- 172.21.15.156:0/3775774731 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2da00682f0 0x7f2da006a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2da4006fd0 tx=0x7f2da4008040 comp rx=0 tx=0).stop 2023-11-28T13:08:41.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.522+0000 7f2dba9ff700 1 -- 172.21.15.156:0/3775774731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2db415c9b0 msgr2=0x7f2db4146d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:41.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.522+0000 7f2dba9ff700 1 --2- 172.21.15.156:0/3775774731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2db415c9b0 0x7f2db4146d60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2db40675d0 tx=0x7f2da8004750 comp rx=0 tx=0).stop 2023-11-28T13:08:41.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.522+0000 7f2dba9ff700 1 -- 172.21.15.156:0/3775774731 shutdown_connections 2023-11-28T13:08:41.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.522+0000 7f2dba9ff700 1 --2- 172.21.15.156:0/3775774731 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2da00682f0 0x7f2da006a7a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.522+0000 7f2dba9ff700 1 --2- 172.21.15.156:0/3775774731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2db4161340 0x7f2db41579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.522+0000 7f2dba9ff700 1 --2- 172.21.15.156:0/3775774731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2db40d2320 0x7f2db40c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.522+0000 7f2dba9ff700 1 --2- 172.21.15.156:0/3775774731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2db415c9b0 0x7f2db4146d60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.522+0000 7f2dba9ff700 1 -- 172.21.15.156:0/3775774731 >> 172.21.15.156:0/3775774731 conn(0x7f2db4072600 msgr2=0x7f2db40b8570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:41.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.522+0000 7f2dba9ff700 1 -- 172.21.15.156:0/3775774731 shutdown_connections 2023-11-28T13:08:41.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.522+0000 7f2dba9ff700 1 -- 172.21.15.156:0/3775774731 wait complete. 2023-11-28T13:08:41.542 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:08:41.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.737+0000 7fb3541ff700 1 Processor -- start 2023-11-28T13:08:41.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.737+0000 7fb3541ff700 1 -- start start 2023-11-28T13:08:41.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.739+0000 7fb3541ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb34c0d1330 0x7fb34c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:41.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.739+0000 7fb3541ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb34c0b20b0 0x7fb34c0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:41.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.739+0000 7fb3541ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb34c0b0230 con 0x7fb34c0d01b0 2023-11-28T13:08:41.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.739+0000 7fb3541ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34c15cec0 con 0x7fb34c0b20b0 2023-11-28T13:08:41.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.739+0000 7fb3541ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34c0b4570 con 0x7fb34c0d1330 2023-11-28T13:08:41.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.739+0000 7fb35279c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb34c0b20b0 0x7fb34c0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:41.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.739+0000 7fb35279c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb34c0b20b0 0x7fb34c0b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41614/0 (socket says 172.21.15.156:41614) 2023-11-28T13:08:41.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.739+0000 7fb35279c700 1 -- 172.21.15.156:0/2446782504 learned_addr learned my addr 172.21.15.156:0/2446782504 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:41.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.739+0000 7fb35179a700 1 --2- 172.21.15.156:0/2446782504 >> v2:172.21.15.186:3300/0 conn(0x7fb34c0d1330 0x7fb34c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:41.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.739+0000 7fb35279c700 1 -- 172.21.15.156:0/2446782504 >> v2:172.21.15.186:3300/0 conn(0x7fb34c0d1330 msgr2=0x7fb34c0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:41.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.739+0000 7fb35279c700 1 --2- 172.21.15.156:0/2446782504 >> v2:172.21.15.186:3300/0 conn(0x7fb34c0d1330 0x7fb34c0b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.739+0000 7fb35279c700 1 -- 172.21.15.156:0/2446782504 >> v1:172.21.15.186:6789/0 conn(0x7fb34c0d01b0 legacy=0x7fb34c0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:41.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.739+0000 7fb35279c700 1 -- 172.21.15.156:0/2446782504 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c0b4850 con 0x7fb34c0b20b0 2023-11-28T13:08:41.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.740+0000 7fb35279c700 1 --2- 172.21.15.156:0/2446782504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb34c0b20b0 0x7fb34c0b0c00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0ca0b0 tx=0x7fb34400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f62abe0708f2ab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:41.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.740+0000 7fb350f99700 1 -- 172.21.15.156:0/2446782504 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34400e070 con 0x7fb34c0b20b0 2023-11-28T13:08:41.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.740+0000 7fb350f99700 1 -- 172.21.15.156:0/2446782504 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34400b910 con 0x7fb34c0b20b0 2023-11-28T13:08:41.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.740+0000 7fb350f99700 1 -- 172.21.15.156:0/2446782504 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34400bcc0 con 0x7fb34c0b20b0 2023-11-28T13:08:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.740+0000 7fb3541ff700 1 -- 172.21.15.156:0/2446782504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb34c0b20b0 msgr2=0x7fb34c0b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.740+0000 7fb3541ff700 1 --2- 172.21.15.156:0/2446782504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb34c0b20b0 0x7fb34c0b0c00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0ca0b0 tx=0x7fb34400b330 comp rx=0 tx=0).stop 2023-11-28T13:08:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.741+0000 7fb3541ff700 1 -- 172.21.15.156:0/2446782504 shutdown_connections 2023-11-28T13:08:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.741+0000 7fb3541ff700 1 --2- 172.21.15.156:0/2446782504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb34c0b20b0 0x7fb34c0b0c00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.741+0000 7fb3541ff700 1 --2- 172.21.15.156:0/2446782504 >> v2:172.21.15.186:3300/0 conn(0x7fb34c0d1330 0x7fb34c0b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.741+0000 7fb3541ff700 1 -- 172.21.15.156:0/2446782504 >> 172.21.15.156:0/2446782504 conn(0x7fb34c09c3f0 msgr2=0x7fb34c0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.741+0000 7fb3541ff700 1 -- 172.21.15.156:0/2446782504 shutdown_connections 2023-11-28T13:08:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.741+0000 7fb3541ff700 1 -- 172.21.15.156:0/2446782504 wait complete. 2023-11-28T13:08:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.742+0000 7fb3541ff700 1 Processor -- start 2023-11-28T13:08:41.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.742+0000 7fb3541ff700 1 -- start start 2023-11-28T13:08:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.743+0000 7fb3541ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb34c0b20b0 0x7fb34c15b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.743+0000 7fb3541ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb34c0d01b0 0x7fb34c09f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.743+0000 7fb3541ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb34c0d1330 0x7fb34c09fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:41.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.743+0000 7fb3541ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c1e67e0 con 0x7fb34c0d01b0 2023-11-28T13:08:41.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.743+0000 7fb3541ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c09d000 con 0x7fb34c0d1330 2023-11-28T13:08:41.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.743+0000 7fb3541ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c0a0250 con 0x7fb34c0b20b0 2023-11-28T13:08:41.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.744+0000 7fb35179a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb34c0d01b0 0x7fb34c09f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:41.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.744+0000 7fb35179a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb34c0d01b0 0x7fb34c09f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41622/0 (socket says 172.21.15.156:41622) 2023-11-28T13:08:41.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.744+0000 7fb35179a700 1 -- 172.21.15.156:0/2119250523 learned_addr learned my addr 172.21.15.156:0/2119250523 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:41.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.744+0000 7fb35279c700 1 --2- 172.21.15.156:0/2119250523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb34c0d1330 0x7fb34c09fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:41.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.744+0000 7fb351f9b700 1 --2- 172.21.15.156:0/2119250523 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb34c0b20b0 0x7fb34c15b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:41.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.744+0000 7fb35179a700 1 -- 172.21.15.156:0/2119250523 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb34c0b20b0 msgr2=0x7fb34c15b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:41.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.744+0000 7fb35179a700 1 --2- 172.21.15.156:0/2119250523 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb34c0b20b0 0x7fb34c15b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.744+0000 7fb35179a700 1 -- 172.21.15.156:0/2119250523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb34c0d1330 msgr2=0x7fb34c09fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:41.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.744+0000 7fb35179a700 1 --2- 172.21.15.156:0/2119250523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb34c0d1330 0x7fb34c09fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.744+0000 7fb35179a700 1 -- 172.21.15.156:0/2119250523 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb344009b30 con 0x7fb34c0d01b0 2023-11-28T13:08:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.744+0000 7fb351f9b700 1 --2- 172.21.15.156:0/2119250523 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb34c0b20b0 0x7fb34c15b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.744+0000 7fb35179a700 1 --2- 172.21.15.156:0/2119250523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb34c0d01b0 0x7fb34c09f6e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00e840 tx=0x7fb33c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.744+0000 7fb3427fc700 1 -- 172.21.15.156:0/2119250523 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c009590 con 0x7fb34c0d01b0 2023-11-28T13:08:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.745+0000 7fb3427fc700 1 -- 172.21.15.156:0/2119250523 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c013680 con 0x7fb34c0d01b0 2023-11-28T13:08:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.745+0000 7fb3541ff700 1 -- 172.21.15.156:0/2119250523 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34c0afec0 con 0x7fb34c0d01b0 2023-11-28T13:08:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.745+0000 7fb3427fc700 1 -- 172.21.15.156:0/2119250523 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c013a30 con 0x7fb34c0d01b0 2023-11-28T13:08:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.745+0000 7fb3541ff700 1 -- 172.21.15.156:0/2119250523 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34c0afbe0 con 0x7fb34c0d01b0 2023-11-28T13:08:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.746+0000 7fb3541ff700 1 -- 172.21.15.156:0/2119250523 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34c0b2600 con 0x7fb34c0d01b0 2023-11-28T13:08:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.747+0000 7fb3427fc700 1 -- 172.21.15.156:0/2119250523 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb33c019030 con 0x7fb34c0d01b0 2023-11-28T13:08:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.747+0000 7fb3427fc700 1 --2- 172.21.15.156:0/2119250523 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3300682f0 0x7fb33006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.747+0000 7fb3427fc700 1 -- 172.21.15.156:0/2119250523 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb33c085b00 con 0x7fb34c0d01b0 2023-11-28T13:08:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.748+0000 7fb351f9b700 1 --2- 172.21.15.156:0/2119250523 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3300682f0 0x7fb33006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:41.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.749+0000 7fb351f9b700 1 --2- 172.21.15.156:0/2119250523 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3300682f0 0x7fb33006a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb348000f60 tx=0x7fb348009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:41.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.751+0000 7fb3427fc700 1 -- 172.21.15.156:0/2119250523 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb33c087e60 con 0x7fb34c0d01b0 2023-11-28T13:08:41.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.957+0000 7fb3541ff700 1 -- 172.21.15.156:0/2119250523 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb34c068690 con 0x7fb34c0d01b0 2023-11-28T13:08:41.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.957+0000 7fb3427fc700 1 -- 172.21.15.156:0/2119250523 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb33c052270 con 0x7fb34c0d01b0 2023-11-28T13:08:41.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.960+0000 7fb3541ff700 1 -- 172.21.15.156:0/2119250523 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3300682f0 msgr2=0x7fb33006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:41.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.960+0000 7fb3541ff700 1 --2- 172.21.15.156:0/2119250523 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3300682f0 0x7fb33006a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb348000f60 tx=0x7fb348009040 comp rx=0 tx=0).stop 2023-11-28T13:08:41.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.960+0000 7fb3541ff700 1 -- 172.21.15.156:0/2119250523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb34c0d01b0 msgr2=0x7fb34c09f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:41.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.960+0000 7fb3541ff700 1 --2- 172.21.15.156:0/2119250523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb34c0d01b0 0x7fb34c09f6e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00e840 tx=0x7fb33c00eb50 comp rx=0 tx=0).stop 2023-11-28T13:08:41.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.960+0000 7fb3541ff700 1 -- 172.21.15.156:0/2119250523 shutdown_connections 2023-11-28T13:08:41.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.960+0000 7fb3541ff700 1 --2- 172.21.15.156:0/2119250523 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3300682f0 0x7fb33006a7a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.960+0000 7fb3541ff700 1 --2- 172.21.15.156:0/2119250523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb34c0d1330 0x7fb34c09fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.960+0000 7fb3541ff700 1 --2- 172.21.15.156:0/2119250523 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb34c0b20b0 0x7fb34c15b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.960+0000 7fb3541ff700 1 --2- 172.21.15.156:0/2119250523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb34c0d01b0 0x7fb34c09f6e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:41.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.960+0000 7fb3541ff700 1 -- 172.21.15.156:0/2119250523 >> 172.21.15.156:0/2119250523 conn(0x7fb34c09c3f0 msgr2=0x7fb34c06b150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:41.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.960+0000 7fb3541ff700 1 -- 172.21.15.156:0/2119250523 shutdown_connections 2023-11-28T13:08:41.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:41.960+0000 7fb3541ff700 1 -- 172.21.15.156:0/2119250523 wait complete. 2023-11-28T13:08:41.963 INFO:teuthology.orchestra.run.smithi156.stderr:listed 0 entries 2023-11-28T13:08:41.980 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T13:08:41.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:41.980 DEBUG:teuthology.orchestra.run.smithi156:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-28T13:08:42.056 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T13:08:42.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:42.056 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T13:08:42.115 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:08:42.115 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:08:42.164 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:08:42.164 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:08:42.250 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:08:42.250 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:08:42.335 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:08:42.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.531+0000 7efc2ffff700 1 Processor -- start 2023-11-28T13:08:42.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.531+0000 7efc2ffff700 1 -- start start 2023-11-28T13:08:42.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.532+0000 7efc2ffff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7efc2815c9b0 0x7efc280b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:42.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.532+0000 7efc2ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efc28161340 0x7efc280d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:42.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.532+0000 7efc2ffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc280d2320 0x7efc280cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:42.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.532+0000 7efc2ffff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc280b5800 con 0x7efc28161340 2023-11-28T13:08:42.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.532+0000 7efc2ffff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc280d1950 con 0x7efc2815c9b0 2023-11-28T13:08:42.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.533+0000 7efc2ffff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc280b5520 con 0x7efc280d2320 2023-11-28T13:08:42.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.533+0000 7efc2d59a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efc28161340 0x7efc280d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:42.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.533+0000 7efc2d59a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efc28161340 0x7efc280d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41630/0 (socket says 172.21.15.156:41630) 2023-11-28T13:08:42.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.533+0000 7efc2d59a700 1 -- 172.21.15.156:0/653934542 learned_addr learned my addr 172.21.15.156:0/653934542 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:42.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.533+0000 7efc2dd9b700 1 --2- 172.21.15.156:0/653934542 >> v2:172.21.15.186:3300/0 conn(0x7efc2815c9b0 0x7efc280b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:42.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.533+0000 7efc2e59c700 1 --2- 172.21.15.156:0/653934542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc280d2320 0x7efc280cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:42.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.533+0000 7efc2d59a700 1 -- 172.21.15.156:0/653934542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc280d2320 msgr2=0x7efc280cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:42.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.533+0000 7efc2d59a700 1 --2- 172.21.15.156:0/653934542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc280d2320 0x7efc280cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.533+0000 7efc2d59a700 1 -- 172.21.15.156:0/653934542 >> v2:172.21.15.186:3300/0 conn(0x7efc2815c9b0 msgr2=0x7efc280b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:42.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.533+0000 7efc2d59a700 1 --2- 172.21.15.156:0/653934542 >> v2:172.21.15.186:3300/0 conn(0x7efc2815c9b0 0x7efc280b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.533+0000 7efc2d59a700 1 -- 172.21.15.156:0/653934542 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc280d1300 con 0x7efc28161340 2023-11-28T13:08:42.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.533+0000 7efc2e59c700 1 --2- 172.21.15.156:0/653934542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc280d2320 0x7efc280cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:42.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.534+0000 7efc2d59a700 1 --2- 172.21.15.156:0/653934542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc28161340 0x7efc280d1de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efc28067460 tx=0x7efc18009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3de9e8769a6810 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:42.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.534+0000 7efc2cd99700 1 -- 172.21.15.156:0/653934542 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18011830 con 0x7efc28161340 2023-11-28T13:08:42.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.534+0000 7efc2cd99700 1 -- 172.21.15.156:0/653934542 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc18011990 con 0x7efc28161340 2023-11-28T13:08:42.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.534+0000 7efc2cd99700 1 -- 172.21.15.156:0/653934542 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18011d20 con 0x7efc28161340 2023-11-28T13:08:42.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.534+0000 7efc2ffff700 1 -- 172.21.15.156:0/653934542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc28161340 msgr2=0x7efc280d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:42.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.534+0000 7efc2ffff700 1 --2- 172.21.15.156:0/653934542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc28161340 0x7efc280d1de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efc28067460 tx=0x7efc18009bd0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.534+0000 7efc2ffff700 1 -- 172.21.15.156:0/653934542 shutdown_connections 2023-11-28T13:08:42.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.534+0000 7efc2ffff700 1 --2- 172.21.15.156:0/653934542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc280d2320 0x7efc280cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.534+0000 7efc2ffff700 1 --2- 172.21.15.156:0/653934542 >> v2:172.21.15.186:3300/0 conn(0x7efc2815c9b0 0x7efc280b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.534+0000 7efc2ffff700 1 --2- 172.21.15.156:0/653934542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc28161340 0x7efc280d1de0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.534+0000 7efc2ffff700 1 -- 172.21.15.156:0/653934542 >> 172.21.15.156:0/653934542 conn(0x7efc28072600 msgr2=0x7efc280cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:42.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.535+0000 7efc2ffff700 1 -- 172.21.15.156:0/653934542 shutdown_connections 2023-11-28T13:08:42.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.535+0000 7efc2ffff700 1 -- 172.21.15.156:0/653934542 wait complete. 2023-11-28T13:08:42.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.536+0000 7efc2ffff700 1 Processor -- start 2023-11-28T13:08:42.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.536+0000 7efc2ffff700 1 -- start start 2023-11-28T13:08:42.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.537+0000 7efc2ffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc280d2320 0x7efc281572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:42.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.537+0000 7efc2ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc2815c9b0 0x7efc280c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:42.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.537+0000 7efc2ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc28161340 0x7efc28154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:42.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.537+0000 7efc2ffff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc280c6480 con 0x7efc28161340 2023-11-28T13:08:42.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.537+0000 7efc2ffff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc280c8240 con 0x7efc2815c9b0 2023-11-28T13:08:42.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.537+0000 7efc2ffff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc280c84e0 con 0x7efc280d2320 2023-11-28T13:08:42.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.538+0000 7efc2e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc28161340 0x7efc28154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:42.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.538+0000 7efc2dd9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc280d2320 0x7efc281572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:42.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.538+0000 7efc2e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc28161340 0x7efc28154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41646/0 (socket says 172.21.15.156:41646) 2023-11-28T13:08:42.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.538+0000 7efc2e59c700 1 -- 172.21.15.156:0/1699004988 learned_addr learned my addr 172.21.15.156:0/1699004988 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:42.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.538+0000 7efc2d59a700 1 --2- 172.21.15.156:0/1699004988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc2815c9b0 0x7efc280c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:42.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.538+0000 7efc2e59c700 1 -- 172.21.15.156:0/1699004988 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc280d2320 msgr2=0x7efc281572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:42.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.538+0000 7efc2e59c700 1 --2- 172.21.15.156:0/1699004988 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc280d2320 0x7efc281572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.538+0000 7efc2e59c700 1 -- 172.21.15.156:0/1699004988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc2815c9b0 msgr2=0x7efc280c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:42.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.538+0000 7efc2e59c700 1 --2- 172.21.15.156:0/1699004988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc2815c9b0 0x7efc280c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.538+0000 7efc2e59c700 1 -- 172.21.15.156:0/1699004988 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc18009550 con 0x7efc28161340 2023-11-28T13:08:42.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.538+0000 7efc2e59c700 1 --2- 172.21.15.156:0/1699004988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc28161340 0x7efc28154310 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efc20007e40 tx=0x7efc2000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:42.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.538+0000 7efc1e7fc700 1 -- 172.21.15.156:0/1699004988 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2000dd20 con 0x7efc28161340 2023-11-28T13:08:42.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.539+0000 7efc1e7fc700 1 -- 172.21.15.156:0/1699004988 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2000f040 con 0x7efc28161340 2023-11-28T13:08:42.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.539+0000 7efc1e7fc700 1 -- 172.21.15.156:0/1699004988 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2000dea0 con 0x7efc28161340 2023-11-28T13:08:42.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.539+0000 7efc2ffff700 1 -- 172.21.15.156:0/1699004988 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc280b5130 con 0x7efc28161340 2023-11-28T13:08:42.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.539+0000 7efc2ffff700 1 -- 172.21.15.156:0/1699004988 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc280d15e0 con 0x7efc28161340 2023-11-28T13:08:42.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.540+0000 7efc2ffff700 1 -- 172.21.15.156:0/1699004988 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc280b52c0 con 0x7efc28161340 2023-11-28T13:08:42.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.541+0000 7efc1e7fc700 1 -- 172.21.15.156:0/1699004988 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efc2001c030 con 0x7efc28161340 2023-11-28T13:08:42.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.541+0000 7efc1e7fc700 1 --2- 172.21.15.156:0/1699004988 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efc0c0682f0 0x7efc0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:42.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.541+0000 7efc1e7fc700 1 -- 172.21.15.156:0/1699004988 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efc20085c80 con 0x7efc28161340 2023-11-28T13:08:42.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.542+0000 7efc2dd9b700 1 --2- 172.21.15.156:0/1699004988 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efc0c0682f0 0x7efc0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:42.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.542+0000 7efc2dd9b700 1 --2- 172.21.15.156:0/1699004988 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efc0c0682f0 0x7efc0c06a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efc24009b60 tx=0x7efc24009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:42.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.546+0000 7efc1e7fc700 1 -- 172.21.15.156:0/1699004988 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efc20052300 con 0x7efc28161340 2023-11-28T13:08:42.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.742+0000 7efc2ffff700 1 -- 172.21.15.156:0/1699004988 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc280662b0 con 0x7efc28161340 2023-11-28T13:08:42.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.743+0000 7efc1e7fc700 1 -- 172.21.15.156:0/1699004988 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17897 (secure 0 0 0) 0x7efc20055920 con 0x7efc28161340 2023-11-28T13:08:42.745 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:42.745 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":21,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:08:04.060053+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:08:00.903485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:08:02.037854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:08:42.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.745+0000 7efc2ffff700 1 -- 172.21.15.156:0/1699004988 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efc0c0682f0 msgr2=0x7efc0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:42.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.745+0000 7efc2ffff700 1 --2- 172.21.15.156:0/1699004988 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efc0c0682f0 0x7efc0c06a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efc24009b60 tx=0x7efc24009210 comp rx=0 tx=0).stop 2023-11-28T13:08:42.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.745+0000 7efc2ffff700 1 -- 172.21.15.156:0/1699004988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc28161340 msgr2=0x7efc28154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:42.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.745+0000 7efc2ffff700 1 --2- 172.21.15.156:0/1699004988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc28161340 0x7efc28154310 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efc20007e40 tx=0x7efc2000d3b0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.745+0000 7efc2ffff700 1 -- 172.21.15.156:0/1699004988 shutdown_connections 2023-11-28T13:08:42.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.745+0000 7efc2ffff700 1 --2- 172.21.15.156:0/1699004988 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efc0c0682f0 0x7efc0c06a7a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.745+0000 7efc2ffff700 1 --2- 172.21.15.156:0/1699004988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc28161340 0x7efc28154310 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.745+0000 7efc2ffff700 1 --2- 172.21.15.156:0/1699004988 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc280d2320 0x7efc281572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.745+0000 7efc2ffff700 1 --2- 172.21.15.156:0/1699004988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc2815c9b0 0x7efc280c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.745+0000 7efc2ffff700 1 -- 172.21.15.156:0/1699004988 >> 172.21.15.156:0/1699004988 conn(0x7efc28072600 msgr2=0x7efc28160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:42.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.745+0000 7efc2ffff700 1 -- 172.21.15.156:0/1699004988 shutdown_connections 2023-11-28T13:08:42.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.745+0000 7efc2ffff700 1 -- 172.21.15.156:0/1699004988 wait complete. 2023-11-28T13:08:42.766 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:08:00.903485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:08:02.037854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:08:42.766 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:08:42.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.964+0000 7f0576fff700 1 Processor -- start 2023-11-28T13:08:42.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.965+0000 7f0576fff700 1 -- start start 2023-11-28T13:08:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.966+0000 7f0576fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f057015c9b0 0x7f05700b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.966+0000 7f0576fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0570161340 0x7f05700d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.966+0000 7f0576fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05700d1950 con 0x7f05700d2320 2023-11-28T13:08:42.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.966+0000 7f0576fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05700b5800 con 0x7f0570161340 2023-11-28T13:08:42.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.966+0000 7f0576fff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05700b5520 con 0x7f057015c9b0 2023-11-28T13:08:42.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.966+0000 7f056ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0570161340 0x7f05700d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:42.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.966+0000 7f056ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0570161340 0x7f05700d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41648/0 (socket says 172.21.15.156:41648) 2023-11-28T13:08:42.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.966+0000 7f056ffff700 1 -- 172.21.15.156:0/2036848140 learned_addr learned my addr 172.21.15.156:0/2036848140 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:42.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.966+0000 7f0574d9b700 1 --2- 172.21.15.156:0/2036848140 >> v2:172.21.15.186:3300/0 conn(0x7f057015c9b0 0x7f05700b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:42.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.966+0000 7f056ffff700 1 -- 172.21.15.156:0/2036848140 >> v2:172.21.15.186:3300/0 conn(0x7f057015c9b0 msgr2=0x7f05700b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:42.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.966+0000 7f056ffff700 1 --2- 172.21.15.156:0/2036848140 >> v2:172.21.15.186:3300/0 conn(0x7f057015c9b0 0x7f05700b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.966+0000 7f056ffff700 1 -- 172.21.15.156:0/2036848140 >> v1:172.21.15.186:6789/0 conn(0x7f05700d2320 legacy=0x7f05700cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:42.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.966+0000 7f056ffff700 1 -- 172.21.15.156:0/2036848140 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05700d1300 con 0x7f0570161340 2023-11-28T13:08:42.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.967+0000 7f056ffff700 1 --2- 172.21.15.156:0/2036848140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0570161340 0x7f05700d1de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0570067460 tx=0x7f055c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97def7be7f3c7fe4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:42.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.967+0000 7f056f7fe700 1 -- 172.21.15.156:0/2036848140 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c004230 con 0x7f0570161340 2023-11-28T13:08:42.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.967+0000 7f056f7fe700 1 -- 172.21.15.156:0/2036848140 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c004390 con 0x7f0570161340 2023-11-28T13:08:42.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.967+0000 7f056f7fe700 1 -- 172.21.15.156:0/2036848140 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c0036a0 con 0x7f0570161340 2023-11-28T13:08:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.967+0000 7f0576fff700 1 -- 172.21.15.156:0/2036848140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0570161340 msgr2=0x7f05700d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.967+0000 7f0576fff700 1 --2- 172.21.15.156:0/2036848140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0570161340 0x7f05700d1de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0570067460 tx=0x7f055c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.967+0000 7f0576fff700 1 -- 172.21.15.156:0/2036848140 shutdown_connections 2023-11-28T13:08:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.967+0000 7f0576fff700 1 --2- 172.21.15.156:0/2036848140 >> v2:172.21.15.186:3300/0 conn(0x7f057015c9b0 0x7f05700b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.967+0000 7f0576fff700 1 --2- 172.21.15.156:0/2036848140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0570161340 0x7f05700d1de0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.968+0000 7f0576fff700 1 -- 172.21.15.156:0/2036848140 >> 172.21.15.156:0/2036848140 conn(0x7f0570072600 msgr2=0x7f05700cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.968+0000 7f0576fff700 1 -- 172.21.15.156:0/2036848140 shutdown_connections 2023-11-28T13:08:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.968+0000 7f0576fff700 1 -- 172.21.15.156:0/2036848140 wait complete. 2023-11-28T13:08:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.969+0000 7f0576fff700 1 Processor -- start 2023-11-28T13:08:42.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.969+0000 7f0576fff700 1 -- start start 2023-11-28T13:08:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.970+0000 7f0576fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05700d2320 0x7f05701f6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:42.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.970+0000 7f0576fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f057015c9b0 0x7f0570146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.970+0000 7f0576fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0570161340 0x7f05701f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.970+0000 7f0576fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05701f7440 con 0x7f057015c9b0 2023-11-28T13:08:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.970+0000 7f0576fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0570146840 con 0x7f05700d2320 2023-11-28T13:08:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.970+0000 7f0576fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05701f6ba0 con 0x7f0570161340 2023-11-28T13:08:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.971+0000 7f056ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f057015c9b0 0x7f0570146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.971+0000 7f056ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f057015c9b0 0x7f0570146300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41652/0 (socket says 172.21.15.156:41652) 2023-11-28T13:08:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.971+0000 7f056ffff700 1 -- 172.21.15.156:0/1713298740 learned_addr learned my addr 172.21.15.156:0/1713298740 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.971+0000 7f0574d9b700 1 --2- 172.21.15.156:0/1713298740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05700d2320 0x7f05701f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.971+0000 7f057559c700 1 --2- 172.21.15.156:0/1713298740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0570161340 0x7f05701f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:42.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.971+0000 7f056ffff700 1 -- 172.21.15.156:0/1713298740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0570161340 msgr2=0x7f05701f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:42.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.971+0000 7f056ffff700 1 --2- 172.21.15.156:0/1713298740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0570161340 0x7f05701f3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.971+0000 7f056ffff700 1 -- 172.21.15.156:0/1713298740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05700d2320 msgr2=0x7f05701f6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:42.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.971+0000 7f056ffff700 1 --2- 172.21.15.156:0/1713298740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05700d2320 0x7f05701f6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:42.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.971+0000 7f056ffff700 1 -- 172.21.15.156:0/1713298740 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c009550 con 0x7f057015c9b0 2023-11-28T13:08:42.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.971+0000 7f057559c700 1 --2- 172.21.15.156:0/1713298740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0570161340 0x7f05701f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:42.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.971+0000 7f056ffff700 1 --2- 172.21.15.156:0/1713298740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f057015c9b0 0x7f0570146300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f055c009c10 tx=0x7f055c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:42.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.971+0000 7f056d7fa700 1 -- 172.21.15.156:0/1713298740 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c003c90 con 0x7f057015c9b0 2023-11-28T13:08:42.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.972+0000 7f056d7fa700 1 -- 172.21.15.156:0/1713298740 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c01f070 con 0x7f057015c9b0 2023-11-28T13:08:42.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.972+0000 7f0576fff700 1 -- 172.21.15.156:0/1713298740 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05700b5130 con 0x7f057015c9b0 2023-11-28T13:08:42.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.972+0000 7f056d7fa700 1 -- 172.21.15.156:0/1713298740 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c011870 con 0x7f057015c9b0 2023-11-28T13:08:42.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.972+0000 7f0576fff700 1 -- 172.21.15.156:0/1713298740 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05700d15e0 con 0x7f057015c9b0 2023-11-28T13:08:42.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.973+0000 7f0576fff700 1 -- 172.21.15.156:0/1713298740 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05700b52c0 con 0x7f057015c9b0 2023-11-28T13:08:42.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.974+0000 7f056d7fa700 1 -- 172.21.15.156:0/1713298740 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f055c017020 con 0x7f057015c9b0 2023-11-28T13:08:42.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.975+0000 7f056d7fa700 1 --2- 172.21.15.156:0/1713298740 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f05540682f0 0x7f055406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:42.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.975+0000 7f056d7fa700 1 -- 172.21.15.156:0/1713298740 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f055c087320 con 0x7f057015c9b0 2023-11-28T13:08:42.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.975+0000 7f0574d9b700 1 --2- 172.21.15.156:0/1713298740 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f05540682f0 0x7f055406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:42.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.976+0000 7f0574d9b700 1 --2- 172.21.15.156:0/1713298740 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f05540682f0 0x7f055406a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0570150dd0 tx=0x7f0564008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:42.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:42.979+0000 7f056d7fa700 1 -- 172.21.15.156:0/1713298740 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f055c053b80 con 0x7f057015c9b0 2023-11-28T13:08:43.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.176+0000 7f0576fff700 1 -- 172.21.15.156:0/1713298740 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f05700ced90 con 0x7f057015c9b0 2023-11-28T13:08:43.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.176+0000 7f056d7fa700 1 -- 172.21.15.156:0/1713298740 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f055c0571a0 con 0x7f057015c9b0 2023-11-28T13:08:43.178 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:08:43.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.179+0000 7f0576fff700 1 -- 172.21.15.156:0/1713298740 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f05540682f0 msgr2=0x7f055406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:43.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.179+0000 7f0576fff700 1 --2- 172.21.15.156:0/1713298740 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f05540682f0 0x7f055406a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0570150dd0 tx=0x7f0564008040 comp rx=0 tx=0).stop 2023-11-28T13:08:43.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.179+0000 7f0576fff700 1 -- 172.21.15.156:0/1713298740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f057015c9b0 msgr2=0x7f0570146300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:43.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.179+0000 7f0576fff700 1 --2- 172.21.15.156:0/1713298740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f057015c9b0 0x7f0570146300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f055c009c10 tx=0x7f055c004750 comp rx=0 tx=0).stop 2023-11-28T13:08:43.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.179+0000 7f0576fff700 1 -- 172.21.15.156:0/1713298740 shutdown_connections 2023-11-28T13:08:43.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.179+0000 7f0576fff700 1 --2- 172.21.15.156:0/1713298740 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f05540682f0 0x7f055406a7a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:43.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.179+0000 7f0576fff700 1 --2- 172.21.15.156:0/1713298740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0570161340 0x7f05701f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:43.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.179+0000 7f0576fff700 1 --2- 172.21.15.156:0/1713298740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05700d2320 0x7f05701f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:43.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.179+0000 7f0576fff700 1 --2- 172.21.15.156:0/1713298740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f057015c9b0 0x7f0570146300 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:43.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.179+0000 7f0576fff700 1 -- 172.21.15.156:0/1713298740 >> 172.21.15.156:0/1713298740 conn(0x7f0570072600 msgr2=0x7f057015cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:43.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.179+0000 7f0576fff700 1 -- 172.21.15.156:0/1713298740 shutdown_connections 2023-11-28T13:08:43.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.179+0000 7f0576fff700 1 -- 172.21.15.156:0/1713298740 wait complete. 2023-11-28T13:08:43.198 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:08:43.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.397+0000 7f58bd7ff700 1 Processor -- start 2023-11-28T13:08:43.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.397+0000 7f58bd7ff700 1 -- start start 2023-11-28T13:08:43.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.398+0000 7f58bd7ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f58b8161340 0x7f58b80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:43.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.398+0000 7f58bd7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58b80d2320 0x7f58b80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:43.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.398+0000 7f58bd7ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b80d1950 con 0x7f58b815c9b0 2023-11-28T13:08:43.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.398+0000 7f58bd7ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b80b5800 con 0x7f58b8161340 2023-11-28T13:08:43.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.398+0000 7f58bd7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80b5300 con 0x7f58b80d2320 2023-11-28T13:08:43.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.398+0000 7f58b67fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f58b8161340 0x7f58b80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:43.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.398+0000 7f58b67fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f58b8161340 0x7f58b80d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41668/0 (socket says 172.21.15.156:41668) 2023-11-28T13:08:43.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.398+0000 7f58b67fc700 1 -- 172.21.15.156:0/3502262330 learned_addr learned my addr 172.21.15.156:0/3502262330 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:43.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.399+0000 7f58b77fe700 1 --2- 172.21.15.156:0/3502262330 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58b80d2320 0x7f58b80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:43.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.399+0000 7f58b67fc700 1 -- 172.21.15.156:0/3502262330 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58b80d2320 msgr2=0x7f58b80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:43.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.399+0000 7f58b67fc700 1 --2- 172.21.15.156:0/3502262330 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58b80d2320 0x7f58b80cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:43.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.399+0000 7f58b67fc700 1 -- 172.21.15.156:0/3502262330 >> v1:172.21.15.186:6789/0 conn(0x7f58b815c9b0 legacy=0x7f58b80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:43.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.399+0000 7f58b67fc700 1 -- 172.21.15.156:0/3502262330 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b80b5520 con 0x7f58b8161340 2023-11-28T13:08:43.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.399+0000 7f58b5ffb700 1 -- 172.21.15.156:0/3502262330 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668273420 0 0) 0x7f58b80d1950 con 0x7f58b815c9b0 2023-11-28T13:08:43.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.399+0000 7f58b67fc700 1 --2- 172.21.15.156:0/3502262330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b8161340 0x7f58b80d1de0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0098f0 tx=0x7f58ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f01f70cd7fe6866 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:43.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.399+0000 7f58b5ffb700 1 -- 172.21.15.156:0/3502262330 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac011830 con 0x7f58b8161340 2023-11-28T13:08:43.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.400+0000 7f58b5ffb700 1 -- 172.21.15.156:0/3502262330 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac011990 con 0x7f58b8161340 2023-11-28T13:08:43.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.400+0000 7f58b5ffb700 1 -- 172.21.15.156:0/3502262330 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac011d20 con 0x7f58b8161340 2023-11-28T13:08:43.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.400+0000 7f58bd7ff700 1 -- 172.21.15.156:0/3502262330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b8161340 msgr2=0x7f58b80d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:43.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.400+0000 7f58bd7ff700 1 --2- 172.21.15.156:0/3502262330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b8161340 0x7f58b80d1de0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0098f0 tx=0x7f58ac009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:43.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.400+0000 7f58bd7ff700 1 -- 172.21.15.156:0/3502262330 shutdown_connections 2023-11-28T13:08:43.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.400+0000 7f58bd7ff700 1 --2- 172.21.15.156:0/3502262330 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58b80d2320 0x7f58b80cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:43.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.400+0000 7f58bd7ff700 1 --2- 172.21.15.156:0/3502262330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b8161340 0x7f58b80d1de0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:43.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.400+0000 7f58bd7ff700 1 -- 172.21.15.156:0/3502262330 >> 172.21.15.156:0/3502262330 conn(0x7f58b8072600 msgr2=0x7f58b80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:43.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.400+0000 7f58bd7ff700 1 -- 172.21.15.156:0/3502262330 shutdown_connections 2023-11-28T13:08:43.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.400+0000 7f58bd7ff700 1 -- 172.21.15.156:0/3502262330 wait complete. 2023-11-28T13:08:43.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.402+0000 7f58bd7ff700 1 Processor -- start 2023-11-28T13:08:43.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.402+0000 7f58bd7ff700 1 -- start start 2023-11-28T13:08:43.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.403+0000 7f58bd7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b80d2320 0x7f58b80c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:43.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.403+0000 7f58bd7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58b815c9b0 0x7f58b80cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:43.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.403+0000 7f58bd7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58b8161340 0x7f58b81516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:43.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.403+0000 7f58bd7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80c7f40 con 0x7f58b80d2320 2023-11-28T13:08:43.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.403+0000 7f58bd7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80ce1b0 con 0x7f58b8161340 2023-11-28T13:08:43.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.403+0000 7f58bd7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80c7330 con 0x7f58b815c9b0 2023-11-28T13:08:43.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.403+0000 7f58b6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b80d2320 0x7f58b80c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:43.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.403+0000 7f58b6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b80d2320 0x7f58b80c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41678/0 (socket says 172.21.15.156:41678) 2023-11-28T13:08:43.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.403+0000 7f58b6ffd700 1 -- 172.21.15.156:0/2108701244 learned_addr learned my addr 172.21.15.156:0/2108701244 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:43.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.404+0000 7f58b67fc700 1 --2- 172.21.15.156:0/2108701244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58b815c9b0 0x7f58b80cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:43.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.404+0000 7f58b77fe700 1 --2- 172.21.15.156:0/2108701244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58b8161340 0x7f58b81516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:43.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.404+0000 7f58b6ffd700 1 -- 172.21.15.156:0/2108701244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58b815c9b0 msgr2=0x7f58b80cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:43.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.404+0000 7f58b6ffd700 1 --2- 172.21.15.156:0/2108701244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58b815c9b0 0x7f58b80cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:43.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.404+0000 7f58b6ffd700 1 -- 172.21.15.156:0/2108701244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58b8161340 msgr2=0x7f58b81516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:43.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.404+0000 7f58b6ffd700 1 --2- 172.21.15.156:0/2108701244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58b8161340 0x7f58b81516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:43.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.404+0000 7f58b6ffd700 1 -- 172.21.15.156:0/2108701244 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac009550 con 0x7f58b80d2320 2023-11-28T13:08:43.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.404+0000 7f58b77fe700 1 --2- 172.21.15.156:0/2108701244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58b8161340 0x7f58b81516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:43.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.404+0000 7f58b6ffd700 1 --2- 172.21.15.156:0/2108701244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b80d2320 0x7f58b80c6ca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f58a800c550 tx=0x7f58a800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:43.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.404+0000 7f589f7fe700 1 -- 172.21.15.156:0/2108701244 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a80054e0 con 0x7f58b80d2320 2023-11-28T13:08:43.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.404+0000 7f589f7fe700 1 -- 172.21.15.156:0/2108701244 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a8003680 con 0x7f58b80d2320 2023-11-28T13:08:43.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.404+0000 7f58bd7ff700 1 -- 172.21.15.156:0/2108701244 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b80d15e0 con 0x7f58b80d2320 2023-11-28T13:08:43.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.404+0000 7f589f7fe700 1 -- 172.21.15.156:0/2108701244 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8003a30 con 0x7f58b80d2320 2023-11-28T13:08:43.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.404+0000 7f58bd7ff700 1 -- 172.21.15.156:0/2108701244 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b80d1300 con 0x7f58b80d2320 2023-11-28T13:08:43.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.405+0000 7f58bd7ff700 1 -- 172.21.15.156:0/2108701244 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b815cd80 con 0x7f58b80d2320 2023-11-28T13:08:43.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.406+0000 7f589f7fe700 1 -- 172.21.15.156:0/2108701244 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f58a8019030 con 0x7f58b80d2320 2023-11-28T13:08:43.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.407+0000 7f589f7fe700 1 --2- 172.21.15.156:0/2108701244 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f58a40682f0 0x7f58a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:43.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.407+0000 7f589f7fe700 1 -- 172.21.15.156:0/2108701244 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f58a8084ff0 con 0x7f58b80d2320 2023-11-28T13:08:43.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.408+0000 7f58b67fc700 1 --2- 172.21.15.156:0/2108701244 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f58a40682f0 0x7f58a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:43.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.408+0000 7f58b67fc700 1 --2- 172.21.15.156:0/2108701244 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f58a40682f0 0x7f58a406a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009c40 tx=0x7f58ac008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:43.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.411+0000 7f589f7fe700 1 -- 172.21.15.156:0/2108701244 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f58a80517c0 con 0x7f58b80d2320 2023-11-28T13:08:43.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.605+0000 7f58bd7ff700 1 -- 172.21.15.156:0/2108701244 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f58b80745c0 con 0x7f58b80d2320 2023-11-28T13:08:43.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.606+0000 7f589f7fe700 1 -- 172.21.15.156:0/2108701244 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f58a8017070 con 0x7f58b80d2320 2023-11-28T13:08:43.608 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:08:43.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.608+0000 7f58bd7ff700 1 -- 172.21.15.156:0/2108701244 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f58a40682f0 msgr2=0x7f58a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:43.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.608+0000 7f58bd7ff700 1 --2- 172.21.15.156:0/2108701244 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f58a40682f0 0x7f58a406a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009c40 tx=0x7f58ac008030 comp rx=0 tx=0).stop 2023-11-28T13:08:43.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.609+0000 7f58bd7ff700 1 -- 172.21.15.156:0/2108701244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b80d2320 msgr2=0x7f58b80c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:43.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.609+0000 7f58bd7ff700 1 --2- 172.21.15.156:0/2108701244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b80d2320 0x7f58b80c6ca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f58a800c550 tx=0x7f58a800c860 comp rx=0 tx=0).stop 2023-11-28T13:08:43.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.609+0000 7f58bd7ff700 1 -- 172.21.15.156:0/2108701244 shutdown_connections 2023-11-28T13:08:43.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.609+0000 7f58bd7ff700 1 --2- 172.21.15.156:0/2108701244 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f58a40682f0 0x7f58a406a7a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:43.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.609+0000 7f58bd7ff700 1 --2- 172.21.15.156:0/2108701244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58b8161340 0x7f58b81516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:43.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.609+0000 7f58bd7ff700 1 --2- 172.21.15.156:0/2108701244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b80d2320 0x7f58b80c6ca0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:43.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.609+0000 7f58bd7ff700 1 --2- 172.21.15.156:0/2108701244 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58b815c9b0 0x7f58b80cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:43.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.609+0000 7f58bd7ff700 1 -- 172.21.15.156:0/2108701244 >> 172.21.15.156:0/2108701244 conn(0x7f58b8072600 msgr2=0x7f58b80b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:43.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.609+0000 7f58bd7ff700 1 -- 172.21.15.156:0/2108701244 shutdown_connections 2023-11-28T13:08:43.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.609+0000 7f58bd7ff700 1 -- 172.21.15.156:0/2108701244 wait complete. 2023-11-28T13:08:43.629 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:08:43.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.827+0000 7f567b1ff700 1 Processor -- start 2023-11-28T13:08:43.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.827+0000 7f567b1ff700 1 -- start start 2023-11-28T13:08:43.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.828+0000 7f567b1ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5674161340 0x7f56740d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:43.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.828+0000 7f567b1ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56740d1950 con 0x7f56740d2320 2023-11-28T13:08:43.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.828+0000 7f567b1ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56740b5520 con 0x7f567415c9b0 2023-11-28T13:08:43.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.828+0000 7f567b1ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56740b5800 con 0x7f5674161340 2023-11-28T13:08:43.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.828+0000 7f5673fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5674161340 0x7f56740d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:43.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.828+0000 7f567979c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f56740d2320 0x7f56740cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:52006/0 (socket says 172.21.15.156:52006) 2023-11-28T13:08:43.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.828+0000 7f567979c700 1 -- 172.21.15.156:0/4058000664 learned_addr learned my addr 172.21.15.156:0/4058000664 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:43.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.829+0000 7f5673fff700 1 -- 172.21.15.156:0/4058000664 >> v1:172.21.15.186:6789/0 conn(0x7f567415c9b0 legacy=0x7f56740b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:43.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.829+0000 7f5673fff700 1 -- 172.21.15.156:0/4058000664 >> v1:172.21.15.156:6789/0 conn(0x7f56740d2320 legacy=0x7f56740cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:43.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.829+0000 7f5673fff700 1 -- 172.21.15.156:0/4058000664 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56740d1300 con 0x7f5674161340 2023-11-28T13:08:43.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.829+0000 7f56737fe700 1 -- 172.21.15.156:0/4058000664 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4013595195 0 0) 0x7f56740d1950 con 0x7f56740d2320 2023-11-28T13:08:43.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.829+0000 7f5673fff700 1 --2- 172.21.15.156:0/4058000664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5674161340 0x7f56740d1de0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5660009840 tx=0x7f5660009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abb1abb98c5626c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:43.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.829+0000 7f56737fe700 1 -- 172.21.15.156:0/4058000664 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660004230 con 0x7f5674161340 2023-11-28T13:08:43.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.829+0000 7f56737fe700 1 -- 172.21.15.156:0/4058000664 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5660004390 con 0x7f5674161340 2023-11-28T13:08:43.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.829+0000 7f56737fe700 1 -- 172.21.15.156:0/4058000664 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56600036a0 con 0x7f5674161340 2023-11-28T13:08:43.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.830+0000 7f567b1ff700 1 -- 172.21.15.156:0/4058000664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5674161340 msgr2=0x7f56740d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:43.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.830+0000 7f567b1ff700 1 --2- 172.21.15.156:0/4058000664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5674161340 0x7f56740d1de0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5660009840 tx=0x7f5660009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:43.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.830+0000 7f567b1ff700 1 -- 172.21.15.156:0/4058000664 shutdown_connections 2023-11-28T13:08:43.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.830+0000 7f567b1ff700 1 --2- 172.21.15.156:0/4058000664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5674161340 0x7f56740d1de0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:43.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.830+0000 7f567b1ff700 1 -- 172.21.15.156:0/4058000664 >> 172.21.15.156:0/4058000664 conn(0x7f5674072600 msgr2=0x7f56740cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:43.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.830+0000 7f567b1ff700 1 -- 172.21.15.156:0/4058000664 shutdown_connections 2023-11-28T13:08:43.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.830+0000 7f567b1ff700 1 -- 172.21.15.156:0/4058000664 wait complete. 2023-11-28T13:08:43.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.832+0000 7f567b1ff700 1 Processor -- start 2023-11-28T13:08:43.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.832+0000 7f567b1ff700 1 -- start start 2023-11-28T13:08:43.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.833+0000 7f567b1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56740d2320 0x7f56741f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:43.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.833+0000 7f567b1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f567415c9b0 0x7f567406ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:43.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.833+0000 7f567b1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5674161340 0x7f56740cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:43.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.833+0000 7f567b1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56741f68a0 con 0x7f567415c9b0 2023-11-28T13:08:43.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.833+0000 7f567b1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56741e7010 con 0x7f56740d2320 2023-11-28T13:08:43.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.833+0000 7f567b1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56741e72f0 con 0x7f5674161340 2023-11-28T13:08:43.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.833+0000 7f5673fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f567415c9b0 0x7f567406ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:43.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.833+0000 7f5673fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f567415c9b0 0x7f567406ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41708/0 (socket says 172.21.15.156:41708) 2023-11-28T13:08:43.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.833+0000 7f5673fff700 1 -- 172.21.15.156:0/1399627937 learned_addr learned my addr 172.21.15.156:0/1399627937 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:43.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.833+0000 7f567979c700 1 --2- 172.21.15.156:0/1399627937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5674161340 0x7f56740cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:43.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.833+0000 7f5673fff700 1 -- 172.21.15.156:0/1399627937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5674161340 msgr2=0x7f56740cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:43.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.833+0000 7f5678f9b700 1 --2- 172.21.15.156:0/1399627937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56740d2320 0x7f56741f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:43.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.834+0000 7f5673fff700 1 --2- 172.21.15.156:0/1399627937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5674161340 0x7f56740cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:43.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.834+0000 7f5673fff700 1 -- 172.21.15.156:0/1399627937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56740d2320 msgr2=0x7f56741f6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:43.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.834+0000 7f5673fff700 1 --2- 172.21.15.156:0/1399627937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56740d2320 0x7f56741f6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:43.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.834+0000 7f5673fff700 1 -- 172.21.15.156:0/1399627937 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56600094a0 con 0x7f567415c9b0 2023-11-28T13:08:43.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.834+0000 7f567979c700 1 --2- 172.21.15.156:0/1399627937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5674161340 0x7f56740cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:43.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.834+0000 7f5678f9b700 1 --2- 172.21.15.156:0/1399627937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56740d2320 0x7f56741f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:43.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.834+0000 7f5673fff700 1 --2- 172.21.15.156:0/1399627937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f567415c9b0 0x7f567406ab70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5660000c00 tx=0x7f5660004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:43.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.834+0000 7f56717fa700 1 -- 172.21.15.156:0/1399627937 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660011630 con 0x7f567415c9b0 2023-11-28T13:08:43.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.834+0000 7f56717fa700 1 -- 172.21.15.156:0/1399627937 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566001e070 con 0x7f567415c9b0 2023-11-28T13:08:43.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.834+0000 7f56717fa700 1 -- 172.21.15.156:0/1399627937 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56600119f0 con 0x7f567415c9b0 2023-11-28T13:08:43.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.834+0000 7f567b1ff700 1 -- 172.21.15.156:0/1399627937 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f567415cd80 con 0x7f567415c9b0 2023-11-28T13:08:43.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.834+0000 7f567b1ff700 1 -- 172.21.15.156:0/1399627937 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56740d15e0 con 0x7f567415c9b0 2023-11-28T13:08:43.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.835+0000 7f567b1ff700 1 -- 172.21.15.156:0/1399627937 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f567415cf10 con 0x7f567415c9b0 2023-11-28T13:08:43.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.836+0000 7f56717fa700 1 -- 172.21.15.156:0/1399627937 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5660017020 con 0x7f567415c9b0 2023-11-28T13:08:43.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.837+0000 7f56717fa700 1 --2- 172.21.15.156:0/1399627937 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56580682f0 0x7f565806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:43.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.837+0000 7f56717fa700 1 -- 172.21.15.156:0/1399627937 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5660087140 con 0x7f567415c9b0 2023-11-28T13:08:43.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.838+0000 7f5678f9b700 1 --2- 172.21.15.156:0/1399627937 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56580682f0 0x7f565806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:43.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.838+0000 7f5678f9b700 1 --2- 172.21.15.156:0/1399627937 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56580682f0 0x7f565806a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5668003f40 tx=0x7f5668009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:43.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:43.841+0000 7f56717fa700 1 -- 172.21.15.156:0/1399627937 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f56600538b0 con 0x7f567415c9b0 2023-11-28T13:08:44.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.029+0000 7f567b1ff700 1 -- 172.21.15.156:0/1399627937 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f56740745c0 con 0x7f567415c9b0 2023-11-28T13:08:44.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.030+0000 7f56717fa700 1 -- 172.21.15.156:0/1399627937 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5660020020 con 0x7f567415c9b0 2023-11-28T13:08:44.031 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T13:08:44.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.032+0000 7f567b1ff700 1 -- 172.21.15.156:0/1399627937 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56580682f0 msgr2=0x7f565806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:44.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.032+0000 7f567b1ff700 1 --2- 172.21.15.156:0/1399627937 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56580682f0 0x7f565806a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5668003f40 tx=0x7f5668009040 comp rx=0 tx=0).stop 2023-11-28T13:08:44.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.032+0000 7f567b1ff700 1 -- 172.21.15.156:0/1399627937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f567415c9b0 msgr2=0x7f567406ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:44.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.032+0000 7f567b1ff700 1 --2- 172.21.15.156:0/1399627937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f567415c9b0 0x7f567406ab70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5660000c00 tx=0x7f5660004750 comp rx=0 tx=0).stop 2023-11-28T13:08:44.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.032+0000 7f567b1ff700 1 -- 172.21.15.156:0/1399627937 shutdown_connections 2023-11-28T13:08:44.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.032+0000 7f567b1ff700 1 --2- 172.21.15.156:0/1399627937 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56580682f0 0x7f565806a7a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.032+0000 7f567b1ff700 1 --2- 172.21.15.156:0/1399627937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5674161340 0x7f56740cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.032+0000 7f567b1ff700 1 --2- 172.21.15.156:0/1399627937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56740d2320 0x7f56741f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.032+0000 7f567b1ff700 1 --2- 172.21.15.156:0/1399627937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f567415c9b0 0x7f567406ab70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.032+0000 7f567b1ff700 1 -- 172.21.15.156:0/1399627937 >> 172.21.15.156:0/1399627937 conn(0x7f5674072600 msgr2=0x7f567415f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:44.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.032+0000 7f567b1ff700 1 -- 172.21.15.156:0/1399627937 shutdown_connections 2023-11-28T13:08:44.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.032+0000 7f567b1ff700 1 -- 172.21.15.156:0/1399627937 wait complete. 2023-11-28T13:08:44.052 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:08:44.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.249+0000 7fb5cfdff700 1 Processor -- start 2023-11-28T13:08:44.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.250+0000 7fb5cfdff700 1 -- start start 2023-11-28T13:08:44.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.251+0000 7fb5cfdff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb5c815c9b0 0x7fb5c80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:44.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.251+0000 7fb5cfdff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c80d1950 con 0x7fb5c80cd7c0 2023-11-28T13:08:44.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.251+0000 7fb5cfdff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c80d24f0 con 0x7fb5c8161340 2023-11-28T13:08:44.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.251+0000 7fb5cfdff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c80b5800 con 0x7fb5c815c9b0 2023-11-28T13:08:44.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.251+0000 7fb5cdb9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb5c815c9b0 0x7fb5c80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:44.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.251+0000 7fb5cdb9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb5c815c9b0 0x7fb5c80b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41712/0 (socket says 172.21.15.156:41712) 2023-11-28T13:08:44.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.251+0000 7fb5ce39c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb5c80cd7c0 0x7fb5c80b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:52020/0 (socket says 172.21.15.156:52020) 2023-11-28T13:08:44.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.251+0000 7fb5cdb9b700 1 -- 172.21.15.156:0/3265730870 learned_addr learned my addr 172.21.15.156:0/3265730870 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:44.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.251+0000 7fb5cdb9b700 1 -- 172.21.15.156:0/3265730870 >> v1:172.21.15.186:6789/0 conn(0x7fb5c8161340 legacy=0x7fb5c80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:44.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.251+0000 7fb5cdb9b700 1 -- 172.21.15.156:0/3265730870 >> v1:172.21.15.156:6789/0 conn(0x7fb5c80cd7c0 legacy=0x7fb5c80b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:44.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.251+0000 7fb5cdb9b700 1 -- 172.21.15.156:0/3265730870 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c80b5520 con 0x7fb5c815c9b0 2023-11-28T13:08:44.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.252+0000 7fb5cdb9b700 1 --2- 172.21.15.156:0/3265730870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5c815c9b0 0x7fb5c80b4bf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40098f0 tx=0x7fb5c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7beeca850756d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:44.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.252+0000 7fb5ccb99700 1 -- 172.21.15.156:0/3265730870 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c400b8a0 con 0x7fb5c815c9b0 2023-11-28T13:08:44.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.252+0000 7fb5ccb99700 1 -- 172.21.15.156:0/3265730870 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c400ba00 con 0x7fb5c815c9b0 2023-11-28T13:08:44.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.252+0000 7fb5ccb99700 1 -- 172.21.15.156:0/3265730870 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c400bd90 con 0x7fb5c815c9b0 2023-11-28T13:08:44.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.252+0000 7fb5cfdff700 1 -- 172.21.15.156:0/3265730870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5c815c9b0 msgr2=0x7fb5c80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:44.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.252+0000 7fb5cfdff700 1 --2- 172.21.15.156:0/3265730870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5c815c9b0 0x7fb5c80b4bf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40098f0 tx=0x7fb5c4009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:44.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.253+0000 7fb5cfdff700 1 -- 172.21.15.156:0/3265730870 shutdown_connections 2023-11-28T13:08:44.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.253+0000 7fb5cfdff700 1 --2- 172.21.15.156:0/3265730870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5c815c9b0 0x7fb5c80b4bf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.253+0000 7fb5cfdff700 1 -- 172.21.15.156:0/3265730870 >> 172.21.15.156:0/3265730870 conn(0x7fb5c8072600 msgr2=0x7fb5c80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:44.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.253+0000 7fb5cfdff700 1 -- 172.21.15.156:0/3265730870 shutdown_connections 2023-11-28T13:08:44.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.253+0000 7fb5cfdff700 1 -- 172.21.15.156:0/3265730870 wait complete. 2023-11-28T13:08:44.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.254+0000 7fb5cfdff700 1 Processor -- start 2023-11-28T13:08:44.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.254+0000 7fb5cfdff700 1 -- start start 2023-11-28T13:08:44.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.255+0000 7fb5cfdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5c80cd7c0 0x7fb5c815f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:44.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.255+0000 7fb5cfdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb5c815c9b0 0x7fb5c814e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:44.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.255+0000 7fb5cfdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5c8161340 0x7fb5c80b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:44.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.255+0000 7fb5cfdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c814eb70 con 0x7fb5c80cd7c0 2023-11-28T13:08:44.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.255+0000 7fb5cfdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c815e3d0 con 0x7fb5c8161340 2023-11-28T13:08:44.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.255+0000 7fb5cfdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c815dde0 con 0x7fb5c815c9b0 2023-11-28T13:08:44.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.256+0000 7fb5cdb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5c80cd7c0 0x7fb5c815f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:44.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.256+0000 7fb5cdb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5c80cd7c0 0x7fb5c815f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41728/0 (socket says 172.21.15.156:41728) 2023-11-28T13:08:44.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.256+0000 7fb5cdb9b700 1 -- 172.21.15.156:0/1969151522 learned_addr learned my addr 172.21.15.156:0/1969151522 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:44.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.256+0000 7fb5ce39c700 1 --2- 172.21.15.156:0/1969151522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5c8161340 0x7fb5c80b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:44.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.256+0000 7fb5cd39a700 1 --2- 172.21.15.156:0/1969151522 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb5c815c9b0 0x7fb5c814e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:44.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.256+0000 7fb5cdb9b700 1 -- 172.21.15.156:0/1969151522 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb5c815c9b0 msgr2=0x7fb5c814e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:44.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.256+0000 7fb5cdb9b700 1 --2- 172.21.15.156:0/1969151522 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb5c815c9b0 0x7fb5c814e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.256+0000 7fb5cdb9b700 1 -- 172.21.15.156:0/1969151522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5c8161340 msgr2=0x7fb5c80b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:44.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.256+0000 7fb5cdb9b700 1 --2- 172.21.15.156:0/1969151522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5c8161340 0x7fb5c80b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.256+0000 7fb5cdb9b700 1 -- 172.21.15.156:0/1969151522 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c4009550 con 0x7fb5c80cd7c0 2023-11-28T13:08:44.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.256+0000 7fb5ce39c700 1 --2- 172.21.15.156:0/1969151522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5c8161340 0x7fb5c80b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:44.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.256+0000 7fb5cdb9b700 1 --2- 172.21.15.156:0/1969151522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5c80cd7c0 0x7fb5c815f100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009130 tx=0x7fb5c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:44.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.257+0000 7fb5be7fc700 1 -- 172.21.15.156:0/1969151522 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40036a0 con 0x7fb5c80cd7c0 2023-11-28T13:08:44.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.257+0000 7fb5be7fc700 1 -- 172.21.15.156:0/1969151522 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c401e070 con 0x7fb5c80cd7c0 2023-11-28T13:08:44.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.257+0000 7fb5cfdff700 1 -- 172.21.15.156:0/1969151522 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c80d15e0 con 0x7fb5c80cd7c0 2023-11-28T13:08:44.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.257+0000 7fb5be7fc700 1 -- 172.21.15.156:0/1969151522 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4003a60 con 0x7fb5c80cd7c0 2023-11-28T13:08:44.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.257+0000 7fb5cfdff700 1 -- 172.21.15.156:0/1969151522 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c80d1300 con 0x7fb5c80cd7c0 2023-11-28T13:08:44.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.258+0000 7fb5cfdff700 1 -- 172.21.15.156:0/1969151522 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c80b5130 con 0x7fb5c80cd7c0 2023-11-28T13:08:44.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.259+0000 7fb5be7fc700 1 -- 172.21.15.156:0/1969151522 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb5c400f050 con 0x7fb5c80cd7c0 2023-11-28T13:08:44.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.259+0000 7fb5be7fc700 1 --2- 172.21.15.156:0/1969151522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb5ac0682f0 0x7fb5ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:44.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.260+0000 7fb5be7fc700 1 -- 172.21.15.156:0/1969151522 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb5c4020070 con 0x7fb5c80cd7c0 2023-11-28T13:08:44.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.260+0000 7fb5cd39a700 1 --2- 172.21.15.156:0/1969151522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb5ac0682f0 0x7fb5ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:44.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.260+0000 7fb5cd39a700 1 --2- 172.21.15.156:0/1969151522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb5ac0682f0 0x7fb5ac06a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8154770 tx=0x7fb5b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:44.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.263+0000 7fb5be7fc700 1 -- 172.21.15.156:0/1969151522 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb5c4053b30 con 0x7fb5c80cd7c0 2023-11-28T13:08:44.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.451+0000 7fb5cfdff700 1 -- 172.21.15.156:0/1969151522 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb5c80745c0 con 0x7fb5c80cd7c0 2023-11-28T13:08:44.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.451+0000 7fb5be7fc700 1 -- 172.21.15.156:0/1969151522 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb5c4015020 con 0x7fb5c80cd7c0 2023-11-28T13:08:44.453 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T13:08:44.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.454+0000 7fb5cfdff700 1 -- 172.21.15.156:0/1969151522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb5ac0682f0 msgr2=0x7fb5ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:44.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.454+0000 7fb5cfdff700 1 --2- 172.21.15.156:0/1969151522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb5ac0682f0 0x7fb5ac06a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8154770 tx=0x7fb5b8009040 comp rx=0 tx=0).stop 2023-11-28T13:08:44.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.454+0000 7fb5cfdff700 1 -- 172.21.15.156:0/1969151522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5c80cd7c0 msgr2=0x7fb5c815f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:44.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.454+0000 7fb5cfdff700 1 --2- 172.21.15.156:0/1969151522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5c80cd7c0 0x7fb5c815f100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009130 tx=0x7fb5c4004750 comp rx=0 tx=0).stop 2023-11-28T13:08:44.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.454+0000 7fb5cfdff700 1 -- 172.21.15.156:0/1969151522 shutdown_connections 2023-11-28T13:08:44.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.454+0000 7fb5cfdff700 1 --2- 172.21.15.156:0/1969151522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb5ac0682f0 0x7fb5ac06a7a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.454+0000 7fb5cfdff700 1 --2- 172.21.15.156:0/1969151522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5c8161340 0x7fb5c80b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.454+0000 7fb5cfdff700 1 --2- 172.21.15.156:0/1969151522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5c80cd7c0 0x7fb5c815f100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.454+0000 7fb5cfdff700 1 --2- 172.21.15.156:0/1969151522 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb5c815c9b0 0x7fb5c814e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.454+0000 7fb5cfdff700 1 -- 172.21.15.156:0/1969151522 >> 172.21.15.156:0/1969151522 conn(0x7fb5c8072600 msgr2=0x7fb5c80d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:44.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.454+0000 7fb5cfdff700 1 -- 172.21.15.156:0/1969151522 shutdown_connections 2023-11-28T13:08:44.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.454+0000 7fb5cfdff700 1 -- 172.21.15.156:0/1969151522 wait complete. 2023-11-28T13:08:44.474 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:08:44.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.671+0000 7f492b7ff700 1 Processor -- start 2023-11-28T13:08:44.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.671+0000 7f492b7ff700 1 -- start start 2023-11-28T13:08:44.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.672+0000 7f492b7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4924158200 0x7f4924070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:44.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.673+0000 7f492b7ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492415e1a0 con 0x7f49240caa70 2023-11-28T13:08:44.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.673+0000 7f492b7ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49241585d0 con 0x7f49240b0d80 2023-11-28T13:08:44.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.673+0000 7f492b7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492415f070 con 0x7f4924158200 2023-11-28T13:08:44.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.673+0000 7f4928d9a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f49240caa70 0x7f49240b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:52028/0 (socket says 172.21.15.156:52028) 2023-11-28T13:08:44.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.673+0000 7f4928d9a700 1 -- 172.21.15.156:0/3587123797 learned_addr learned my addr 172.21.15.156:0/3587123797 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:44.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.673+0000 7f492959b700 1 --2- 172.21.15.156:0/3587123797 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4924158200 0x7f4924070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:44.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.673+0000 7f491bfff700 1 -- 172.21.15.156:0/3587123797 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407069322 0 0) 0x7f492415e1a0 con 0x7f49240caa70 2023-11-28T13:08:44.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.673+0000 7f491bfff700 1 -- 172.21.15.156:0/3587123797 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49100035b0 con 0x7f49240caa70 2023-11-28T13:08:44.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.673+0000 7f491bfff700 1 -- 172.21.15.156:0/3587123797 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3773611499 0 0) 0x7f49241585d0 con 0x7f49240b0d80 2023-11-28T13:08:44.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.673+0000 7f491bfff700 1 -- 172.21.15.156:0/3587123797 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f492415e1a0 con 0x7f49240b0d80 2023-11-28T13:08:44.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.674+0000 7f492959b700 1 -- 172.21.15.156:0/3587123797 >> v1:172.21.15.186:6789/0 conn(0x7f49240b0d80 legacy=0x7f492415add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:44.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.674+0000 7f491bfff700 1 -- 172.21.15.156:0/3587123797 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3627985716 0 0) 0x7f49100035b0 con 0x7f49240caa70 2023-11-28T13:08:44.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.674+0000 7f492959b700 1 -- 172.21.15.156:0/3587123797 >> v1:172.21.15.156:6789/0 conn(0x7f49240caa70 legacy=0x7f49240b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:44.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.674+0000 7f492959b700 1 -- 172.21.15.156:0/3587123797 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492415e3f0 con 0x7f4924158200 2023-11-28T13:08:44.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.674+0000 7f492959b700 1 --2- 172.21.15.156:0/3587123797 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4924158200 0x7f4924070c50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4920009870 tx=0x7f4920009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b28b89e483e88de2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:44.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.674+0000 7f491bfff700 1 -- 172.21.15.156:0/3587123797 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4914001670 con 0x7f4924158200 2023-11-28T13:08:44.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.674+0000 7f491bfff700 1 -- 172.21.15.156:0/3587123797 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4920004030 con 0x7f4924158200 2023-11-28T13:08:44.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.674+0000 7f491bfff700 1 -- 172.21.15.156:0/3587123797 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49200043c0 con 0x7f4924158200 2023-11-28T13:08:44.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.675+0000 7f492b7ff700 1 -- 172.21.15.156:0/3587123797 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4924158200 msgr2=0x7f4924070c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:44.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.675+0000 7f492b7ff700 1 --2- 172.21.15.156:0/3587123797 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4924158200 0x7f4924070c50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4920009870 tx=0x7f4920009b80 comp rx=0 tx=0).stop 2023-11-28T13:08:44.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.675+0000 7f492b7ff700 1 -- 172.21.15.156:0/3587123797 shutdown_connections 2023-11-28T13:08:44.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.675+0000 7f492b7ff700 1 --2- 172.21.15.156:0/3587123797 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4924158200 0x7f4924070c50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.675+0000 7f492b7ff700 1 -- 172.21.15.156:0/3587123797 >> 172.21.15.156:0/3587123797 conn(0x7f49240cb670 msgr2=0x7f49240cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:44.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.675+0000 7f492b7ff700 1 -- 172.21.15.156:0/3587123797 shutdown_connections 2023-11-28T13:08:44.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.675+0000 7f492b7ff700 1 -- 172.21.15.156:0/3587123797 wait complete. 2023-11-28T13:08:44.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.677+0000 7f492b7ff700 1 Processor -- start 2023-11-28T13:08:44.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.677+0000 7f492b7ff700 1 -- start start 2023-11-28T13:08:44.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.679+0000 7f492b7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49240b0d80 0x7f49240c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:44.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.679+0000 7f492b7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49240caa70 0x7f49241efdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:44.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.679+0000 7f492b7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4924158200 0x7f492415c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:44.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.679+0000 7f492b7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4924071190 con 0x7f4924158200 2023-11-28T13:08:44.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.679+0000 7f492b7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492415be20 con 0x7f49240b0d80 2023-11-28T13:08:44.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.679+0000 7f492b7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492415dbb0 con 0x7f49240caa70 2023-11-28T13:08:44.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.679+0000 7f4929d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4924158200 0x7f492415c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:44.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.679+0000 7f4929d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4924158200 0x7f492415c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41738/0 (socket says 172.21.15.156:41738) 2023-11-28T13:08:44.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.679+0000 7f4929d9c700 1 -- 172.21.15.156:0/2354541255 learned_addr learned my addr 172.21.15.156:0/2354541255 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:44.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.679+0000 7f492959b700 1 --2- 172.21.15.156:0/2354541255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49240b0d80 0x7f49240c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:44.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.679+0000 7f4928d9a700 1 --2- 172.21.15.156:0/2354541255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49240caa70 0x7f49241efdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:44.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.679+0000 7f4929d9c700 1 -- 172.21.15.156:0/2354541255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49240caa70 msgr2=0x7f49241efdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:44.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.679+0000 7f4929d9c700 1 --2- 172.21.15.156:0/2354541255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49240caa70 0x7f49241efdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.679+0000 7f4929d9c700 1 -- 172.21.15.156:0/2354541255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49240b0d80 msgr2=0x7f49240c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:44.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.680+0000 7f4929d9c700 1 --2- 172.21.15.156:0/2354541255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49240b0d80 0x7f49240c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.680+0000 7f4929d9c700 1 -- 172.21.15.156:0/2354541255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49200094a0 con 0x7f4924158200 2023-11-28T13:08:44.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.680+0000 7f4929d9c700 1 --2- 172.21.15.156:0/2354541255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4924158200 0x7f492415c550 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f491c00cc50 tx=0x7f491c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:44.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.680+0000 7f4919ffb700 1 -- 172.21.15.156:0/2354541255 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c003c00 con 0x7f4924158200 2023-11-28T13:08:44.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.680+0000 7f4919ffb700 1 -- 172.21.15.156:0/2354541255 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c003d60 con 0x7f4924158200 2023-11-28T13:08:44.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.680+0000 7f492b7ff700 1 -- 172.21.15.156:0/2354541255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492415e9b0 con 0x7f4924158200 2023-11-28T13:08:44.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.680+0000 7f4919ffb700 1 -- 172.21.15.156:0/2354541255 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c00aa50 con 0x7f4924158200 2023-11-28T13:08:44.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.680+0000 7f492b7ff700 1 -- 172.21.15.156:0/2354541255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492415e6d0 con 0x7f4924158200 2023-11-28T13:08:44.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.681+0000 7f492b7ff700 1 -- 172.21.15.156:0/2354541255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492415ec90 con 0x7f4924158200 2023-11-28T13:08:44.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.682+0000 7f4919ffb700 1 -- 172.21.15.156:0/2354541255 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f491c010050 con 0x7f4924158200 2023-11-28T13:08:44.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.683+0000 7f4919ffb700 1 --2- 172.21.15.156:0/2354541255 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f49080682f0 0x7f490806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:44.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.683+0000 7f4919ffb700 1 -- 172.21.15.156:0/2354541255 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f491c085ad0 con 0x7f4924158200 2023-11-28T13:08:44.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.683+0000 7f492959b700 1 --2- 172.21.15.156:0/2354541255 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f49080682f0 0x7f490806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:44.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.684+0000 7f492959b700 1 --2- 172.21.15.156:0/2354541255 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f49080682f0 0x7f490806a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f49200097c0 tx=0x7f4920008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:44.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.687+0000 7f4919ffb700 1 -- 172.21.15.156:0/2354541255 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f491c052330 con 0x7f4924158200 2023-11-28T13:08:44.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.894+0000 7f492b7ff700 1 -- 172.21.15.156:0/2354541255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f492406fd60 con 0x7f4924158200 2023-11-28T13:08:44.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.895+0000 7f4919ffb700 1 -- 172.21.15.156:0/2354541255 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f491c055950 con 0x7f4924158200 2023-11-28T13:08:44.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.897+0000 7f492b7ff700 1 -- 172.21.15.156:0/2354541255 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f49080682f0 msgr2=0x7f490806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:44.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.897+0000 7f492b7ff700 1 --2- 172.21.15.156:0/2354541255 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f49080682f0 0x7f490806a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f49200097c0 tx=0x7f4920008030 comp rx=0 tx=0).stop 2023-11-28T13:08:44.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.897+0000 7f492b7ff700 1 -- 172.21.15.156:0/2354541255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4924158200 msgr2=0x7f492415c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:44.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.897+0000 7f492b7ff700 1 --2- 172.21.15.156:0/2354541255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4924158200 0x7f492415c550 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f491c00cc50 tx=0x7f491c00cf60 comp rx=0 tx=0).stop 2023-11-28T13:08:44.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.897+0000 7f492b7ff700 1 -- 172.21.15.156:0/2354541255 shutdown_connections 2023-11-28T13:08:44.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.897+0000 7f492b7ff700 1 --2- 172.21.15.156:0/2354541255 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f49080682f0 0x7f490806a7a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.897+0000 7f492b7ff700 1 --2- 172.21.15.156:0/2354541255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4924158200 0x7f492415c550 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.897+0000 7f492b7ff700 1 --2- 172.21.15.156:0/2354541255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49240b0d80 0x7f49240c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.897+0000 7f492b7ff700 1 --2- 172.21.15.156:0/2354541255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49240caa70 0x7f49241efdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:44.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.897+0000 7f492b7ff700 1 -- 172.21.15.156:0/2354541255 >> 172.21.15.156:0/2354541255 conn(0x7f49240cb670 msgr2=0x7f492415a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:44.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.897+0000 7f492b7ff700 1 -- 172.21.15.156:0/2354541255 shutdown_connections 2023-11-28T13:08:44.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:44.897+0000 7f492b7ff700 1 -- 172.21.15.156:0/2354541255 wait complete. 2023-11-28T13:08:44.901 INFO:teuthology.orchestra.run.smithi156.stderr:listed 0 entries 2023-11-28T13:08:44.917 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-28T13:08:44.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:44.918 DEBUG:teuthology.orchestra.run.smithi186:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-28T13:08:45.015 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T13:08:45.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:45.015 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T13:08:45.079 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:08:45.080 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:08:45.124 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:08:45.124 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:08:45.208 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:08:45.208 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:08:45.293 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:08:45.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.527+0000 7f44a1dff700 1 Processor -- start 2023-11-28T13:08:45.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.528+0000 7f44a1dff700 1 -- start start 2023-11-28T13:08:45.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.529+0000 7f44a1dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f449c0b0670 0x7f449c15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:45.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.529+0000 7f44a1dff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f449c0ad0a0 0x7f449c0c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:45.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.529+0000 7f44a1dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449c0ce100 con 0x7f449c0cc730 2023-11-28T13:08:45.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.529+0000 7f44a1dff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f449c0acc10 con 0x7f449c0ad0a0 2023-11-28T13:08:45.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.529+0000 7f44a1dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449c15dd30 con 0x7f449c0b0670 2023-11-28T13:08:45.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.529+0000 7f449b7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f449c0b0670 0x7f449c15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:45.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.529+0000 7f449b7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f449c0b0670 0x7f449c15d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:33480/0 (socket says 172.21.15.156:33480) 2023-11-28T13:08:45.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.529+0000 7f449b7fe700 1 -- 172.21.15.156:0/177247974 learned_addr learned my addr 172.21.15.156:0/177247974 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:45.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.529+0000 7f449bfff700 1 --2- 172.21.15.156:0/177247974 >> v2:172.21.15.186:3300/0 conn(0x7f449c0ad0a0 0x7f449c0c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:45.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.529+0000 7f449affd700 1 --1- 172.21.15.156:0/177247974 >> v1:172.21.15.186:6789/0 conn(0x7f449c0cc730 0x7f449c0b0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:50070/0 (socket says 172.21.15.156:50070) 2023-11-28T13:08:45.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.530+0000 7f449b7fe700 1 -- 172.21.15.156:0/177247974 >> v2:172.21.15.186:3300/0 conn(0x7f449c0ad0a0 msgr2=0x7f449c0c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:45.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.530+0000 7f449b7fe700 1 --2- 172.21.15.156:0/177247974 >> v2:172.21.15.186:3300/0 conn(0x7f449c0ad0a0 0x7f449c0c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:45.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.530+0000 7f449a7fc700 1 -- 172.21.15.156:0/177247974 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466299358 0 0) 0x7f449c0ce100 con 0x7f449c0cc730 2023-11-28T13:08:45.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.530+0000 7f449b7fe700 1 -- 172.21.15.156:0/177247974 >> v1:172.21.15.186:6789/0 conn(0x7f449c0cc730 legacy=0x7f449c0b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:45.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.530+0000 7f449b7fe700 1 -- 172.21.15.156:0/177247974 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c0ac8c0 con 0x7f449c0b0670 2023-11-28T13:08:45.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.530+0000 7f449b7fe700 1 --2- 172.21.15.156:0/177247974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f449c0b0670 0x7f449c15d7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f448c0098f0 tx=0x7f448c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4907e5db26e8d3d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:45.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.530+0000 7f449a7fc700 1 -- 172.21.15.156:0/177247974 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c011830 con 0x7f449c0b0670 2023-11-28T13:08:45.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.530+0000 7f449a7fc700 1 -- 172.21.15.156:0/177247974 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c011990 con 0x7f449c0b0670 2023-11-28T13:08:45.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.530+0000 7f449a7fc700 1 -- 172.21.15.156:0/177247974 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c011d20 con 0x7f449c0b0670 2023-11-28T13:08:45.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.531+0000 7f44a1dff700 1 -- 172.21.15.156:0/177247974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f449c0b0670 msgr2=0x7f449c15d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:45.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.531+0000 7f44a1dff700 1 --2- 172.21.15.156:0/177247974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f449c0b0670 0x7f449c15d7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f448c0098f0 tx=0x7f448c009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:45.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.531+0000 7f44a1dff700 1 -- 172.21.15.156:0/177247974 shutdown_connections 2023-11-28T13:08:45.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.531+0000 7f44a1dff700 1 --2- 172.21.15.156:0/177247974 >> v2:172.21.15.186:3300/0 conn(0x7f449c0ad0a0 0x7f449c0c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:45.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.531+0000 7f44a1dff700 1 --2- 172.21.15.156:0/177247974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f449c0b0670 0x7f449c15d7f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:45.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.531+0000 7f44a1dff700 1 -- 172.21.15.156:0/177247974 >> 172.21.15.156:0/177247974 conn(0x7f449c0cb670 msgr2=0x7f449c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:45.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.531+0000 7f44a1dff700 1 -- 172.21.15.156:0/177247974 shutdown_connections 2023-11-28T13:08:45.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.531+0000 7f44a1dff700 1 -- 172.21.15.156:0/177247974 wait complete. 2023-11-28T13:08:45.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.533+0000 7f44a1dff700 1 Processor -- start 2023-11-28T13:08:45.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.533+0000 7f44a1dff700 1 -- start start 2023-11-28T13:08:45.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.534+0000 7f44a1dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f449c0ad0a0 0x7f449c0c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:45.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.534+0000 7f44a1dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f449c0b0670 0x7f449c15c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:45.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.534+0000 7f44a1dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f449c0cc730 0x7f449c15f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:45.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.534+0000 7f44a1dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449c0c4b90 con 0x7f449c0ad0a0 2023-11-28T13:08:45.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.534+0000 7f44a1dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449c0c41c0 con 0x7f449c0cc730 2023-11-28T13:08:45.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.534+0000 7f44a1dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449c0c3330 con 0x7f449c0b0670 2023-11-28T13:08:45.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.534+0000 7f449b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f449c0ad0a0 0x7f449c0c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:45.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.534+0000 7f449b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f449c0ad0a0 0x7f449c0c4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41746/0 (socket says 172.21.15.156:41746) 2023-11-28T13:08:45.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.534+0000 7f449b7fe700 1 -- 172.21.15.156:0/2081029638 learned_addr learned my addr 172.21.15.156:0/2081029638 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:45.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.534+0000 7f449affd700 1 --2- 172.21.15.156:0/2081029638 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f449c0b0670 0x7f449c15c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:45.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.534+0000 7f449bfff700 1 --2- 172.21.15.156:0/2081029638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f449c0cc730 0x7f449c15f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:45.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.535+0000 7f449b7fe700 1 -- 172.21.15.156:0/2081029638 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f449c0b0670 msgr2=0x7f449c15c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:45.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.535+0000 7f449b7fe700 1 --2- 172.21.15.156:0/2081029638 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f449c0b0670 0x7f449c15c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:45.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.535+0000 7f449b7fe700 1 -- 172.21.15.156:0/2081029638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f449c0cc730 msgr2=0x7f449c15f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:45.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.535+0000 7f449b7fe700 1 --2- 172.21.15.156:0/2081029638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f449c0cc730 0x7f449c15f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:45.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.535+0000 7f449b7fe700 1 -- 172.21.15.156:0/2081029638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c009550 con 0x7f449c0ad0a0 2023-11-28T13:08:45.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.535+0000 7f449b7fe700 1 --2- 172.21.15.156:0/2081029638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f449c0ad0a0 0x7f449c0c4650 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f449c15de70 tx=0x7f448c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:45.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.535+0000 7f4483fff700 1 -- 172.21.15.156:0/2081029638 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c004510 con 0x7f449c0ad0a0 2023-11-28T13:08:45.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.535+0000 7f4483fff700 1 -- 172.21.15.156:0/2081029638 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c01e070 con 0x7f449c0ad0a0 2023-11-28T13:08:45.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.535+0000 7f44a1dff700 1 -- 172.21.15.156:0/2081029638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449c0ce610 con 0x7f449c0ad0a0 2023-11-28T13:08:45.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.535+0000 7f4483fff700 1 -- 172.21.15.156:0/2081029638 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c0038c0 con 0x7f449c0ad0a0 2023-11-28T13:08:45.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.535+0000 7f44a1dff700 1 -- 172.21.15.156:0/2081029638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449c0ce330 con 0x7f449c0ad0a0 2023-11-28T13:08:45.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.536+0000 7f44a1dff700 1 -- 172.21.15.156:0/2081029638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f449c0ce8f0 con 0x7f449c0ad0a0 2023-11-28T13:08:45.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.537+0000 7f4483fff700 1 -- 172.21.15.156:0/2081029638 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f448c00f050 con 0x7f449c0ad0a0 2023-11-28T13:08:45.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.538+0000 7f4483fff700 1 --2- 172.21.15.156:0/2081029638 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f44880682f0 0x7f448806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:45.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.538+0000 7f4483fff700 1 -- 172.21.15.156:0/2081029638 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f448c086b80 con 0x7f449c0ad0a0 2023-11-28T13:08:45.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.538+0000 7f449affd700 1 --2- 172.21.15.156:0/2081029638 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f44880682f0 0x7f448806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:45.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.539+0000 7f449affd700 1 --2- 172.21.15.156:0/2081029638 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f44880682f0 0x7f448806a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4490000f60 tx=0x7f4490007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:45.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.542+0000 7f4483fff700 1 -- 172.21.15.156:0/2081029638 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f448c053360 con 0x7f449c0ad0a0 2023-11-28T13:08:45.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.735+0000 7f44a1dff700 1 -- 172.21.15.156:0/2081029638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f449c0c44a0 con 0x7f449c0ad0a0 2023-11-28T13:08:45.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.736+0000 7f4483fff700 1 -- 172.21.15.156:0/2081029638 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17897 (secure 0 0 0) 0x7f448c020090 con 0x7f449c0ad0a0 2023-11-28T13:08:45.738 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:45.738 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":21,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:08:04.060053+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:08:00.903485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:08:02.037854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:08:45.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.738+0000 7f44a1dff700 1 -- 172.21.15.156:0/2081029638 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f44880682f0 msgr2=0x7f448806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:45.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.738+0000 7f44a1dff700 1 --2- 172.21.15.156:0/2081029638 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f44880682f0 0x7f448806a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4490000f60 tx=0x7f4490007e70 comp rx=0 tx=0).stop 2023-11-28T13:08:45.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.738+0000 7f44a1dff700 1 -- 172.21.15.156:0/2081029638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f449c0ad0a0 msgr2=0x7f449c0c4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:45.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.738+0000 7f44a1dff700 1 --2- 172.21.15.156:0/2081029638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f449c0ad0a0 0x7f449c0c4650 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f449c15de70 tx=0x7f448c004750 comp rx=0 tx=0).stop 2023-11-28T13:08:45.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.738+0000 7f44a1dff700 1 -- 172.21.15.156:0/2081029638 shutdown_connections 2023-11-28T13:08:45.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.738+0000 7f44a1dff700 1 --2- 172.21.15.156:0/2081029638 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f44880682f0 0x7f448806a7a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:45.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.738+0000 7f44a1dff700 1 --2- 172.21.15.156:0/2081029638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f449c0cc730 0x7f449c15f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:45.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.738+0000 7f44a1dff700 1 --2- 172.21.15.156:0/2081029638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f449c0ad0a0 0x7f449c0c4650 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:45.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.738+0000 7f44a1dff700 1 --2- 172.21.15.156:0/2081029638 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f449c0b0670 0x7f449c15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:45.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.738+0000 7f44a1dff700 1 -- 172.21.15.156:0/2081029638 >> 172.21.15.156:0/2081029638 conn(0x7f449c0cb670 msgr2=0x7f449c0aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:45.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.739+0000 7f44a1dff700 1 -- 172.21.15.156:0/2081029638 shutdown_connections 2023-11-28T13:08:45.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.739+0000 7f44a1dff700 1 -- 172.21.15.156:0/2081029638 wait complete. 2023-11-28T13:08:45.759 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:08:00.903485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:08:02.037854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:08:45.760 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:08:45.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.959+0000 7f394bdff700 1 Processor -- start 2023-11-28T13:08:45.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.959+0000 7f394bdff700 1 -- start start 2023-11-28T13:08:45.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.960+0000 7f394bdff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3944161340 0x7f39440d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:45.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.960+0000 7f394bdff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39440d1950 con 0x7f394415c9b0 2023-11-28T13:08:45.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.960+0000 7f394bdff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39440b5520 con 0x7f39440d2320 2023-11-28T13:08:45.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.960+0000 7f394bdff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39440b5800 con 0x7f3944161340 2023-11-28T13:08:45.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.961+0000 7f3949b9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f394415c9b0 0x7f39440b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39430/0 (socket says 172.21.15.156:39430) 2023-11-28T13:08:45.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.961+0000 7f3949b9b700 1 -- 172.21.15.156:0/1441852688 learned_addr learned my addr 172.21.15.156:0/1441852688 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:45.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.961+0000 7f3948b99700 1 -- 172.21.15.156:0/1441852688 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561247328 0 0) 0x7f39440d1950 con 0x7f394415c9b0 2023-11-28T13:08:45.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.961+0000 7f3948b99700 1 -- 172.21.15.156:0/1441852688 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39300035b0 con 0x7f394415c9b0 2023-11-28T13:08:45.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.961+0000 7f394939a700 1 --2- 172.21.15.156:0/1441852688 >> v2:172.21.15.186:3300/0 conn(0x7f3944161340 0x7f39440d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:45.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.961+0000 7f3948b99700 1 -- 172.21.15.156:0/1441852688 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3200830345 0 0) 0x7f39300035b0 con 0x7f394415c9b0 2023-11-28T13:08:45.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.961+0000 7f3948b99700 1 -- 172.21.15.156:0/1441852688 >> v2:172.21.15.186:3300/0 conn(0x7f3944161340 msgr2=0x7f39440d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:45.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.961+0000 7f3948b99700 1 --2- 172.21.15.156:0/1441852688 >> v2:172.21.15.186:3300/0 conn(0x7f3944161340 0x7f39440d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:45.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.961+0000 7f3948b99700 1 -- 172.21.15.156:0/1441852688 >> v1:172.21.15.186:6789/0 conn(0x7f39440d2320 legacy=0x7f39440cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:45.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.961+0000 7f3948b99700 1 -- 172.21.15.156:0/1441852688 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39440d1300 con 0x7f394415c9b0 2023-11-28T13:08:45.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.961+0000 7f3948b99700 1 -- 172.21.15.156:0/1441852688 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f3940001670 con 0x7f394415c9b0 2023-11-28T13:08:45.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.962+0000 7f3948b99700 1 -- 172.21.15.156:0/1441852688 >> v1:172.21.15.156:6789/0 conn(0x7f394415c9b0 legacy=0x7f39440b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:45.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.962+0000 7f3948b99700 1 --2- 172.21.15.156:0/1441852688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3930003a00 0x7f3930003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:45.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.962+0000 7f3948b99700 1 --2- 172.21.15.156:0/1441852688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3930004850 0x7f3930004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:45.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.962+0000 7f3948b99700 1 --2- 172.21.15.156:0/1441852688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3930005200 0x7f393000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:45.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.962+0000 7f3948b99700 1 -- 172.21.15.156:0/1441852688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39440b5800 con 0x7f3930004850 2023-11-28T13:08:45.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.962+0000 7f3948b99700 1 -- 172.21.15.156:0/1441852688 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393000c110 con 0x7f3930003a00 2023-11-28T13:08:45.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.962+0000 7f3948b99700 1 -- 172.21.15.156:0/1441852688 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393000c2d0 con 0x7f3930005200 2023-11-28T13:08:45.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.962+0000 7f394939a700 1 --2- 172.21.15.156:0/1441852688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3930004850 0x7f3930004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:45.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.962+0000 7f3949b9b700 1 --2- 172.21.15.156:0/1441852688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3930003a00 0x7f3930003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:45.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.962+0000 7f394a39c700 1 --2- 172.21.15.156:0/1441852688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3930005200 0x7f393000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:45.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.962+0000 7f394939a700 1 -- 172.21.15.156:0/1441852688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3930005200 msgr2=0x7f393000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:45.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.962+0000 7f394939a700 1 --2- 172.21.15.156:0/1441852688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3930005200 0x7f393000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:45.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.962+0000 7f394939a700 1 -- 172.21.15.156:0/1441852688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3930003a00 msgr2=0x7f3930003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:45.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.962+0000 7f394939a700 1 --2- 172.21.15.156:0/1441852688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3930003a00 0x7f3930003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:45.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.962+0000 7f394939a700 1 -- 172.21.15.156:0/1441852688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39300035b0 con 0x7f3930004850 2023-11-28T13:08:45.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.962+0000 7f394a39c700 1 -- 172.21.15.156:0/1441852688 reap_dead start 2023-11-28T13:08:45.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.962+0000 7f394939a700 1 --2- 172.21.15.156:0/1441852688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3930004850 0x7f3930004cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3934009b60 tx=0x7f3930005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854970991bdb2b7d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:45.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.963+0000 7f3948b99700 1 -- 172.21.15.156:0/1441852688 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393400e6c0 con 0x7f3930004850 2023-11-28T13:08:45.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.963+0000 7f3948b99700 1 -- 172.21.15.156:0/1441852688 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393400e820 con 0x7f3930004850 2023-11-28T13:08:45.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.963+0000 7f3948b99700 1 -- 172.21.15.156:0/1441852688 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393400ebb0 con 0x7f3930004850 2023-11-28T13:08:45.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.963+0000 7f394bdff700 1 -- 172.21.15.156:0/1441852688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3930004850 msgr2=0x7f3930004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:45.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.963+0000 7f394bdff700 1 --2- 172.21.15.156:0/1441852688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3930004850 0x7f3930004cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3934009b60 tx=0x7f3930005880 comp rx=0 tx=0).stop 2023-11-28T13:08:45.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.963+0000 7f394bdff700 1 -- 172.21.15.156:0/1441852688 shutdown_connections 2023-11-28T13:08:45.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.963+0000 7f394bdff700 1 --2- 172.21.15.156:0/1441852688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3930004850 0x7f3930004cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:45.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.963+0000 7f394bdff700 1 -- 172.21.15.156:0/1441852688 >> 172.21.15.156:0/1441852688 conn(0x7f3944072600 msgr2=0x7f39440cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:45.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.963+0000 7f394bdff700 1 -- 172.21.15.156:0/1441852688 shutdown_connections 2023-11-28T13:08:45.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.963+0000 7f394bdff700 1 -- 172.21.15.156:0/1441852688 wait complete. 2023-11-28T13:08:45.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.965+0000 7f394bdff700 1 Processor -- start 2023-11-28T13:08:45.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.965+0000 7f394bdff700 1 -- start start 2023-11-28T13:08:45.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.966+0000 7f394bdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3930004850 0x7f39441f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:45.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.966+0000 7f394bdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39440c8010 0x7f39440cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:45.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.966+0000 7f394bdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39440cde00 0x7f394415e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:45.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.966+0000 7f394bdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39441f2ce0 con 0x7f39440cde00 2023-11-28T13:08:45.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.966+0000 7f394bdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39441f3af0 con 0x7f39440c8010 2023-11-28T13:08:45.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.966+0000 7f394bdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39441f2ee0 con 0x7f3930004850 2023-11-28T13:08:45.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.966+0000 7f394a39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39440cde00 0x7f394415e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:45.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.966+0000 7f394a39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39440cde00 0x7f394415e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39642/0 (socket says 172.21.15.156:39642) 2023-11-28T13:08:45.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.966+0000 7f394a39c700 1 -- 172.21.15.156:0/2878450124 learned_addr learned my addr 172.21.15.156:0/2878450124 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:45.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.967+0000 7f394a39c700 1 -- 172.21.15.156:0/2878450124 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3930004850 msgr2=0x7f39441f27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:45.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.967+0000 7f3949b9b700 1 --2- 172.21.15.156:0/2878450124 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3930004850 0x7f39441f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:45.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.967+0000 7f394939a700 1 --2- 172.21.15.156:0/2878450124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39440c8010 0x7f39440cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:45.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.967+0000 7f394a39c700 1 --2- 172.21.15.156:0/2878450124 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3930004850 0x7f39441f27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:45.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.967+0000 7f394a39c700 1 -- 172.21.15.156:0/2878450124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39440c8010 msgr2=0x7f39440cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:45.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.967+0000 7f394a39c700 1 --2- 172.21.15.156:0/2878450124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39440c8010 0x7f39440cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:45.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.967+0000 7f394a39c700 1 -- 172.21.15.156:0/2878450124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394415cd80 con 0x7f39440cde00 2023-11-28T13:08:45.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.967+0000 7f394939a700 1 --2- 172.21.15.156:0/2878450124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39440c8010 0x7f39440cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:45.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.967+0000 7f394a39c700 1 --2- 172.21.15.156:0/2878450124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39440cde00 0x7f394415e6f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f39440cfbc0 tx=0x7f393c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:45.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.967+0000 7f393a7fc700 1 -- 172.21.15.156:0/2878450124 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c003c10 con 0x7f39440cde00 2023-11-28T13:08:45.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.967+0000 7f393a7fc700 1 -- 172.21.15.156:0/2878450124 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c003d70 con 0x7f39440cde00 2023-11-28T13:08:45.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.967+0000 7f394bdff700 1 -- 172.21.15.156:0/2878450124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39440d15e0 con 0x7f39440cde00 2023-11-28T13:08:45.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.967+0000 7f393a7fc700 1 -- 172.21.15.156:0/2878450124 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c005310 con 0x7f39440cde00 2023-11-28T13:08:45.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.967+0000 7f394bdff700 1 -- 172.21.15.156:0/2878450124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394415cf10 con 0x7f39440cde00 2023-11-28T13:08:45.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.968+0000 7f394bdff700 1 -- 172.21.15.156:0/2878450124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39441f3c80 con 0x7f39440cde00 2023-11-28T13:08:45.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.969+0000 7f393a7fc700 1 -- 172.21.15.156:0/2878450124 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f393c005470 con 0x7f39440cde00 2023-11-28T13:08:45.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.970+0000 7f393a7fc700 1 --2- 172.21.15.156:0/2878450124 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39280682f0 0x7f392806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:45.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.970+0000 7f393a7fc700 1 -- 172.21.15.156:0/2878450124 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f393c085df0 con 0x7f39440cde00 2023-11-28T13:08:45.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.970+0000 7f3949b9b700 1 --2- 172.21.15.156:0/2878450124 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39280682f0 0x7f392806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:45.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.971+0000 7f3949b9b700 1 --2- 172.21.15.156:0/2878450124 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39280682f0 0x7f392806a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3940001050 tx=0x7f3940009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:45.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:45.974+0000 7f393a7fc700 1 -- 172.21.15.156:0/2878450124 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f393c052650 con 0x7f39440cde00 2023-11-28T13:08:46.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.163+0000 7f394bdff700 1 -- 172.21.15.156:0/2878450124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f39440745c0 con 0x7f39440cde00 2023-11-28T13:08:46.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.164+0000 7f393a7fc700 1 -- 172.21.15.156:0/2878450124 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f393c055c70 con 0x7f39440cde00 2023-11-28T13:08:46.165 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:08:46.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.166+0000 7f394bdff700 1 -- 172.21.15.156:0/2878450124 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39280682f0 msgr2=0x7f392806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:46.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.166+0000 7f394bdff700 1 --2- 172.21.15.156:0/2878450124 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39280682f0 0x7f392806a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3940001050 tx=0x7f3940009040 comp rx=0 tx=0).stop 2023-11-28T13:08:46.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.166+0000 7f394bdff700 1 -- 172.21.15.156:0/2878450124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39440cde00 msgr2=0x7f394415e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:46.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.166+0000 7f394bdff700 1 --2- 172.21.15.156:0/2878450124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39440cde00 0x7f394415e6f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f39440cfbc0 tx=0x7f393c00ee00 comp rx=0 tx=0).stop 2023-11-28T13:08:46.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.166+0000 7f394bdff700 1 -- 172.21.15.156:0/2878450124 shutdown_connections 2023-11-28T13:08:46.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.166+0000 7f394bdff700 1 --2- 172.21.15.156:0/2878450124 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f39280682f0 0x7f392806a7a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.166+0000 7f394bdff700 1 --2- 172.21.15.156:0/2878450124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39440cde00 0x7f394415e6f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.166+0000 7f394bdff700 1 --2- 172.21.15.156:0/2878450124 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3930004850 0x7f39441f27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.166+0000 7f394bdff700 1 --2- 172.21.15.156:0/2878450124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39440c8010 0x7f39440cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.166+0000 7f394bdff700 1 -- 172.21.15.156:0/2878450124 >> 172.21.15.156:0/2878450124 conn(0x7f3944072600 msgr2=0x7f39440b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:46.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.166+0000 7f394bdff700 1 -- 172.21.15.156:0/2878450124 shutdown_connections 2023-11-28T13:08:46.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.166+0000 7f394bdff700 1 -- 172.21.15.156:0/2878450124 wait complete. 2023-11-28T13:08:46.186 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:08:46.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.384+0000 7f4f475ff700 1 Processor -- start 2023-11-28T13:08:46.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.384+0000 7f4f475ff700 1 -- start start 2023-11-28T13:08:46.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.385+0000 7f4f475ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4f4015c9b0 0x7f4f400b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:46.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.385+0000 7f4f475ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f40161340 0x7f4f400d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:46.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.385+0000 7f4f475ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f400d1950 con 0x7f4f400d2320 2023-11-28T13:08:46.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.385+0000 7f4f475ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f400b5800 con 0x7f4f4015c9b0 2023-11-28T13:08:46.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.385+0000 7f4f475ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f400b5520 con 0x7f4f40161340 2023-11-28T13:08:46.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.386+0000 7f4f44b9a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f40161340 0x7f4f400d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:46.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.386+0000 7f4f44b9a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f40161340 0x7f4f400d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:56312/0 (socket says 172.21.15.156:56312) 2023-11-28T13:08:46.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.386+0000 7f4f4539b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4f4015c9b0 0x7f4f400b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:46.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.386+0000 7f4f44b9a700 1 -- 172.21.15.156:0/888833595 learned_addr learned my addr 172.21.15.156:0/888833595 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:46.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.386+0000 7f4f45b9c700 1 --1- 172.21.15.156:0/888833595 >> v1:172.21.15.186:6789/0 conn(0x7f4f400d2320 0x7f4f400cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:33244/0 (socket says 172.21.15.156:33244) 2023-11-28T13:08:46.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.386+0000 7f4f44b9a700 1 -- 172.21.15.156:0/888833595 >> v2:172.21.15.186:3300/0 conn(0x7f4f4015c9b0 msgr2=0x7f4f400b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:46.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.386+0000 7f4f44b9a700 1 --2- 172.21.15.156:0/888833595 >> v2:172.21.15.186:3300/0 conn(0x7f4f4015c9b0 0x7f4f400b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.386+0000 7f4f44b9a700 1 -- 172.21.15.156:0/888833595 >> v1:172.21.15.186:6789/0 conn(0x7f4f400d2320 legacy=0x7f4f400cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:46.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.386+0000 7f4f44b9a700 1 -- 172.21.15.156:0/888833595 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f400d1300 con 0x7f4f40161340 2023-11-28T13:08:46.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.386+0000 7f4f44b9a700 1 --2- 172.21.15.156:0/888833595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f40161340 0x7f4f400d1de0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f300098f0 tx=0x7f4f30009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1ee591966809193 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:46.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.387+0000 7f4f37fff700 1 -- 172.21.15.156:0/888833595 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30011830 con 0x7f4f40161340 2023-11-28T13:08:46.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.387+0000 7f4f37fff700 1 -- 172.21.15.156:0/888833595 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f30011990 con 0x7f4f40161340 2023-11-28T13:08:46.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.387+0000 7f4f37fff700 1 -- 172.21.15.156:0/888833595 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30011d20 con 0x7f4f40161340 2023-11-28T13:08:46.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.387+0000 7f4f475ff700 1 -- 172.21.15.156:0/888833595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f40161340 msgr2=0x7f4f400d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:46.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.387+0000 7f4f475ff700 1 --2- 172.21.15.156:0/888833595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f40161340 0x7f4f400d1de0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f300098f0 tx=0x7f4f30009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:46.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.387+0000 7f4f475ff700 1 -- 172.21.15.156:0/888833595 shutdown_connections 2023-11-28T13:08:46.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.387+0000 7f4f475ff700 1 --2- 172.21.15.156:0/888833595 >> v2:172.21.15.186:3300/0 conn(0x7f4f4015c9b0 0x7f4f400b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.387+0000 7f4f475ff700 1 --2- 172.21.15.156:0/888833595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f40161340 0x7f4f400d1de0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.387+0000 7f4f475ff700 1 -- 172.21.15.156:0/888833595 >> 172.21.15.156:0/888833595 conn(0x7f4f40072600 msgr2=0x7f4f400cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:46.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.387+0000 7f4f475ff700 1 -- 172.21.15.156:0/888833595 shutdown_connections 2023-11-28T13:08:46.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.387+0000 7f4f475ff700 1 -- 172.21.15.156:0/888833595 wait complete. 2023-11-28T13:08:46.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.389+0000 7f4f475ff700 1 Processor -- start 2023-11-28T13:08:46.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.389+0000 7f4f475ff700 1 -- start start 2023-11-28T13:08:46.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.390+0000 7f4f475ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f400d2320 0x7f4f400c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:46.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.390+0000 7f4f475ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4015c9b0 0x7f4f400c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:46.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.390+0000 7f4f475ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f40161340 0x7f4f400c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:46.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.390+0000 7f4f475ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f40155d10 con 0x7f4f40161340 2023-11-28T13:08:46.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.390+0000 7f4f475ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f40155a00 con 0x7f4f4015c9b0 2023-11-28T13:08:46.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.390+0000 7f4f475ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f400c7bc0 con 0x7f4f400d2320 2023-11-28T13:08:46.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.390+0000 7f4f45b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f40161340 0x7f4f400c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:46.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.390+0000 7f4f45b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f40161340 0x7f4f400c5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39648/0 (socket says 172.21.15.156:39648) 2023-11-28T13:08:46.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.390+0000 7f4f45b9c700 1 -- 172.21.15.156:0/3518667896 learned_addr learned my addr 172.21.15.156:0/3518667896 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:46.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.390+0000 7f4f44b9a700 1 --2- 172.21.15.156:0/3518667896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4015c9b0 0x7f4f400c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:46.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.390+0000 7f4f4539b700 1 --2- 172.21.15.156:0/3518667896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f400d2320 0x7f4f400c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:46.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.391+0000 7f4f45b9c700 1 -- 172.21.15.156:0/3518667896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f400d2320 msgr2=0x7f4f400c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:46.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.391+0000 7f4f45b9c700 1 --2- 172.21.15.156:0/3518667896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f400d2320 0x7f4f400c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.391+0000 7f4f45b9c700 1 -- 172.21.15.156:0/3518667896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4015c9b0 msgr2=0x7f4f400c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:46.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.391+0000 7f4f45b9c700 1 --2- 172.21.15.156:0/3518667896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4015c9b0 0x7f4f400c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.391+0000 7f4f45b9c700 1 -- 172.21.15.156:0/3518667896 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f30009550 con 0x7f4f40161340 2023-11-28T13:08:46.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.391+0000 7f4f45b9c700 1 --2- 172.21.15.156:0/3518667896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f40161340 0x7f4f400c5e60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800cbc0 tx=0x7f4f3800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:46.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.391+0000 7f4f35ffb700 1 -- 172.21.15.156:0/3518667896 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3800a980 con 0x7f4f40161340 2023-11-28T13:08:46.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.391+0000 7f4f35ffb700 1 -- 172.21.15.156:0/3518667896 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3800aae0 con 0x7f4f40161340 2023-11-28T13:08:46.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.391+0000 7f4f475ff700 1 -- 172.21.15.156:0/3518667896 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f400b5130 con 0x7f4f40161340 2023-11-28T13:08:46.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.391+0000 7f4f35ffb700 1 -- 172.21.15.156:0/3518667896 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3800ae50 con 0x7f4f40161340 2023-11-28T13:08:46.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.392+0000 7f4f475ff700 1 -- 172.21.15.156:0/3518667896 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f400d15e0 con 0x7f4f40161340 2023-11-28T13:08:46.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.392+0000 7f4f475ff700 1 -- 172.21.15.156:0/3518667896 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f400b52c0 con 0x7f4f40161340 2023-11-28T13:08:46.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.393+0000 7f4f35ffb700 1 -- 172.21.15.156:0/3518667896 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4f38019030 con 0x7f4f40161340 2023-11-28T13:08:46.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.394+0000 7f4f35ffb700 1 --2- 172.21.15.156:0/3518667896 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4f240682f0 0x7f4f2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:46.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.394+0000 7f4f35ffb700 1 -- 172.21.15.156:0/3518667896 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4f380857a0 con 0x7f4f40161340 2023-11-28T13:08:46.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.395+0000 7f4f4539b700 1 --2- 172.21.15.156:0/3518667896 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4f240682f0 0x7f4f2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:46.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.395+0000 7f4f4539b700 1 --2- 172.21.15.156:0/3518667896 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4f240682f0 0x7f4f2406a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009be0 tx=0x7f4f3c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:46.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.398+0000 7f4f35ffb700 1 -- 172.21.15.156:0/3518667896 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4f3804b5e0 con 0x7f4f40161340 2023-11-28T13:08:46.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.596+0000 7f4f475ff700 1 -- 172.21.15.156:0/3518667896 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f40155e50 con 0x7f4f40161340 2023-11-28T13:08:46.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.596+0000 7f4f35ffb700 1 -- 172.21.15.156:0/3518667896 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f38017020 con 0x7f4f40161340 2023-11-28T13:08:46.598 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:08:46.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.599+0000 7f4f475ff700 1 -- 172.21.15.156:0/3518667896 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4f240682f0 msgr2=0x7f4f2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:46.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.599+0000 7f4f475ff700 1 --2- 172.21.15.156:0/3518667896 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4f240682f0 0x7f4f2406a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009be0 tx=0x7f4f3c009210 comp rx=0 tx=0).stop 2023-11-28T13:08:46.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.599+0000 7f4f475ff700 1 -- 172.21.15.156:0/3518667896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f40161340 msgr2=0x7f4f400c5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:46.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.599+0000 7f4f475ff700 1 --2- 172.21.15.156:0/3518667896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f40161340 0x7f4f400c5e60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800cbc0 tx=0x7f4f3800ced0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.599+0000 7f4f475ff700 1 -- 172.21.15.156:0/3518667896 shutdown_connections 2023-11-28T13:08:46.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.599+0000 7f4f475ff700 1 --2- 172.21.15.156:0/3518667896 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4f240682f0 0x7f4f2406a7a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.599+0000 7f4f475ff700 1 --2- 172.21.15.156:0/3518667896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f40161340 0x7f4f400c5e60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.599+0000 7f4f475ff700 1 --2- 172.21.15.156:0/3518667896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f400d2320 0x7f4f400c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.599+0000 7f4f475ff700 1 --2- 172.21.15.156:0/3518667896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4015c9b0 0x7f4f400c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.599+0000 7f4f475ff700 1 -- 172.21.15.156:0/3518667896 >> 172.21.15.156:0/3518667896 conn(0x7f4f40072600 msgr2=0x7f4f400b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:46.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.599+0000 7f4f475ff700 1 -- 172.21.15.156:0/3518667896 shutdown_connections 2023-11-28T13:08:46.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.599+0000 7f4f475ff700 1 -- 172.21.15.156:0/3518667896 wait complete. 2023-11-28T13:08:46.619 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:08:46.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.817+0000 7f885fdff700 1 Processor -- start 2023-11-28T13:08:46.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.817+0000 7f885fdff700 1 -- start start 2023-11-28T13:08:46.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.818+0000 7f885fdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f885815c9b0 0x7f88580b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:46.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.818+0000 7f885fdff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8858161340 0x7f88580d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:46.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.818+0000 7f885fdff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88580d1950 con 0x7f88580d2320 2023-11-28T13:08:46.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.818+0000 7f885fdff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88580b5800 con 0x7f8858161340 2023-11-28T13:08:46.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.818+0000 7f885fdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88580b5520 con 0x7f885815c9b0 2023-11-28T13:08:46.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.818+0000 7f885d39a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8858161340 0x7f88580d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:46.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.818+0000 7f885e39c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f88580d2320 0x7f88580cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39442/0 (socket says 172.21.15.156:39442) 2023-11-28T13:08:46.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.818+0000 7f885e39c700 1 -- 172.21.15.156:0/356902868 learned_addr learned my addr 172.21.15.156:0/356902868 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:46.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.819+0000 7f885db9b700 1 --2- 172.21.15.156:0/356902868 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f885815c9b0 0x7f88580b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:46.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.819+0000 7f885d39a700 1 -- 172.21.15.156:0/356902868 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f885815c9b0 msgr2=0x7f88580b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:46.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.819+0000 7f885d39a700 1 --2- 172.21.15.156:0/356902868 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f885815c9b0 0x7f88580b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.819+0000 7f885d39a700 1 -- 172.21.15.156:0/356902868 >> v1:172.21.15.156:6789/0 conn(0x7f88580d2320 legacy=0x7f88580cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:46.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.819+0000 7f885d39a700 1 -- 172.21.15.156:0/356902868 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88580d1300 con 0x7f8858161340 2023-11-28T13:08:46.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.819+0000 7f885db9b700 1 --2- 172.21.15.156:0/356902868 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f885815c9b0 0x7f88580b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:46.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.819+0000 7f885cb99700 1 -- 172.21.15.156:0/356902868 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041101870 0 0) 0x7f88580d1950 con 0x7f88580d2320 2023-11-28T13:08:46.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.819+0000 7f885d39a700 1 --2- 172.21.15.156:0/356902868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8858161340 0x7f88580d1de0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8848009840 tx=0x7f8848009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da40344bf170a11 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:46.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.819+0000 7f885cb99700 1 -- 172.21.15.156:0/356902868 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848011830 con 0x7f8858161340 2023-11-28T13:08:46.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.819+0000 7f885cb99700 1 -- 172.21.15.156:0/356902868 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8848011990 con 0x7f8858161340 2023-11-28T13:08:46.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.819+0000 7f885cb99700 1 -- 172.21.15.156:0/356902868 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848011d20 con 0x7f8858161340 2023-11-28T13:08:46.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.820+0000 7f885fdff700 1 -- 172.21.15.156:0/356902868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8858161340 msgr2=0x7f88580d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:46.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.820+0000 7f885fdff700 1 --2- 172.21.15.156:0/356902868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8858161340 0x7f88580d1de0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8848009840 tx=0x7f8848009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:46.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.820+0000 7f885fdff700 1 -- 172.21.15.156:0/356902868 shutdown_connections 2023-11-28T13:08:46.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.820+0000 7f885fdff700 1 --2- 172.21.15.156:0/356902868 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f885815c9b0 0x7f88580b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.820+0000 7f885fdff700 1 --2- 172.21.15.156:0/356902868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8858161340 0x7f88580d1de0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.820+0000 7f885fdff700 1 -- 172.21.15.156:0/356902868 >> 172.21.15.156:0/356902868 conn(0x7f8858072600 msgr2=0x7f88580cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.820+0000 7f885fdff700 1 -- 172.21.15.156:0/356902868 shutdown_connections 2023-11-28T13:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.820+0000 7f885fdff700 1 -- 172.21.15.156:0/356902868 wait complete. 2023-11-28T13:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.821+0000 7f885fdff700 1 Processor -- start 2023-11-28T13:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.822+0000 7f885fdff700 1 -- start start 2023-11-28T13:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.822+0000 7f885fdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88580d2320 0x7f8858149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.823+0000 7f885fdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f885815c9b0 0x7f8858155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.823+0000 7f885fdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8858161340 0x7f8858155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.823+0000 7f885fdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8858155030 con 0x7f8858161340 2023-11-28T13:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.823+0000 7f885fdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885815fca0 con 0x7f885815c9b0 2023-11-28T13:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.823+0000 7f885fdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885815f6e0 con 0x7f88580d2320 2023-11-28T13:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.823+0000 7f885e39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8858161340 0x7f8858155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:46.827 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.823+0000 7f885e39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8858161340 0x7f8858155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39666/0 (socket says 172.21.15.156:39666) 2023-11-28T13:08:46.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.823+0000 7f885e39c700 1 -- 172.21.15.156:0/2268302733 learned_addr learned my addr 172.21.15.156:0/2268302733 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:46.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.823+0000 7f885e39c700 1 -- 172.21.15.156:0/2268302733 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88580d2320 msgr2=0x7f8858149790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:46.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.823+0000 7f885e39c700 1 --2- 172.21.15.156:0/2268302733 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88580d2320 0x7f8858149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.823+0000 7f885e39c700 1 -- 172.21.15.156:0/2268302733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f885815c9b0 msgr2=0x7f8858155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:46.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.823+0000 7f885e39c700 1 --2- 172.21.15.156:0/2268302733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f885815c9b0 0x7f8858155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:46.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.823+0000 7f885e39c700 1 -- 172.21.15.156:0/2268302733 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88480094a0 con 0x7f8858161340 2023-11-28T13:08:46.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.823+0000 7f885e39c700 1 --2- 172.21.15.156:0/2268302733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8858161340 0x7f8858155fb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f885000cba0 tx=0x7f885000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:46.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.824+0000 7f884e7fc700 1 -- 172.21.15.156:0/2268302733 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885000ab80 con 0x7f8858161340 2023-11-28T13:08:46.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.824+0000 7f884e7fc700 1 -- 172.21.15.156:0/2268302733 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885000ad00 con 0x7f8858161340 2023-11-28T13:08:46.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.824+0000 7f885fdff700 1 -- 172.21.15.156:0/2268302733 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88580b5130 con 0x7f8858161340 2023-11-28T13:08:46.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.824+0000 7f884e7fc700 1 -- 172.21.15.156:0/2268302733 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850007310 con 0x7f8858161340 2023-11-28T13:08:46.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.824+0000 7f885fdff700 1 -- 172.21.15.156:0/2268302733 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88580d15e0 con 0x7f8858161340 2023-11-28T13:08:46.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.825+0000 7f885fdff700 1 -- 172.21.15.156:0/2268302733 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88580b52c0 con 0x7f8858161340 2023-11-28T13:08:46.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.826+0000 7f884e7fc700 1 -- 172.21.15.156:0/2268302733 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8850010070 con 0x7f8858161340 2023-11-28T13:08:46.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.826+0000 7f884e7fc700 1 --2- 172.21.15.156:0/2268302733 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f883c0682f0 0x7f883c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:46.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.827+0000 7f884e7fc700 1 -- 172.21.15.156:0/2268302733 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f88500855f0 con 0x7f8858161340 2023-11-28T13:08:46.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.827+0000 7f885db9b700 1 --2- 172.21.15.156:0/2268302733 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f883c0682f0 0x7f883c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:46.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.828+0000 7f885db9b700 1 --2- 172.21.15.156:0/2268302733 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f883c0682f0 0x7f883c06a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f885400a8b0 tx=0x7f8854008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:46.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:46.831+0000 7f884e7fc700 1 -- 172.21.15.156:0/2268302733 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8850051e50 con 0x7f8858161340 2023-11-28T13:08:47.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.022+0000 7f885fdff700 1 -- 172.21.15.156:0/2268302733 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8858155170 con 0x7f8858161340 2023-11-28T13:08:47.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.023+0000 7f884e7fc700 1 -- 172.21.15.156:0/2268302733 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8858155170 con 0x7f8858161340 2023-11-28T13:08:47.024 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T13:08:47.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.025+0000 7f885fdff700 1 -- 172.21.15.156:0/2268302733 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f883c0682f0 msgr2=0x7f883c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:47.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.025+0000 7f885fdff700 1 --2- 172.21.15.156:0/2268302733 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f883c0682f0 0x7f883c06a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f885400a8b0 tx=0x7f8854008040 comp rx=0 tx=0).stop 2023-11-28T13:08:47.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.025+0000 7f885fdff700 1 -- 172.21.15.156:0/2268302733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8858161340 msgr2=0x7f8858155fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:47.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.025+0000 7f885fdff700 1 --2- 172.21.15.156:0/2268302733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8858161340 0x7f8858155fb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f885000cba0 tx=0x7f885000cf60 comp rx=0 tx=0).stop 2023-11-28T13:08:47.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.025+0000 7f885fdff700 1 -- 172.21.15.156:0/2268302733 shutdown_connections 2023-11-28T13:08:47.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.025+0000 7f885fdff700 1 --2- 172.21.15.156:0/2268302733 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f883c0682f0 0x7f883c06a7a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.025+0000 7f885fdff700 1 --2- 172.21.15.156:0/2268302733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8858161340 0x7f8858155fb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.025+0000 7f885fdff700 1 --2- 172.21.15.156:0/2268302733 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88580d2320 0x7f8858149790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.025+0000 7f885fdff700 1 --2- 172.21.15.156:0/2268302733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f885815c9b0 0x7f8858155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.025+0000 7f885fdff700 1 -- 172.21.15.156:0/2268302733 >> 172.21.15.156:0/2268302733 conn(0x7f8858072600 msgr2=0x7f885815cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:47.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.025+0000 7f885fdff700 1 -- 172.21.15.156:0/2268302733 shutdown_connections 2023-11-28T13:08:47.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.025+0000 7f885fdff700 1 -- 172.21.15.156:0/2268302733 wait complete. 2023-11-28T13:08:47.045 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:08:47.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.244+0000 7f264f3ff700 1 Processor -- start 2023-11-28T13:08:47.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.244+0000 7f264f3ff700 1 -- start start 2023-11-28T13:08:47.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.245+0000 7f264f3ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f264815c9b0 0x7f26480b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:47.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.245+0000 7f264f3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2648161340 0x7f26480d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:47.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.245+0000 7f264f3ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26480d1950 con 0x7f26480d2320 2023-11-28T13:08:47.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.246+0000 7f264f3ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26480b5800 con 0x7f264815c9b0 2023-11-28T13:08:47.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.246+0000 7f264f3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26480b5520 con 0x7f2648161340 2023-11-28T13:08:47.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.246+0000 7f264d19b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f264815c9b0 0x7f26480b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:47.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.246+0000 7f264d19b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f264815c9b0 0x7f26480b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39680/0 (socket says 172.21.15.156:39680) 2023-11-28T13:08:47.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.246+0000 7f264d19b700 1 -- 172.21.15.156:0/2877753126 learned_addr learned my addr 172.21.15.156:0/2877753126 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:47.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.246+0000 7f264c99a700 1 --2- 172.21.15.156:0/2877753126 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2648161340 0x7f26480d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:47.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.246+0000 7f264d19b700 1 -- 172.21.15.156:0/2877753126 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2648161340 msgr2=0x7f26480d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:47.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.246+0000 7f264d19b700 1 --2- 172.21.15.156:0/2877753126 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2648161340 0x7f26480d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.246+0000 7f264d19b700 1 -- 172.21.15.156:0/2877753126 >> v1:172.21.15.186:6789/0 conn(0x7f26480d2320 legacy=0x7f26480cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:47.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.246+0000 7f264d19b700 1 -- 172.21.15.156:0/2877753126 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26480d1300 con 0x7f264815c9b0 2023-11-28T13:08:47.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.247+0000 7f264d19b700 1 --2- 172.21.15.156:0/2877753126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f264815c9b0 0x7f26480b4bf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f26440098f0 tx=0x7f2644009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55d8390875e649e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:47.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.247+0000 7f263ffff700 1 -- 172.21.15.156:0/2877753126 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2644011830 con 0x7f264815c9b0 2023-11-28T13:08:47.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.247+0000 7f263ffff700 1 -- 172.21.15.156:0/2877753126 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2644011990 con 0x7f264815c9b0 2023-11-28T13:08:47.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.247+0000 7f263ffff700 1 -- 172.21.15.156:0/2877753126 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2644011d20 con 0x7f264815c9b0 2023-11-28T13:08:47.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.247+0000 7f264f3ff700 1 -- 172.21.15.156:0/2877753126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f264815c9b0 msgr2=0x7f26480b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:47.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.247+0000 7f264f3ff700 1 --2- 172.21.15.156:0/2877753126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f264815c9b0 0x7f26480b4bf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f26440098f0 tx=0x7f2644009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:47.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.247+0000 7f264f3ff700 1 -- 172.21.15.156:0/2877753126 shutdown_connections 2023-11-28T13:08:47.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.247+0000 7f264f3ff700 1 --2- 172.21.15.156:0/2877753126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f264815c9b0 0x7f26480b4bf0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.247+0000 7f264f3ff700 1 --2- 172.21.15.156:0/2877753126 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2648161340 0x7f26480d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.247+0000 7f264f3ff700 1 -- 172.21.15.156:0/2877753126 >> 172.21.15.156:0/2877753126 conn(0x7f2648072600 msgr2=0x7f26480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:47.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.247+0000 7f264f3ff700 1 -- 172.21.15.156:0/2877753126 shutdown_connections 2023-11-28T13:08:47.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.247+0000 7f264f3ff700 1 -- 172.21.15.156:0/2877753126 wait complete. 2023-11-28T13:08:47.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.249+0000 7f264f3ff700 1 Processor -- start 2023-11-28T13:08:47.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.249+0000 7f264f3ff700 1 -- start start 2023-11-28T13:08:47.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.250+0000 7f264f3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26480d2320 0x7f2648151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:47.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.250+0000 7f264f3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f264815c9b0 0x7f2648150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:47.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.250+0000 7f264f3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2648161340 0x7f26481e8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:47.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.250+0000 7f264f3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2648150760 con 0x7f26480d2320 2023-11-28T13:08:47.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.250+0000 7f264f3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2648154af0 con 0x7f264815c9b0 2023-11-28T13:08:47.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.250+0000 7f264f3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2648154dd0 con 0x7f2648161340 2023-11-28T13:08:47.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.250+0000 7f264d19b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26480d2320 0x7f2648151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:47.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.250+0000 7f264d19b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26480d2320 0x7f2648151700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39686/0 (socket says 172.21.15.156:39686) 2023-11-28T13:08:47.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.250+0000 7f264d19b700 1 -- 172.21.15.156:0/2032749543 learned_addr learned my addr 172.21.15.156:0/2032749543 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:47.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.251+0000 7f264c99a700 1 --2- 172.21.15.156:0/2032749543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f264815c9b0 0x7f2648150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:47.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.251+0000 7f264d99c700 1 --2- 172.21.15.156:0/2032749543 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2648161340 0x7f26481e8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:47.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.251+0000 7f264d19b700 1 -- 172.21.15.156:0/2032749543 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2648161340 msgr2=0x7f26481e8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:47.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.251+0000 7f264d19b700 1 --2- 172.21.15.156:0/2032749543 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2648161340 0x7f26481e8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.251+0000 7f264d19b700 1 -- 172.21.15.156:0/2032749543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f264815c9b0 msgr2=0x7f2648150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:47.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.251+0000 7f264d19b700 1 --2- 172.21.15.156:0/2032749543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f264815c9b0 0x7f2648150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.251+0000 7f264d19b700 1 -- 172.21.15.156:0/2032749543 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2644009550 con 0x7f26480d2320 2023-11-28T13:08:47.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.251+0000 7f264d19b700 1 --2- 172.21.15.156:0/2032749543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26480d2320 0x7f2648151700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2644000c00 tx=0x7f2644004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:47.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.251+0000 7f263dffb700 1 -- 172.21.15.156:0/2032749543 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2644004570 con 0x7f26480d2320 2023-11-28T13:08:47.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.251+0000 7f263dffb700 1 -- 172.21.15.156:0/2032749543 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264401f070 con 0x7f26480d2320 2023-11-28T13:08:47.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.251+0000 7f264f3ff700 1 -- 172.21.15.156:0/2032749543 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26480b5130 con 0x7f26480d2320 2023-11-28T13:08:47.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.251+0000 7f263dffb700 1 -- 172.21.15.156:0/2032749543 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26440038c0 con 0x7f26480d2320 2023-11-28T13:08:47.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.252+0000 7f264f3ff700 1 -- 172.21.15.156:0/2032749543 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26480d15e0 con 0x7f26480d2320 2023-11-28T13:08:47.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.253+0000 7f264f3ff700 1 -- 172.21.15.156:0/2032749543 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26480b52c0 con 0x7f26480d2320 2023-11-28T13:08:47.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.253+0000 7f263dffb700 1 -- 172.21.15.156:0/2032749543 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f264402a020 con 0x7f26480d2320 2023-11-28T13:08:47.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.255+0000 7f263dffb700 1 --2- 172.21.15.156:0/2032749543 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f262c0682f0 0x7f262c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:47.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.255+0000 7f263dffb700 1 -- 172.21.15.156:0/2032749543 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2644087300 con 0x7f26480d2320 2023-11-28T13:08:47.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.255+0000 7f264c99a700 1 --2- 172.21.15.156:0/2032749543 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f262c0682f0 0x7f262c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:47.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.256+0000 7f264c99a700 1 --2- 172.21.15.156:0/2032749543 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f262c0682f0 0x7f262c06a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f26480ce3a0 tx=0x7f2638008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:47.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.259+0000 7f263dffb700 1 -- 172.21.15.156:0/2032749543 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2644053b60 con 0x7f26480d2320 2023-11-28T13:08:47.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.449+0000 7f264f3ff700 1 -- 172.21.15.156:0/2032749543 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2648066d60 con 0x7f26480d2320 2023-11-28T13:08:47.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.449+0000 7f263dffb700 1 -- 172.21.15.156:0/2032749543 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2644057180 con 0x7f26480d2320 2023-11-28T13:08:47.451 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T13:08:47.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.451+0000 7f264f3ff700 1 -- 172.21.15.156:0/2032749543 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f262c0682f0 msgr2=0x7f262c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:47.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.451+0000 7f264f3ff700 1 --2- 172.21.15.156:0/2032749543 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f262c0682f0 0x7f262c06a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f26480ce3a0 tx=0x7f2638008040 comp rx=0 tx=0).stop 2023-11-28T13:08:47.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.451+0000 7f264f3ff700 1 -- 172.21.15.156:0/2032749543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26480d2320 msgr2=0x7f2648151700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:47.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.451+0000 7f264f3ff700 1 --2- 172.21.15.156:0/2032749543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26480d2320 0x7f2648151700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2644000c00 tx=0x7f2644004750 comp rx=0 tx=0).stop 2023-11-28T13:08:47.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.451+0000 7f264f3ff700 1 -- 172.21.15.156:0/2032749543 shutdown_connections 2023-11-28T13:08:47.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.451+0000 7f264f3ff700 1 --2- 172.21.15.156:0/2032749543 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f262c0682f0 0x7f262c06a7a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.451+0000 7f264f3ff700 1 --2- 172.21.15.156:0/2032749543 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2648161340 0x7f26481e8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.451+0000 7f264f3ff700 1 --2- 172.21.15.156:0/2032749543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26480d2320 0x7f2648151700 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.451+0000 7f264f3ff700 1 --2- 172.21.15.156:0/2032749543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f264815c9b0 0x7f2648150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.451+0000 7f264f3ff700 1 -- 172.21.15.156:0/2032749543 >> 172.21.15.156:0/2032749543 conn(0x7f2648072600 msgr2=0x7f2648160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:47.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.451+0000 7f264f3ff700 1 -- 172.21.15.156:0/2032749543 shutdown_connections 2023-11-28T13:08:47.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.451+0000 7f264f3ff700 1 -- 172.21.15.156:0/2032749543 wait complete. 2023-11-28T13:08:47.471 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:08:47.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.668+0000 7fd1effff700 1 Processor -- start 2023-11-28T13:08:47.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.668+0000 7fd1effff700 1 -- start start 2023-11-28T13:08:47.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.670+0000 7fd1effff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd1e80d1330 0x7fd1e80b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:47.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.670+0000 7fd1effff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd1e80b20b0 0x7fd1e80b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:47.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.670+0000 7fd1effff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e80b0230 con 0x7fd1e80d01b0 2023-11-28T13:08:47.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.670+0000 7fd1effff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1e815cec0 con 0x7fd1e80b20b0 2023-11-28T13:08:47.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.670+0000 7fd1effff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1e80b4570 con 0x7fd1e80d1330 2023-11-28T13:08:47.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.670+0000 7fd1edd9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd1e80d01b0 0x7fd1e80d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39452/0 (socket says 172.21.15.156:39452) 2023-11-28T13:08:47.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.670+0000 7fd1edd9b700 1 -- 172.21.15.156:0/35778172 learned_addr learned my addr 172.21.15.156:0/35778172 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:47.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.670+0000 7fd1ee59c700 1 --2- 172.21.15.156:0/35778172 >> v2:172.21.15.156:3300/0 conn(0x7fd1e80b20b0 0x7fd1e80b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:47.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.670+0000 7fd1ed59a700 1 --2- 172.21.15.156:0/35778172 >> v2:172.21.15.186:3300/0 conn(0x7fd1e80d1330 0x7fd1e80b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:47.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.670+0000 7fd1ee59c700 1 -- 172.21.15.156:0/35778172 >> v2:172.21.15.186:3300/0 conn(0x7fd1e80d1330 msgr2=0x7fd1e80b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:47.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.670+0000 7fd1ee59c700 1 --2- 172.21.15.156:0/35778172 >> v2:172.21.15.186:3300/0 conn(0x7fd1e80d1330 0x7fd1e80b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.670+0000 7fd1ee59c700 1 -- 172.21.15.156:0/35778172 >> v1:172.21.15.156:6789/0 conn(0x7fd1e80d01b0 legacy=0x7fd1e80d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:47.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.670+0000 7fd1ee59c700 1 -- 172.21.15.156:0/35778172 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e80b4850 con 0x7fd1e80b20b0 2023-11-28T13:08:47.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.670+0000 7fd1ed59a700 1 --2- 172.21.15.156:0/35778172 >> v2:172.21.15.186:3300/0 conn(0x7fd1e80d1330 0x7fd1e80b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:47.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.670+0000 7fd1ecd99700 1 -- 172.21.15.156:0/35778172 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936570410 0 0) 0x7fd1e80b0230 con 0x7fd1e80d01b0 2023-11-28T13:08:47.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.671+0000 7fd1ee59c700 1 --2- 172.21.15.156:0/35778172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1e80b20b0 0x7fd1e80b0c00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80ca0b0 tx=0x7fd1e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e3f542a26897c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:47.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.671+0000 7fd1ecd99700 1 -- 172.21.15.156:0/35778172 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e000e070 con 0x7fd1e80b20b0 2023-11-28T13:08:47.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.671+0000 7fd1ecd99700 1 -- 172.21.15.156:0/35778172 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e000bbd0 con 0x7fd1e80b20b0 2023-11-28T13:08:47.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.671+0000 7fd1ecd99700 1 -- 172.21.15.156:0/35778172 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0012430 con 0x7fd1e80b20b0 2023-11-28T13:08:47.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.671+0000 7fd1effff700 1 -- 172.21.15.156:0/35778172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1e80b20b0 msgr2=0x7fd1e80b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:47.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.671+0000 7fd1effff700 1 --2- 172.21.15.156:0/35778172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1e80b20b0 0x7fd1e80b0c00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80ca0b0 tx=0x7fd1e000b330 comp rx=0 tx=0).stop 2023-11-28T13:08:47.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.671+0000 7fd1effff700 1 -- 172.21.15.156:0/35778172 shutdown_connections 2023-11-28T13:08:47.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.671+0000 7fd1effff700 1 --2- 172.21.15.156:0/35778172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1e80b20b0 0x7fd1e80b0c00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.671+0000 7fd1effff700 1 --2- 172.21.15.156:0/35778172 >> v2:172.21.15.186:3300/0 conn(0x7fd1e80d1330 0x7fd1e80b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.671+0000 7fd1effff700 1 -- 172.21.15.156:0/35778172 >> 172.21.15.156:0/35778172 conn(0x7fd1e809c3f0 msgr2=0x7fd1e80d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:47.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.672+0000 7fd1effff700 1 -- 172.21.15.156:0/35778172 shutdown_connections 2023-11-28T13:08:47.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.672+0000 7fd1effff700 1 -- 172.21.15.156:0/35778172 wait complete. 2023-11-28T13:08:47.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.673+0000 7fd1effff700 1 Processor -- start 2023-11-28T13:08:47.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.673+0000 7fd1effff700 1 -- start start 2023-11-28T13:08:47.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.674+0000 7fd1effff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1e80b20b0 0x7fd1e81585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:47.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.674+0000 7fd1effff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1e80d01b0 0x7fd1e814c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:47.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.674+0000 7fd1effff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1e80d1330 0x7fd1e815d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:47.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.674+0000 7fd1effff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e815e0f0 con 0x7fd1e80d1330 2023-11-28T13:08:47.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.674+0000 7fd1effff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e815d850 con 0x7fd1e80b20b0 2023-11-28T13:08:47.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.674+0000 7fd1effff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e815db30 con 0x7fd1e80d01b0 2023-11-28T13:08:47.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.674+0000 7fd1ee59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1e80d1330 0x7fd1e815d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:47.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.675+0000 7fd1ee59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1e80d1330 0x7fd1e815d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39710/0 (socket says 172.21.15.156:39710) 2023-11-28T13:08:47.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.675+0000 7fd1ee59c700 1 -- 172.21.15.156:0/818537919 learned_addr learned my addr 172.21.15.156:0/818537919 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:47.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.675+0000 7fd1ee59c700 1 -- 172.21.15.156:0/818537919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1e80d01b0 msgr2=0x7fd1e814c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:47.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.675+0000 7fd1ee59c700 1 --2- 172.21.15.156:0/818537919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1e80d01b0 0x7fd1e814c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.675+0000 7fd1ee59c700 1 -- 172.21.15.156:0/818537919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1e80b20b0 msgr2=0x7fd1e81585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:47.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.675+0000 7fd1ee59c700 1 --2- 172.21.15.156:0/818537919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1e80b20b0 0x7fd1e81585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.675+0000 7fd1ee59c700 1 -- 172.21.15.156:0/818537919 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e0009a70 con 0x7fd1e80d1330 2023-11-28T13:08:47.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.675+0000 7fd1ee59c700 1 --2- 172.21.15.156:0/818537919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1e80d1330 0x7fd1e815d000 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0013000 tx=0x7fd1e0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:47.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.675+0000 7fd1de7fc700 1 -- 172.21.15.156:0/818537919 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0012430 con 0x7fd1e80d1330 2023-11-28T13:08:47.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.675+0000 7fd1de7fc700 1 -- 172.21.15.156:0/818537919 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e0020070 con 0x7fd1e80d1330 2023-11-28T13:08:47.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.675+0000 7fd1effff700 1 -- 172.21.15.156:0/818537919 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e80afec0 con 0x7fd1e80d1330 2023-11-28T13:08:47.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.675+0000 7fd1de7fc700 1 -- 172.21.15.156:0/818537919 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0004280 con 0x7fd1e80d1330 2023-11-28T13:08:47.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.676+0000 7fd1effff700 1 -- 172.21.15.156:0/818537919 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e80afbe0 con 0x7fd1e80d1330 2023-11-28T13:08:47.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.677+0000 7fd1effff700 1 -- 172.21.15.156:0/818537919 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e80b2600 con 0x7fd1e80d1330 2023-11-28T13:08:47.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.677+0000 7fd1de7fc700 1 -- 172.21.15.156:0/818537919 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd1e00043e0 con 0x7fd1e80d1330 2023-11-28T13:08:47.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.679+0000 7fd1de7fc700 1 --2- 172.21.15.156:0/818537919 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd1cc0682f0 0x7fd1cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:47.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.679+0000 7fd1de7fc700 1 -- 172.21.15.156:0/818537919 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd1e0022070 con 0x7fd1e80d1330 2023-11-28T13:08:47.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.679+0000 7fd1edd9b700 1 --2- 172.21.15.156:0/818537919 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd1cc0682f0 0x7fd1cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:47.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.680+0000 7fd1edd9b700 1 --2- 172.21.15.156:0/818537919 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd1cc0682f0 0x7fd1cc06a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40011e0 tx=0x7fd1e40079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:47.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.683+0000 7fd1de7fc700 1 -- 172.21.15.156:0/818537919 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd1e0054df0 con 0x7fd1e80d1330 2023-11-28T13:08:47.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.892+0000 7fd1effff700 1 -- 172.21.15.156:0/818537919 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd1e8068690 con 0x7fd1e80d1330 2023-11-28T13:08:47.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.892+0000 7fd1de7fc700 1 -- 172.21.15.156:0/818537919 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd1e0058410 con 0x7fd1e80d1330 2023-11-28T13:08:47.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.895+0000 7fd1effff700 1 -- 172.21.15.156:0/818537919 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd1cc0682f0 msgr2=0x7fd1cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:47.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.895+0000 7fd1effff700 1 --2- 172.21.15.156:0/818537919 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd1cc0682f0 0x7fd1cc06a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40011e0 tx=0x7fd1e40079b0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.895+0000 7fd1effff700 1 -- 172.21.15.156:0/818537919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1e80d1330 msgr2=0x7fd1e815d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:47.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.895+0000 7fd1effff700 1 --2- 172.21.15.156:0/818537919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1e80d1330 0x7fd1e815d000 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0013000 tx=0x7fd1e0004ad0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.895+0000 7fd1effff700 1 -- 172.21.15.156:0/818537919 shutdown_connections 2023-11-28T13:08:47.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.895+0000 7fd1effff700 1 --2- 172.21.15.156:0/818537919 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd1cc0682f0 0x7fd1cc06a7a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.895+0000 7fd1effff700 1 --2- 172.21.15.156:0/818537919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1e80d1330 0x7fd1e815d000 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.895+0000 7fd1effff700 1 --2- 172.21.15.156:0/818537919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1e80b20b0 0x7fd1e81585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.895+0000 7fd1effff700 1 --2- 172.21.15.156:0/818537919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1e80d01b0 0x7fd1e814c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:47.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.895+0000 7fd1effff700 1 -- 172.21.15.156:0/818537919 >> 172.21.15.156:0/818537919 conn(0x7fd1e809c3f0 msgr2=0x7fd1e806b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:47.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.895+0000 7fd1effff700 1 -- 172.21.15.156:0/818537919 shutdown_connections 2023-11-28T13:08:47.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:47.895+0000 7fd1effff700 1 -- 172.21.15.156:0/818537919 wait complete. 2023-11-28T13:08:47.898 INFO:teuthology.orchestra.run.smithi156.stderr:listed 0 entries 2023-11-28T13:08:47.915 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-28T13:08:47.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:47.915 DEBUG:teuthology.orchestra.run.smithi156:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-28T13:08:47.987 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T13:08:47.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:47.988 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T13:08:48.034 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:08:48.034 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:08:48.083 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:08:48.084 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:08:48.170 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:08:48.170 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:08:48.254 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:08:48.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.450+0000 7fba5b3ff700 1 Processor -- start 2023-11-28T13:08:48.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.450+0000 7fba5b3ff700 1 -- start start 2023-11-28T13:08:48.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.451+0000 7fba5b3ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fba54161340 0x7fba540d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:48.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.451+0000 7fba5b3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba540d2320 0x7fba540cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:48.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.451+0000 7fba5b3ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba540d1950 con 0x7fba5415c9b0 2023-11-28T13:08:48.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.451+0000 7fba5b3ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba540b5800 con 0x7fba54161340 2023-11-28T13:08:48.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.451+0000 7fba5b3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba540b5300 con 0x7fba540d2320 2023-11-28T13:08:48.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.452+0000 7fba5899a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fba54161340 0x7fba540d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:48.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.452+0000 7fba5899a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fba54161340 0x7fba540d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39724/0 (socket says 172.21.15.156:39724) 2023-11-28T13:08:48.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.452+0000 7fba5899a700 1 -- 172.21.15.156:0/939391703 learned_addr learned my addr 172.21.15.156:0/939391703 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:48.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.452+0000 7fba5999c700 1 --2- 172.21.15.156:0/939391703 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba540d2320 0x7fba540cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:48.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.452+0000 7fba5899a700 1 -- 172.21.15.156:0/939391703 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba540d2320 msgr2=0x7fba540cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:48.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.452+0000 7fba5899a700 1 --2- 172.21.15.156:0/939391703 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba540d2320 0x7fba540cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:48.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.452+0000 7fba5899a700 1 -- 172.21.15.156:0/939391703 >> v1:172.21.15.186:6789/0 conn(0x7fba5415c9b0 legacy=0x7fba540b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:48.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.452+0000 7fba5899a700 1 -- 172.21.15.156:0/939391703 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba540b5520 con 0x7fba54161340 2023-11-28T13:08:48.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.452+0000 7fba5899a700 1 --2- 172.21.15.156:0/939391703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba54161340 0x7fba540d1de0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fba440098f0 tx=0x7fba44009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd37e9314e808c17 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:48.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.453+0000 7fba4bfff700 1 -- 172.21.15.156:0/939391703 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4400b8a0 con 0x7fba54161340 2023-11-28T13:08:48.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.453+0000 7fba4bfff700 1 -- 172.21.15.156:0/939391703 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4400ba00 con 0x7fba54161340 2023-11-28T13:08:48.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.453+0000 7fba4bfff700 1 -- 172.21.15.156:0/939391703 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4400bd90 con 0x7fba54161340 2023-11-28T13:08:48.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.453+0000 7fba5b3ff700 1 -- 172.21.15.156:0/939391703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba54161340 msgr2=0x7fba540d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:48.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.453+0000 7fba5b3ff700 1 --2- 172.21.15.156:0/939391703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba54161340 0x7fba540d1de0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fba440098f0 tx=0x7fba44009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:48.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.453+0000 7fba5b3ff700 1 -- 172.21.15.156:0/939391703 shutdown_connections 2023-11-28T13:08:48.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.453+0000 7fba5b3ff700 1 --2- 172.21.15.156:0/939391703 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba540d2320 0x7fba540cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:48.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.453+0000 7fba5b3ff700 1 --2- 172.21.15.156:0/939391703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba54161340 0x7fba540d1de0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:48.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.453+0000 7fba5b3ff700 1 -- 172.21.15.156:0/939391703 >> 172.21.15.156:0/939391703 conn(0x7fba54072600 msgr2=0x7fba540cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:48.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.453+0000 7fba5b3ff700 1 -- 172.21.15.156:0/939391703 shutdown_connections 2023-11-28T13:08:48.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.453+0000 7fba5b3ff700 1 -- 172.21.15.156:0/939391703 wait complete. 2023-11-28T13:08:48.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.455+0000 7fba5b3ff700 1 Processor -- start 2023-11-28T13:08:48.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.455+0000 7fba5b3ff700 1 -- start start 2023-11-28T13:08:48.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.456+0000 7fba5b3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba540d2320 0x7fba5414feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:48.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.456+0000 7fba5b3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba5415c9b0 0x7fba54159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:48.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.456+0000 7fba5b3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba54161340 0x7fba54159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:48.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.456+0000 7fba5b3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba541503f0 con 0x7fba5415c9b0 2023-11-28T13:08:48.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.456+0000 7fba5b3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba540c6ca0 con 0x7fba540d2320 2023-11-28T13:08:48.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.456+0000 7fba5b3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba540c6ef0 con 0x7fba54161340 2023-11-28T13:08:48.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.456+0000 7fba5899a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba5415c9b0 0x7fba54159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:48.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.457+0000 7fba5899a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba5415c9b0 0x7fba54159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39732/0 (socket says 172.21.15.156:39732) 2023-11-28T13:08:48.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.457+0000 7fba5899a700 1 -- 172.21.15.156:0/3424771161 learned_addr learned my addr 172.21.15.156:0/3424771161 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:48.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.457+0000 7fba5919b700 1 --2- 172.21.15.156:0/3424771161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba540d2320 0x7fba5414feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:48.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.457+0000 7fba5999c700 1 --2- 172.21.15.156:0/3424771161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba54161340 0x7fba54159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:48.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.457+0000 7fba5899a700 1 -- 172.21.15.156:0/3424771161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba54161340 msgr2=0x7fba54159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:48.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.457+0000 7fba5899a700 1 --2- 172.21.15.156:0/3424771161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba54161340 0x7fba54159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:48.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.457+0000 7fba5899a700 1 -- 172.21.15.156:0/3424771161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba540d2320 msgr2=0x7fba5414feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:48.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.457+0000 7fba5899a700 1 --2- 172.21.15.156:0/3424771161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba540d2320 0x7fba5414feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:48.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.457+0000 7fba5899a700 1 -- 172.21.15.156:0/3424771161 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba44009550 con 0x7fba5415c9b0 2023-11-28T13:08:48.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.457+0000 7fba5899a700 1 --2- 172.21.15.156:0/3424771161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba5415c9b0 0x7fba54159340 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fba44000c00 tx=0x7fba44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:48.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.457+0000 7fba49ffb700 1 -- 172.21.15.156:0/3424771161 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba44004300 con 0x7fba5415c9b0 2023-11-28T13:08:48.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.457+0000 7fba49ffb700 1 -- 172.21.15.156:0/3424771161 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4401f070 con 0x7fba5415c9b0 2023-11-28T13:08:48.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.457+0000 7fba49ffb700 1 -- 172.21.15.156:0/3424771161 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba440036a0 con 0x7fba5415c9b0 2023-11-28T13:08:48.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.458+0000 7fba5b3ff700 1 -- 172.21.15.156:0/3424771161 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba540d15e0 con 0x7fba5415c9b0 2023-11-28T13:08:48.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.458+0000 7fba5b3ff700 1 -- 172.21.15.156:0/3424771161 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba540d1300 con 0x7fba5415c9b0 2023-11-28T13:08:48.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.459+0000 7fba5b3ff700 1 -- 172.21.15.156:0/3424771161 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba5415cd80 con 0x7fba5415c9b0 2023-11-28T13:08:48.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.459+0000 7fba49ffb700 1 -- 172.21.15.156:0/3424771161 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fba4402a020 con 0x7fba5415c9b0 2023-11-28T13:08:48.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.460+0000 7fba49ffb700 1 --2- 172.21.15.156:0/3424771161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fba380682f0 0x7fba3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:48.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.460+0000 7fba49ffb700 1 -- 172.21.15.156:0/3424771161 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fba440870a0 con 0x7fba5415c9b0 2023-11-28T13:08:48.462 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.461+0000 7fba5919b700 1 --2- 172.21.15.156:0/3424771161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fba380682f0 0x7fba3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:48.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.461+0000 7fba5919b700 1 --2- 172.21.15.156:0/3424771161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fba380682f0 0x7fba3806a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fba541e8d50 tx=0x7fba50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:48.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.465+0000 7fba49ffb700 1 -- 172.21.15.156:0/3424771161 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fba44053880 con 0x7fba5415c9b0 2023-11-28T13:08:48.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.659+0000 7fba5b3ff700 1 -- 172.21.15.156:0/3424771161 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba540745c0 con 0x7fba5415c9b0 2023-11-28T13:08:48.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.661+0000 7fba49ffb700 1 -- 172.21.15.156:0/3424771161 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17897 (secure 0 0 0) 0x7fba44056ea0 con 0x7fba5415c9b0 2023-11-28T13:08:48.662 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:48.663 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":21,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:08:04.060053+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:08:00.903485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:08:02.037854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:08:48.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.663+0000 7fba5b3ff700 1 -- 172.21.15.156:0/3424771161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fba380682f0 msgr2=0x7fba3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:48.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.663+0000 7fba5b3ff700 1 --2- 172.21.15.156:0/3424771161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fba380682f0 0x7fba3806a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fba541e8d50 tx=0x7fba50009040 comp rx=0 tx=0).stop 2023-11-28T13:08:48.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.663+0000 7fba5b3ff700 1 -- 172.21.15.156:0/3424771161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba5415c9b0 msgr2=0x7fba54159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:48.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.663+0000 7fba5b3ff700 1 --2- 172.21.15.156:0/3424771161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba5415c9b0 0x7fba54159340 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fba44000c00 tx=0x7fba44004750 comp rx=0 tx=0).stop 2023-11-28T13:08:48.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.663+0000 7fba5b3ff700 1 -- 172.21.15.156:0/3424771161 shutdown_connections 2023-11-28T13:08:48.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.663+0000 7fba5b3ff700 1 --2- 172.21.15.156:0/3424771161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fba380682f0 0x7fba3806a7a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:48.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.663+0000 7fba5b3ff700 1 --2- 172.21.15.156:0/3424771161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba54161340 0x7fba54159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:48.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.663+0000 7fba5b3ff700 1 --2- 172.21.15.156:0/3424771161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba540d2320 0x7fba5414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:48.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.663+0000 7fba5b3ff700 1 --2- 172.21.15.156:0/3424771161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba5415c9b0 0x7fba54159340 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:48.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.663+0000 7fba5b3ff700 1 -- 172.21.15.156:0/3424771161 >> 172.21.15.156:0/3424771161 conn(0x7fba54072600 msgr2=0x7fba54160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:48.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.663+0000 7fba5b3ff700 1 -- 172.21.15.156:0/3424771161 shutdown_connections 2023-11-28T13:08:48.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.663+0000 7fba5b3ff700 1 -- 172.21.15.156:0/3424771161 wait complete. 2023-11-28T13:08:48.683 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:08:00.903485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:08:02.037854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:08:48.683 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:08:48.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.881+0000 7fa64b1ff700 1 Processor -- start 2023-11-28T13:08:48.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.881+0000 7fa64b1ff700 1 -- start start 2023-11-28T13:08:48.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.883+0000 7fa64b1ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa644161340 0x7fa6440d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:48.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.883+0000 7fa64b1ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6440d1950 con 0x7fa6440d2320 2023-11-28T13:08:48.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.883+0000 7fa64b1ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6440b5520 con 0x7fa64415c9b0 2023-11-28T13:08:48.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.883+0000 7fa64b1ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6440b5800 con 0x7fa644161340 2023-11-28T13:08:48.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.883+0000 7fa643fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa644161340 0x7fa6440d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:48.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.883+0000 7fa64979c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa6440d2320 0x7fa6440cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39464/0 (socket says 172.21.15.156:39464) 2023-11-28T13:08:48.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.883+0000 7fa64979c700 1 -- 172.21.15.156:0/3996220923 learned_addr learned my addr 172.21.15.156:0/3996220923 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:48.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.883+0000 7fa643fff700 1 -- 172.21.15.156:0/3996220923 >> v1:172.21.15.186:6789/0 conn(0x7fa64415c9b0 legacy=0x7fa6440b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:48.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.883+0000 7fa643fff700 1 -- 172.21.15.156:0/3996220923 >> v1:172.21.15.156:6789/0 conn(0x7fa6440d2320 legacy=0x7fa6440cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:48.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.883+0000 7fa643fff700 1 -- 172.21.15.156:0/3996220923 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6440d1300 con 0x7fa644161340 2023-11-28T13:08:48.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.883+0000 7fa6437fe700 1 -- 172.21.15.156:0/3996220923 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941581812 0 0) 0x7fa6440d1950 con 0x7fa6440d2320 2023-11-28T13:08:48.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.884+0000 7fa643fff700 1 --2- 172.21.15.156:0/3996220923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa644161340 0x7fa6440d1de0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa630009840 tx=0x7fa630009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d770b67ef41f120 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:48.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.884+0000 7fa6437fe700 1 -- 172.21.15.156:0/3996220923 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630011830 con 0x7fa644161340 2023-11-28T13:08:48.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.884+0000 7fa6437fe700 1 -- 172.21.15.156:0/3996220923 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa630011990 con 0x7fa644161340 2023-11-28T13:08:48.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.884+0000 7fa6437fe700 1 -- 172.21.15.156:0/3996220923 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630011d20 con 0x7fa644161340 2023-11-28T13:08:48.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.884+0000 7fa64b1ff700 1 -- 172.21.15.156:0/3996220923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa644161340 msgr2=0x7fa6440d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:48.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.884+0000 7fa64b1ff700 1 --2- 172.21.15.156:0/3996220923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa644161340 0x7fa6440d1de0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa630009840 tx=0x7fa630009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:48.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.884+0000 7fa64b1ff700 1 -- 172.21.15.156:0/3996220923 shutdown_connections 2023-11-28T13:08:48.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.884+0000 7fa64b1ff700 1 --2- 172.21.15.156:0/3996220923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa644161340 0x7fa6440d1de0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:48.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.884+0000 7fa64b1ff700 1 -- 172.21.15.156:0/3996220923 >> 172.21.15.156:0/3996220923 conn(0x7fa644072600 msgr2=0x7fa6440cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:48.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.885+0000 7fa64b1ff700 1 -- 172.21.15.156:0/3996220923 shutdown_connections 2023-11-28T13:08:48.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.885+0000 7fa64b1ff700 1 -- 172.21.15.156:0/3996220923 wait complete. 2023-11-28T13:08:48.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.886+0000 7fa64b1ff700 1 Processor -- start 2023-11-28T13:08:48.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.886+0000 7fa64b1ff700 1 -- start start 2023-11-28T13:08:48.890 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.887+0000 7fa64b1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6440d2320 0x7fa64414a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:48.890 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.887+0000 7fa64b1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa64415c9b0 0x7fa6440b6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:48.890 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.887+0000 7fa64b1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa644161340 0x7fa6440b7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:48.890 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.887+0000 7fa64b1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64414aac0 con 0x7fa6440d2320 2023-11-28T13:08:48.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.887+0000 7fa64b1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6441e6e70 con 0x7fa64415c9b0 2023-11-28T13:08:48.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.887+0000 7fa64b1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6441e70c0 con 0x7fa644161340 2023-11-28T13:08:48.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.887+0000 7fa648f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6440d2320 0x7fa64414a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:48.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.887+0000 7fa648f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6440d2320 0x7fa64414a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39748/0 (socket says 172.21.15.156:39748) 2023-11-28T13:08:48.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.888+0000 7fa648f9b700 1 -- 172.21.15.156:0/88273372 learned_addr learned my addr 172.21.15.156:0/88273372 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:48.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.888+0000 7fa643fff700 1 --2- 172.21.15.156:0/88273372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa64415c9b0 0x7fa6440b6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:48.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.888+0000 7fa64979c700 1 --2- 172.21.15.156:0/88273372 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa644161340 0x7fa6440b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:48.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.888+0000 7fa648f9b700 1 -- 172.21.15.156:0/88273372 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa644161340 msgr2=0x7fa6440b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:48.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.888+0000 7fa648f9b700 1 --2- 172.21.15.156:0/88273372 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa644161340 0x7fa6440b7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:48.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.888+0000 7fa648f9b700 1 -- 172.21.15.156:0/88273372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa64415c9b0 msgr2=0x7fa6440b6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:48.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.888+0000 7fa648f9b700 1 --2- 172.21.15.156:0/88273372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa64415c9b0 0x7fa6440b6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:48.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.888+0000 7fa648f9b700 1 -- 172.21.15.156:0/88273372 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6300094a0 con 0x7fa6440d2320 2023-11-28T13:08:48.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.888+0000 7fa64979c700 1 --2- 172.21.15.156:0/88273372 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa644161340 0x7fa6440b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:48.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.888+0000 7fa648f9b700 1 --2- 172.21.15.156:0/88273372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6440d2320 0x7fa64414a580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa63800c4e0 tx=0x7fa63800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:48.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.888+0000 7fa6417fa700 1 -- 172.21.15.156:0/88273372 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6380052d0 con 0x7fa6440d2320 2023-11-28T13:08:48.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.888+0000 7fa6417fa700 1 -- 172.21.15.156:0/88273372 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa638005430 con 0x7fa6440d2320 2023-11-28T13:08:48.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.888+0000 7fa64b1ff700 1 -- 172.21.15.156:0/88273372 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64415cd80 con 0x7fa6440d2320 2023-11-28T13:08:48.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.888+0000 7fa6417fa700 1 -- 172.21.15.156:0/88273372 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6380038c0 con 0x7fa6440d2320 2023-11-28T13:08:48.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.889+0000 7fa64b1ff700 1 -- 172.21.15.156:0/88273372 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6440d15e0 con 0x7fa6440d2320 2023-11-28T13:08:48.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.890+0000 7fa64b1ff700 1 -- 172.21.15.156:0/88273372 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64415cf10 con 0x7fa6440d2320 2023-11-28T13:08:48.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.890+0000 7fa6417fa700 1 -- 172.21.15.156:0/88273372 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa638019030 con 0x7fa6440d2320 2023-11-28T13:08:48.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.892+0000 7fa6417fa700 1 --2- 172.21.15.156:0/88273372 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa6280682f0 0x7fa62806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:48.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.892+0000 7fa6417fa700 1 -- 172.21.15.156:0/88273372 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa638084fc0 con 0x7fa6440d2320 2023-11-28T13:08:48.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.893+0000 7fa643fff700 1 --2- 172.21.15.156:0/88273372 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa6280682f0 0x7fa62806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:48.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.893+0000 7fa643fff700 1 --2- 172.21.15.156:0/88273372 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa6280682f0 0x7fa62806a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa6300090b0 tx=0x7fa63001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:48.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:48.896+0000 7fa6417fa700 1 -- 172.21.15.156:0/88273372 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa638051820 con 0x7fa6440d2320 2023-11-28T13:08:49.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.090+0000 7fa64b1ff700 1 -- 172.21.15.156:0/88273372 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa6440745c0 con 0x7fa6440d2320 2023-11-28T13:08:49.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.090+0000 7fa6417fa700 1 -- 172.21.15.156:0/88273372 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa638054e40 con 0x7fa6440d2320 2023-11-28T13:08:49.093 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:08:49.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.093+0000 7fa64b1ff700 1 -- 172.21.15.156:0/88273372 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa6280682f0 msgr2=0x7fa62806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:49.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.093+0000 7fa64b1ff700 1 --2- 172.21.15.156:0/88273372 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa6280682f0 0x7fa62806a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa6300090b0 tx=0x7fa63001c040 comp rx=0 tx=0).stop 2023-11-28T13:08:49.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.093+0000 7fa64b1ff700 1 -- 172.21.15.156:0/88273372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6440d2320 msgr2=0x7fa64414a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.093+0000 7fa64b1ff700 1 --2- 172.21.15.156:0/88273372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6440d2320 0x7fa64414a580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa63800c4e0 tx=0x7fa63800c7f0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.093+0000 7fa64b1ff700 1 -- 172.21.15.156:0/88273372 shutdown_connections 2023-11-28T13:08:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.093+0000 7fa64b1ff700 1 --2- 172.21.15.156:0/88273372 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa6280682f0 0x7fa62806a7a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.093+0000 7fa64b1ff700 1 --2- 172.21.15.156:0/88273372 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa644161340 0x7fa6440b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.093+0000 7fa64b1ff700 1 --2- 172.21.15.156:0/88273372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6440d2320 0x7fa64414a580 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.093+0000 7fa64b1ff700 1 --2- 172.21.15.156:0/88273372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa64415c9b0 0x7fa6440b6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.093+0000 7fa64b1ff700 1 -- 172.21.15.156:0/88273372 >> 172.21.15.156:0/88273372 conn(0x7fa644072600 msgr2=0x7fa64415f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.093+0000 7fa64b1ff700 1 -- 172.21.15.156:0/88273372 shutdown_connections 2023-11-28T13:08:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.093+0000 7fa64b1ff700 1 -- 172.21.15.156:0/88273372 wait complete. 2023-11-28T13:08:49.113 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:08:49.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.311+0000 7f7960dff700 1 Processor -- start 2023-11-28T13:08:49.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.311+0000 7f7960dff700 1 -- start start 2023-11-28T13:08:49.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.312+0000 7f7960dff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f795c159490 0x7f795c0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:49.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.312+0000 7f7960dff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f795c0ccc00 con 0x7f795c159860 2023-11-28T13:08:49.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.312+0000 7f7960dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f795c0ccec0 con 0x7f795c0cdc80 2023-11-28T13:08:49.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.312+0000 7f7960dff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f795c0cd1a0 con 0x7f795c159490 2023-11-28T13:08:49.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.313+0000 7f795a59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f795c159490 0x7f795c0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:49.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.313+0000 7f7953fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f795c159860 0x7f795c0c94c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39474/0 (socket says 172.21.15.156:39474) 2023-11-28T13:08:49.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.313+0000 7f7953fff700 1 -- 172.21.15.156:0/3725072001 learned_addr learned my addr 172.21.15.156:0/3725072001 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:49.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.313+0000 7f795a59c700 1 -- 172.21.15.156:0/3725072001 >> v1:172.21.15.186:6789/0 conn(0x7f795c0cdc80 legacy=0x7f795c06fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:49.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.313+0000 7f795a59c700 1 -- 172.21.15.156:0/3725072001 >> v1:172.21.15.156:6789/0 conn(0x7f795c159860 legacy=0x7f795c0c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:49.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.313+0000 7f7959d9b700 1 -- 172.21.15.156:0/3725072001 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179907579 0 0) 0x7f795c0ccc00 con 0x7f795c159860 2023-11-28T13:08:49.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.313+0000 7f795a59c700 1 -- 172.21.15.156:0/3725072001 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795c0cd480 con 0x7f795c159490 2023-11-28T13:08:49.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.313+0000 7f795a59c700 1 --2- 172.21.15.156:0/3725072001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f795c159490 0x7f795c0b0670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f794c009840 tx=0x7f794c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d965b85de4c5653 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:49.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.314+0000 7f7959d9b700 1 -- 172.21.15.156:0/3725072001 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c004230 con 0x7f795c159490 2023-11-28T13:08:49.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.314+0000 7f7959d9b700 1 -- 172.21.15.156:0/3725072001 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c004390 con 0x7f795c159490 2023-11-28T13:08:49.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.314+0000 7f7959d9b700 1 -- 172.21.15.156:0/3725072001 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c0036a0 con 0x7f795c159490 2023-11-28T13:08:49.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.314+0000 7f7960dff700 1 -- 172.21.15.156:0/3725072001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f795c159490 msgr2=0x7f795c0b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:49.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.314+0000 7f7960dff700 1 --2- 172.21.15.156:0/3725072001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f795c159490 0x7f795c0b0670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f794c009840 tx=0x7f794c009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:49.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.314+0000 7f7960dff700 1 -- 172.21.15.156:0/3725072001 shutdown_connections 2023-11-28T13:08:49.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.314+0000 7f7960dff700 1 --2- 172.21.15.156:0/3725072001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f795c159490 0x7f795c0b0670 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.314+0000 7f7960dff700 1 -- 172.21.15.156:0/3725072001 >> 172.21.15.156:0/3725072001 conn(0x7f795c0cb670 msgr2=0x7f795c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:49.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.314+0000 7f7960dff700 1 -- 172.21.15.156:0/3725072001 shutdown_connections 2023-11-28T13:08:49.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.314+0000 7f7960dff700 1 -- 172.21.15.156:0/3725072001 wait complete. 2023-11-28T13:08:49.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.316+0000 7f7960dff700 1 Processor -- start 2023-11-28T13:08:49.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.316+0000 7f7960dff700 1 -- start start 2023-11-28T13:08:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.317+0000 7f7960dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f795c0cdc80 0x7f795c148df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.317+0000 7f7960dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f795c159490 0x7f795c0c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.317+0000 7f7960dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f795c155a00 0x7f795c0c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.317+0000 7f7960dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c0c5cc0 con 0x7f795c0cdc80 2023-11-28T13:08:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.317+0000 7f7960dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c149330 con 0x7f795c155a00 2023-11-28T13:08:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.317+0000 7f7960dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795c155f70 con 0x7f795c159490 2023-11-28T13:08:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.317+0000 7f795a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f795c0cdc80 0x7f795c148df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.317+0000 7f795a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f795c0cdc80 0x7f795c148df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39766/0 (socket says 172.21.15.156:39766) 2023-11-28T13:08:49.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.317+0000 7f795a59c700 1 -- 172.21.15.156:0/3109638514 learned_addr learned my addr 172.21.15.156:0/3109638514 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:49.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.318+0000 7f795a59c700 1 -- 172.21.15.156:0/3109638514 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f795c159490 msgr2=0x7f795c0c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:49.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.318+0000 7f795a59c700 1 --2- 172.21.15.156:0/3109638514 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f795c159490 0x7f795c0c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.318+0000 7f795a59c700 1 -- 172.21.15.156:0/3109638514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f795c155a00 msgr2=0x7f795c0c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:49.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.318+0000 7f795ad9d700 1 --2- 172.21.15.156:0/3109638514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f795c155a00 0x7f795c0c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:49.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.318+0000 7f795a59c700 1 --2- 172.21.15.156:0/3109638514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f795c155a00 0x7f795c0c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.318+0000 7f795a59c700 1 -- 172.21.15.156:0/3109638514 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c0094a0 con 0x7f795c0cdc80 2023-11-28T13:08:49.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.318+0000 7f795ad9d700 1 --2- 172.21.15.156:0/3109638514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f795c155a00 0x7f795c0c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:49.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.318+0000 7f795a59c700 1 --2- 172.21.15.156:0/3109638514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f795c0cdc80 0x7f795c148df0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f795c067360 tx=0x7f794c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:49.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.318+0000 7f7952ffd700 1 -- 172.21.15.156:0/3109638514 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c011630 con 0x7f795c0cdc80 2023-11-28T13:08:49.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.318+0000 7f7952ffd700 1 -- 172.21.15.156:0/3109638514 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c01e070 con 0x7f795c0cdc80 2023-11-28T13:08:49.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.318+0000 7f7960dff700 1 -- 172.21.15.156:0/3109638514 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795c0b0bb0 con 0x7f795c0cdc80 2023-11-28T13:08:49.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.318+0000 7f7952ffd700 1 -- 172.21.15.156:0/3109638514 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c0119b0 con 0x7f795c0cdc80 2023-11-28T13:08:49.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.318+0000 7f7960dff700 1 -- 172.21.15.156:0/3109638514 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795c0cd760 con 0x7f795c0cdc80 2023-11-28T13:08:49.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.319+0000 7f7960dff700 1 -- 172.21.15.156:0/3109638514 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f795c0b0d40 con 0x7f795c0cdc80 2023-11-28T13:08:49.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.320+0000 7f7952ffd700 1 -- 172.21.15.156:0/3109638514 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f794c017020 con 0x7f795c0cdc80 2023-11-28T13:08:49.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.321+0000 7f7952ffd700 1 --2- 172.21.15.156:0/3109638514 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f793c0682f0 0x7f793c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:49.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.322+0000 7f7952ffd700 1 -- 172.21.15.156:0/3109638514 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f794c087150 con 0x7f795c0cdc80 2023-11-28T13:08:49.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.322+0000 7f7953fff700 1 --2- 172.21.15.156:0/3109638514 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f793c0682f0 0x7f793c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:49.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.323+0000 7f7953fff700 1 --2- 172.21.15.156:0/3109638514 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f793c0682f0 0x7f793c06a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f79440011e0 tx=0x7f79440079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:49.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.326+0000 7f7952ffd700 1 -- 172.21.15.156:0/3109638514 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f794c0538c0 con 0x7f795c0cdc80 2023-11-28T13:08:49.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.517+0000 7f7960dff700 1 -- 172.21.15.156:0/3109638514 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f795c0745c0 con 0x7f795c0cdc80 2023-11-28T13:08:49.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.517+0000 7f7952ffd700 1 -- 172.21.15.156:0/3109638514 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f794c020020 con 0x7f795c0cdc80 2023-11-28T13:08:49.519 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:08:49.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.520+0000 7f7960dff700 1 -- 172.21.15.156:0/3109638514 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f793c0682f0 msgr2=0x7f793c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:49.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.520+0000 7f7960dff700 1 --2- 172.21.15.156:0/3109638514 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f793c0682f0 0x7f793c06a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f79440011e0 tx=0x7f79440079b0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.520+0000 7f7960dff700 1 -- 172.21.15.156:0/3109638514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f795c0cdc80 msgr2=0x7f795c148df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:49.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.520+0000 7f7960dff700 1 --2- 172.21.15.156:0/3109638514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f795c0cdc80 0x7f795c148df0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f795c067360 tx=0x7f794c004750 comp rx=0 tx=0).stop 2023-11-28T13:08:49.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.520+0000 7f7960dff700 1 -- 172.21.15.156:0/3109638514 shutdown_connections 2023-11-28T13:08:49.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.520+0000 7f7960dff700 1 --2- 172.21.15.156:0/3109638514 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f793c0682f0 0x7f793c06a7a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.520+0000 7f7960dff700 1 --2- 172.21.15.156:0/3109638514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f795c155a00 0x7f795c0c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.520+0000 7f7960dff700 1 --2- 172.21.15.156:0/3109638514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f795c0cdc80 0x7f795c148df0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.520+0000 7f7960dff700 1 --2- 172.21.15.156:0/3109638514 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f795c159490 0x7f795c0c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.520+0000 7f7960dff700 1 -- 172.21.15.156:0/3109638514 >> 172.21.15.156:0/3109638514 conn(0x7f795c0cb670 msgr2=0x7f795c15bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:49.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.520+0000 7f7960dff700 1 -- 172.21.15.156:0/3109638514 shutdown_connections 2023-11-28T13:08:49.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.520+0000 7f7960dff700 1 -- 172.21.15.156:0/3109638514 wait complete. 2023-11-28T13:08:49.541 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:08:49.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.739+0000 7fc4005ff700 1 Processor -- start 2023-11-28T13:08:49.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.739+0000 7fc4005ff700 1 -- start start 2023-11-28T13:08:49.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.740+0000 7fc4005ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc3f815c9b0 0x7fc3f80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:49.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.740+0000 7fc4005ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3f8161340 0x7fc3f80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:49.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.740+0000 7fc4005ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc3f80d2320 0x7fc3f80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:49.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.740+0000 7fc4005ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f80b5800 con 0x7fc3f815c9b0 2023-11-28T13:08:49.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.740+0000 7fc4005ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f80d1950 con 0x7fc3f80d2320 2023-11-28T13:08:49.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.740+0000 7fc4005ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f80b5520 con 0x7fc3f8161340 2023-11-28T13:08:49.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.740+0000 7fc3fe39b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc3f815c9b0 0x7fc3f80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:49.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.740+0000 7fc3fe39b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc3f815c9b0 0x7fc3f80b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39774/0 (socket says 172.21.15.156:39774) 2023-11-28T13:08:49.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.740+0000 7fc3fe39b700 1 -- 172.21.15.156:0/1035945987 learned_addr learned my addr 172.21.15.156:0/1035945987 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:49.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.740+0000 7fc3feb9c700 1 --2- 172.21.15.156:0/1035945987 >> v2:172.21.15.186:3300/0 conn(0x7fc3f80d2320 0x7fc3f80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:49.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.740+0000 7fc3fdb9a700 1 --2- 172.21.15.156:0/1035945987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3f8161340 0x7fc3f80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:49.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.741+0000 7fc3fe39b700 1 -- 172.21.15.156:0/1035945987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3f8161340 msgr2=0x7fc3f80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:49.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.741+0000 7fc3fe39b700 1 --2- 172.21.15.156:0/1035945987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3f8161340 0x7fc3f80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.741+0000 7fc3fe39b700 1 -- 172.21.15.156:0/1035945987 >> v2:172.21.15.186:3300/0 conn(0x7fc3f80d2320 msgr2=0x7fc3f80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:49.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.741+0000 7fc3fe39b700 1 --2- 172.21.15.156:0/1035945987 >> v2:172.21.15.186:3300/0 conn(0x7fc3f80d2320 0x7fc3f80cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.741+0000 7fc3fe39b700 1 -- 172.21.15.156:0/1035945987 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f80d1300 con 0x7fc3f815c9b0 2023-11-28T13:08:49.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.741+0000 7fc3fdb9a700 1 --2- 172.21.15.156:0/1035945987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3f8161340 0x7fc3f80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:49.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.741+0000 7fc3feb9c700 1 --2- 172.21.15.156:0/1035945987 >> v2:172.21.15.186:3300/0 conn(0x7fc3f80d2320 0x7fc3f80cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:49.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.741+0000 7fc3fe39b700 1 --2- 172.21.15.156:0/1035945987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f815c9b0 0x7fc3f80b4bf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc3f808c4d0 tx=0x7fc3f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2436c50ffeaccf12 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:49.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.741+0000 7fc3fd399700 1 -- 172.21.15.156:0/1035945987 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f400b870 con 0x7fc3f815c9b0 2023-11-28T13:08:49.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.741+0000 7fc3fd399700 1 -- 172.21.15.156:0/1035945987 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f400b9d0 con 0x7fc3f815c9b0 2023-11-28T13:08:49.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.741+0000 7fc3fd399700 1 -- 172.21.15.156:0/1035945987 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f400bd60 con 0x7fc3f815c9b0 2023-11-28T13:08:49.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.742+0000 7fc4005ff700 1 -- 172.21.15.156:0/1035945987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f815c9b0 msgr2=0x7fc3f80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:49.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.742+0000 7fc4005ff700 1 --2- 172.21.15.156:0/1035945987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f815c9b0 0x7fc3f80b4bf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc3f808c4d0 tx=0x7fc3f4009bd0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.742+0000 7fc4005ff700 1 -- 172.21.15.156:0/1035945987 shutdown_connections 2023-11-28T13:08:49.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.742+0000 7fc4005ff700 1 --2- 172.21.15.156:0/1035945987 >> v2:172.21.15.186:3300/0 conn(0x7fc3f80d2320 0x7fc3f80cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.742+0000 7fc4005ff700 1 --2- 172.21.15.156:0/1035945987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f815c9b0 0x7fc3f80b4bf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.742+0000 7fc4005ff700 1 --2- 172.21.15.156:0/1035945987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3f8161340 0x7fc3f80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.742+0000 7fc4005ff700 1 -- 172.21.15.156:0/1035945987 >> 172.21.15.156:0/1035945987 conn(0x7fc3f8072600 msgr2=0x7fc3f80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:49.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.742+0000 7fc4005ff700 1 -- 172.21.15.156:0/1035945987 shutdown_connections 2023-11-28T13:08:49.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.742+0000 7fc4005ff700 1 -- 172.21.15.156:0/1035945987 wait complete. 2023-11-28T13:08:49.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.744+0000 7fc4005ff700 1 Processor -- start 2023-11-28T13:08:49.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.744+0000 7fc4005ff700 1 -- start start 2023-11-28T13:08:49.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.745+0000 7fc4005ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3f80d2320 0x7fc3f81f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:49.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.745+0000 7fc4005ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f815c9b0 0x7fc3f814dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:49.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.745+0000 7fc4005ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc3f8161340 0x7fc3f81e8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:49.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.745+0000 7fc4005ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f81548a0 con 0x7fc3f815c9b0 2023-11-28T13:08:49.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.745+0000 7fc4005ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f81e8d30 con 0x7fc3f8161340 2023-11-28T13:08:49.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.745+0000 7fc4005ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f81e9010 con 0x7fc3f80d2320 2023-11-28T13:08:49.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.745+0000 7fc3fdb9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f815c9b0 0x7fc3f814dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:49.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.745+0000 7fc3fdb9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f815c9b0 0x7fc3f814dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39780/0 (socket says 172.21.15.156:39780) 2023-11-28T13:08:49.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.745+0000 7fc3fdb9a700 1 -- 172.21.15.156:0/1152508291 learned_addr learned my addr 172.21.15.156:0/1152508291 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:49.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.745+0000 7fc3fe39b700 1 --2- 172.21.15.156:0/1152508291 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3f80d2320 0x7fc3f81f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:49.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.745+0000 7fc3feb9c700 1 --2- 172.21.15.156:0/1152508291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc3f8161340 0x7fc3f81e8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:49.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.745+0000 7fc3fdb9a700 1 -- 172.21.15.156:0/1152508291 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3f80d2320 msgr2=0x7fc3f81f6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:49.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.745+0000 7fc3fdb9a700 1 --2- 172.21.15.156:0/1152508291 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3f80d2320 0x7fc3f81f6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.745+0000 7fc3fdb9a700 1 -- 172.21.15.156:0/1152508291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc3f8161340 msgr2=0x7fc3f81e8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:49.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.745+0000 7fc3fdb9a700 1 --2- 172.21.15.156:0/1152508291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc3f8161340 0x7fc3f81e8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.746+0000 7fc3fdb9a700 1 -- 172.21.15.156:0/1152508291 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f4009550 con 0x7fc3f815c9b0 2023-11-28T13:08:49.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.746+0000 7fc3feb9c700 1 --2- 172.21.15.156:0/1152508291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc3f8161340 0x7fc3f81e8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:49.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.746+0000 7fc3fdb9a700 1 --2- 172.21.15.156:0/1152508291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f815c9b0 0x7fc3f814dcd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800e810 tx=0x7fc3e800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:49.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.746+0000 7fc3eeffd700 1 -- 172.21.15.156:0/1152508291 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8013610 con 0x7fc3f815c9b0 2023-11-28T13:08:49.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.746+0000 7fc3eeffd700 1 -- 172.21.15.156:0/1152508291 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e8013770 con 0x7fc3f815c9b0 2023-11-28T13:08:49.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.746+0000 7fc4005ff700 1 -- 172.21.15.156:0/1152508291 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f80b5130 con 0x7fc3f815c9b0 2023-11-28T13:08:49.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.746+0000 7fc3eeffd700 1 -- 172.21.15.156:0/1152508291 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8013b20 con 0x7fc3f815c9b0 2023-11-28T13:08:49.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.746+0000 7fc4005ff700 1 -- 172.21.15.156:0/1152508291 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f80d15e0 con 0x7fc3f815c9b0 2023-11-28T13:08:49.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.747+0000 7fc4005ff700 1 -- 172.21.15.156:0/1152508291 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f80b52c0 con 0x7fc3f815c9b0 2023-11-28T13:08:49.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.748+0000 7fc3eeffd700 1 -- 172.21.15.156:0/1152508291 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc3e8023070 con 0x7fc3f815c9b0 2023-11-28T13:08:49.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.749+0000 7fc3eeffd700 1 --2- 172.21.15.156:0/1152508291 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc3dc0682f0 0x7fc3dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:49.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.750+0000 7fc3eeffd700 1 -- 172.21.15.156:0/1152508291 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc3e8085890 con 0x7fc3f815c9b0 2023-11-28T13:08:49.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.750+0000 7fc3fe39b700 1 --2- 172.21.15.156:0/1152508291 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc3dc0682f0 0x7fc3dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:49.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.751+0000 7fc3fe39b700 1 --2- 172.21.15.156:0/1152508291 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc3dc0682f0 0x7fc3dc06a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4014040 tx=0x7fc3f4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:49.755 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.754+0000 7fc3eeffd700 1 -- 172.21.15.156:0/1152508291 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc3e8052090 con 0x7fc3f815c9b0 2023-11-28T13:08:49.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.943+0000 7fc4005ff700 1 -- 172.21.15.156:0/1152508291 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc3f80745c0 con 0x7fc3f815c9b0 2023-11-28T13:08:49.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.944+0000 7fc3eeffd700 1 -- 172.21.15.156:0/1152508291 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc3e80556b0 con 0x7fc3f815c9b0 2023-11-28T13:08:49.945 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T13:08:49.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.946+0000 7fc4005ff700 1 -- 172.21.15.156:0/1152508291 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc3dc0682f0 msgr2=0x7fc3dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:49.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.946+0000 7fc4005ff700 1 --2- 172.21.15.156:0/1152508291 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc3dc0682f0 0x7fc3dc06a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4014040 tx=0x7fc3f4008030 comp rx=0 tx=0).stop 2023-11-28T13:08:49.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.946+0000 7fc4005ff700 1 -- 172.21.15.156:0/1152508291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f815c9b0 msgr2=0x7fc3f814dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:49.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.946+0000 7fc4005ff700 1 --2- 172.21.15.156:0/1152508291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f815c9b0 0x7fc3f814dcd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800e810 tx=0x7fc3e800eb20 comp rx=0 tx=0).stop 2023-11-28T13:08:49.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.946+0000 7fc4005ff700 1 -- 172.21.15.156:0/1152508291 shutdown_connections 2023-11-28T13:08:49.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.946+0000 7fc4005ff700 1 --2- 172.21.15.156:0/1152508291 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc3dc0682f0 0x7fc3dc06a7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.946+0000 7fc4005ff700 1 --2- 172.21.15.156:0/1152508291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc3f8161340 0x7fc3f81e8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.946+0000 7fc4005ff700 1 --2- 172.21.15.156:0/1152508291 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc3f80d2320 0x7fc3f81f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.946+0000 7fc4005ff700 1 --2- 172.21.15.156:0/1152508291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f815c9b0 0x7fc3f814dcd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:49.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.946+0000 7fc4005ff700 1 -- 172.21.15.156:0/1152508291 >> 172.21.15.156:0/1152508291 conn(0x7fc3f8072600 msgr2=0x7fc3f815d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:49.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.946+0000 7fc4005ff700 1 -- 172.21.15.156:0/1152508291 shutdown_connections 2023-11-28T13:08:49.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:49.946+0000 7fc4005ff700 1 -- 172.21.15.156:0/1152508291 wait complete. 2023-11-28T13:08:49.966 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:08:50.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.165+0000 7ff180bff700 1 Processor -- start 2023-11-28T13:08:50.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.165+0000 7ff180bff700 1 -- start start 2023-11-28T13:08:50.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.166+0000 7ff180bff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff17c0cd7c0 0x7ff17c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:50.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.166+0000 7ff180bff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17c0d1950 con 0x7ff17c15c9b0 2023-11-28T13:08:50.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.166+0000 7ff180bff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17c0ce140 con 0x7ff17c161340 2023-11-28T13:08:50.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.166+0000 7ff180bff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff17c0b5800 con 0x7ff17c0cd7c0 2023-11-28T13:08:50.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.167+0000 7ff17a59c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff17c15c9b0 0x7ff17c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39490/0 (socket says 172.21.15.156:39490) 2023-11-28T13:08:50.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.167+0000 7ff17a59c700 1 -- 172.21.15.156:0/2085911403 learned_addr learned my addr 172.21.15.156:0/2085911403 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:50.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.167+0000 7ff17ad9d700 1 --2- 172.21.15.156:0/2085911403 >> v2:172.21.15.186:3300/0 conn(0x7ff17c0cd7c0 0x7ff17c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:50.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.167+0000 7ff17959a700 1 -- 172.21.15.156:0/2085911403 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 465490431 0 0) 0x7ff17c0d1950 con 0x7ff17c15c9b0 2023-11-28T13:08:50.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.167+0000 7ff17959a700 1 -- 172.21.15.156:0/2085911403 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1640035b0 con 0x7ff17c15c9b0 2023-11-28T13:08:50.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.167+0000 7ff17959a700 1 -- 172.21.15.156:0/2085911403 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093788974 0 0) 0x7ff17c0ce140 con 0x7ff17c161340 2023-11-28T13:08:50.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.167+0000 7ff17959a700 1 -- 172.21.15.156:0/2085911403 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff17c0d1950 con 0x7ff17c161340 2023-11-28T13:08:50.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.167+0000 7ff17959a700 1 -- 172.21.15.156:0/2085911403 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1615482356 0 0) 0x7ff1640035b0 con 0x7ff17c15c9b0 2023-11-28T13:08:50.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.167+0000 7ff17959a700 1 -- 172.21.15.156:0/2085911403 >> v2:172.21.15.186:3300/0 conn(0x7ff17c0cd7c0 msgr2=0x7ff17c0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:50.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.167+0000 7ff17959a700 1 --2- 172.21.15.156:0/2085911403 >> v2:172.21.15.186:3300/0 conn(0x7ff17c0cd7c0 0x7ff17c0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.167+0000 7ff17959a700 1 -- 172.21.15.156:0/2085911403 >> v1:172.21.15.186:6789/0 conn(0x7ff17c161340 legacy=0x7ff17c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:50.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.167+0000 7ff17959a700 1 -- 172.21.15.156:0/2085911403 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c0d24f0 con 0x7ff17c15c9b0 2023-11-28T13:08:50.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.167+0000 7ff17959a700 1 -- 172.21.15.156:0/2085911403 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7ff16c001670 con 0x7ff17c15c9b0 2023-11-28T13:08:50.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.167+0000 7ff17ad9d700 1 --2- 172.21.15.156:0/2085911403 >> v2:172.21.15.186:3300/0 conn(0x7ff17c0cd7c0 0x7ff17c0cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:50.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.167+0000 7ff17959a700 1 -- 172.21.15.156:0/2085911403 >> v1:172.21.15.156:6789/0 conn(0x7ff17c15c9b0 legacy=0x7ff17c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:50.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.168+0000 7ff17959a700 1 --2- 172.21.15.156:0/2085911403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff164003be0 0x7ff164003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:50.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.168+0000 7ff17959a700 1 --2- 172.21.15.156:0/2085911403 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff164004850 0x7ff164004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:50.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.168+0000 7ff17959a700 1 --2- 172.21.15.156:0/2085911403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff164005250 0x7ff16400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:50.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.168+0000 7ff17959a700 1 -- 172.21.15.156:0/2085911403 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c0b5800 con 0x7ff164005250 2023-11-28T13:08:50.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.168+0000 7ff17959a700 1 -- 172.21.15.156:0/2085911403 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16400c160 con 0x7ff164003be0 2023-11-28T13:08:50.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.168+0000 7ff17959a700 1 -- 172.21.15.156:0/2085911403 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16400c320 con 0x7ff164004850 2023-11-28T13:08:50.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.168+0000 7ff17a59c700 1 --2- 172.21.15.156:0/2085911403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff164003be0 0x7ff164003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:50.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.168+0000 7ff17ad9d700 1 --2- 172.21.15.156:0/2085911403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff164005250 0x7ff16400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:50.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.168+0000 7ff179d9b700 1 --2- 172.21.15.156:0/2085911403 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff164004850 0x7ff164004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:50.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.168+0000 7ff17ad9d700 1 -- 172.21.15.156:0/2085911403 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff164004850 msgr2=0x7ff164004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:50.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.168+0000 7ff17ad9d700 1 --2- 172.21.15.156:0/2085911403 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff164004850 0x7ff164004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.168+0000 7ff17ad9d700 1 -- 172.21.15.156:0/2085911403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff164003be0 msgr2=0x7ff164003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:50.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.168+0000 7ff17ad9d700 1 --2- 172.21.15.156:0/2085911403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff164003be0 0x7ff164003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.168+0000 7ff17ad9d700 1 -- 172.21.15.156:0/2085911403 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1640035b0 con 0x7ff164005250 2023-11-28T13:08:50.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.168+0000 7ff17ad9d700 1 -- 172.21.15.156:0/2085911403 reap_dead start 2023-11-28T13:08:50.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.169+0000 7ff17ad9d700 1 --2- 172.21.15.156:0/2085911403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff164005250 0x7ff16400ba10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff17400e320 tx=0x7ff164005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d829228b88a4f4f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:50.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.169+0000 7ff17959a700 1 -- 172.21.15.156:0/2085911403 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17400edc0 con 0x7ff164005250 2023-11-28T13:08:50.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.169+0000 7ff17959a700 1 -- 172.21.15.156:0/2085911403 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17400f040 con 0x7ff164005250 2023-11-28T13:08:50.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.169+0000 7ff17959a700 1 -- 172.21.15.156:0/2085911403 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1740146c0 con 0x7ff164005250 2023-11-28T13:08:50.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.169+0000 7ff180bff700 1 -- 172.21.15.156:0/2085911403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff164005250 msgr2=0x7ff16400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:50.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.169+0000 7ff180bff700 1 --2- 172.21.15.156:0/2085911403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff164005250 0x7ff16400ba10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff17400e320 tx=0x7ff164005880 comp rx=0 tx=0).stop 2023-11-28T13:08:50.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.169+0000 7ff180bff700 1 -- 172.21.15.156:0/2085911403 shutdown_connections 2023-11-28T13:08:50.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.169+0000 7ff180bff700 1 --2- 172.21.15.156:0/2085911403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff164005250 0x7ff16400ba10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.169+0000 7ff180bff700 1 -- 172.21.15.156:0/2085911403 >> 172.21.15.156:0/2085911403 conn(0x7ff17c072600 msgr2=0x7ff17c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:50.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.169+0000 7ff180bff700 1 -- 172.21.15.156:0/2085911403 shutdown_connections 2023-11-28T13:08:50.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.170+0000 7ff180bff700 1 -- 172.21.15.156:0/2085911403 wait complete. 2023-11-28T13:08:50.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.172+0000 7ff180bff700 1 Processor -- start 2023-11-28T13:08:50.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.172+0000 7ff180bff700 1 -- start start 2023-11-28T13:08:50.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.173+0000 7ff180bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff164005250 0x7ff17c0c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:50.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.173+0000 7ff180bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff17c0c9c40 0x7ff17c1e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:50.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.173+0000 7ff180bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff17c15e610 0x7ff17c0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:50.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.173+0000 7ff180bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c1e8120 con 0x7ff164005250 2023-11-28T13:08:50.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.173+0000 7ff180bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c0c8710 con 0x7ff17c15e610 2023-11-28T13:08:50.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.173+0000 7ff180bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c1e7930 con 0x7ff17c0c9c40 2023-11-28T13:08:50.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.174+0000 7ff17a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff164005250 0x7ff17c0c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:50.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.174+0000 7ff17a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff164005250 0x7ff17c0c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39802/0 (socket says 172.21.15.156:39802) 2023-11-28T13:08:50.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.174+0000 7ff17a59c700 1 -- 172.21.15.156:0/430520805 learned_addr learned my addr 172.21.15.156:0/430520805 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:50.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.174+0000 7ff179d9b700 1 --2- 172.21.15.156:0/430520805 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff17c0c9c40 0x7ff17c1e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:50.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.174+0000 7ff17ad9d700 1 --2- 172.21.15.156:0/430520805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff17c15e610 0x7ff17c0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:50.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.174+0000 7ff17a59c700 1 -- 172.21.15.156:0/430520805 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff17c0c9c40 msgr2=0x7ff17c1e6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:50.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.174+0000 7ff17a59c700 1 --2- 172.21.15.156:0/430520805 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff17c0c9c40 0x7ff17c1e6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.174+0000 7ff17a59c700 1 -- 172.21.15.156:0/430520805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff17c15e610 msgr2=0x7ff17c0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:50.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.174+0000 7ff17a59c700 1 --2- 172.21.15.156:0/430520805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff17c15e610 0x7ff17c0ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.174+0000 7ff17a59c700 1 -- 172.21.15.156:0/430520805 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c0d1300 con 0x7ff164005250 2023-11-28T13:08:50.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.174+0000 7ff17a59c700 1 --2- 172.21.15.156:0/430520805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff164005250 0x7ff17c0c81d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff16c004f80 tx=0x7ff16c007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:50.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.174+0000 7ff16affd700 1 -- 172.21.15.156:0/430520805 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c012560 con 0x7ff164005250 2023-11-28T13:08:50.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.175+0000 7ff16affd700 1 -- 172.21.15.156:0/430520805 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c0126e0 con 0x7ff164005250 2023-11-28T13:08:50.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.175+0000 7ff180bff700 1 -- 172.21.15.156:0/430520805 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17c0b5520 con 0x7ff164005250 2023-11-28T13:08:50.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.175+0000 7ff16affd700 1 -- 172.21.15.156:0/430520805 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c012a90 con 0x7ff164005250 2023-11-28T13:08:50.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.175+0000 7ff180bff700 1 -- 172.21.15.156:0/430520805 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17c0d15e0 con 0x7ff164005250 2023-11-28T13:08:50.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.176+0000 7ff180bff700 1 -- 172.21.15.156:0/430520805 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17c0745c0 con 0x7ff164005250 2023-11-28T13:08:50.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.177+0000 7ff16affd700 1 -- 172.21.15.156:0/430520805 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff16c012bf0 con 0x7ff164005250 2023-11-28T13:08:50.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.177+0000 7ff16affd700 1 --2- 172.21.15.156:0/430520805 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff15c0682f0 0x7ff15c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:50.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.177+0000 7ff16affd700 1 -- 172.21.15.156:0/430520805 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff16c085c50 con 0x7ff164005250 2023-11-28T13:08:50.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.178+0000 7ff179d9b700 1 --2- 172.21.15.156:0/430520805 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff15c0682f0 0x7ff15c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:50.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.178+0000 7ff179d9b700 1 --2- 172.21.15.156:0/430520805 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff15c0682f0 0x7ff15c06a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff17c15ef90 tx=0x7ff170009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:50.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.181+0000 7ff16affd700 1 -- 172.21.15.156:0/430520805 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff16c0524b0 con 0x7ff164005250 2023-11-28T13:08:50.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.373+0000 7ff180bff700 1 -- 172.21.15.156:0/430520805 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff17c0d08a0 con 0x7ff164005250 2023-11-28T13:08:50.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.373+0000 7ff16affd700 1 -- 172.21.15.156:0/430520805 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff16c055ad0 con 0x7ff164005250 2023-11-28T13:08:50.375 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T13:08:50.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.375+0000 7ff180bff700 1 -- 172.21.15.156:0/430520805 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff15c0682f0 msgr2=0x7ff15c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:50.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.375+0000 7ff180bff700 1 --2- 172.21.15.156:0/430520805 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff15c0682f0 0x7ff15c06a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff17c15ef90 tx=0x7ff170009040 comp rx=0 tx=0).stop 2023-11-28T13:08:50.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.375+0000 7ff180bff700 1 -- 172.21.15.156:0/430520805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff164005250 msgr2=0x7ff17c0c81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:50.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.375+0000 7ff180bff700 1 --2- 172.21.15.156:0/430520805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff164005250 0x7ff17c0c81d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff16c004f80 tx=0x7ff16c007340 comp rx=0 tx=0).stop 2023-11-28T13:08:50.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.375+0000 7ff180bff700 1 -- 172.21.15.156:0/430520805 shutdown_connections 2023-11-28T13:08:50.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.375+0000 7ff180bff700 1 --2- 172.21.15.156:0/430520805 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff15c0682f0 0x7ff15c06a7a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.375+0000 7ff180bff700 1 --2- 172.21.15.156:0/430520805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff17c15e610 0x7ff17c0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.375+0000 7ff180bff700 1 --2- 172.21.15.156:0/430520805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff164005250 0x7ff17c0c81d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.375+0000 7ff180bff700 1 --2- 172.21.15.156:0/430520805 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff17c0c9c40 0x7ff17c1e6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.375+0000 7ff180bff700 1 -- 172.21.15.156:0/430520805 >> 172.21.15.156:0/430520805 conn(0x7ff17c072600 msgr2=0x7ff17c06a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:50.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.375+0000 7ff180bff700 1 -- 172.21.15.156:0/430520805 shutdown_connections 2023-11-28T13:08:50.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.376+0000 7ff180bff700 1 -- 172.21.15.156:0/430520805 wait complete. 2023-11-28T13:08:50.395 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:08:50.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.595+0000 7fb71d5ff700 1 Processor -- start 2023-11-28T13:08:50.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.595+0000 7fb71d5ff700 1 -- start start 2023-11-28T13:08:50.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.596+0000 7fb71d5ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb71815c9b0 0x7fb7180b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:50.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.596+0000 7fb71d5ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb718161340 0x7fb7180d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:50.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.597+0000 7fb71d5ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7180d1950 con 0x7fb7180d2320 2023-11-28T13:08:50.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.597+0000 7fb71d5ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7180b5800 con 0x7fb71815c9b0 2023-11-28T13:08:50.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.597+0000 7fb71d5ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7180b5520 con 0x7fb718161340 2023-11-28T13:08:50.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.597+0000 7fb716ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb71815c9b0 0x7fb7180b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:50.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.597+0000 7fb716ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb71815c9b0 0x7fb7180b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39818/0 (socket says 172.21.15.156:39818) 2023-11-28T13:08:50.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.597+0000 7fb716ffd700 1 -- 172.21.15.156:0/794784052 learned_addr learned my addr 172.21.15.156:0/794784052 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:50.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.597+0000 7fb70ffff700 1 --2- 172.21.15.156:0/794784052 >> v2:172.21.15.186:3300/0 conn(0x7fb718161340 0x7fb7180d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:50.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.597+0000 7fb716ffd700 1 -- 172.21.15.156:0/794784052 >> v2:172.21.15.186:3300/0 conn(0x7fb718161340 msgr2=0x7fb7180d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:50.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.597+0000 7fb716ffd700 1 --2- 172.21.15.156:0/794784052 >> v2:172.21.15.186:3300/0 conn(0x7fb718161340 0x7fb7180d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.597+0000 7fb716ffd700 1 -- 172.21.15.156:0/794784052 >> v1:172.21.15.186:6789/0 conn(0x7fb7180d2320 legacy=0x7fb7180cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:50.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.597+0000 7fb716ffd700 1 -- 172.21.15.156:0/794784052 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7180d1300 con 0x7fb71815c9b0 2023-11-28T13:08:50.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.597+0000 7fb70ffff700 1 --2- 172.21.15.156:0/794784052 >> v2:172.21.15.186:3300/0 conn(0x7fb718161340 0x7fb7180d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:50.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.597+0000 7fb716ffd700 1 --2- 172.21.15.156:0/794784052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71815c9b0 0x7fb7180b4bf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb7180685c0 tx=0x7fb708009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a36cbe3fe552ae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:50.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.598+0000 7fb7167fc700 1 -- 172.21.15.156:0/794784052 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708004230 con 0x7fb71815c9b0 2023-11-28T13:08:50.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.598+0000 7fb7167fc700 1 -- 172.21.15.156:0/794784052 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb708004390 con 0x7fb71815c9b0 2023-11-28T13:08:50.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.598+0000 7fb7167fc700 1 -- 172.21.15.156:0/794784052 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7080036a0 con 0x7fb71815c9b0 2023-11-28T13:08:50.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.598+0000 7fb71d5ff700 1 -- 172.21.15.156:0/794784052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71815c9b0 msgr2=0x7fb7180b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:50.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.598+0000 7fb71d5ff700 1 --2- 172.21.15.156:0/794784052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71815c9b0 0x7fb7180b4bf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb7180685c0 tx=0x7fb708009bd0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.598+0000 7fb71d5ff700 1 -- 172.21.15.156:0/794784052 shutdown_connections 2023-11-28T13:08:50.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.598+0000 7fb71d5ff700 1 --2- 172.21.15.156:0/794784052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71815c9b0 0x7fb7180b4bf0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.598+0000 7fb71d5ff700 1 --2- 172.21.15.156:0/794784052 >> v2:172.21.15.186:3300/0 conn(0x7fb718161340 0x7fb7180d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.598+0000 7fb71d5ff700 1 -- 172.21.15.156:0/794784052 >> 172.21.15.156:0/794784052 conn(0x7fb718072600 msgr2=0x7fb7180cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:50.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.599+0000 7fb71d5ff700 1 -- 172.21.15.156:0/794784052 shutdown_connections 2023-11-28T13:08:50.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.599+0000 7fb71d5ff700 1 -- 172.21.15.156:0/794784052 wait complete. 2023-11-28T13:08:50.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.600+0000 7fb71d5ff700 1 Processor -- start 2023-11-28T13:08:50.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.600+0000 7fb71d5ff700 1 -- start start 2023-11-28T13:08:50.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.601+0000 7fb71d5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7180d2320 0x7fb7180c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:50.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.601+0000 7fb71d5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb71815c9b0 0x7fb718154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:50.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.601+0000 7fb71d5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb718161340 0x7fb718150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:50.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.601+0000 7fb71d5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb718150860 con 0x7fb7180d2320 2023-11-28T13:08:50.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.601+0000 7fb71d5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb718150d60 con 0x7fb718161340 2023-11-28T13:08:50.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.601+0000 7fb71d5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7181547c0 con 0x7fb71815c9b0 2023-11-28T13:08:50.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.601+0000 7fb716ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7180d2320 0x7fb7180c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:50.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.601+0000 7fb716ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7180d2320 0x7fb7180c9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39834/0 (socket says 172.21.15.156:39834) 2023-11-28T13:08:50.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.601+0000 7fb716ffd700 1 -- 172.21.15.156:0/2428548079 learned_addr learned my addr 172.21.15.156:0/2428548079 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:50.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.602+0000 7fb70ffff700 1 --2- 172.21.15.156:0/2428548079 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb71815c9b0 0x7fb718154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:50.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.602+0000 7fb7177fe700 1 --2- 172.21.15.156:0/2428548079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb718161340 0x7fb718150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:50.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.602+0000 7fb716ffd700 1 -- 172.21.15.156:0/2428548079 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb71815c9b0 msgr2=0x7fb718154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:50.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.602+0000 7fb716ffd700 1 --2- 172.21.15.156:0/2428548079 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb71815c9b0 0x7fb718154130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.602+0000 7fb716ffd700 1 -- 172.21.15.156:0/2428548079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb718161340 msgr2=0x7fb718150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:50.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.602+0000 7fb716ffd700 1 --2- 172.21.15.156:0/2428548079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb718161340 0x7fb718150320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.602+0000 7fb716ffd700 1 -- 172.21.15.156:0/2428548079 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb708009550 con 0x7fb7180d2320 2023-11-28T13:08:50.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.602+0000 7fb7177fe700 1 --2- 172.21.15.156:0/2428548079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb718161340 0x7fb718150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:50.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.602+0000 7fb716ffd700 1 --2- 172.21.15.156:0/2428548079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7180d2320 0x7fb7180c9200 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb708009130 tx=0x7fb708004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:50.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.602+0000 7fb70f7fe700 1 -- 172.21.15.156:0/2428548079 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708003d10 con 0x7fb7180d2320 2023-11-28T13:08:50.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.602+0000 7fb70f7fe700 1 -- 172.21.15.156:0/2428548079 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70801f070 con 0x7fb7180d2320 2023-11-28T13:08:50.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.602+0000 7fb71d5ff700 1 -- 172.21.15.156:0/2428548079 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7180b5130 con 0x7fb7180d2320 2023-11-28T13:08:50.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.602+0000 7fb70f7fe700 1 -- 172.21.15.156:0/2428548079 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708011870 con 0x7fb7180d2320 2023-11-28T13:08:50.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.603+0000 7fb71d5ff700 1 -- 172.21.15.156:0/2428548079 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7180d15e0 con 0x7fb7180d2320 2023-11-28T13:08:50.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.603+0000 7fb71d5ff700 1 -- 172.21.15.156:0/2428548079 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7180b52c0 con 0x7fb7180d2320 2023-11-28T13:08:50.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.604+0000 7fb70f7fe700 1 -- 172.21.15.156:0/2428548079 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb708017020 con 0x7fb7180d2320 2023-11-28T13:08:50.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.605+0000 7fb70f7fe700 1 --2- 172.21.15.156:0/2428548079 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb6f80682f0 0x7fb6f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:50.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.605+0000 7fb70f7fe700 1 -- 172.21.15.156:0/2428548079 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb708087540 con 0x7fb7180d2320 2023-11-28T13:08:50.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.606+0000 7fb70ffff700 1 --2- 172.21.15.156:0/2428548079 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb6f80682f0 0x7fb6f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:50.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.606+0000 7fb70ffff700 1 --2- 172.21.15.156:0/2428548079 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb6f80682f0 0x7fb6f806a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb718154aa0 tx=0x7fb700008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:50.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.609+0000 7fb70f7fe700 1 -- 172.21.15.156:0/2428548079 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb708053c70 con 0x7fb7180d2320 2023-11-28T13:08:50.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.815+0000 7fb71d5ff700 1 -- 172.21.15.156:0/2428548079 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb7180745c0 con 0x7fb7180d2320 2023-11-28T13:08:50.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.815+0000 7fb70f7fe700 1 -- 172.21.15.156:0/2428548079 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb708057290 con 0x7fb7180d2320 2023-11-28T13:08:50.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.817+0000 7fb71d5ff700 1 -- 172.21.15.156:0/2428548079 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb6f80682f0 msgr2=0x7fb6f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:50.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.817+0000 7fb71d5ff700 1 --2- 172.21.15.156:0/2428548079 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb6f80682f0 0x7fb6f806a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb718154aa0 tx=0x7fb700008040 comp rx=0 tx=0).stop 2023-11-28T13:08:50.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.817+0000 7fb71d5ff700 1 -- 172.21.15.156:0/2428548079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7180d2320 msgr2=0x7fb7180c9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:50.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.817+0000 7fb71d5ff700 1 --2- 172.21.15.156:0/2428548079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7180d2320 0x7fb7180c9200 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb708009130 tx=0x7fb708004750 comp rx=0 tx=0).stop 2023-11-28T13:08:50.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.817+0000 7fb71d5ff700 1 -- 172.21.15.156:0/2428548079 shutdown_connections 2023-11-28T13:08:50.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.817+0000 7fb71d5ff700 1 --2- 172.21.15.156:0/2428548079 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb6f80682f0 0x7fb6f806a7a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.817+0000 7fb71d5ff700 1 --2- 172.21.15.156:0/2428548079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb718161340 0x7fb718150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.817+0000 7fb71d5ff700 1 --2- 172.21.15.156:0/2428548079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7180d2320 0x7fb7180c9200 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.817+0000 7fb71d5ff700 1 --2- 172.21.15.156:0/2428548079 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb71815c9b0 0x7fb718154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:50.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.817+0000 7fb71d5ff700 1 -- 172.21.15.156:0/2428548079 >> 172.21.15.156:0/2428548079 conn(0x7fb718072600 msgr2=0x7fb7180b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:50.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.818+0000 7fb71d5ff700 1 -- 172.21.15.156:0/2428548079 shutdown_connections 2023-11-28T13:08:50.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:50.818+0000 7fb71d5ff700 1 -- 172.21.15.156:0/2428548079 wait complete. 2023-11-28T13:08:50.821 INFO:teuthology.orchestra.run.smithi156.stderr:listed 0 entries 2023-11-28T13:08:50.837 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-28T13:08:50.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:50.837 DEBUG:teuthology.orchestra.run.smithi186:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-28T13:08:50.915 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T13:08:50.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:08:50.916 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T13:08:50.960 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T13:08:50.961 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:08:51.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.160+0000 7f8db27ff700 1 Processor -- start 2023-11-28T13:08:51.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.160+0000 7f8db27ff700 1 -- start start 2023-11-28T13:08:51.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.161+0000 7f8db27ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8dac15c9b0 0x7f8dac0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:51.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.161+0000 7f8db27ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8dac161340 0x7f8dac0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:51.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.162+0000 7f8db27ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dac0d1950 con 0x7f8dac0d2320 2023-11-28T13:08:51.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.162+0000 7f8db27ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dac0b5800 con 0x7f8dac15c9b0 2023-11-28T13:08:51.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.162+0000 7f8db27ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac0b5520 con 0x7f8dac161340 2023-11-28T13:08:51.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.162+0000 7f8dab7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8dac161340 0x7f8dac0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:51.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.162+0000 7f8dabfff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8dac15c9b0 0x7f8dac0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:51.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.162+0000 7f8dab7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8dac161340 0x7f8dac0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:56440/0 (socket says 172.21.15.156:56440) 2023-11-28T13:08:51.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.162+0000 7f8dab7fe700 1 -- 172.21.15.156:0/1501745352 learned_addr learned my addr 172.21.15.156:0/1501745352 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:51.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.162+0000 7f8db0d9c700 1 --1- 172.21.15.156:0/1501745352 >> v1:172.21.15.186:6789/0 conn(0x7f8dac0d2320 0x7f8dac0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:33304/0 (socket says 172.21.15.156:33304) 2023-11-28T13:08:51.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.163+0000 7f8dab7fe700 1 -- 172.21.15.156:0/1501745352 >> v2:172.21.15.186:3300/0 conn(0x7f8dac15c9b0 msgr2=0x7f8dac0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:51.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.163+0000 7f8dab7fe700 1 --2- 172.21.15.156:0/1501745352 >> v2:172.21.15.186:3300/0 conn(0x7f8dac15c9b0 0x7f8dac0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.163+0000 7f8dab7fe700 1 -- 172.21.15.156:0/1501745352 >> v1:172.21.15.186:6789/0 conn(0x7f8dac0d2320 legacy=0x7f8dac0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:51.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.163+0000 7f8dab7fe700 1 -- 172.21.15.156:0/1501745352 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac0d1300 con 0x7f8dac161340 2023-11-28T13:08:51.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.163+0000 7f8dabfff700 1 --2- 172.21.15.156:0/1501745352 >> v2:172.21.15.186:3300/0 conn(0x7f8dac15c9b0 0x7f8dac0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:51.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.163+0000 7f8dab7fe700 1 --2- 172.21.15.156:0/1501745352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8dac161340 0x7f8dac0d1de0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8da00098f0 tx=0x7f8da0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bf81b3acba7d0de server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:51.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.163+0000 7f8daaffd700 1 -- 172.21.15.156:0/1501745352 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0011830 con 0x7f8dac161340 2023-11-28T13:08:51.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.163+0000 7f8daaffd700 1 -- 172.21.15.156:0/1501745352 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da0011990 con 0x7f8dac161340 2023-11-28T13:08:51.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.163+0000 7f8daaffd700 1 -- 172.21.15.156:0/1501745352 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0011d20 con 0x7f8dac161340 2023-11-28T13:08:51.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.164+0000 7f8db27ff700 1 -- 172.21.15.156:0/1501745352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8dac161340 msgr2=0x7f8dac0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:51.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.164+0000 7f8db27ff700 1 --2- 172.21.15.156:0/1501745352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8dac161340 0x7f8dac0d1de0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8da00098f0 tx=0x7f8da0009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:51.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.164+0000 7f8db27ff700 1 -- 172.21.15.156:0/1501745352 shutdown_connections 2023-11-28T13:08:51.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.164+0000 7f8db27ff700 1 --2- 172.21.15.156:0/1501745352 >> v2:172.21.15.186:3300/0 conn(0x7f8dac15c9b0 0x7f8dac0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.164+0000 7f8db27ff700 1 --2- 172.21.15.156:0/1501745352 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8dac161340 0x7f8dac0d1de0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.164+0000 7f8db27ff700 1 -- 172.21.15.156:0/1501745352 >> 172.21.15.156:0/1501745352 conn(0x7f8dac072600 msgr2=0x7f8dac0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:51.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.164+0000 7f8db27ff700 1 -- 172.21.15.156:0/1501745352 shutdown_connections 2023-11-28T13:08:51.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.164+0000 7f8db27ff700 1 -- 172.21.15.156:0/1501745352 wait complete. 2023-11-28T13:08:51.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.165+0000 7f8db27ff700 1 Processor -- start 2023-11-28T13:08:51.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.166+0000 7f8db27ff700 1 -- start start 2023-11-28T13:08:51.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.166+0000 7f8db27ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8dac0d2320 0x7f8dac150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:51.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.166+0000 7f8db27ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dac15c9b0 0x7f8dac1576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:51.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.166+0000 7f8db27ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8dac161340 0x7f8dac14fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:51.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.166+0000 7f8db27ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac157ed0 con 0x7f8dac15c9b0 2023-11-28T13:08:51.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.166+0000 7f8db27ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac151710 con 0x7f8dac161340 2023-11-28T13:08:51.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.166+0000 7f8db27ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac157c30 con 0x7f8dac0d2320 2023-11-28T13:08:51.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.167+0000 7f8dab7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dac15c9b0 0x7f8dac1576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:51.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.167+0000 7f8dabfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8dac0d2320 0x7f8dac150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:51.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.167+0000 7f8dab7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dac15c9b0 0x7f8dac1576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39848/0 (socket says 172.21.15.156:39848) 2023-11-28T13:08:51.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.167+0000 7f8dab7fe700 1 -- 172.21.15.156:0/2140612493 learned_addr learned my addr 172.21.15.156:0/2140612493 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:51.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.167+0000 7f8db0d9c700 1 --2- 172.21.15.156:0/2140612493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8dac161340 0x7f8dac14fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:51.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.167+0000 7f8dab7fe700 1 -- 172.21.15.156:0/2140612493 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8dac0d2320 msgr2=0x7f8dac150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:51.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.167+0000 7f8dab7fe700 1 --2- 172.21.15.156:0/2140612493 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8dac0d2320 0x7f8dac150880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.167+0000 7f8dab7fe700 1 -- 172.21.15.156:0/2140612493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8dac161340 msgr2=0x7f8dac14fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:51.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.167+0000 7f8dab7fe700 1 --2- 172.21.15.156:0/2140612493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8dac161340 0x7f8dac14fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.167+0000 7f8dab7fe700 1 -- 172.21.15.156:0/2140612493 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c0094a0 con 0x7f8dac15c9b0 2023-11-28T13:08:51.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.167+0000 7f8dab7fe700 1 --2- 172.21.15.156:0/2140612493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dac15c9b0 0x7f8dac1576f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8da0000c00 tx=0x7f8da0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:51.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.168+0000 7f8da8ff9700 1 -- 172.21.15.156:0/2140612493 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0004400 con 0x7f8dac15c9b0 2023-11-28T13:08:51.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.168+0000 7f8da8ff9700 1 -- 172.21.15.156:0/2140612493 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da001f070 con 0x7f8dac15c9b0 2023-11-28T13:08:51.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.168+0000 7f8db27ff700 1 -- 172.21.15.156:0/2140612493 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da0009550 con 0x7f8dac15c9b0 2023-11-28T13:08:51.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.168+0000 7f8da8ff9700 1 -- 172.21.15.156:0/2140612493 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da00038e0 con 0x7f8dac15c9b0 2023-11-28T13:08:51.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.168+0000 7f8db27ff700 1 -- 172.21.15.156:0/2140612493 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dac0b5130 con 0x7f8dac15c9b0 2023-11-28T13:08:51.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.169+0000 7f8db27ff700 1 -- 172.21.15.156:0/2140612493 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dac0d15e0 con 0x7f8dac15c9b0 2023-11-28T13:08:51.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.170+0000 7f8da8ff9700 1 -- 172.21.15.156:0/2140612493 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8da002a020 con 0x7f8dac15c9b0 2023-11-28T13:08:51.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.170+0000 7f8da8ff9700 1 --2- 172.21.15.156:0/2140612493 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8d980682f0 0x7f8d9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:51.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.171+0000 7f8da8ff9700 1 -- 172.21.15.156:0/2140612493 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8da00872f0 con 0x7f8dac15c9b0 2023-11-28T13:08:51.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.171+0000 7f8dabfff700 1 --2- 172.21.15.156:0/2140612493 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8d980682f0 0x7f8d9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:51.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.172+0000 7f8dabfff700 1 --2- 172.21.15.156:0/2140612493 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8d980682f0 0x7f8d9806a7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009d70 tx=0x7f8d9c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:51.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.175+0000 7f8da8ff9700 1 -- 172.21.15.156:0/2140612493 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8da00539a0 con 0x7f8dac15c9b0 2023-11-28T13:08:51.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.409+0000 7f8db27ff700 1 -- 172.21.15.156:0/2140612493 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8dac0b52c0 con 0x7f8dac15c9b0 2023-11-28T13:08:51.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.409+0000 7f8da8ff9700 1 -- 172.21.15.156:0/2140612493 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4651 (secure 0 0 0) 0x7f8da0056fc0 con 0x7f8dac15c9b0 2023-11-28T13:08:51.411 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:51.411 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1283906891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1283906891},{"type":"v1","addr":"172.21.15.156:6835","nonce":1283906891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1738881422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1738881422},{"type":"v1","addr":"172.21.15.156:6833","nonce":1738881422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2235657629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2235657629},{"type":"v1","addr":"172.21.15.186:6836","nonce":2235657629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:08:03.058116+0000","modified":"2023-11-28T13:08:06.077600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/402651964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":402651964},{"type":"v1","addr":"172.21.15.186:6837","nonce":402651964}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:08:51.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.411+0000 7f8db27ff700 1 -- 172.21.15.156:0/2140612493 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8d980682f0 msgr2=0x7f8d9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:51.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.411+0000 7f8db27ff700 1 --2- 172.21.15.156:0/2140612493 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8d980682f0 0x7f8d9806a7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009d70 tx=0x7f8d9c009210 comp rx=0 tx=0).stop 2023-11-28T13:08:51.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.411+0000 7f8db27ff700 1 -- 172.21.15.156:0/2140612493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dac15c9b0 msgr2=0x7f8dac1576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:51.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.411+0000 7f8db27ff700 1 --2- 172.21.15.156:0/2140612493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dac15c9b0 0x7f8dac1576f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8da0000c00 tx=0x7f8da0004750 comp rx=0 tx=0).stop 2023-11-28T13:08:51.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.412+0000 7f8db27ff700 1 -- 172.21.15.156:0/2140612493 shutdown_connections 2023-11-28T13:08:51.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.412+0000 7f8db27ff700 1 --2- 172.21.15.156:0/2140612493 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8d980682f0 0x7f8d9806a7a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.412+0000 7f8db27ff700 1 --2- 172.21.15.156:0/2140612493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8dac161340 0x7f8dac14fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.412+0000 7f8db27ff700 1 --2- 172.21.15.156:0/2140612493 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8dac0d2320 0x7f8dac150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.412+0000 7f8db27ff700 1 --2- 172.21.15.156:0/2140612493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dac15c9b0 0x7f8dac1576f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.412+0000 7f8db27ff700 1 -- 172.21.15.156:0/2140612493 >> 172.21.15.156:0/2140612493 conn(0x7f8dac072600 msgr2=0x7f8dac15cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:51.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.412+0000 7f8db27ff700 1 -- 172.21.15.156:0/2140612493 shutdown_connections 2023-11-28T13:08:51.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.412+0000 7f8db27ff700 1 -- 172.21.15.156:0/2140612493 wait complete. 2023-11-28T13:08:51.415 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 8 2023-11-28T13:08:51.432 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:08:51.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.630+0000 7ff0a07ff700 1 Processor -- start 2023-11-28T13:08:51.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.630+0000 7ff0a07ff700 1 -- start start 2023-11-28T13:08:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.631+0000 7ff0a07ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09815c9b0 0x7ff0980b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.631+0000 7ff0a07ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff098161340 0x7ff0980d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.632+0000 7ff0a07ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0980d1950 con 0x7ff0980d2320 2023-11-28T13:08:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.632+0000 7ff0a07ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0980b5800 con 0x7ff098161340 2023-11-28T13:08:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.632+0000 7ff0a07ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0980b5520 con 0x7ff09815c9b0 2023-11-28T13:08:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.632+0000 7ff09ed9c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff0980d2320 0x7ff0980cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39506/0 (socket says 172.21.15.156:39506) 2023-11-28T13:08:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.632+0000 7ff09ed9c700 1 -- 172.21.15.156:0/4060120900 learned_addr learned my addr 172.21.15.156:0/4060120900 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:51.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.632+0000 7ff09e59b700 1 --2- 172.21.15.156:0/4060120900 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09815c9b0 0x7ff0980b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:51.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.632+0000 7ff09dd9a700 1 --2- 172.21.15.156:0/4060120900 >> v2:172.21.15.186:3300/0 conn(0x7ff098161340 0x7ff0980d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:51.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.632+0000 7ff09d599700 1 -- 172.21.15.156:0/4060120900 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620402556 0 0) 0x7ff0980d1950 con 0x7ff0980d2320 2023-11-28T13:08:51.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.632+0000 7ff09d599700 1 -- 172.21.15.156:0/4060120900 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0840035b0 con 0x7ff0980d2320 2023-11-28T13:08:51.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.632+0000 7ff09dd9a700 1 -- 172.21.15.156:0/4060120900 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09815c9b0 msgr2=0x7ff0980b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:51.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.632+0000 7ff09dd9a700 1 --2- 172.21.15.156:0/4060120900 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09815c9b0 0x7ff0980b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.632+0000 7ff09dd9a700 1 -- 172.21.15.156:0/4060120900 >> v1:172.21.15.156:6789/0 conn(0x7ff0980d2320 legacy=0x7ff0980cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:51.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.632+0000 7ff09dd9a700 1 -- 172.21.15.156:0/4060120900 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0980d1300 con 0x7ff098161340 2023-11-28T13:08:51.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.632+0000 7ff09e59b700 1 --2- 172.21.15.156:0/4060120900 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09815c9b0 0x7ff0980b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:51.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.633+0000 7ff09dd9a700 1 --2- 172.21.15.156:0/4060120900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff098161340 0x7ff0980d1de0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff088009840 tx=0x7ff088009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c3b94c8c65d52b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:51.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.633+0000 7ff09d599700 1 -- 172.21.15.156:0/4060120900 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088011830 con 0x7ff098161340 2023-11-28T13:08:51.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.633+0000 7ff09d599700 1 -- 172.21.15.156:0/4060120900 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff088011990 con 0x7ff098161340 2023-11-28T13:08:51.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.633+0000 7ff09d599700 1 -- 172.21.15.156:0/4060120900 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088011d20 con 0x7ff098161340 2023-11-28T13:08:51.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.633+0000 7ff0a07ff700 1 -- 172.21.15.156:0/4060120900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff098161340 msgr2=0x7ff0980d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:51.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.633+0000 7ff0a07ff700 1 --2- 172.21.15.156:0/4060120900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff098161340 0x7ff0980d1de0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff088009840 tx=0x7ff088009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:51.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.634+0000 7ff0a07ff700 1 -- 172.21.15.156:0/4060120900 shutdown_connections 2023-11-28T13:08:51.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.634+0000 7ff0a07ff700 1 --2- 172.21.15.156:0/4060120900 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff09815c9b0 0x7ff0980b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.634+0000 7ff0a07ff700 1 --2- 172.21.15.156:0/4060120900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff098161340 0x7ff0980d1de0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.634+0000 7ff0a07ff700 1 -- 172.21.15.156:0/4060120900 >> 172.21.15.156:0/4060120900 conn(0x7ff098072600 msgr2=0x7ff0980cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:51.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.634+0000 7ff0a07ff700 1 -- 172.21.15.156:0/4060120900 shutdown_connections 2023-11-28T13:08:51.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.634+0000 7ff0a07ff700 1 -- 172.21.15.156:0/4060120900 wait complete. 2023-11-28T13:08:51.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.635+0000 7ff0a07ff700 1 Processor -- start 2023-11-28T13:08:51.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.636+0000 7ff0a07ff700 1 -- start start 2023-11-28T13:08:51.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.636+0000 7ff0a07ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0980d2320 0x7ff0981f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:51.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.636+0000 7ff0a07ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09815c9b0 0x7ff0980c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:51.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.636+0000 7ff0a07ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff098161340 0x7ff0980c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:51.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.636+0000 7ff0a07ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09813d0c0 con 0x7ff09815c9b0 2023-11-28T13:08:51.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.636+0000 7ff0a07ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09813aba0 con 0x7ff0980d2320 2023-11-28T13:08:51.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.636+0000 7ff0a07ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09813fed0 con 0x7ff098161340 2023-11-28T13:08:51.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.637+0000 7ff09dd9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09815c9b0 0x7ff0980c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:51.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.637+0000 7ff09dd9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09815c9b0 0x7ff0980c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39862/0 (socket says 172.21.15.156:39862) 2023-11-28T13:08:51.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.637+0000 7ff09dd9a700 1 -- 172.21.15.156:0/3710904190 learned_addr learned my addr 172.21.15.156:0/3710904190 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:51.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.637+0000 7ff09ed9c700 1 --2- 172.21.15.156:0/3710904190 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff098161340 0x7ff0980c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:51.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.637+0000 7ff09e59b700 1 --2- 172.21.15.156:0/3710904190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0980d2320 0x7ff0981f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:51.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.637+0000 7ff09dd9a700 1 -- 172.21.15.156:0/3710904190 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff098161340 msgr2=0x7ff0980c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:51.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.637+0000 7ff09dd9a700 1 --2- 172.21.15.156:0/3710904190 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff098161340 0x7ff0980c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.637+0000 7ff09dd9a700 1 -- 172.21.15.156:0/3710904190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0980d2320 msgr2=0x7ff0981f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:51.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.637+0000 7ff09dd9a700 1 --2- 172.21.15.156:0/3710904190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0980d2320 0x7ff0981f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.637+0000 7ff09dd9a700 1 -- 172.21.15.156:0/3710904190 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0940094a0 con 0x7ff09815c9b0 2023-11-28T13:08:51.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.637+0000 7ff09e59b700 1 --2- 172.21.15.156:0/3710904190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0980d2320 0x7ff0981f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:51.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.638+0000 7ff09dd9a700 1 --2- 172.21.15.156:0/3710904190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09815c9b0 0x7ff0980c7d20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff0880090b0 tx=0x7ff088004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:51.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.638+0000 7ff08effd700 1 -- 172.21.15.156:0/3710904190 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0880036a0 con 0x7ff09815c9b0 2023-11-28T13:08:51.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.638+0000 7ff08effd700 1 -- 172.21.15.156:0/3710904190 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08801f070 con 0x7ff09815c9b0 2023-11-28T13:08:51.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.638+0000 7ff0a07ff700 1 -- 172.21.15.156:0/3710904190 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0880094a0 con 0x7ff09815c9b0 2023-11-28T13:08:51.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.638+0000 7ff08effd700 1 -- 172.21.15.156:0/3710904190 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088003a60 con 0x7ff09815c9b0 2023-11-28T13:08:51.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.638+0000 7ff0a07ff700 1 -- 172.21.15.156:0/3710904190 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0980b5130 con 0x7ff09815c9b0 2023-11-28T13:08:51.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.639+0000 7ff0a07ff700 1 -- 172.21.15.156:0/3710904190 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0980d15e0 con 0x7ff09815c9b0 2023-11-28T13:08:51.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.640+0000 7ff08effd700 1 -- 172.21.15.156:0/3710904190 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff08800f050 con 0x7ff09815c9b0 2023-11-28T13:08:51.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.641+0000 7ff08effd700 1 --2- 172.21.15.156:0/3710904190 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff07c0682f0 0x7ff07c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:51.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.641+0000 7ff08effd700 1 -- 172.21.15.156:0/3710904190 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff088021070 con 0x7ff09815c9b0 2023-11-28T13:08:51.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.641+0000 7ff09e59b700 1 --2- 172.21.15.156:0/3710904190 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff07c0682f0 0x7ff07c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:51.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.642+0000 7ff09e59b700 1 --2- 172.21.15.156:0/3710904190 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff07c0682f0 0x7ff07c06a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff094009dc0 tx=0x7ff094008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:51.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.645+0000 7ff08effd700 1 -- 172.21.15.156:0/3710904190 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff088053d30 con 0x7ff09815c9b0 2023-11-28T13:08:51.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.838+0000 7ff0a07ff700 1 -- 172.21.15.156:0/3710904190 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0980b52c0 con 0x7ff09815c9b0 2023-11-28T13:08:51.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.839+0000 7ff08effd700 1 -- 172.21.15.156:0/3710904190 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17993 (secure 0 0 0) 0x7ff088057350 con 0x7ff09815c9b0 2023-11-28T13:08:51.841 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:51.842 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:08:51.536161+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:08:00.903485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:08:02.037854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1c","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:08:51.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.842+0000 7ff0a07ff700 1 -- 172.21.15.156:0/3710904190 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff07c0682f0 msgr2=0x7ff07c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:51.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.842+0000 7ff0a07ff700 1 --2- 172.21.15.156:0/3710904190 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff07c0682f0 0x7ff07c06a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff094009dc0 tx=0x7ff094008040 comp rx=0 tx=0).stop 2023-11-28T13:08:51.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.842+0000 7ff0a07ff700 1 -- 172.21.15.156:0/3710904190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09815c9b0 msgr2=0x7ff0980c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:51.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.842+0000 7ff0a07ff700 1 --2- 172.21.15.156:0/3710904190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09815c9b0 0x7ff0980c7d20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff0880090b0 tx=0x7ff088004750 comp rx=0 tx=0).stop 2023-11-28T13:08:51.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.842+0000 7ff0a07ff700 1 -- 172.21.15.156:0/3710904190 shutdown_connections 2023-11-28T13:08:51.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.842+0000 7ff0a07ff700 1 --2- 172.21.15.156:0/3710904190 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff07c0682f0 0x7ff07c06a7a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.842+0000 7ff0a07ff700 1 --2- 172.21.15.156:0/3710904190 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff098161340 0x7ff0980c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.842+0000 7ff0a07ff700 1 --2- 172.21.15.156:0/3710904190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0980d2320 0x7ff0981f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.842+0000 7ff0a07ff700 1 --2- 172.21.15.156:0/3710904190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09815c9b0 0x7ff0980c7d20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:51.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.842+0000 7ff0a07ff700 1 -- 172.21.15.156:0/3710904190 >> 172.21.15.156:0/3710904190 conn(0x7ff098072600 msgr2=0x7ff09815cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:51.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.842+0000 7ff0a07ff700 1 -- 172.21.15.156:0/3710904190 shutdown_connections 2023-11-28T13:08:51.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:51.842+0000 7ff0a07ff700 1 -- 172.21.15.156:0/3710904190 wait complete. 2023-11-28T13:08:51.862 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:08:00.903485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:08:02.037854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:08:51.863 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:08:52.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.061+0000 7fb1585ff700 1 Processor -- start 2023-11-28T13:08:52.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.061+0000 7fb1585ff700 1 -- start start 2023-11-28T13:08:52.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.062+0000 7fb1585ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb150161340 0x7fb1500d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:52.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.062+0000 7fb1585ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb1500d2320 0x7fb1500cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:52.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.062+0000 7fb1585ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1500d1950 con 0x7fb15015c9b0 2023-11-28T13:08:52.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.062+0000 7fb1585ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1500b5800 con 0x7fb1500d2320 2023-11-28T13:08:52.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.062+0000 7fb1585ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1500b5300 con 0x7fb150161340 2023-11-28T13:08:52.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.062+0000 7fb156b9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb1500d2320 0x7fb1500cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:52.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.062+0000 7fb15639b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb15015c9b0 0x7fb1500b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39518/0 (socket says 172.21.15.156:39518) 2023-11-28T13:08:52.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.062+0000 7fb15639b700 1 -- 172.21.15.156:0/3814443996 learned_addr learned my addr 172.21.15.156:0/3814443996 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:52.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.062+0000 7fb155b9a700 1 --2- 172.21.15.156:0/3814443996 >> v2:172.21.15.186:3300/0 conn(0x7fb150161340 0x7fb1500d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:52.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.063+0000 7fb156b9c700 1 -- 172.21.15.156:0/3814443996 >> v2:172.21.15.186:3300/0 conn(0x7fb150161340 msgr2=0x7fb1500d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:52.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.063+0000 7fb156b9c700 1 --2- 172.21.15.156:0/3814443996 >> v2:172.21.15.186:3300/0 conn(0x7fb150161340 0x7fb1500d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.063+0000 7fb156b9c700 1 -- 172.21.15.156:0/3814443996 >> v1:172.21.15.156:6789/0 conn(0x7fb15015c9b0 legacy=0x7fb1500b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:52.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.063+0000 7fb156b9c700 1 -- 172.21.15.156:0/3814443996 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1500b5520 con 0x7fb1500d2320 2023-11-28T13:08:52.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.063+0000 7fb155399700 1 -- 172.21.15.156:0/3814443996 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1236146947 0 0) 0x7fb1500d1950 con 0x7fb15015c9b0 2023-11-28T13:08:52.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.063+0000 7fb155b9a700 1 --2- 172.21.15.156:0/3814443996 >> v2:172.21.15.186:3300/0 conn(0x7fb150161340 0x7fb1500d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:52.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.063+0000 7fb156b9c700 1 --2- 172.21.15.156:0/3814443996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1500d2320 0x7fb1500cd7c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb1500685c0 tx=0x7fb14800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27e9b3262077ef9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:52.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.063+0000 7fb155399700 1 -- 172.21.15.156:0/3814443996 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14800e070 con 0x7fb1500d2320 2023-11-28T13:08:52.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.063+0000 7fb155399700 1 -- 172.21.15.156:0/3814443996 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14800bb90 con 0x7fb1500d2320 2023-11-28T13:08:52.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.063+0000 7fb155399700 1 -- 172.21.15.156:0/3814443996 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb148012430 con 0x7fb1500d2320 2023-11-28T13:08:52.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.064+0000 7fb1585ff700 1 -- 172.21.15.156:0/3814443996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1500d2320 msgr2=0x7fb1500cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:52.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.064+0000 7fb1585ff700 1 --2- 172.21.15.156:0/3814443996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1500d2320 0x7fb1500cd7c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb1500685c0 tx=0x7fb14800b330 comp rx=0 tx=0).stop 2023-11-28T13:08:52.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.064+0000 7fb1585ff700 1 -- 172.21.15.156:0/3814443996 shutdown_connections 2023-11-28T13:08:52.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.064+0000 7fb1585ff700 1 --2- 172.21.15.156:0/3814443996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1500d2320 0x7fb1500cd7c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.064+0000 7fb1585ff700 1 --2- 172.21.15.156:0/3814443996 >> v2:172.21.15.186:3300/0 conn(0x7fb150161340 0x7fb1500d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.064+0000 7fb1585ff700 1 -- 172.21.15.156:0/3814443996 >> 172.21.15.156:0/3814443996 conn(0x7fb150072600 msgr2=0x7fb1500cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:52.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.064+0000 7fb1585ff700 1 -- 172.21.15.156:0/3814443996 shutdown_connections 2023-11-28T13:08:52.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.064+0000 7fb1585ff700 1 -- 172.21.15.156:0/3814443996 wait complete. 2023-11-28T13:08:52.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.066+0000 7fb1585ff700 1 Processor -- start 2023-11-28T13:08:52.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.066+0000 7fb1585ff700 1 -- start start 2023-11-28T13:08:52.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.067+0000 7fb1585ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1500d2320 0x7fb1501465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:52.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.067+0000 7fb1585ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb15015c9b0 0x7fb150154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:52.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.067+0000 7fb1585ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb150161340 0x7fb150157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:52.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.067+0000 7fb1585ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb150157dd0 con 0x7fb15015c9b0 2023-11-28T13:08:52.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.067+0000 7fb1585ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1501f6120 con 0x7fb1500d2320 2023-11-28T13:08:52.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.067+0000 7fb1585ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1501f63c0 con 0x7fb150161340 2023-11-28T13:08:52.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.067+0000 7fb155b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb15015c9b0 0x7fb150154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:52.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.067+0000 7fb155b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb15015c9b0 0x7fb150154a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39882/0 (socket says 172.21.15.156:39882) 2023-11-28T13:08:52.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.067+0000 7fb155b9a700 1 -- 172.21.15.156:0/651230386 learned_addr learned my addr 172.21.15.156:0/651230386 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:52.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.067+0000 7fb15639b700 1 --2- 172.21.15.156:0/651230386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1500d2320 0x7fb1501465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:52.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.067+0000 7fb156b9c700 1 --2- 172.21.15.156:0/651230386 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb150161340 0x7fb150157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:52.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.067+0000 7fb155b9a700 1 -- 172.21.15.156:0/651230386 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb150161340 msgr2=0x7fb150157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:52.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.067+0000 7fb155b9a700 1 --2- 172.21.15.156:0/651230386 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb150161340 0x7fb150157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.067+0000 7fb155b9a700 1 -- 172.21.15.156:0/651230386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1500d2320 msgr2=0x7fb1501465b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:52.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.067+0000 7fb155b9a700 1 --2- 172.21.15.156:0/651230386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1500d2320 0x7fb1501465b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.067+0000 7fb155b9a700 1 -- 172.21.15.156:0/651230386 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb148009a70 con 0x7fb15015c9b0 2023-11-28T13:08:52.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.067+0000 7fb156b9c700 1 --2- 172.21.15.156:0/651230386 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb150161340 0x7fb150157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:52.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.068+0000 7fb155b9a700 1 --2- 172.21.15.156:0/651230386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb15015c9b0 0x7fb150154a60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb14000e810 tx=0x7fb14000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:52.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.068+0000 7fb146ffd700 1 -- 172.21.15.156:0/651230386 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb140013610 con 0x7fb15015c9b0 2023-11-28T13:08:52.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.068+0000 7fb146ffd700 1 -- 172.21.15.156:0/651230386 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb140013770 con 0x7fb15015c9b0 2023-11-28T13:08:52.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.068+0000 7fb146ffd700 1 -- 172.21.15.156:0/651230386 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb140013b20 con 0x7fb15015c9b0 2023-11-28T13:08:52.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.068+0000 7fb1585ff700 1 -- 172.21.15.156:0/651230386 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1500d15e0 con 0x7fb15015c9b0 2023-11-28T13:08:52.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.068+0000 7fb1585ff700 1 -- 172.21.15.156:0/651230386 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1500d1300 con 0x7fb15015c9b0 2023-11-28T13:08:52.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.069+0000 7fb1585ff700 1 -- 172.21.15.156:0/651230386 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15015cd80 con 0x7fb15015c9b0 2023-11-28T13:08:52.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.070+0000 7fb146ffd700 1 -- 172.21.15.156:0/651230386 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb140023070 con 0x7fb15015c9b0 2023-11-28T13:08:52.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.071+0000 7fb146ffd700 1 --2- 172.21.15.156:0/651230386 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb1340682f0 0x7fb13406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:52.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.071+0000 7fb146ffd700 1 -- 172.21.15.156:0/651230386 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb140085a20 con 0x7fb15015c9b0 2023-11-28T13:08:52.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.071+0000 7fb15639b700 1 --2- 172.21.15.156:0/651230386 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb1340682f0 0x7fb13406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:52.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.072+0000 7fb15639b700 1 --2- 172.21.15.156:0/651230386 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb1340682f0 0x7fb13406a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb14c000f60 tx=0x7fb14c007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:52.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.075+0000 7fb146ffd700 1 -- 172.21.15.156:0/651230386 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb140052150 con 0x7fb15015c9b0 2023-11-28T13:08:52.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.262+0000 7fb1585ff700 1 -- 172.21.15.156:0/651230386 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb1501f6cf0 con 0x7fb15015c9b0 2023-11-28T13:08:52.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.262+0000 7fb146ffd700 1 -- 172.21.15.156:0/651230386 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb140055770 con 0x7fb15015c9b0 2023-11-28T13:08:52.264 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:08:52.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.264+0000 7fb1585ff700 1 -- 172.21.15.156:0/651230386 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb1340682f0 msgr2=0x7fb13406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:52.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.264+0000 7fb1585ff700 1 --2- 172.21.15.156:0/651230386 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb1340682f0 0x7fb13406a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb14c000f60 tx=0x7fb14c007e70 comp rx=0 tx=0).stop 2023-11-28T13:08:52.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.265+0000 7fb1585ff700 1 -- 172.21.15.156:0/651230386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb15015c9b0 msgr2=0x7fb150154a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:52.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.265+0000 7fb1585ff700 1 --2- 172.21.15.156:0/651230386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb15015c9b0 0x7fb150154a60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb14000e810 tx=0x7fb14000eb20 comp rx=0 tx=0).stop 2023-11-28T13:08:52.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.265+0000 7fb1585ff700 1 -- 172.21.15.156:0/651230386 shutdown_connections 2023-11-28T13:08:52.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.265+0000 7fb1585ff700 1 --2- 172.21.15.156:0/651230386 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb1340682f0 0x7fb13406a7a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.265+0000 7fb1585ff700 1 --2- 172.21.15.156:0/651230386 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb150161340 0x7fb150157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.265+0000 7fb1585ff700 1 --2- 172.21.15.156:0/651230386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1500d2320 0x7fb1501465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.265+0000 7fb1585ff700 1 --2- 172.21.15.156:0/651230386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb15015c9b0 0x7fb150154a60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.265+0000 7fb1585ff700 1 -- 172.21.15.156:0/651230386 >> 172.21.15.156:0/651230386 conn(0x7fb150072600 msgr2=0x7fb15015ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:52.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.265+0000 7fb1585ff700 1 -- 172.21.15.156:0/651230386 shutdown_connections 2023-11-28T13:08:52.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.265+0000 7fb1585ff700 1 -- 172.21.15.156:0/651230386 wait complete. 2023-11-28T13:08:52.285 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:08:52.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.485+0000 7f4e36fff700 1 Processor -- start 2023-11-28T13:08:52.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.485+0000 7f4e36fff700 1 -- start start 2023-11-28T13:08:52.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.486+0000 7f4e36fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4e3015c9b0 0x7f4e300b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:52.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.486+0000 7f4e36fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e300d1950 con 0x7f4e30161340 2023-11-28T13:08:52.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.486+0000 7f4e36fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e300d24f0 con 0x7f4e300cd7c0 2023-11-28T13:08:52.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.486+0000 7f4e36fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e300b5800 con 0x7f4e3015c9b0 2023-11-28T13:08:52.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.486+0000 7f4e34d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4e3015c9b0 0x7f4e300b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:52.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.486+0000 7f4e34d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4e3015c9b0 0x7f4e300b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39894/0 (socket says 172.21.15.156:39894) 2023-11-28T13:08:52.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.486+0000 7f4e34d9b700 1 -- 172.21.15.156:0/2347873917 learned_addr learned my addr 172.21.15.156:0/2347873917 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:52.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.486+0000 7f4e2ffff700 1 --1- 172.21.15.156:0/2347873917 >> v1:172.21.15.156:6789/0 conn(0x7f4e30161340 0x7f4e300d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39532/0 (socket says 172.21.15.156:39532) 2023-11-28T13:08:52.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.487+0000 7f4e34d9b700 1 -- 172.21.15.156:0/2347873917 >> v1:172.21.15.186:6789/0 conn(0x7f4e300cd7c0 legacy=0x7f4e300b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:52.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.487+0000 7f4e34d9b700 1 -- 172.21.15.156:0/2347873917 >> v1:172.21.15.156:6789/0 conn(0x7f4e30161340 legacy=0x7f4e300d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:52.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.487+0000 7f4e34d9b700 1 -- 172.21.15.156:0/2347873917 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e300b5520 con 0x7f4e3015c9b0 2023-11-28T13:08:52.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.487+0000 7f4e2f7fe700 1 -- 172.21.15.156:0/2347873917 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164420581 0 0) 0x7f4e300d1950 con 0x7f4e30161340 2023-11-28T13:08:52.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.487+0000 7f4e34d9b700 1 --2- 172.21.15.156:0/2347873917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e3015c9b0 0x7f4e300b4bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4e240098f0 tx=0x7f4e24009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397552a8a5371e17 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:52.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.487+0000 7f4e2f7fe700 1 -- 172.21.15.156:0/2347873917 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2400b8a0 con 0x7f4e3015c9b0 2023-11-28T13:08:52.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.487+0000 7f4e2f7fe700 1 -- 172.21.15.156:0/2347873917 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2400ba00 con 0x7f4e3015c9b0 2023-11-28T13:08:52.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.487+0000 7f4e2f7fe700 1 -- 172.21.15.156:0/2347873917 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2400bd90 con 0x7f4e3015c9b0 2023-11-28T13:08:52.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.488+0000 7f4e36fff700 1 -- 172.21.15.156:0/2347873917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e3015c9b0 msgr2=0x7f4e300b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:52.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.488+0000 7f4e36fff700 1 --2- 172.21.15.156:0/2347873917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e3015c9b0 0x7f4e300b4bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4e240098f0 tx=0x7f4e24009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:52.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.488+0000 7f4e36fff700 1 -- 172.21.15.156:0/2347873917 shutdown_connections 2023-11-28T13:08:52.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.488+0000 7f4e36fff700 1 --2- 172.21.15.156:0/2347873917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e3015c9b0 0x7f4e300b4bf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.488+0000 7f4e36fff700 1 -- 172.21.15.156:0/2347873917 >> 172.21.15.156:0/2347873917 conn(0x7f4e30072600 msgr2=0x7f4e300cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:52.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.488+0000 7f4e36fff700 1 -- 172.21.15.156:0/2347873917 shutdown_connections 2023-11-28T13:08:52.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.488+0000 7f4e36fff700 1 -- 172.21.15.156:0/2347873917 wait complete. 2023-11-28T13:08:52.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.490+0000 7f4e36fff700 1 Processor -- start 2023-11-28T13:08:52.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.490+0000 7f4e36fff700 1 -- start start 2023-11-28T13:08:52.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.491+0000 7f4e36fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e300cd7c0 0x7f4e3014d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:52.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.491+0000 7f4e36fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e3015c9b0 0x7f4e3015ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:52.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.491+0000 7f4e36fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e30161340 0x7f4e3015e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:52.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.491+0000 7f4e36fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e300c80c0 con 0x7f4e30161340 2023-11-28T13:08:52.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.491+0000 7f4e36fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e300ca240 con 0x7f4e300cd7c0 2023-11-28T13:08:52.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.491+0000 7f4e36fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e300cada0 con 0x7f4e3015c9b0 2023-11-28T13:08:52.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.491+0000 7f4e3559c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e30161340 0x7f4e3015e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:52.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.491+0000 7f4e3559c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e30161340 0x7f4e3015e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39902/0 (socket says 172.21.15.156:39902) 2023-11-28T13:08:52.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.491+0000 7f4e3559c700 1 -- 172.21.15.156:0/2183752634 learned_addr learned my addr 172.21.15.156:0/2183752634 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:52.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.491+0000 7f4e2ffff700 1 --2- 172.21.15.156:0/2183752634 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e3015c9b0 0x7f4e3015ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:52.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.491+0000 7f4e34d9b700 1 --2- 172.21.15.156:0/2183752634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e300cd7c0 0x7f4e3014d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:52.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.491+0000 7f4e3559c700 1 -- 172.21.15.156:0/2183752634 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e3015c9b0 msgr2=0x7f4e3015ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:52.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.491+0000 7f4e3559c700 1 --2- 172.21.15.156:0/2183752634 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e3015c9b0 0x7f4e3015ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.491+0000 7f4e3559c700 1 -- 172.21.15.156:0/2183752634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e300cd7c0 msgr2=0x7f4e3014d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:52.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.491+0000 7f4e3559c700 1 --2- 172.21.15.156:0/2183752634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e300cd7c0 0x7f4e3014d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.491+0000 7f4e3559c700 1 -- 172.21.15.156:0/2183752634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e24009550 con 0x7f4e30161340 2023-11-28T13:08:52.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.491+0000 7f4e2ffff700 1 --2- 172.21.15.156:0/2183752634 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e3015c9b0 0x7f4e3015ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:52.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.492+0000 7f4e3559c700 1 --2- 172.21.15.156:0/2183752634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e30161340 0x7f4e3015e310 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800cb50 tx=0x7f4e2800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:52.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.492+0000 7f4e2d7fa700 1 -- 172.21.15.156:0/2183752634 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28003c00 con 0x7f4e30161340 2023-11-28T13:08:52.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.492+0000 7f4e2d7fa700 1 -- 172.21.15.156:0/2183752634 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e28003d60 con 0x7f4e30161340 2023-11-28T13:08:52.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.492+0000 7f4e36fff700 1 -- 172.21.15.156:0/2183752634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e300d15e0 con 0x7f4e30161340 2023-11-28T13:08:52.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.492+0000 7f4e2d7fa700 1 -- 172.21.15.156:0/2183752634 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800aac0 con 0x7f4e30161340 2023-11-28T13:08:52.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.492+0000 7f4e36fff700 1 -- 172.21.15.156:0/2183752634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e300d1300 con 0x7f4e30161340 2023-11-28T13:08:52.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.493+0000 7f4e36fff700 1 -- 172.21.15.156:0/2183752634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e300b5130 con 0x7f4e30161340 2023-11-28T13:08:52.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.494+0000 7f4e2d7fa700 1 -- 172.21.15.156:0/2183752634 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4e28010050 con 0x7f4e30161340 2023-11-28T13:08:52.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.495+0000 7f4e2d7fa700 1 --2- 172.21.15.156:0/2183752634 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4e140682f0 0x7f4e1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:52.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.495+0000 7f4e2d7fa700 1 -- 172.21.15.156:0/2183752634 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4e28085c00 con 0x7f4e30161340 2023-11-28T13:08:52.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.495+0000 7f4e34d9b700 1 --2- 172.21.15.156:0/2183752634 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4e140682f0 0x7f4e1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:52.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.496+0000 7f4e34d9b700 1 --2- 172.21.15.156:0/2183752634 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4e140682f0 0x7f4e1406a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4e24014040 tx=0x7f4e24008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:52.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.499+0000 7f4e2d7fa700 1 -- 172.21.15.156:0/2183752634 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4e28052330 con 0x7f4e30161340 2023-11-28T13:08:52.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.694+0000 7f4e36fff700 1 -- 172.21.15.156:0/2183752634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e300745c0 con 0x7f4e30161340 2023-11-28T13:08:52.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.694+0000 7f4e2d7fa700 1 -- 172.21.15.156:0/2183752634 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e28055950 con 0x7f4e30161340 2023-11-28T13:08:52.696 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:08:52.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.696+0000 7f4e36fff700 1 -- 172.21.15.156:0/2183752634 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4e140682f0 msgr2=0x7f4e1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:52.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.696+0000 7f4e36fff700 1 --2- 172.21.15.156:0/2183752634 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4e140682f0 0x7f4e1406a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4e24014040 tx=0x7f4e24008030 comp rx=0 tx=0).stop 2023-11-28T13:08:52.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.696+0000 7f4e36fff700 1 -- 172.21.15.156:0/2183752634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e30161340 msgr2=0x7f4e3015e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:52.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.696+0000 7f4e36fff700 1 --2- 172.21.15.156:0/2183752634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e30161340 0x7f4e3015e310 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800cb50 tx=0x7f4e2800ce60 comp rx=0 tx=0).stop 2023-11-28T13:08:52.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.696+0000 7f4e36fff700 1 -- 172.21.15.156:0/2183752634 shutdown_connections 2023-11-28T13:08:52.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.696+0000 7f4e36fff700 1 --2- 172.21.15.156:0/2183752634 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4e140682f0 0x7f4e1406a7a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.696+0000 7f4e36fff700 1 --2- 172.21.15.156:0/2183752634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e30161340 0x7f4e3015e310 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.696+0000 7f4e36fff700 1 --2- 172.21.15.156:0/2183752634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e300cd7c0 0x7f4e3014d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.696+0000 7f4e36fff700 1 --2- 172.21.15.156:0/2183752634 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e3015c9b0 0x7f4e3015ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.696+0000 7f4e36fff700 1 -- 172.21.15.156:0/2183752634 >> 172.21.15.156:0/2183752634 conn(0x7f4e30072600 msgr2=0x7f4e30160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:52.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.696+0000 7f4e36fff700 1 -- 172.21.15.156:0/2183752634 shutdown_connections 2023-11-28T13:08:52.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.696+0000 7f4e36fff700 1 -- 172.21.15.156:0/2183752634 wait complete. 2023-11-28T13:08:52.717 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:08:52.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.915+0000 7ff7f37ff700 1 Processor -- start 2023-11-28T13:08:52.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.915+0000 7ff7f37ff700 1 -- start start 2023-11-28T13:08:52.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.916+0000 7ff7f37ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff7ec15c9b0 0x7ff7ec0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:52.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.916+0000 7ff7f37ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff7ec0cd7c0 0x7ff7ec0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:52.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.916+0000 7ff7f37ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ec0d1950 con 0x7ff7ec161340 2023-11-28T13:08:52.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.916+0000 7ff7f37ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ec0b5800 con 0x7ff7ec15c9b0 2023-11-28T13:08:52.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.916+0000 7ff7f37ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ec0ce140 con 0x7ff7ec0cd7c0 2023-11-28T13:08:52.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.917+0000 7ff7f159b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff7ec15c9b0 0x7ff7ec0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:52.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.917+0000 7ff7f159b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff7ec15c9b0 0x7ff7ec0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39910/0 (socket says 172.21.15.156:39910) 2023-11-28T13:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.917+0000 7ff7f159b700 1 -- 172.21.15.156:0/1894353788 learned_addr learned my addr 172.21.15.156:0/1894353788 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.917+0000 7ff7f1d9c700 1 --2- 172.21.15.156:0/1894353788 >> v2:172.21.15.186:3300/0 conn(0x7ff7ec0cd7c0 0x7ff7ec0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.917+0000 7ff7f159b700 1 -- 172.21.15.156:0/1894353788 >> v2:172.21.15.186:3300/0 conn(0x7ff7ec0cd7c0 msgr2=0x7ff7ec0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.917+0000 7ff7f159b700 1 --2- 172.21.15.156:0/1894353788 >> v2:172.21.15.186:3300/0 conn(0x7ff7ec0cd7c0 0x7ff7ec0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.917+0000 7ff7f159b700 1 -- 172.21.15.156:0/1894353788 >> v1:172.21.15.186:6789/0 conn(0x7ff7ec161340 legacy=0x7ff7ec0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.917+0000 7ff7f159b700 1 -- 172.21.15.156:0/1894353788 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ec0d24f0 con 0x7ff7ec15c9b0 2023-11-28T13:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.917+0000 7ff7f159b700 1 --2- 172.21.15.156:0/1894353788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7ec15c9b0 0x7ff7ec0b4bf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec06c170 tx=0x7ff7e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203619b97ecdbed0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.918+0000 7ff7e3fff700 1 -- 172.21.15.156:0/1894353788 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800b870 con 0x7ff7ec15c9b0 2023-11-28T13:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.918+0000 7ff7e3fff700 1 -- 172.21.15.156:0/1894353788 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e800b9d0 con 0x7ff7ec15c9b0 2023-11-28T13:08:52.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.918+0000 7ff7e3fff700 1 -- 172.21.15.156:0/1894353788 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800bd60 con 0x7ff7ec15c9b0 2023-11-28T13:08:52.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.918+0000 7ff7f37ff700 1 -- 172.21.15.156:0/1894353788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7ec15c9b0 msgr2=0x7ff7ec0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:52.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.918+0000 7ff7f37ff700 1 --2- 172.21.15.156:0/1894353788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7ec15c9b0 0x7ff7ec0b4bf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec06c170 tx=0x7ff7e8009bd0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.918+0000 7ff7f37ff700 1 -- 172.21.15.156:0/1894353788 shutdown_connections 2023-11-28T13:08:52.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.918+0000 7ff7f37ff700 1 --2- 172.21.15.156:0/1894353788 >> v2:172.21.15.186:3300/0 conn(0x7ff7ec0cd7c0 0x7ff7ec0cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.918+0000 7ff7f37ff700 1 --2- 172.21.15.156:0/1894353788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7ec15c9b0 0x7ff7ec0b4bf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.918+0000 7ff7f37ff700 1 -- 172.21.15.156:0/1894353788 >> 172.21.15.156:0/1894353788 conn(0x7ff7ec072600 msgr2=0x7ff7ec0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:52.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.918+0000 7ff7f37ff700 1 -- 172.21.15.156:0/1894353788 shutdown_connections 2023-11-28T13:08:52.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.918+0000 7ff7f37ff700 1 -- 172.21.15.156:0/1894353788 wait complete. 2023-11-28T13:08:52.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.920+0000 7ff7f37ff700 1 Processor -- start 2023-11-28T13:08:52.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.920+0000 7ff7f37ff700 1 -- start start 2023-11-28T13:08:52.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.921+0000 7ff7f37ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7ec0cd7c0 0x7ff7ec157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:52.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.921+0000 7ff7f37ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7ec15c9b0 0x7ff7ec0c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:52.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.921+0000 7ff7f37ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff7ec161340 0x7ff7ec15e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:52.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.921+0000 7ff7f37ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec0ca240 con 0x7ff7ec15c9b0 2023-11-28T13:08:52.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.921+0000 7ff7f37ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec157590 con 0x7ff7ec0cd7c0 2023-11-28T13:08:52.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.921+0000 7ff7f37ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec0ca490 con 0x7ff7ec161340 2023-11-28T13:08:52.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.921+0000 7ff7f0d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7ec15c9b0 0x7ff7ec0c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:52.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.921+0000 7ff7f0d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7ec15c9b0 0x7ff7ec0c9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39912/0 (socket says 172.21.15.156:39912) 2023-11-28T13:08:52.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.921+0000 7ff7f0d9a700 1 -- 172.21.15.156:0/4292909660 learned_addr learned my addr 172.21.15.156:0/4292909660 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:52.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.921+0000 7ff7f159b700 1 --2- 172.21.15.156:0/4292909660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7ec0cd7c0 0x7ff7ec157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:52.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.921+0000 7ff7f1d9c700 1 --2- 172.21.15.156:0/4292909660 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff7ec161340 0x7ff7ec15e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:52.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.922+0000 7ff7f0d9a700 1 -- 172.21.15.156:0/4292909660 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff7ec161340 msgr2=0x7ff7ec15e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:52.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.922+0000 7ff7f0d9a700 1 --2- 172.21.15.156:0/4292909660 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff7ec161340 0x7ff7ec15e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.922+0000 7ff7f0d9a700 1 -- 172.21.15.156:0/4292909660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7ec0cd7c0 msgr2=0x7ff7ec157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:52.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.922+0000 7ff7f0d9a700 1 --2- 172.21.15.156:0/4292909660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7ec0cd7c0 0x7ff7ec157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:52.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.922+0000 7ff7f0d9a700 1 -- 172.21.15.156:0/4292909660 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e8009550 con 0x7ff7ec15c9b0 2023-11-28T13:08:52.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.922+0000 7ff7f0d9a700 1 --2- 172.21.15.156:0/4292909660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7ec15c9b0 0x7ff7ec0c9710 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00c450 tx=0x7ff7dc00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:52.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.922+0000 7ff7e1ffb700 1 -- 172.21.15.156:0/4292909660 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc0052d0 con 0x7ff7ec15c9b0 2023-11-28T13:08:52.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.922+0000 7ff7e1ffb700 1 -- 172.21.15.156:0/4292909660 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7dc005430 con 0x7ff7ec15c9b0 2023-11-28T13:08:52.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.922+0000 7ff7f37ff700 1 -- 172.21.15.156:0/4292909660 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ec0d1300 con 0x7ff7ec15c9b0 2023-11-28T13:08:52.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.922+0000 7ff7e1ffb700 1 -- 172.21.15.156:0/4292909660 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc0038c0 con 0x7ff7ec15c9b0 2023-11-28T13:08:52.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.922+0000 7ff7f37ff700 1 -- 172.21.15.156:0/4292909660 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ec0b5520 con 0x7ff7ec15c9b0 2023-11-28T13:08:52.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.923+0000 7ff7f37ff700 1 -- 172.21.15.156:0/4292909660 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ec0d15e0 con 0x7ff7ec15c9b0 2023-11-28T13:08:52.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.924+0000 7ff7e1ffb700 1 -- 172.21.15.156:0/4292909660 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff7dc019030 con 0x7ff7ec15c9b0 2023-11-28T13:08:52.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.925+0000 7ff7e1ffb700 1 --2- 172.21.15.156:0/4292909660 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff7d0068320 0x7ff7d006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:52.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.925+0000 7ff7e1ffb700 1 -- 172.21.15.156:0/4292909660 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff7dc084f60 con 0x7ff7ec15c9b0 2023-11-28T13:08:52.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.926+0000 7ff7f159b700 1 --2- 172.21.15.156:0/4292909660 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff7d0068320 0x7ff7d006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:52.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.926+0000 7ff7f159b700 1 --2- 172.21.15.156:0/4292909660 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff7d0068320 0x7ff7d006a7d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8014040 tx=0x7ff7e8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:52.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:52.929+0000 7ff7e1ffb700 1 -- 172.21.15.156:0/4292909660 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff7dc051690 con 0x7ff7ec15c9b0 2023-11-28T13:08:53.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.122+0000 7ff7f37ff700 1 -- 172.21.15.156:0/4292909660 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff7ec0745c0 con 0x7ff7ec15c9b0 2023-11-28T13:08:53.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.122+0000 7ff7e1ffb700 1 -- 172.21.15.156:0/4292909660 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff7dc017020 con 0x7ff7ec15c9b0 2023-11-28T13:08:53.124 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T13:08:53.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.125+0000 7ff7f37ff700 1 -- 172.21.15.156:0/4292909660 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff7d0068320 msgr2=0x7ff7d006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:53.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.125+0000 7ff7f37ff700 1 --2- 172.21.15.156:0/4292909660 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff7d0068320 0x7ff7d006a7d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8014040 tx=0x7ff7e8008030 comp rx=0 tx=0).stop 2023-11-28T13:08:53.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.125+0000 7ff7f37ff700 1 -- 172.21.15.156:0/4292909660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7ec15c9b0 msgr2=0x7ff7ec0c9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:53.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.125+0000 7ff7f37ff700 1 --2- 172.21.15.156:0/4292909660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7ec15c9b0 0x7ff7ec0c9710 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00c450 tx=0x7ff7dc00c760 comp rx=0 tx=0).stop 2023-11-28T13:08:53.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.125+0000 7ff7f37ff700 1 -- 172.21.15.156:0/4292909660 shutdown_connections 2023-11-28T13:08:53.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.125+0000 7ff7f37ff700 1 --2- 172.21.15.156:0/4292909660 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff7d0068320 0x7ff7d006a7d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.125+0000 7ff7f37ff700 1 --2- 172.21.15.156:0/4292909660 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff7ec161340 0x7ff7ec15e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.125+0000 7ff7f37ff700 1 --2- 172.21.15.156:0/4292909660 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7ec0cd7c0 0x7ff7ec157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.125+0000 7ff7f37ff700 1 --2- 172.21.15.156:0/4292909660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7ec15c9b0 0x7ff7ec0c9710 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.125+0000 7ff7f37ff700 1 -- 172.21.15.156:0/4292909660 >> 172.21.15.156:0/4292909660 conn(0x7ff7ec072600 msgr2=0x7ff7ec0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:53.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.125+0000 7ff7f37ff700 1 -- 172.21.15.156:0/4292909660 shutdown_connections 2023-11-28T13:08:53.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.125+0000 7ff7f37ff700 1 -- 172.21.15.156:0/4292909660 wait complete. 2023-11-28T13:08:53.144 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:08:53.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.342+0000 7fb0f9bff700 1 Processor -- start 2023-11-28T13:08:53.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.343+0000 7fb0f9bff700 1 -- start start 2023-11-28T13:08:53.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.344+0000 7fb0f9bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb0f415c9b0 0x7fb0f40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:53.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.344+0000 7fb0f9bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0f4161340 0x7fb0f40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:53.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.344+0000 7fb0f9bff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb0f40d2320 0x7fb0f40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:53.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.344+0000 7fb0f9bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f40b5800 con 0x7fb0f415c9b0 2023-11-28T13:08:53.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.344+0000 7fb0f9bff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f40d1950 con 0x7fb0f40d2320 2023-11-28T13:08:53.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.344+0000 7fb0f9bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f40b5520 con 0x7fb0f4161340 2023-11-28T13:08:53.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.344+0000 7fb0f37fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb0f415c9b0 0x7fb0f40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:53.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.344+0000 7fb0f37fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb0f415c9b0 0x7fb0f40b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39920/0 (socket says 172.21.15.156:39920) 2023-11-28T13:08:53.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.344+0000 7fb0f37fe700 1 -- 172.21.15.156:0/2638160358 learned_addr learned my addr 172.21.15.156:0/2638160358 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:53.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.344+0000 7fb0f37fe700 1 -- 172.21.15.156:0/2638160358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0f4161340 msgr2=0x7fb0f40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:53.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.344+0000 7fb0f37fe700 1 --2- 172.21.15.156:0/2638160358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0f4161340 0x7fb0f40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.344+0000 7fb0f37fe700 1 -- 172.21.15.156:0/2638160358 >> v2:172.21.15.186:3300/0 conn(0x7fb0f40d2320 msgr2=0x7fb0f40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:53.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.344+0000 7fb0f37fe700 1 --2- 172.21.15.156:0/2638160358 >> v2:172.21.15.186:3300/0 conn(0x7fb0f40d2320 0x7fb0f40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.344+0000 7fb0f37fe700 1 -- 172.21.15.156:0/2638160358 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f40d1300 con 0x7fb0f415c9b0 2023-11-28T13:08:53.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.345+0000 7fb0f37fe700 1 --2- 172.21.15.156:0/2638160358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f415c9b0 0x7fb0f40b4bf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb0f408c4d0 tx=0x7fb0e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166ecfea306d71f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:53.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.345+0000 7fb0f2ffd700 1 -- 172.21.15.156:0/2638160358 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e400b870 con 0x7fb0f415c9b0 2023-11-28T13:08:53.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.345+0000 7fb0f2ffd700 1 -- 172.21.15.156:0/2638160358 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e400b9d0 con 0x7fb0f415c9b0 2023-11-28T13:08:53.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.345+0000 7fb0f2ffd700 1 -- 172.21.15.156:0/2638160358 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e400bd60 con 0x7fb0f415c9b0 2023-11-28T13:08:53.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.345+0000 7fb0f9bff700 1 -- 172.21.15.156:0/2638160358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f415c9b0 msgr2=0x7fb0f40b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:53.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.345+0000 7fb0f9bff700 1 --2- 172.21.15.156:0/2638160358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f415c9b0 0x7fb0f40b4bf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb0f408c4d0 tx=0x7fb0e4009bd0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.345+0000 7fb0f9bff700 1 -- 172.21.15.156:0/2638160358 shutdown_connections 2023-11-28T13:08:53.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.345+0000 7fb0f9bff700 1 --2- 172.21.15.156:0/2638160358 >> v2:172.21.15.186:3300/0 conn(0x7fb0f40d2320 0x7fb0f40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.345+0000 7fb0f9bff700 1 --2- 172.21.15.156:0/2638160358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f415c9b0 0x7fb0f40b4bf0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.345+0000 7fb0f9bff700 1 --2- 172.21.15.156:0/2638160358 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0f4161340 0x7fb0f40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.345+0000 7fb0f9bff700 1 -- 172.21.15.156:0/2638160358 >> 172.21.15.156:0/2638160358 conn(0x7fb0f4072600 msgr2=0x7fb0f40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:53.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.345+0000 7fb0f9bff700 1 -- 172.21.15.156:0/2638160358 shutdown_connections 2023-11-28T13:08:53.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.346+0000 7fb0f9bff700 1 -- 172.21.15.156:0/2638160358 wait complete. 2023-11-28T13:08:53.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.347+0000 7fb0f9bff700 1 Processor -- start 2023-11-28T13:08:53.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.347+0000 7fb0f9bff700 1 -- start start 2023-11-28T13:08:53.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.348+0000 7fb0f9bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f40d2320 0x7fb0f40c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:53.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.348+0000 7fb0f9bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0f415c9b0 0x7fb0f415ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:53.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.348+0000 7fb0f9bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0f4161340 0x7fb0f415f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:53.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.348+0000 7fb0f9bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f4146f10 con 0x7fb0f40d2320 2023-11-28T13:08:53.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.348+0000 7fb0f9bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f41471f0 con 0x7fb0f415c9b0 2023-11-28T13:08:53.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.348+0000 7fb0f9bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f41474d0 con 0x7fb0f4161340 2023-11-28T13:08:53.351 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.348+0000 7fb0f37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f40d2320 0x7fb0f40c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:53.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.349+0000 7fb0f37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f40d2320 0x7fb0f40c62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39932/0 (socket says 172.21.15.156:39932) 2023-11-28T13:08:53.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.349+0000 7fb0f37fe700 1 -- 172.21.15.156:0/975573846 learned_addr learned my addr 172.21.15.156:0/975573846 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:53.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.349+0000 7fb0ebfff700 1 --2- 172.21.15.156:0/975573846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0f415c9b0 0x7fb0f415ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:53.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.349+0000 7fb0f3fff700 1 --2- 172.21.15.156:0/975573846 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0f4161340 0x7fb0f415f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:53.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.349+0000 7fb0f37fe700 1 -- 172.21.15.156:0/975573846 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0f4161340 msgr2=0x7fb0f415f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:53.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.349+0000 7fb0f37fe700 1 --2- 172.21.15.156:0/975573846 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0f4161340 0x7fb0f415f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.349+0000 7fb0f37fe700 1 -- 172.21.15.156:0/975573846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0f415c9b0 msgr2=0x7fb0f415ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:53.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.349+0000 7fb0f37fe700 1 --2- 172.21.15.156:0/975573846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0f415c9b0 0x7fb0f415ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.349+0000 7fb0f37fe700 1 -- 172.21.15.156:0/975573846 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e4009550 con 0x7fb0f40d2320 2023-11-28T13:08:53.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.349+0000 7fb0f37fe700 1 --2- 172.21.15.156:0/975573846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f40d2320 0x7fb0f40c62f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4014040 tx=0x7fb0e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:53.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.349+0000 7fb0f0ff9700 1 -- 172.21.15.156:0/975573846 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4004310 con 0x7fb0f40d2320 2023-11-28T13:08:53.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.349+0000 7fb0f0ff9700 1 -- 172.21.15.156:0/975573846 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e401e070 con 0x7fb0f40d2320 2023-11-28T13:08:53.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.349+0000 7fb0f9bff700 1 -- 172.21.15.156:0/975573846 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f40b5130 con 0x7fb0f40d2320 2023-11-28T13:08:53.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.349+0000 7fb0f0ff9700 1 -- 172.21.15.156:0/975573846 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e40036a0 con 0x7fb0f40d2320 2023-11-28T13:08:53.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.350+0000 7fb0f9bff700 1 -- 172.21.15.156:0/975573846 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f40d15e0 con 0x7fb0f40d2320 2023-11-28T13:08:53.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.351+0000 7fb0f9bff700 1 -- 172.21.15.156:0/975573846 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f40b52c0 con 0x7fb0f40d2320 2023-11-28T13:08:53.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.351+0000 7fb0f0ff9700 1 -- 172.21.15.156:0/975573846 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb0e4029020 con 0x7fb0f40d2320 2023-11-28T13:08:53.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.352+0000 7fb0f0ff9700 1 --2- 172.21.15.156:0/975573846 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0d40682f0 0x7fb0d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:53.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.352+0000 7fb0f0ff9700 1 -- 172.21.15.156:0/975573846 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb0e4086ea0 con 0x7fb0f40d2320 2023-11-28T13:08:53.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.353+0000 7fb0ebfff700 1 --2- 172.21.15.156:0/975573846 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0d40682f0 0x7fb0d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:53.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.353+0000 7fb0ebfff700 1 --2- 172.21.15.156:0/975573846 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0d40682f0 0x7fb0d406a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4154140 tx=0x7fb0dc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:53.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.356+0000 7fb0f0ff9700 1 -- 172.21.15.156:0/975573846 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb0e40536c0 con 0x7fb0f40d2320 2023-11-28T13:08:53.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.544+0000 7fb0f9bff700 1 -- 172.21.15.156:0/975573846 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0f40745c0 con 0x7fb0f40d2320 2023-11-28T13:08:53.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.545+0000 7fb0f0ff9700 1 -- 172.21.15.156:0/975573846 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb0e4056ce0 con 0x7fb0f40d2320 2023-11-28T13:08:53.547 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T13:08:53.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.547+0000 7fb0f9bff700 1 -- 172.21.15.156:0/975573846 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0d40682f0 msgr2=0x7fb0d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:53.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.547+0000 7fb0f9bff700 1 --2- 172.21.15.156:0/975573846 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0d40682f0 0x7fb0d406a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4154140 tx=0x7fb0dc009190 comp rx=0 tx=0).stop 2023-11-28T13:08:53.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.547+0000 7fb0f9bff700 1 -- 172.21.15.156:0/975573846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f40d2320 msgr2=0x7fb0f40c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:53.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.547+0000 7fb0f9bff700 1 --2- 172.21.15.156:0/975573846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f40d2320 0x7fb0f40c62f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4014040 tx=0x7fb0e4004750 comp rx=0 tx=0).stop 2023-11-28T13:08:53.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.547+0000 7fb0f9bff700 1 -- 172.21.15.156:0/975573846 shutdown_connections 2023-11-28T13:08:53.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.547+0000 7fb0f9bff700 1 --2- 172.21.15.156:0/975573846 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0d40682f0 0x7fb0d406a7a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.547+0000 7fb0f9bff700 1 --2- 172.21.15.156:0/975573846 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0f4161340 0x7fb0f415f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.547+0000 7fb0f9bff700 1 --2- 172.21.15.156:0/975573846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f40d2320 0x7fb0f40c62f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.547+0000 7fb0f9bff700 1 --2- 172.21.15.156:0/975573846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0f415c9b0 0x7fb0f415ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.547+0000 7fb0f9bff700 1 -- 172.21.15.156:0/975573846 >> 172.21.15.156:0/975573846 conn(0x7fb0f4072600 msgr2=0x7fb0f415ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:53.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.547+0000 7fb0f9bff700 1 -- 172.21.15.156:0/975573846 shutdown_connections 2023-11-28T13:08:53.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.547+0000 7fb0f9bff700 1 -- 172.21.15.156:0/975573846 wait complete. 2023-11-28T13:08:53.568 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:08:53.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.764+0000 7fb02ddff700 1 Processor -- start 2023-11-28T13:08:53.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.765+0000 7fb02ddff700 1 -- start start 2023-11-28T13:08:53.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.766+0000 7fb02ddff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb028161340 0x7fb0280d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:53.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.766+0000 7fb02ddff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb0280d2320 0x7fb0280cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:53.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.766+0000 7fb02ddff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0280d1950 con 0x7fb02815c9b0 2023-11-28T13:08:53.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.766+0000 7fb02ddff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0280b5800 con 0x7fb0280d2320 2023-11-28T13:08:53.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.766+0000 7fb02ddff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0280b5300 con 0x7fb028161340 2023-11-28T13:08:53.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.766+0000 7fb0277fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb02815c9b0 0x7fb0280b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39542/0 (socket says 172.21.15.156:39542) 2023-11-28T13:08:53.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.766+0000 7fb0277fe700 1 -- 172.21.15.156:0/1585199874 learned_addr learned my addr 172.21.15.156:0/1585199874 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:53.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.766+0000 7fb027fff700 1 --2- 172.21.15.156:0/1585199874 >> v2:172.21.15.156:3300/0 conn(0x7fb0280d2320 0x7fb0280cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:53.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.766+0000 7fb026ffd700 1 --2- 172.21.15.156:0/1585199874 >> v2:172.21.15.186:3300/0 conn(0x7fb028161340 0x7fb0280d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:53.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.766+0000 7fb0267fc700 1 -- 172.21.15.156:0/1585199874 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 677131002 0 0) 0x7fb0280d1950 con 0x7fb02815c9b0 2023-11-28T13:08:53.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.766+0000 7fb027fff700 1 -- 172.21.15.156:0/1585199874 >> v2:172.21.15.186:3300/0 conn(0x7fb028161340 msgr2=0x7fb0280d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:53.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.766+0000 7fb027fff700 1 --2- 172.21.15.156:0/1585199874 >> v2:172.21.15.186:3300/0 conn(0x7fb028161340 0x7fb0280d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.766+0000 7fb027fff700 1 -- 172.21.15.156:0/1585199874 >> v1:172.21.15.156:6789/0 conn(0x7fb02815c9b0 legacy=0x7fb0280b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:53.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.766+0000 7fb027fff700 1 -- 172.21.15.156:0/1585199874 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0280b5520 con 0x7fb0280d2320 2023-11-28T13:08:53.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.767+0000 7fb027fff700 1 --2- 172.21.15.156:0/1585199874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0280d2320 0x7fb0280cd7c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb0280685c0 tx=0x7fb02000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ff5a0988e53529 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:53.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.767+0000 7fb0267fc700 1 -- 172.21.15.156:0/1585199874 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02000e070 con 0x7fb0280d2320 2023-11-28T13:08:53.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.767+0000 7fb0267fc700 1 -- 172.21.15.156:0/1585199874 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02000bb90 con 0x7fb0280d2320 2023-11-28T13:08:53.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.767+0000 7fb0267fc700 1 -- 172.21.15.156:0/1585199874 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020012430 con 0x7fb0280d2320 2023-11-28T13:08:53.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.767+0000 7fb02ddff700 1 -- 172.21.15.156:0/1585199874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0280d2320 msgr2=0x7fb0280cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:53.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.767+0000 7fb02ddff700 1 --2- 172.21.15.156:0/1585199874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0280d2320 0x7fb0280cd7c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb0280685c0 tx=0x7fb02000b330 comp rx=0 tx=0).stop 2023-11-28T13:08:53.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.767+0000 7fb02ddff700 1 -- 172.21.15.156:0/1585199874 shutdown_connections 2023-11-28T13:08:53.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.767+0000 7fb02ddff700 1 --2- 172.21.15.156:0/1585199874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0280d2320 0x7fb0280cd7c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.767+0000 7fb02ddff700 1 --2- 172.21.15.156:0/1585199874 >> v2:172.21.15.186:3300/0 conn(0x7fb028161340 0x7fb0280d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.767+0000 7fb02ddff700 1 -- 172.21.15.156:0/1585199874 >> 172.21.15.156:0/1585199874 conn(0x7fb028072600 msgr2=0x7fb0280cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:53.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.768+0000 7fb02ddff700 1 -- 172.21.15.156:0/1585199874 shutdown_connections 2023-11-28T13:08:53.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.768+0000 7fb02ddff700 1 -- 172.21.15.156:0/1585199874 wait complete. 2023-11-28T13:08:53.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.769+0000 7fb02ddff700 1 Processor -- start 2023-11-28T13:08:53.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.769+0000 7fb02ddff700 1 -- start start 2023-11-28T13:08:53.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.770+0000 7fb02ddff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0280d2320 0x7fb028146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:53.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.770+0000 7fb02ddff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb02815c9b0 0x7fb028151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:53.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.770+0000 7fb02ddff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb028161340 0x7fb028157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:53.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.770+0000 7fb02ddff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb028157d40 con 0x7fb028161340 2023-11-28T13:08:53.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.770+0000 7fb02ddff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb028156760 con 0x7fb02815c9b0 2023-11-28T13:08:53.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.770+0000 7fb02ddff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb028156a40 con 0x7fb0280d2320 2023-11-28T13:08:53.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.771+0000 7fb027fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb028161340 0x7fb028157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:53.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.771+0000 7fb027fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb028161340 0x7fb028157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39952/0 (socket says 172.21.15.156:39952) 2023-11-28T13:08:53.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.771+0000 7fb027fff700 1 -- 172.21.15.156:0/436403559 learned_addr learned my addr 172.21.15.156:0/436403559 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:53.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.771+0000 7fb026ffd700 1 --2- 172.21.15.156:0/436403559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb02815c9b0 0x7fb028151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:53.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.771+0000 7fb0277fe700 1 --2- 172.21.15.156:0/436403559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0280d2320 0x7fb028146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:53.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.771+0000 7fb027fff700 1 -- 172.21.15.156:0/436403559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0280d2320 msgr2=0x7fb028146540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:53.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.771+0000 7fb027fff700 1 --2- 172.21.15.156:0/436403559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0280d2320 0x7fb028146540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.771+0000 7fb027fff700 1 -- 172.21.15.156:0/436403559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb02815c9b0 msgr2=0x7fb028151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:53.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.771+0000 7fb027fff700 1 --2- 172.21.15.156:0/436403559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb02815c9b0 0x7fb028151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:53.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.771+0000 7fb027fff700 1 -- 172.21.15.156:0/436403559 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb020009a70 con 0x7fb028161340 2023-11-28T13:08:53.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.771+0000 7fb027fff700 1 --2- 172.21.15.156:0/436403559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb028161340 0x7fb028157800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb020014040 tx=0x7fb020004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:53.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.772+0000 7fb00ffff700 1 -- 172.21.15.156:0/436403559 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020012430 con 0x7fb028161340 2023-11-28T13:08:53.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.772+0000 7fb00ffff700 1 -- 172.21.15.156:0/436403559 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb020020070 con 0x7fb028161340 2023-11-28T13:08:53.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.772+0000 7fb02ddff700 1 -- 172.21.15.156:0/436403559 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0280d15e0 con 0x7fb028161340 2023-11-28T13:08:53.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.772+0000 7fb00ffff700 1 -- 172.21.15.156:0/436403559 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020004280 con 0x7fb028161340 2023-11-28T13:08:53.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.772+0000 7fb02ddff700 1 -- 172.21.15.156:0/436403559 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0280d1300 con 0x7fb028161340 2023-11-28T13:08:53.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.773+0000 7fb02ddff700 1 -- 172.21.15.156:0/436403559 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02815cd80 con 0x7fb028161340 2023-11-28T13:08:53.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.774+0000 7fb00ffff700 1 -- 172.21.15.156:0/436403559 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb0200043e0 con 0x7fb028161340 2023-11-28T13:08:53.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.775+0000 7fb00ffff700 1 --2- 172.21.15.156:0/436403559 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0140682f0 0x7fb01406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:53.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.775+0000 7fb00ffff700 1 -- 172.21.15.156:0/436403559 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb020022070 con 0x7fb028161340 2023-11-28T13:08:53.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.775+0000 7fb0277fe700 1 --2- 172.21.15.156:0/436403559 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0140682f0 0x7fb01406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:53.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.776+0000 7fb0277fe700 1 --2- 172.21.15.156:0/436403559 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0140682f0 0x7fb01406a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb018000f60 tx=0x7fb018007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:53.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:53.779+0000 7fb00ffff700 1 -- 172.21.15.156:0/436403559 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb0200548b0 con 0x7fb028161340 2023-11-28T13:08:54.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.012+0000 7fb02ddff700 1 -- 172.21.15.156:0/436403559 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0280745c0 con 0x7fb028161340 2023-11-28T13:08:54.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.012+0000 7fb00ffff700 1 -- 172.21.15.156:0/436403559 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4651 (secure 0 0 0) 0x7fb020057ed0 con 0x7fb028161340 2023-11-28T13:08:54.014 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:54.014 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1283906891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1283906891},{"type":"v1","addr":"172.21.15.156:6835","nonce":1283906891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1738881422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1738881422},{"type":"v1","addr":"172.21.15.156:6833","nonce":1738881422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2235657629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2235657629},{"type":"v1","addr":"172.21.15.186:6836","nonce":2235657629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:08:03.058116+0000","modified":"2023-11-28T13:08:06.077600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/402651964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":402651964},{"type":"v1","addr":"172.21.15.186:6837","nonce":402651964}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:08:54.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.015+0000 7fb02ddff700 1 -- 172.21.15.156:0/436403559 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0140682f0 msgr2=0x7fb01406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:54.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.015+0000 7fb02ddff700 1 --2- 172.21.15.156:0/436403559 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0140682f0 0x7fb01406a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb018000f60 tx=0x7fb018007c90 comp rx=0 tx=0).stop 2023-11-28T13:08:54.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.015+0000 7fb02ddff700 1 -- 172.21.15.156:0/436403559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb028161340 msgr2=0x7fb028157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:54.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.015+0000 7fb02ddff700 1 --2- 172.21.15.156:0/436403559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb028161340 0x7fb028157800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb020014040 tx=0x7fb020004ad0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.015+0000 7fb02ddff700 1 -- 172.21.15.156:0/436403559 shutdown_connections 2023-11-28T13:08:54.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.015+0000 7fb02ddff700 1 --2- 172.21.15.156:0/436403559 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0140682f0 0x7fb01406a7a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.015+0000 7fb02ddff700 1 --2- 172.21.15.156:0/436403559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb028161340 0x7fb028157800 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.015+0000 7fb02ddff700 1 --2- 172.21.15.156:0/436403559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0280d2320 0x7fb028146540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.015+0000 7fb02ddff700 1 --2- 172.21.15.156:0/436403559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb02815c9b0 0x7fb028151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.015+0000 7fb02ddff700 1 -- 172.21.15.156:0/436403559 >> 172.21.15.156:0/436403559 conn(0x7fb028072600 msgr2=0x7fb02815f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:54.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.015+0000 7fb02ddff700 1 -- 172.21.15.156:0/436403559 shutdown_connections 2023-11-28T13:08:54.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.015+0000 7fb02ddff700 1 -- 172.21.15.156:0/436403559 wait complete. 2023-11-28T13:08:54.018 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 8 2023-11-28T13:08:54.035 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:08:54.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.230+0000 7f1a38dff700 1 Processor -- start 2023-11-28T13:08:54.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.231+0000 7f1a38dff700 1 -- start start 2023-11-28T13:08:54.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.232+0000 7f1a38dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a3415c9b0 0x7f1a340b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:54.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.232+0000 7f1a38dff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1a34161340 0x7f1a340d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:54.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.232+0000 7f1a38dff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a340d1950 con 0x7f1a340d2320 2023-11-28T13:08:54.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.232+0000 7f1a38dff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a340b5800 con 0x7f1a34161340 2023-11-28T13:08:54.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.232+0000 7f1a38dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340b5520 con 0x7f1a3415c9b0 2023-11-28T13:08:54.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.232+0000 7f1a2bfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1a34161340 0x7f1a340d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:54.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.232+0000 7f1a32d9d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f1a340d2320 0x7f1a340cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39558/0 (socket says 172.21.15.156:39558) 2023-11-28T13:08:54.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.232+0000 7f1a32d9d700 1 -- 172.21.15.156:0/1337546110 learned_addr learned my addr 172.21.15.156:0/1337546110 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:54.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.232+0000 7f1a3259c700 1 --2- 172.21.15.156:0/1337546110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a3415c9b0 0x7f1a340b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:54.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.232+0000 7f1a2bfff700 1 -- 172.21.15.156:0/1337546110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a3415c9b0 msgr2=0x7f1a340b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:54.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.232+0000 7f1a2bfff700 1 --2- 172.21.15.156:0/1337546110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a3415c9b0 0x7f1a340b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.232+0000 7f1a2bfff700 1 -- 172.21.15.156:0/1337546110 >> v1:172.21.15.156:6789/0 conn(0x7f1a340d2320 legacy=0x7f1a340cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:54.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.233+0000 7f1a2bfff700 1 -- 172.21.15.156:0/1337546110 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a340d1300 con 0x7f1a34161340 2023-11-28T13:08:54.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.233+0000 7f1a3259c700 1 --2- 172.21.15.156:0/1337546110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a3415c9b0 0x7f1a340b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:08:54.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.233+0000 7f1a31d9b700 1 -- 172.21.15.156:0/1337546110 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1246092866 0 0) 0x7f1a340d1950 con 0x7f1a340d2320 2023-11-28T13:08:54.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.233+0000 7f1a2bfff700 1 --2- 172.21.15.156:0/1337546110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a34161340 0x7f1a340d1de0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009840 tx=0x7f1a1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a13769b3f757a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:54.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.233+0000 7f1a31d9b700 1 -- 172.21.15.156:0/1337546110 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c011830 con 0x7f1a34161340 2023-11-28T13:08:54.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.233+0000 7f1a31d9b700 1 -- 172.21.15.156:0/1337546110 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c011990 con 0x7f1a34161340 2023-11-28T13:08:54.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.233+0000 7f1a31d9b700 1 -- 172.21.15.156:0/1337546110 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c011d20 con 0x7f1a34161340 2023-11-28T13:08:54.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.233+0000 7f1a38dff700 1 -- 172.21.15.156:0/1337546110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a34161340 msgr2=0x7f1a340d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:54.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.233+0000 7f1a38dff700 1 --2- 172.21.15.156:0/1337546110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a34161340 0x7f1a340d1de0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009840 tx=0x7f1a1c009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:54.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.234+0000 7f1a38dff700 1 -- 172.21.15.156:0/1337546110 shutdown_connections 2023-11-28T13:08:54.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.234+0000 7f1a38dff700 1 --2- 172.21.15.156:0/1337546110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a3415c9b0 0x7f1a340b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.234+0000 7f1a38dff700 1 --2- 172.21.15.156:0/1337546110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a34161340 0x7f1a340d1de0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.234+0000 7f1a38dff700 1 -- 172.21.15.156:0/1337546110 >> 172.21.15.156:0/1337546110 conn(0x7f1a34072600 msgr2=0x7f1a340cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:54.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.234+0000 7f1a38dff700 1 -- 172.21.15.156:0/1337546110 shutdown_connections 2023-11-28T13:08:54.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.234+0000 7f1a38dff700 1 -- 172.21.15.156:0/1337546110 wait complete. 2023-11-28T13:08:54.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.235+0000 7f1a38dff700 1 Processor -- start 2023-11-28T13:08:54.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.236+0000 7f1a38dff700 1 -- start start 2023-11-28T13:08:54.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.236+0000 7f1a38dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a340d2320 0x7f1a3414c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:54.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.236+0000 7f1a38dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a3415c9b0 0x7f1a340cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:54.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.236+0000 7f1a38dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a34161340 0x7f1a340ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:54.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.236+0000 7f1a38dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340cd460 con 0x7f1a340d2320 2023-11-28T13:08:54.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.236+0000 7f1a38dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340cd740 con 0x7f1a34161340 2023-11-28T13:08:54.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.236+0000 7f1a38dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340cda20 con 0x7f1a3415c9b0 2023-11-28T13:08:54.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.237+0000 7f1a3259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a340d2320 0x7f1a3414c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:54.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.237+0000 7f1a3259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a340d2320 0x7f1a3414c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39978/0 (socket says 172.21.15.156:39978) 2023-11-28T13:08:54.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.237+0000 7f1a3259c700 1 -- 172.21.15.156:0/1233502080 learned_addr learned my addr 172.21.15.156:0/1233502080 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:54.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.237+0000 7f1a32d9d700 1 --2- 172.21.15.156:0/1233502080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a34161340 0x7f1a340ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:54.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.237+0000 7f1a2bfff700 1 --2- 172.21.15.156:0/1233502080 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a3415c9b0 0x7f1a340cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:54.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.237+0000 7f1a3259c700 1 -- 172.21.15.156:0/1233502080 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a3415c9b0 msgr2=0x7f1a340cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:54.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.237+0000 7f1a3259c700 1 --2- 172.21.15.156:0/1233502080 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a3415c9b0 0x7f1a340cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.237+0000 7f1a3259c700 1 -- 172.21.15.156:0/1233502080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a34161340 msgr2=0x7f1a340ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:54.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.237+0000 7f1a3259c700 1 --2- 172.21.15.156:0/1233502080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a34161340 0x7f1a340ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.237+0000 7f1a3259c700 1 -- 172.21.15.156:0/1233502080 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1c0094a0 con 0x7f1a340d2320 2023-11-28T13:08:54.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.237+0000 7f1a3259c700 1 --2- 172.21.15.156:0/1233502080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a340d2320 0x7f1a3414c470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400e810 tx=0x7f1a2400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:54.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.237+0000 7f1a2affd700 1 -- 172.21.15.156:0/1233502080 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24013610 con 0x7f1a340d2320 2023-11-28T13:08:54.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.237+0000 7f1a2affd700 1 -- 172.21.15.156:0/1233502080 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a24013770 con 0x7f1a340d2320 2023-11-28T13:08:54.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.238+0000 7f1a38dff700 1 -- 172.21.15.156:0/1233502080 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a340b5130 con 0x7f1a340d2320 2023-11-28T13:08:54.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.238+0000 7f1a2affd700 1 -- 172.21.15.156:0/1233502080 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24013b20 con 0x7f1a340d2320 2023-11-28T13:08:54.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.238+0000 7f1a38dff700 1 -- 172.21.15.156:0/1233502080 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a340d15e0 con 0x7f1a340d2320 2023-11-28T13:08:54.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.239+0000 7f1a38dff700 1 -- 172.21.15.156:0/1233502080 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a340b52c0 con 0x7f1a340d2320 2023-11-28T13:08:54.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.239+0000 7f1a2affd700 1 -- 172.21.15.156:0/1233502080 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1a24023070 con 0x7f1a340d2320 2023-11-28T13:08:54.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.240+0000 7f1a2affd700 1 --2- 172.21.15.156:0/1233502080 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a140682f0 0x7f1a1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:54.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.240+0000 7f1a2affd700 1 -- 172.21.15.156:0/1233502080 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1a24085a70 con 0x7f1a340d2320 2023-11-28T13:08:54.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.241+0000 7f1a2bfff700 1 --2- 172.21.15.156:0/1233502080 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a140682f0 0x7f1a1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:54.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.241+0000 7f1a2bfff700 1 --2- 172.21.15.156:0/1233502080 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a140682f0 0x7f1a1406a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0090b0 tx=0x7f1a1c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:54.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.244+0000 7f1a2affd700 1 -- 172.21.15.156:0/1233502080 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1a240521a0 con 0x7f1a340d2320 2023-11-28T13:08:54.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.437+0000 7f1a38dff700 1 -- 172.21.15.156:0/1233502080 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a340b9150 con 0x7f1a340d2320 2023-11-28T13:08:54.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.439+0000 7f1a2affd700 1 -- 172.21.15.156:0/1233502080 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17993 (secure 0 0 0) 0x7f1a240557c0 con 0x7f1a340d2320 2023-11-28T13:08:54.440 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:54.440 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:08:51.536161+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:08:00.903485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:08:02.037854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1c","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:08:54.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.441+0000 7f1a38dff700 1 -- 172.21.15.156:0/1233502080 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a140682f0 msgr2=0x7f1a1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:54.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.441+0000 7f1a38dff700 1 --2- 172.21.15.156:0/1233502080 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a140682f0 0x7f1a1406a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0090b0 tx=0x7f1a1c008030 comp rx=0 tx=0).stop 2023-11-28T13:08:54.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.441+0000 7f1a38dff700 1 -- 172.21.15.156:0/1233502080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a340d2320 msgr2=0x7f1a3414c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:54.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.441+0000 7f1a38dff700 1 --2- 172.21.15.156:0/1233502080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a340d2320 0x7f1a3414c470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400e810 tx=0x7f1a2400ebd0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.441+0000 7f1a38dff700 1 -- 172.21.15.156:0/1233502080 shutdown_connections 2023-11-28T13:08:54.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.441+0000 7f1a38dff700 1 --2- 172.21.15.156:0/1233502080 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a140682f0 0x7f1a1406a7a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.441+0000 7f1a38dff700 1 --2- 172.21.15.156:0/1233502080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a34161340 0x7f1a340ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.441+0000 7f1a38dff700 1 --2- 172.21.15.156:0/1233502080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a340d2320 0x7f1a3414c470 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.441+0000 7f1a38dff700 1 --2- 172.21.15.156:0/1233502080 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a3415c9b0 0x7f1a340cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.441+0000 7f1a38dff700 1 -- 172.21.15.156:0/1233502080 >> 172.21.15.156:0/1233502080 conn(0x7f1a34072600 msgr2=0x7f1a34160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:54.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.441+0000 7f1a38dff700 1 -- 172.21.15.156:0/1233502080 shutdown_connections 2023-11-28T13:08:54.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.441+0000 7f1a38dff700 1 -- 172.21.15.156:0/1233502080 wait complete. 2023-11-28T13:08:54.461 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T13:08:54.462 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:08:54.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.658+0000 7f13245ff700 1 Processor -- start 2023-11-28T13:08:54.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.658+0000 7f13245ff700 1 -- start start 2023-11-28T13:08:54.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.659+0000 7f13245ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f131c161340 0x7f131c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:54.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.659+0000 7f13245ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f131c0d1950 con 0x7f131c15c9b0 2023-11-28T13:08:54.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.659+0000 7f13245ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f131c0b5520 con 0x7f131c0d2320 2023-11-28T13:08:54.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.659+0000 7f13245ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131c0b5800 con 0x7f131c161340 2023-11-28T13:08:54.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.660+0000 7f132239b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f131c15c9b0 0x7f131c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39566/0 (socket says 172.21.15.156:39566) 2023-11-28T13:08:54.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.660+0000 7f132239b700 1 -- 172.21.15.156:0/2317369632 learned_addr learned my addr 172.21.15.156:0/2317369632 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:54.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.660+0000 7f1321b9a700 1 --2- 172.21.15.156:0/2317369632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f131c161340 0x7f131c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:54.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.660+0000 7f1321399700 1 -- 172.21.15.156:0/2317369632 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3160763368 0 0) 0x7f131c0d1950 con 0x7f131c15c9b0 2023-11-28T13:08:54.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.660+0000 7f1321399700 1 -- 172.21.15.156:0/2317369632 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13080035b0 con 0x7f131c15c9b0 2023-11-28T13:08:54.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.660+0000 7f1321399700 1 -- 172.21.15.156:0/2317369632 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4245166399 0 0) 0x7f13080035b0 con 0x7f131c15c9b0 2023-11-28T13:08:54.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.660+0000 7f1321399700 1 -- 172.21.15.156:0/2317369632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f131c161340 msgr2=0x7f131c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:54.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.660+0000 7f1321399700 1 --2- 172.21.15.156:0/2317369632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f131c161340 0x7f131c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.660+0000 7f1321399700 1 -- 172.21.15.156:0/2317369632 >> v1:172.21.15.186:6789/0 conn(0x7f131c0d2320 legacy=0x7f131c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:54.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.660+0000 7f1321399700 1 -- 172.21.15.156:0/2317369632 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131c0d1300 con 0x7f131c15c9b0 2023-11-28T13:08:54.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.660+0000 7f1321399700 1 -- 172.21.15.156:0/2317369632 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f1318001670 con 0x7f131c15c9b0 2023-11-28T13:08:54.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.661+0000 7f1321399700 1 -- 172.21.15.156:0/2317369632 >> v1:172.21.15.156:6789/0 conn(0x7f131c15c9b0 legacy=0x7f131c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:54.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.661+0000 7f1321399700 1 --2- 172.21.15.156:0/2317369632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1308003a30 0x7f1308003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:54.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.661+0000 7f1321399700 1 --2- 172.21.15.156:0/2317369632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13080043d0 0x7f1308004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:54.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.661+0000 7f1321399700 1 --2- 172.21.15.156:0/2317369632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f131c161340 0x7f1308005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:54.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.661+0000 7f1321399700 1 -- 172.21.15.156:0/2317369632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c0b5800 con 0x7f13080043d0 2023-11-28T13:08:54.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.661+0000 7f1321399700 1 -- 172.21.15.156:0/2317369632 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130800bac0 con 0x7f1308003a30 2023-11-28T13:08:54.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.661+0000 7f1321399700 1 -- 172.21.15.156:0/2317369632 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130800bc50 con 0x7f131c161340 2023-11-28T13:08:54.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.661+0000 7f1321b9a700 1 --2- 172.21.15.156:0/2317369632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13080043d0 0x7f1308004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:54.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.661+0000 7f132239b700 1 --2- 172.21.15.156:0/2317369632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1308003a30 0x7f1308003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:54.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.661+0000 7f1322b9c700 1 --2- 172.21.15.156:0/2317369632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f131c161340 0x7f1308005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:54.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.661+0000 7f1321b9a700 1 -- 172.21.15.156:0/2317369632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f131c161340 msgr2=0x7f1308005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:54.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.661+0000 7f1321b9a700 1 --2- 172.21.15.156:0/2317369632 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f131c161340 0x7f1308005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.661+0000 7f1321b9a700 1 -- 172.21.15.156:0/2317369632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1308003a30 msgr2=0x7f1308003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:54.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.661+0000 7f1321b9a700 1 --2- 172.21.15.156:0/2317369632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1308003a30 0x7f1308003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.661+0000 7f1321b9a700 1 -- 172.21.15.156:0/2317369632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13080035b0 con 0x7f13080043d0 2023-11-28T13:08:54.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.661+0000 7f1321b9a700 1 --2- 172.21.15.156:0/2317369632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13080043d0 0x7f1308004890 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f130c009b90 tx=0x7f1308005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38312be1e7ac1244 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:54.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.662+0000 7f1321399700 1 -- 172.21.15.156:0/2317369632 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c00e6c0 con 0x7f13080043d0 2023-11-28T13:08:54.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.662+0000 7f1321399700 1 -- 172.21.15.156:0/2317369632 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130c00e820 con 0x7f13080043d0 2023-11-28T13:08:54.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.662+0000 7f1321399700 1 -- 172.21.15.156:0/2317369632 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c00eb90 con 0x7f13080043d0 2023-11-28T13:08:54.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.662+0000 7f13245ff700 1 -- 172.21.15.156:0/2317369632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13080043d0 msgr2=0x7f1308004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:54.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.662+0000 7f13245ff700 1 --2- 172.21.15.156:0/2317369632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13080043d0 0x7f1308004890 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f130c009b90 tx=0x7f1308005880 comp rx=0 tx=0).stop 2023-11-28T13:08:54.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.662+0000 7f1322b9c700 1 -- 172.21.15.156:0/2317369632 reap_dead start 2023-11-28T13:08:54.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.662+0000 7f13245ff700 1 -- 172.21.15.156:0/2317369632 shutdown_connections 2023-11-28T13:08:54.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.662+0000 7f13245ff700 1 -- 172.21.15.156:0/2317369632 >> 172.21.15.156:0/2317369632 conn(0x7f131c072600 msgr2=0x7f131c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:54.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.662+0000 7f13245ff700 1 -- 172.21.15.156:0/2317369632 shutdown_connections 2023-11-28T13:08:54.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.662+0000 7f13245ff700 1 -- 172.21.15.156:0/2317369632 wait complete. 2023-11-28T13:08:54.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.664+0000 7f13245ff700 1 Processor -- start 2023-11-28T13:08:54.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.664+0000 7f13245ff700 1 -- start start 2023-11-28T13:08:54.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.665+0000 7f13245ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13080043d0 0x7f131c1e6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:54.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.665+0000 7f13245ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f131c0c6ca0 0x7f131c157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:54.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.665+0000 7f13245ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f131c15e630 0x7f131c15eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:54.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.665+0000 7f13245ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c15efe0 con 0x7f131c15e630 2023-11-28T13:08:54.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.665+0000 7f13245ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c1470c0 con 0x7f13080043d0 2023-11-28T13:08:54.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.665+0000 7f13245ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131c0c9d00 con 0x7f131c0c6ca0 2023-11-28T13:08:54.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.665+0000 7f1322b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f131c15e630 0x7f131c15eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:54.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.665+0000 7f1322b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f131c15e630 0x7f131c15eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:40000/0 (socket says 172.21.15.156:40000) 2023-11-28T13:08:54.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.665+0000 7f1322b9c700 1 -- 172.21.15.156:0/4077456518 learned_addr learned my addr 172.21.15.156:0/4077456518 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:54.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.665+0000 7f132239b700 1 --2- 172.21.15.156:0/4077456518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13080043d0 0x7f131c1e6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:54.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.666+0000 7f1321b9a700 1 --2- 172.21.15.156:0/4077456518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f131c0c6ca0 0x7f131c157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:54.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.666+0000 7f1322b9c700 1 -- 172.21.15.156:0/4077456518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f131c0c6ca0 msgr2=0x7f131c157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:54.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.666+0000 7f1322b9c700 1 --2- 172.21.15.156:0/4077456518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f131c0c6ca0 0x7f131c157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.666+0000 7f1322b9c700 1 -- 172.21.15.156:0/4077456518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13080043d0 msgr2=0x7f131c1e6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:54.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.666+0000 7f1322b9c700 1 --2- 172.21.15.156:0/4077456518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13080043d0 0x7f131c1e6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.666+0000 7f1322b9c700 1 -- 172.21.15.156:0/4077456518 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131c15cd80 con 0x7f131c15e630 2023-11-28T13:08:54.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.666+0000 7f1322b9c700 1 --2- 172.21.15.156:0/4077456518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f131c15e630 0x7f131c15eaa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f131c0d05a0 tx=0x7f131400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:54.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.666+0000 7f1312ffd700 1 -- 172.21.15.156:0/4077456518 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1314003c10 con 0x7f131c15e630 2023-11-28T13:08:54.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.666+0000 7f1312ffd700 1 -- 172.21.15.156:0/4077456518 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1314003d70 con 0x7f131c15e630 2023-11-28T13:08:54.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.666+0000 7f1312ffd700 1 -- 172.21.15.156:0/4077456518 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1314005310 con 0x7f131c15e630 2023-11-28T13:08:54.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.666+0000 7f13245ff700 1 -- 172.21.15.156:0/4077456518 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131c0d15e0 con 0x7f131c15e630 2023-11-28T13:08:54.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.667+0000 7f13245ff700 1 -- 172.21.15.156:0/4077456518 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131c15cf10 con 0x7f131c15e630 2023-11-28T13:08:54.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.667+0000 7f13245ff700 1 -- 172.21.15.156:0/4077456518 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131c147250 con 0x7f131c15e630 2023-11-28T13:08:54.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.668+0000 7f1312ffd700 1 -- 172.21.15.156:0/4077456518 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1314005470 con 0x7f131c15e630 2023-11-28T13:08:54.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.669+0000 7f1312ffd700 1 --2- 172.21.15.156:0/4077456518 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f13000682f0 0x7f130006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:54.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.669+0000 7f1312ffd700 1 -- 172.21.15.156:0/4077456518 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1314085f80 con 0x7f131c15e630 2023-11-28T13:08:54.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.670+0000 7f132239b700 1 --2- 172.21.15.156:0/4077456518 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f13000682f0 0x7f130006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:54.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.670+0000 7f132239b700 1 --2- 172.21.15.156:0/4077456518 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f13000682f0 0x7f130006a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1318000c00 tx=0x7f1318000f80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:54.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.673+0000 7f1312ffd700 1 -- 172.21.15.156:0/4077456518 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f13140526b0 con 0x7f131c15e630 2023-11-28T13:08:54.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.908+0000 7f13245ff700 1 -- 172.21.15.156:0/4077456518 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f131c0745c0 con 0x7f131c15e630 2023-11-28T13:08:54.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.909+0000 7f1312ffd700 1 -- 172.21.15.156:0/4077456518 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4651 (secure 0 0 0) 0x7f1314055cd0 con 0x7f131c15e630 2023-11-28T13:08:54.910 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:54.910 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1283906891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1283906891},{"type":"v1","addr":"172.21.15.156:6835","nonce":1283906891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1738881422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1738881422},{"type":"v1","addr":"172.21.15.156:6833","nonce":1738881422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2235657629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2235657629},{"type":"v1","addr":"172.21.15.186:6836","nonce":2235657629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:08:03.058116+0000","modified":"2023-11-28T13:08:06.077600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/402651964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":402651964},{"type":"v1","addr":"172.21.15.186:6837","nonce":402651964}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:08:54.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.911+0000 7f13245ff700 1 -- 172.21.15.156:0/4077456518 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f13000682f0 msgr2=0x7f130006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:54.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.911+0000 7f13245ff700 1 --2- 172.21.15.156:0/4077456518 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f13000682f0 0x7f130006a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1318000c00 tx=0x7f1318000f80 comp rx=0 tx=0).stop 2023-11-28T13:08:54.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.911+0000 7f13245ff700 1 -- 172.21.15.156:0/4077456518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f131c15e630 msgr2=0x7f131c15eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:54.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.911+0000 7f13245ff700 1 --2- 172.21.15.156:0/4077456518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f131c15e630 0x7f131c15eaa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f131c0d05a0 tx=0x7f131400ee50 comp rx=0 tx=0).stop 2023-11-28T13:08:54.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.911+0000 7f13245ff700 1 -- 172.21.15.156:0/4077456518 shutdown_connections 2023-11-28T13:08:54.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.911+0000 7f13245ff700 1 --2- 172.21.15.156:0/4077456518 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f13000682f0 0x7f130006a7a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.911+0000 7f13245ff700 1 --2- 172.21.15.156:0/4077456518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f131c15e630 0x7f131c15eaa0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.911+0000 7f13245ff700 1 --2- 172.21.15.156:0/4077456518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13080043d0 0x7f131c1e6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.911+0000 7f13245ff700 1 --2- 172.21.15.156:0/4077456518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f131c0c6ca0 0x7f131c157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:54.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.911+0000 7f13245ff700 1 -- 172.21.15.156:0/4077456518 >> 172.21.15.156:0/4077456518 conn(0x7f131c072600 msgr2=0x7f131c0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:54.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.911+0000 7f13245ff700 1 -- 172.21.15.156:0/4077456518 shutdown_connections 2023-11-28T13:08:54.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:54.911+0000 7f13245ff700 1 -- 172.21.15.156:0/4077456518 wait complete. 2023-11-28T13:08:54.914 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 8 2023-11-28T13:08:54.931 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:08:55.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.127+0000 7fa9993ff700 1 Processor -- start 2023-11-28T13:08:55.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.127+0000 7fa9993ff700 1 -- start start 2023-11-28T13:08:55.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.129+0000 7fa9993ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa994161340 0x7fa9940d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:55.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.129+0000 7fa9993ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9940d1950 con 0x7fa9940d2320 2023-11-28T13:08:55.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.129+0000 7fa9993ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9940b5520 con 0x7fa99415c9b0 2023-11-28T13:08:55.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.129+0000 7fa9993ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940b5800 con 0x7fa994161340 2023-11-28T13:08:55.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.129+0000 7fa9937fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa9940d2320 0x7fa9940cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39568/0 (socket says 172.21.15.156:39568) 2023-11-28T13:08:55.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.129+0000 7fa9937fe700 1 -- 172.21.15.156:0/4100567060 learned_addr learned my addr 172.21.15.156:0/4100567060 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:55.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.129+0000 7fa98bfff700 1 --2- 172.21.15.156:0/4100567060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa994161340 0x7fa9940d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:55.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.129+0000 7fa9927fc700 1 -- 172.21.15.156:0/4100567060 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079202257 0 0) 0x7fa9940d1950 con 0x7fa9940d2320 2023-11-28T13:08:55.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.129+0000 7fa9927fc700 1 -- 172.21.15.156:0/4100567060 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9800035b0 con 0x7fa9940d2320 2023-11-28T13:08:55.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.130+0000 7fa9927fc700 1 -- 172.21.15.156:0/4100567060 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4073361664 0 0) 0x7fa9940b5520 con 0x7fa99415c9b0 2023-11-28T13:08:55.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.130+0000 7fa9927fc700 1 -- 172.21.15.156:0/4100567060 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9940d1950 con 0x7fa99415c9b0 2023-11-28T13:08:55.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.130+0000 7fa9927fc700 1 -- 172.21.15.156:0/4100567060 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3234000991 0 0) 0x7fa9800035b0 con 0x7fa9940d2320 2023-11-28T13:08:55.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.130+0000 7fa98bfff700 1 -- 172.21.15.156:0/4100567060 >> v1:172.21.15.186:6789/0 conn(0x7fa99415c9b0 legacy=0x7fa9940b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:55.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.130+0000 7fa98bfff700 1 -- 172.21.15.156:0/4100567060 >> v1:172.21.15.156:6789/0 conn(0x7fa9940d2320 legacy=0x7fa9940cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:55.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.130+0000 7fa98bfff700 1 -- 172.21.15.156:0/4100567060 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9940d1300 con 0x7fa994161340 2023-11-28T13:08:55.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.130+0000 7fa98bfff700 1 --2- 172.21.15.156:0/4100567060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa994161340 0x7fa9940d1de0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009870 tx=0x7fa97c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8894c4199b5c1383 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:55.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.130+0000 7fa9927fc700 1 -- 172.21.15.156:0/4100567060 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0019f0 con 0x7fa994161340 2023-11-28T13:08:55.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.130+0000 7fa9927fc700 1 -- 172.21.15.156:0/4100567060 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c004030 con 0x7fa994161340 2023-11-28T13:08:55.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.130+0000 7fa9927fc700 1 -- 172.21.15.156:0/4100567060 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c0043c0 con 0x7fa994161340 2023-11-28T13:08:55.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.131+0000 7fa9993ff700 1 -- 172.21.15.156:0/4100567060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa994161340 msgr2=0x7fa9940d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:55.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.131+0000 7fa9993ff700 1 --2- 172.21.15.156:0/4100567060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa994161340 0x7fa9940d1de0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009870 tx=0x7fa97c009b80 comp rx=0 tx=0).stop 2023-11-28T13:08:55.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.131+0000 7fa9993ff700 1 -- 172.21.15.156:0/4100567060 shutdown_connections 2023-11-28T13:08:55.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.131+0000 7fa9993ff700 1 --2- 172.21.15.156:0/4100567060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa994161340 0x7fa9940d1de0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:55.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.131+0000 7fa9993ff700 1 -- 172.21.15.156:0/4100567060 >> 172.21.15.156:0/4100567060 conn(0x7fa994072600 msgr2=0x7fa9940cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:55.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.131+0000 7fa9993ff700 1 -- 172.21.15.156:0/4100567060 shutdown_connections 2023-11-28T13:08:55.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.131+0000 7fa9993ff700 1 -- 172.21.15.156:0/4100567060 wait complete. 2023-11-28T13:08:55.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.133+0000 7fa9993ff700 1 Processor -- start 2023-11-28T13:08:55.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.133+0000 7fa9993ff700 1 -- start start 2023-11-28T13:08:55.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.134+0000 7fa9993ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9940d2320 0x7fa994157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:55.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.134+0000 7fa9993ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa99415c9b0 0x7fa9940c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:55.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.134+0000 7fa9993ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa994161340 0x7fa99415a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:55.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.134+0000 7fa9993ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9941e6e00 con 0x7fa9940d2320 2023-11-28T13:08:55.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.134+0000 7fa9993ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940c9150 con 0x7fa994161340 2023-11-28T13:08:55.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.134+0000 7fa9993ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9941e6460 con 0x7fa99415c9b0 2023-11-28T13:08:55.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.134+0000 7fa992ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9940d2320 0x7fa994157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:55.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.134+0000 7fa992ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9940d2320 0x7fa994157150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:40006/0 (socket says 172.21.15.156:40006) 2023-11-28T13:08:55.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.134+0000 7fa992ffd700 1 -- 172.21.15.156:0/2399604353 learned_addr learned my addr 172.21.15.156:0/2399604353 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:55.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.134+0000 7fa98bfff700 1 --2- 172.21.15.156:0/2399604353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa99415c9b0 0x7fa9940c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:55.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.134+0000 7fa9937fe700 1 --2- 172.21.15.156:0/2399604353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa994161340 0x7fa99415a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:55.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.135+0000 7fa992ffd700 1 -- 172.21.15.156:0/2399604353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa99415c9b0 msgr2=0x7fa9940c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:55.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.135+0000 7fa992ffd700 1 --2- 172.21.15.156:0/2399604353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa99415c9b0 0x7fa9940c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:55.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.135+0000 7fa992ffd700 1 -- 172.21.15.156:0/2399604353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa994161340 msgr2=0x7fa99415a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:55.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.135+0000 7fa992ffd700 1 --2- 172.21.15.156:0/2399604353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa994161340 0x7fa99415a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:55.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.135+0000 7fa992ffd700 1 -- 172.21.15.156:0/2399604353 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c0094a0 con 0x7fa9940d2320 2023-11-28T13:08:55.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.135+0000 7fa992ffd700 1 --2- 172.21.15.156:0/2399604353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9940d2320 0x7fa994157150 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c520 tx=0x7fa98400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:55.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.135+0000 7fa98b7fe700 1 -- 172.21.15.156:0/2399604353 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9840052d0 con 0x7fa9940d2320 2023-11-28T13:08:55.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.135+0000 7fa98b7fe700 1 -- 172.21.15.156:0/2399604353 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa984005430 con 0x7fa9940d2320 2023-11-28T13:08:55.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.135+0000 7fa9993ff700 1 -- 172.21.15.156:0/2399604353 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99415cd80 con 0x7fa9940d2320 2023-11-28T13:08:55.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.135+0000 7fa98b7fe700 1 -- 172.21.15.156:0/2399604353 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9840038c0 con 0x7fa9940d2320 2023-11-28T13:08:55.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.135+0000 7fa9993ff700 1 -- 172.21.15.156:0/2399604353 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9940d15e0 con 0x7fa9940d2320 2023-11-28T13:08:55.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.136+0000 7fa9993ff700 1 -- 172.21.15.156:0/2399604353 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99415cf10 con 0x7fa9940d2320 2023-11-28T13:08:55.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.137+0000 7fa98b7fe700 1 -- 172.21.15.156:0/2399604353 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa984019030 con 0x7fa9940d2320 2023-11-28T13:08:55.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.138+0000 7fa98b7fe700 1 --2- 172.21.15.156:0/2399604353 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa9740682f0 0x7fa97406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:55.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.138+0000 7fa98b7fe700 1 -- 172.21.15.156:0/2399604353 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa984084fc0 con 0x7fa9940d2320 2023-11-28T13:08:55.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.138+0000 7fa98bfff700 1 --2- 172.21.15.156:0/2399604353 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa9740682f0 0x7fa97406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:55.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.139+0000 7fa98bfff700 1 --2- 172.21.15.156:0/2399604353 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa9740682f0 0x7fa97406a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0097c0 tx=0x7fa97c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:55.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.142+0000 7fa98b7fe700 1 -- 172.21.15.156:0/2399604353 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa9840517e0 con 0x7fa9940d2320 2023-11-28T13:08:55.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.375+0000 7fa9993ff700 1 -- 172.21.15.156:0/2399604353 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9940745c0 con 0x7fa9940d2320 2023-11-28T13:08:55.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.376+0000 7fa98b7fe700 1 -- 172.21.15.156:0/2399604353 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4651 (secure 0 0 0) 0x7fa984054e00 con 0x7fa9940d2320 2023-11-28T13:08:55.378 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:55.378 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1283906891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1283906891},{"type":"v1","addr":"172.21.15.156:6835","nonce":1283906891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1738881422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1738881422},{"type":"v1","addr":"172.21.15.156:6833","nonce":1738881422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2235657629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2235657629},{"type":"v1","addr":"172.21.15.186:6836","nonce":2235657629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:08:03.058116+0000","modified":"2023-11-28T13:08:06.077600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6837/402651964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":402651964},{"type":"v1","addr":"172.21.15.186:6837","nonce":402651964}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:08:55.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.378+0000 7fa9993ff700 1 -- 172.21.15.156:0/2399604353 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa9740682f0 msgr2=0x7fa97406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:55.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.378+0000 7fa9993ff700 1 --2- 172.21.15.156:0/2399604353 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa9740682f0 0x7fa97406a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0097c0 tx=0x7fa97c008030 comp rx=0 tx=0).stop 2023-11-28T13:08:55.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.378+0000 7fa9993ff700 1 -- 172.21.15.156:0/2399604353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9940d2320 msgr2=0x7fa994157150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:55.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.378+0000 7fa9993ff700 1 --2- 172.21.15.156:0/2399604353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9940d2320 0x7fa994157150 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c520 tx=0x7fa98400c830 comp rx=0 tx=0).stop 2023-11-28T13:08:55.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.378+0000 7fa9993ff700 1 -- 172.21.15.156:0/2399604353 shutdown_connections 2023-11-28T13:08:55.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.378+0000 7fa9993ff700 1 --2- 172.21.15.156:0/2399604353 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa9740682f0 0x7fa97406a7a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:55.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.378+0000 7fa9993ff700 1 --2- 172.21.15.156:0/2399604353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa994161340 0x7fa99415a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:55.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.378+0000 7fa9993ff700 1 --2- 172.21.15.156:0/2399604353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9940d2320 0x7fa994157150 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:55.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.378+0000 7fa9993ff700 1 --2- 172.21.15.156:0/2399604353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa99415c9b0 0x7fa9940c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:55.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.378+0000 7fa9993ff700 1 -- 172.21.15.156:0/2399604353 >> 172.21.15.156:0/2399604353 conn(0x7fa994072600 msgr2=0x7fa99415f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:55.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.379+0000 7fa9993ff700 1 -- 172.21.15.156:0/2399604353 shutdown_connections 2023-11-28T13:08:55.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.379+0000 7fa9993ff700 1 -- 172.21.15.156:0/2399604353 wait complete. 2023-11-28T13:08:55.381 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 8 2023-11-28T13:08:55.398 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:08:55.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.594+0000 7f51a63ff700 1 Processor -- start 2023-11-28T13:08:55.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.594+0000 7f51a63ff700 1 -- start start 2023-11-28T13:08:55.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.595+0000 7f51a63ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f51a015c9b0 0x7f51a00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:55.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.595+0000 7f51a63ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51a0161340 0x7f51a00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:55.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.595+0000 7f51a63ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a00d1950 con 0x7f51a00d2320 2023-11-28T13:08:55.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.595+0000 7f51a63ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a00b5800 con 0x7f51a015c9b0 2023-11-28T13:08:55.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.595+0000 7f51a63ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a00b5520 con 0x7f51a0161340 2023-11-28T13:08:55.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.596+0000 7f51a499c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f51a00d2320 0x7f51a00cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39576/0 (socket says 172.21.15.156:39576) 2023-11-28T13:08:55.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.596+0000 7f51a499c700 1 -- 172.21.15.156:0/2889206287 learned_addr learned my addr 172.21.15.156:0/2889206287 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:55.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.596+0000 7f519f7fe700 1 --2- 172.21.15.156:0/2889206287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51a0161340 0x7f51a00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:55.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.596+0000 7f519ffff700 1 --2- 172.21.15.156:0/2889206287 >> v2:172.21.15.186:3300/0 conn(0x7f51a015c9b0 0x7f51a00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:55.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.596+0000 7f519effd700 1 -- 172.21.15.156:0/2889206287 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056890491 0 0) 0x7f51a00d1950 con 0x7f51a00d2320 2023-11-28T13:08:55.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.596+0000 7f519effd700 1 -- 172.21.15.156:0/2889206287 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51880035b0 con 0x7f51a00d2320 2023-11-28T13:08:55.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.596+0000 7f519f7fe700 1 -- 172.21.15.156:0/2889206287 >> v2:172.21.15.186:3300/0 conn(0x7f51a015c9b0 msgr2=0x7f51a00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:55.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.596+0000 7f519f7fe700 1 --2- 172.21.15.156:0/2889206287 >> v2:172.21.15.186:3300/0 conn(0x7f51a015c9b0 0x7f51a00b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:55.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.596+0000 7f519f7fe700 1 -- 172.21.15.156:0/2889206287 >> v1:172.21.15.156:6789/0 conn(0x7f51a00d2320 legacy=0x7f51a00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:55.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.596+0000 7f519f7fe700 1 -- 172.21.15.156:0/2889206287 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a00d1300 con 0x7f51a0161340 2023-11-28T13:08:55.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.597+0000 7f519f7fe700 1 --2- 172.21.15.156:0/2889206287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51a0161340 0x7f51a00d1de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5194009840 tx=0x7f5194009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d3244cfc9bfcabb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:55.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.597+0000 7f519effd700 1 -- 172.21.15.156:0/2889206287 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194011830 con 0x7f51a0161340 2023-11-28T13:08:55.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.597+0000 7f519effd700 1 -- 172.21.15.156:0/2889206287 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5194011990 con 0x7f51a0161340 2023-11-28T13:08:55.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.597+0000 7f519effd700 1 -- 172.21.15.156:0/2889206287 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194011d00 con 0x7f51a0161340 2023-11-28T13:08:55.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.597+0000 7f51a63ff700 1 -- 172.21.15.156:0/2889206287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51a0161340 msgr2=0x7f51a00d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:55.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.597+0000 7f51a63ff700 1 --2- 172.21.15.156:0/2889206287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51a0161340 0x7f51a00d1de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5194009840 tx=0x7f5194009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:55.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.597+0000 7f51a63ff700 1 -- 172.21.15.156:0/2889206287 shutdown_connections 2023-11-28T13:08:55.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.597+0000 7f51a63ff700 1 --2- 172.21.15.156:0/2889206287 >> v2:172.21.15.186:3300/0 conn(0x7f51a015c9b0 0x7f51a00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:55.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.597+0000 7f51a63ff700 1 --2- 172.21.15.156:0/2889206287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51a0161340 0x7f51a00d1de0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:55.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.597+0000 7f51a63ff700 1 -- 172.21.15.156:0/2889206287 >> 172.21.15.156:0/2889206287 conn(0x7f51a0072600 msgr2=0x7f51a00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:55.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.597+0000 7f51a63ff700 1 -- 172.21.15.156:0/2889206287 shutdown_connections 2023-11-28T13:08:55.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.598+0000 7f51a63ff700 1 -- 172.21.15.156:0/2889206287 wait complete. 2023-11-28T13:08:55.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.599+0000 7f51a63ff700 1 Processor -- start 2023-11-28T13:08:55.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.599+0000 7f51a63ff700 1 -- start start 2023-11-28T13:08:55.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.600+0000 7f51a63ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51a00d2320 0x7f51a0160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:55.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.600+0000 7f51a63ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51a015c9b0 0x7f51a01f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:55.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.600+0000 7f51a63ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51a0161340 0x7f51a00c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:55.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.600+0000 7f51a63ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a0155e30 con 0x7f51a0161340 2023-11-28T13:08:55.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.600+0000 7f51a63ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a0156060 con 0x7f51a015c9b0 2023-11-28T13:08:55.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.600+0000 7f51a63ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a0156340 con 0x7f51a00d2320 2023-11-28T13:08:55.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.600+0000 7f51a499c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51a0161340 0x7f51a00c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:55.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.601+0000 7f51a499c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51a0161340 0x7f51a00c5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:40018/0 (socket says 172.21.15.156:40018) 2023-11-28T13:08:55.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.601+0000 7f51a499c700 1 -- 172.21.15.156:0/816300631 learned_addr learned my addr 172.21.15.156:0/816300631 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:55.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.601+0000 7f519ffff700 1 --2- 172.21.15.156:0/816300631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51a00d2320 0x7f51a0160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:55.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.601+0000 7f519f7fe700 1 --2- 172.21.15.156:0/816300631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51a015c9b0 0x7f51a01f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:55.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.601+0000 7f51a499c700 1 -- 172.21.15.156:0/816300631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51a00d2320 msgr2=0x7f51a0160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:55.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.601+0000 7f51a499c700 1 --2- 172.21.15.156:0/816300631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51a00d2320 0x7f51a0160880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:55.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.601+0000 7f51a499c700 1 -- 172.21.15.156:0/816300631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51a015c9b0 msgr2=0x7f51a01f28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:55.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.601+0000 7f51a499c700 1 --2- 172.21.15.156:0/816300631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51a015c9b0 0x7f51a01f28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:55.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.601+0000 7f51a499c700 1 -- 172.21.15.156:0/816300631 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51940094a0 con 0x7f51a0161340 2023-11-28T13:08:55.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.601+0000 7f51a499c700 1 --2- 172.21.15.156:0/816300631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51a0161340 0x7f51a00c5ac0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f519800caa0 tx=0x7f519800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:55.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.601+0000 7f519cff9700 1 -- 172.21.15.156:0/816300631 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519800a9e0 con 0x7f51a0161340 2023-11-28T13:08:55.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.601+0000 7f519cff9700 1 -- 172.21.15.156:0/816300631 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519800ab40 con 0x7f51a0161340 2023-11-28T13:08:55.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.601+0000 7f51a63ff700 1 -- 172.21.15.156:0/816300631 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a00b5130 con 0x7f51a0161340 2023-11-28T13:08:55.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.601+0000 7f519cff9700 1 -- 172.21.15.156:0/816300631 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198012400 con 0x7f51a0161340 2023-11-28T13:08:55.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.602+0000 7f51a63ff700 1 -- 172.21.15.156:0/816300631 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a00d15e0 con 0x7f51a0161340 2023-11-28T13:08:55.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.603+0000 7f51a63ff700 1 -- 172.21.15.156:0/816300631 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a00b52c0 con 0x7f51a0161340 2023-11-28T13:08:55.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.603+0000 7f519cff9700 1 -- 172.21.15.156:0/816300631 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5198019030 con 0x7f51a0161340 2023-11-28T13:08:55.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.604+0000 7f519cff9700 1 --2- 172.21.15.156:0/816300631 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f518c0682f0 0x7f518c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:55.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.604+0000 7f519cff9700 1 -- 172.21.15.156:0/816300631 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5198085960 con 0x7f51a0161340 2023-11-28T13:08:55.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.605+0000 7f519ffff700 1 --2- 172.21.15.156:0/816300631 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f518c0682f0 0x7f518c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:55.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.605+0000 7f519ffff700 1 --2- 172.21.15.156:0/816300631 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f518c0682f0 0x7f518c06a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5190009b40 tx=0x7f5190009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:55.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.608+0000 7f519cff9700 1 -- 172.21.15.156:0/816300631 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5198052100 con 0x7f51a0161340 2023-11-28T13:08:55.807 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.805+0000 7f51a63ff700 1 -- 172.21.15.156:0/816300631 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51a00745c0 con 0x7f51a0161340 2023-11-28T13:08:55.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.806+0000 7f519cff9700 1 -- 172.21.15.156:0/816300631 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17993 (secure 0 0 0) 0x7f5198055720 con 0x7f51a0161340 2023-11-28T13:08:55.810 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:08:55.810 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:08:51.536161+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:08:00.903485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:08:02.037854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1c","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:08:55.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.809+0000 7f51a63ff700 1 -- 172.21.15.156:0/816300631 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f518c0682f0 msgr2=0x7f518c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:55.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.809+0000 7f51a63ff700 1 --2- 172.21.15.156:0/816300631 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f518c0682f0 0x7f518c06a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5190009b40 tx=0x7f5190009210 comp rx=0 tx=0).stop 2023-11-28T13:08:55.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.809+0000 7f51a63ff700 1 -- 172.21.15.156:0/816300631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51a0161340 msgr2=0x7f51a00c5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:55.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.809+0000 7f51a63ff700 1 --2- 172.21.15.156:0/816300631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51a0161340 0x7f51a00c5ac0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f519800caa0 tx=0x7f519800cdb0 comp rx=0 tx=0).stop 2023-11-28T13:08:55.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.809+0000 7f51a63ff700 1 -- 172.21.15.156:0/816300631 shutdown_connections 2023-11-28T13:08:55.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.809+0000 7f51a63ff700 1 --2- 172.21.15.156:0/816300631 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f518c0682f0 0x7f518c06a7a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:55.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.809+0000 7f51a63ff700 1 --2- 172.21.15.156:0/816300631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51a0161340 0x7f51a00c5ac0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:55.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.809+0000 7f51a63ff700 1 --2- 172.21.15.156:0/816300631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51a00d2320 0x7f51a0160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:55.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.809+0000 7f51a63ff700 1 --2- 172.21.15.156:0/816300631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51a015c9b0 0x7f51a01f28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:55.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.809+0000 7f51a63ff700 1 -- 172.21.15.156:0/816300631 >> 172.21.15.156:0/816300631 conn(0x7f51a0072600 msgr2=0x7f51a00b7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:55.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.809+0000 7f51a63ff700 1 -- 172.21.15.156:0/816300631 shutdown_connections 2023-11-28T13:08:55.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:55.809+0000 7f51a63ff700 1 -- 172.21.15.156:0/816300631 wait complete. 2023-11-28T13:08:55.832 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T13:08:56.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.031+0000 7f801f1ff700 1 Processor -- start 2023-11-28T13:08:56.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.031+0000 7f801f1ff700 1 -- start start 2023-11-28T13:08:56.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.033+0000 7f801f1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8018070c50 0x7f80180b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:56.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.033+0000 7f801f1ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f80180b0bb0 0x7f801815add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:56.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.033+0000 7f801f1ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80180b1010 con 0x7f8018158200 2023-11-28T13:08:56.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.033+0000 7f801f1ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f801815f070 con 0x7f80180b0bb0 2023-11-28T13:08:56.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.033+0000 7f801f1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801815e1a0 con 0x7f8018070c50 2023-11-28T13:08:56.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.033+0000 7f801d79c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f80180b0bb0 0x7f801815add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:56.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.033+0000 7f801d79c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f80180b0bb0 0x7f801815add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41360/0 (socket says 172.21.15.156:41360) 2023-11-28T13:08:56.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.033+0000 7f801d79c700 1 -- 172.21.15.156:0/2569544942 learned_addr learned my addr 172.21.15.156:0/2569544942 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:56.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.033+0000 7f8017fff700 1 --2- 172.21.15.156:0/2569544942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8018070c50 0x7f80180b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:56.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.033+0000 7f801d79c700 1 -- 172.21.15.156:0/2569544942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8018070c50 msgr2=0x7f80180b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:56.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.033+0000 7f801d79c700 1 --2- 172.21.15.156:0/2569544942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8018070c50 0x7f80180b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:56.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.033+0000 7f801d79c700 1 -- 172.21.15.156:0/2569544942 >> v1:172.21.15.186:6789/0 conn(0x7f8018158200 legacy=0x7f80180caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:56.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.033+0000 7f801d79c700 1 -- 172.21.15.156:0/2569544942 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80180cb180 con 0x7f80180b0bb0 2023-11-28T13:08:56.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.034+0000 7f801d79c700 1 --2- 172.21.15.156:0/2569544942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80180b0bb0 0x7f801815add0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8010009e50 tx=0x7f801000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5610af3d6ab71e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:56.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.034+0000 7f80177fe700 1 -- 172.21.15.156:0/2569544942 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801000e070 con 0x7f80180b0bb0 2023-11-28T13:08:56.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.034+0000 7f80177fe700 1 -- 172.21.15.156:0/2569544942 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801000b910 con 0x7f80180b0bb0 2023-11-28T13:08:56.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.034+0000 7f80177fe700 1 -- 172.21.15.156:0/2569544942 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801000bcc0 con 0x7f80180b0bb0 2023-11-28T13:08:56.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.034+0000 7f801f1ff700 1 -- 172.21.15.156:0/2569544942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80180b0bb0 msgr2=0x7f801815add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:56.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.034+0000 7f801f1ff700 1 --2- 172.21.15.156:0/2569544942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80180b0bb0 0x7f801815add0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8010009e50 tx=0x7f801000b330 comp rx=0 tx=0).stop 2023-11-28T13:08:56.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.035+0000 7f801f1ff700 1 -- 172.21.15.156:0/2569544942 shutdown_connections 2023-11-28T13:08:56.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.035+0000 7f801f1ff700 1 --2- 172.21.15.156:0/2569544942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80180b0bb0 0x7f801815add0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:56.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.035+0000 7f801f1ff700 1 --2- 172.21.15.156:0/2569544942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8018070c50 0x7f80180b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:56.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.035+0000 7f801f1ff700 1 -- 172.21.15.156:0/2569544942 >> 172.21.15.156:0/2569544942 conn(0x7f80180cb670 msgr2=0x7f80180cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:56.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.035+0000 7f801f1ff700 1 -- 172.21.15.156:0/2569544942 shutdown_connections 2023-11-28T13:08:56.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.035+0000 7f801f1ff700 1 -- 172.21.15.156:0/2569544942 wait complete. 2023-11-28T13:08:56.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.037+0000 7f801f1ff700 1 Processor -- start 2023-11-28T13:08:56.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.037+0000 7f801f1ff700 1 -- start start 2023-11-28T13:08:56.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.038+0000 7f801f1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8018070c50 0x7f80181f2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:56.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.038+0000 7f801f1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80180b0bb0 0x7f80181f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:56.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.038+0000 7f801f1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8018158200 0x7f80180ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:56.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.038+0000 7f801f1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80181f2fb0 con 0x7f80180b0bb0 2023-11-28T13:08:56.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.038+0000 7f801f1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80180ad0f0 con 0x7f8018158200 2023-11-28T13:08:56.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.038+0000 7f801f1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80180ad3d0 con 0x7f8018070c50 2023-11-28T13:08:56.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.039+0000 7f8017fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80180b0bb0 0x7f80181f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:56.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.039+0000 7f8017fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80180b0bb0 0x7f80181f2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41368/0 (socket says 172.21.15.156:41368) 2023-11-28T13:08:56.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.039+0000 7f8017fff700 1 -- 172.21.15.156:0/2659604268 learned_addr learned my addr 172.21.15.156:0/2659604268 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:56.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.039+0000 7f801d79c700 1 --2- 172.21.15.156:0/2659604268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8018158200 0x7f80180ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:56.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.039+0000 7f801cf9b700 1 --2- 172.21.15.156:0/2659604268 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8018070c50 0x7f80181f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:56.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.039+0000 7f8017fff700 1 -- 172.21.15.156:0/2659604268 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8018070c50 msgr2=0x7f80181f2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:56.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.039+0000 7f8017fff700 1 --2- 172.21.15.156:0/2659604268 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8018070c50 0x7f80181f2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:56.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.039+0000 7f8017fff700 1 -- 172.21.15.156:0/2659604268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8018158200 msgr2=0x7f80180ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:56.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.039+0000 7f8017fff700 1 --2- 172.21.15.156:0/2659604268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8018158200 0x7f80180ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:56.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.039+0000 7f8017fff700 1 -- 172.21.15.156:0/2659604268 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8010009b30 con 0x7f80180b0bb0 2023-11-28T13:08:56.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.039+0000 7f8017fff700 1 --2- 172.21.15.156:0/2659604268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80180b0bb0 0x7f80181f2a70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f800400e870 tx=0x7f800400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:56.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.040+0000 7f80157fa700 1 -- 172.21.15.156:0/2659604268 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004009590 con 0x7f80180b0bb0 2023-11-28T13:08:56.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.040+0000 7f80157fa700 1 -- 172.21.15.156:0/2659604268 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8004013680 con 0x7f80180b0bb0 2023-11-28T13:08:56.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.040+0000 7f801f1ff700 1 -- 172.21.15.156:0/2659604268 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80181585d0 con 0x7f80180b0bb0 2023-11-28T13:08:56.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.040+0000 7f80157fa700 1 -- 172.21.15.156:0/2659604268 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040139f0 con 0x7f80180b0bb0 2023-11-28T13:08:56.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.040+0000 7f801f1ff700 1 -- 172.21.15.156:0/2659604268 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80180710f0 con 0x7f80180b0bb0 2023-11-28T13:08:56.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.041+0000 7f801f1ff700 1 -- 172.21.15.156:0/2659604268 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801815e3f0 con 0x7f80180b0bb0 2023-11-28T13:08:56.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.042+0000 7f80157fa700 1 -- 172.21.15.156:0/2659604268 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8004019030 con 0x7f80180b0bb0 2023-11-28T13:08:56.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.043+0000 7f80157fa700 1 --2- 172.21.15.156:0/2659604268 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7ffc0682f0 0x7f7ffc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:56.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.043+0000 7f80157fa700 1 -- 172.21.15.156:0/2659604268 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8004085b40 con 0x7f80180b0bb0 2023-11-28T13:08:56.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.044+0000 7f801cf9b700 1 --2- 172.21.15.156:0/2659604268 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7ffc0682f0 0x7f7ffc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:56.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.044+0000 7f801cf9b700 1 --2- 172.21.15.156:0/2659604268 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7ffc0682f0 0x7f7ffc06a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f800c000f60 tx=0x7f800c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:56.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.046+0000 7f80157fa700 1 -- 172.21.15.156:0/2659604268 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8004052270 con 0x7f80180b0bb0 2023-11-28T13:08:56.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.281+0000 7f801f1ff700 1 -- 172.21.15.156:0/2659604268 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f801806fd60 con 0x7f80180b0bb0 2023-11-28T13:08:56.559 INFO:tasks.ceph.mon.a.smithi156.stderr:2023-11-28T13:08:56.558+0000 7f61ad1f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T13:08:56.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.564+0000 7f80157fa700 1 -- 172.21.15.156:0/2659604268 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8004055890 con 0x7f80180b0bb0 2023-11-28T13:08:56.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.566+0000 7f801f1ff700 1 -- 172.21.15.156:0/2659604268 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7ffc0682f0 msgr2=0x7f7ffc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:56.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.566+0000 7f801f1ff700 1 --2- 172.21.15.156:0/2659604268 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7ffc0682f0 0x7f7ffc06a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f800c000f60 tx=0x7f800c009040 comp rx=0 tx=0).stop 2023-11-28T13:08:56.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.566+0000 7f801f1ff700 1 -- 172.21.15.156:0/2659604268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80180b0bb0 msgr2=0x7f80181f2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:56.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.566+0000 7f801f1ff700 1 --2- 172.21.15.156:0/2659604268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80180b0bb0 0x7f80181f2a70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f800400e870 tx=0x7f800400eb80 comp rx=0 tx=0).stop 2023-11-28T13:08:56.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.566+0000 7f801f1ff700 1 -- 172.21.15.156:0/2659604268 shutdown_connections 2023-11-28T13:08:56.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.566+0000 7f801f1ff700 1 --2- 172.21.15.156:0/2659604268 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7ffc0682f0 0x7f7ffc06a7a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:56.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.566+0000 7f801f1ff700 1 --2- 172.21.15.156:0/2659604268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8018158200 0x7f80180ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:56.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.566+0000 7f801f1ff700 1 --2- 172.21.15.156:0/2659604268 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8018070c50 0x7f80181f2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:56.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.566+0000 7f801f1ff700 1 --2- 172.21.15.156:0/2659604268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80180b0bb0 0x7f80181f2a70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:56.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.566+0000 7f801f1ff700 1 -- 172.21.15.156:0/2659604268 >> 172.21.15.156:0/2659604268 conn(0x7f80180cb670 msgr2=0x7f801815d220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:56.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.567+0000 7f801f1ff700 1 -- 172.21.15.156:0/2659604268 shutdown_connections 2023-11-28T13:08:56.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.567+0000 7f801f1ff700 1 -- 172.21.15.156:0/2659604268 wait complete. 2023-11-28T13:08:56.569 INFO:teuthology.orchestra.run.smithi156.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T13:08:56.587 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T13:08:56.637 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2023-11-28T13:08:56.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.787+0000 7f72309ff700 1 Processor -- start 2023-11-28T13:08:56.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.787+0000 7f72309ff700 1 -- start start 2023-11-28T13:08:56.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.788+0000 7f72309ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f722c15c9b0 0x7f722c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:56.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.788+0000 7f72309ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f722c0d1950 con 0x7f722c161340 2023-11-28T13:08:56.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.788+0000 7f72309ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f722c0d24f0 con 0x7f722c0cd7c0 2023-11-28T13:08:56.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.788+0000 7f72309ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0b5800 con 0x7f722c15c9b0 2023-11-28T13:08:56.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.789+0000 7f7229d9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f722c161340 0x7f722c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:33346/0 (socket says 172.21.15.156:33346) 2023-11-28T13:08:56.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.789+0000 7f7229d9b700 1 -- 172.21.15.156:0/3222517233 learned_addr learned my addr 172.21.15.156:0/3222517233 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:56.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.789+0000 7f722a59c700 1 --2- 172.21.15.156:0/3222517233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f722c15c9b0 0x7f722c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:56.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.789+0000 7f722959a700 1 -- 172.21.15.156:0/3222517233 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585226815 0 0) 0x7f722c0d1950 con 0x7f722c161340 2023-11-28T13:08:56.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.789+0000 7f722959a700 1 -- 172.21.15.156:0/3222517233 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72140035b0 con 0x7f722c161340 2023-11-28T13:08:56.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.789+0000 7f722959a700 1 -- 172.21.15.156:0/3222517233 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019685890 0 0) 0x7f722c0d24f0 con 0x7f722c0cd7c0 2023-11-28T13:08:56.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.789+0000 7f722a59c700 1 -- 172.21.15.156:0/3222517233 >> v1:172.21.15.186:6789/0 conn(0x7f722c0cd7c0 legacy=0x7f722c0b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:56.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.789+0000 7f722a59c700 1 -- 172.21.15.156:0/3222517233 >> v1:172.21.15.156:6789/0 conn(0x7f722c161340 legacy=0x7f722c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:56.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.789+0000 7f722a59c700 1 -- 172.21.15.156:0/3222517233 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c0b5520 con 0x7f722c15c9b0 2023-11-28T13:08:56.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.790+0000 7f722a59c700 1 --2- 172.21.15.156:0/3222517233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f722c15c9b0 0x7f722c0b4bf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f721c009870 tx=0x7f721c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1fad6a02e59a42f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:56.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.790+0000 7f722959a700 1 -- 172.21.15.156:0/3222517233 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c011830 con 0x7f722c15c9b0 2023-11-28T13:08:56.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.790+0000 7f722959a700 1 -- 172.21.15.156:0/3222517233 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c011990 con 0x7f722c15c9b0 2023-11-28T13:08:56.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.790+0000 7f722959a700 1 -- 172.21.15.156:0/3222517233 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c011d20 con 0x7f722c15c9b0 2023-11-28T13:08:56.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.790+0000 7f72309ff700 1 -- 172.21.15.156:0/3222517233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f722c15c9b0 msgr2=0x7f722c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:56.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.790+0000 7f72309ff700 1 --2- 172.21.15.156:0/3222517233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f722c15c9b0 0x7f722c0b4bf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f721c009870 tx=0x7f721c009b80 comp rx=0 tx=0).stop 2023-11-28T13:08:56.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.790+0000 7f72309ff700 1 -- 172.21.15.156:0/3222517233 shutdown_connections 2023-11-28T13:08:56.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.790+0000 7f72309ff700 1 --2- 172.21.15.156:0/3222517233 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f722c15c9b0 0x7f722c0b4bf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:56.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.790+0000 7f72309ff700 1 -- 172.21.15.156:0/3222517233 >> 172.21.15.156:0/3222517233 conn(0x7f722c072600 msgr2=0x7f722c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:56.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.791+0000 7f72309ff700 1 -- 172.21.15.156:0/3222517233 shutdown_connections 2023-11-28T13:08:56.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.791+0000 7f72309ff700 1 -- 172.21.15.156:0/3222517233 wait complete. 2023-11-28T13:08:56.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.792+0000 7f72309ff700 1 Processor -- start 2023-11-28T13:08:56.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.792+0000 7f72309ff700 1 -- start start 2023-11-28T13:08:56.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.793+0000 7f72309ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f722c0cd7c0 0x7f722c150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:56.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.793+0000 7f72309ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f722c15c9b0 0x7f722c14ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:56.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.793+0000 7f72309ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f722c161340 0x7f722c0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:56.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.793+0000 7f72309ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0c6ca0 con 0x7f722c15c9b0 2023-11-28T13:08:56.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.793+0000 7f72309ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0cce50 con 0x7f722c161340 2023-11-28T13:08:56.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.793+0000 7f72309ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0cd2e0 con 0x7f722c0cd7c0 2023-11-28T13:08:56.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.794+0000 7f7229d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f722c15c9b0 0x7f722c14ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:56.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.794+0000 7f7229d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f722c15c9b0 0x7f722c14ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41384/0 (socket says 172.21.15.156:41384) 2023-11-28T13:08:56.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.794+0000 7f722a59c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f722c0cd7c0 0x7f722c150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:56.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.794+0000 7f7229d9b700 1 -- 172.21.15.156:0/3454874522 learned_addr learned my addr 172.21.15.156:0/3454874522 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:56.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.794+0000 7f722ad9d700 1 --2- 172.21.15.156:0/3454874522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f722c161340 0x7f722c0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:56.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.794+0000 7f7229d9b700 1 -- 172.21.15.156:0/3454874522 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f722c0cd7c0 msgr2=0x7f722c150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:56.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.794+0000 7f7229d9b700 1 --2- 172.21.15.156:0/3454874522 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f722c0cd7c0 0x7f722c150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:56.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.794+0000 7f7229d9b700 1 -- 172.21.15.156:0/3454874522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f722c161340 msgr2=0x7f722c0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:56.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.794+0000 7f7229d9b700 1 --2- 172.21.15.156:0/3454874522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f722c161340 0x7f722c0ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:56.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.794+0000 7f7229d9b700 1 -- 172.21.15.156:0/3454874522 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c0094a0 con 0x7f722c15c9b0 2023-11-28T13:08:56.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.794+0000 7f7229d9b700 1 --2- 172.21.15.156:0/3454874522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f722c15c9b0 0x7f722c14ff60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f722000c420 tx=0x7f722000c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:56.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.795+0000 7f721affd700 1 -- 172.21.15.156:0/3454874522 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7220005d70 con 0x7f722c15c9b0 2023-11-28T13:08:56.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.795+0000 7f721affd700 1 -- 172.21.15.156:0/3454874522 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722000ce30 con 0x7f722c15c9b0 2023-11-28T13:08:56.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.795+0000 7f721affd700 1 -- 172.21.15.156:0/3454874522 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72200072d0 con 0x7f722c15c9b0 2023-11-28T13:08:56.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.795+0000 7f72309ff700 1 -- 172.21.15.156:0/3454874522 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722c0d15e0 con 0x7f722c15c9b0 2023-11-28T13:08:56.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.795+0000 7f72309ff700 1 -- 172.21.15.156:0/3454874522 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722c0d1300 con 0x7f722c15c9b0 2023-11-28T13:08:56.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.796+0000 7f72309ff700 1 -- 172.21.15.156:0/3454874522 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f722c0b5130 con 0x7f722c15c9b0 2023-11-28T13:08:56.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.797+0000 7f721affd700 1 -- 172.21.15.156:0/3454874522 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7220019030 con 0x7f722c15c9b0 2023-11-28T13:08:56.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.798+0000 7f721affd700 1 --2- 172.21.15.156:0/3454874522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f720c0682f0 0x7f720c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:56.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.798+0000 7f721affd700 1 -- 172.21.15.156:0/3454874522 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f7220084ef0 con 0x7f722c15c9b0 2023-11-28T13:08:56.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.799+0000 7f722a59c700 1 --2- 172.21.15.156:0/3454874522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f720c0682f0 0x7f720c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:56.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.799+0000 7f722a59c700 1 --2- 172.21.15.156:0/3454874522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f720c0682f0 0x7f720c06a7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f721c0097c0 tx=0x7f721c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:56.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:56.803+0000 7f721affd700 1 -- 172.21.15.156:0/3454874522 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f722004b5e0 con 0x7f722c15c9b0 2023-11-28T13:08:57.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.038+0000 7f72309ff700 1 -- 172.21.15.156:0/3454874522 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f722c0ccfc0 con 0x7f722c15c9b0 2023-11-28T13:08:57.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.567+0000 7f721affd700 1 -- 172.21.15.156:0/3454874522 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f72200170a0 con 0x7f722c15c9b0 2023-11-28T13:08:57.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.570+0000 7f72309ff700 1 -- 172.21.15.156:0/3454874522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f720c0682f0 msgr2=0x7f720c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:57.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.570+0000 7f72309ff700 1 --2- 172.21.15.156:0/3454874522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f720c0682f0 0x7f720c06a7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f721c0097c0 tx=0x7f721c01c040 comp rx=0 tx=0).stop 2023-11-28T13:08:57.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.570+0000 7f72309ff700 1 -- 172.21.15.156:0/3454874522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f722c15c9b0 msgr2=0x7f722c14ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:57.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.570+0000 7f72309ff700 1 --2- 172.21.15.156:0/3454874522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f722c15c9b0 0x7f722c14ff60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f722000c420 tx=0x7f722000c730 comp rx=0 tx=0).stop 2023-11-28T13:08:57.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.570+0000 7f72309ff700 1 -- 172.21.15.156:0/3454874522 shutdown_connections 2023-11-28T13:08:57.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.570+0000 7f72309ff700 1 --2- 172.21.15.156:0/3454874522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f720c0682f0 0x7f720c06a7a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:57.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.570+0000 7f72309ff700 1 --2- 172.21.15.156:0/3454874522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f722c161340 0x7f722c0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:57.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.570+0000 7f72309ff700 1 --2- 172.21.15.156:0/3454874522 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f722c0cd7c0 0x7f722c150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:57.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.570+0000 7f72309ff700 1 --2- 172.21.15.156:0/3454874522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f722c15c9b0 0x7f722c14ff60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:57.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.570+0000 7f72309ff700 1 -- 172.21.15.156:0/3454874522 >> 172.21.15.156:0/3454874522 conn(0x7f722c072600 msgr2=0x7f722c160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:57.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.570+0000 7f72309ff700 1 -- 172.21.15.156:0/3454874522 shutdown_connections 2023-11-28T13:08:57.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.570+0000 7f72309ff700 1 -- 172.21.15.156:0/3454874522 wait complete. 2023-11-28T13:08:57.592 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T13:08:57.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.796+0000 7f117bfff700 1 Processor -- start 2023-11-28T13:08:57.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.796+0000 7f117bfff700 1 -- start start 2023-11-28T13:08:57.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.797+0000 7f117bfff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f117415a760 0x7f1174161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:57.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.797+0000 7f117bfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11740cded0 0x7f11740b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:57.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.797+0000 7f117bfff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117415e0a0 con 0x7f117415dc30 2023-11-28T13:08:57.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.797+0000 7f117bfff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f117415ab30 con 0x7f117415a760 2023-11-28T13:08:57.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.797+0000 7f117bfff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1174160d70 con 0x7f11740cded0 2023-11-28T13:08:57.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.797+0000 7f117959a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f117415dc30 0x7f11740cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:33362/0 (socket says 172.21.15.156:33362) 2023-11-28T13:08:57.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.797+0000 7f117959a700 1 -- 172.21.15.156:0/126776371 learned_addr learned my addr 172.21.15.156:0/126776371 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:57.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f117a59c700 1 --2- 172.21.15.156:0/126776371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11740cded0 0x7f11740b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:57.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1179d9b700 1 --2- 172.21.15.156:0/126776371 >> v2:172.21.15.186:3300/0 conn(0x7f117415a760 0x7f1174161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:57.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1178d99700 1 -- 172.21.15.156:0/126776371 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277727749 0 0) 0x7f117415e0a0 con 0x7f117415dc30 2023-11-28T13:08:57.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1178d99700 1 -- 172.21.15.156:0/126776371 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11600035b0 con 0x7f117415dc30 2023-11-28T13:08:57.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1178d99700 1 -- 172.21.15.156:0/126776371 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3935635691 0 0) 0x7f11600035b0 con 0x7f117415dc30 2023-11-28T13:08:57.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1178d99700 1 -- 172.21.15.156:0/126776371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11740cded0 msgr2=0x7f11740b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:57.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1178d99700 1 --2- 172.21.15.156:0/126776371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11740cded0 0x7f11740b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:57.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1178d99700 1 -- 172.21.15.156:0/126776371 >> v2:172.21.15.186:3300/0 conn(0x7f117415a760 msgr2=0x7f1174161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:57.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1178d99700 1 --2- 172.21.15.156:0/126776371 >> v2:172.21.15.186:3300/0 conn(0x7f117415a760 0x7f1174161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:57.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1178d99700 1 -- 172.21.15.156:0/126776371 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11740b5130 con 0x7f117415dc30 2023-11-28T13:08:57.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1178d99700 1 -- 172.21.15.156:0/126776371 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f1164001670 con 0x7f117415dc30 2023-11-28T13:08:57.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1178d99700 1 -- 172.21.15.156:0/126776371 >> v1:172.21.15.156:6789/0 conn(0x7f117415dc30 legacy=0x7f11740cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:57.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1178d99700 1 --2- 172.21.15.156:0/126776371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1160003a00 0x7f1160003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:57.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1178d99700 1 --2- 172.21.15.156:0/126776371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1160004430 0x7f11600048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:57.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1178d99700 1 --2- 172.21.15.156:0/126776371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1160004fb0 0x7f116000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:57.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1178d99700 1 -- 172.21.15.156:0/126776371 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117415ab30 con 0x7f1160004fb0 2023-11-28T13:08:57.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1178d99700 1 -- 172.21.15.156:0/126776371 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1174160d70 con 0x7f1160003a00 2023-11-28T13:08:57.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1178d99700 1 -- 172.21.15.156:0/126776371 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116000bf50 con 0x7f1160004430 2023-11-28T13:08:57.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f117a59c700 1 --2- 172.21.15.156:0/126776371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f11740cded0 0x7f11740b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:57.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.798+0000 7f1179d9b700 1 --2- 172.21.15.156:0/126776371 >> v2:172.21.15.186:3300/0 conn(0x7f117415a760 0x7f1174161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:57.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.799+0000 7f117a59c700 1 --2- 172.21.15.156:0/126776371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1160004fb0 0x7f116000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:57.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.799+0000 7f117959a700 1 --2- 172.21.15.156:0/126776371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1160004430 0x7f11600048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:57.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.799+0000 7f1179d9b700 1 --2- 172.21.15.156:0/126776371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1160003a00 0x7f1160003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:57.804 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.799+0000 7f117a59c700 1 -- 172.21.15.156:0/126776371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1160004430 msgr2=0x7f11600048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:57.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.799+0000 7f117a59c700 1 --2- 172.21.15.156:0/126776371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1160004430 0x7f11600048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:57.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.799+0000 7f117a59c700 1 -- 172.21.15.156:0/126776371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1160003a00 msgr2=0x7f1160003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:57.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.799+0000 7f117a59c700 1 --2- 172.21.15.156:0/126776371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1160003a00 0x7f1160003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:57.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.799+0000 7f117a59c700 1 -- 172.21.15.156:0/126776371 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11600035b0 con 0x7f1160004fb0 2023-11-28T13:08:57.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.799+0000 7f117a59c700 1 --2- 172.21.15.156:0/126776371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1160004fb0 0x7f116000ba10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f116c00e1f0 tx=0x7f1160005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c479bc3316a2acd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:57.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.799+0000 7f1178d99700 1 -- 172.21.15.156:0/126776371 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c00ec90 con 0x7f1160004fb0 2023-11-28T13:08:57.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.800+0000 7f1178d99700 1 -- 172.21.15.156:0/126776371 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116c00f040 con 0x7f1160004fb0 2023-11-28T13:08:57.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.800+0000 7f1178d99700 1 -- 172.21.15.156:0/126776371 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c003a20 con 0x7f1160004fb0 2023-11-28T13:08:57.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.800+0000 7f117bfff700 1 -- 172.21.15.156:0/126776371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1160004fb0 msgr2=0x7f116000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:57.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.800+0000 7f117bfff700 1 --2- 172.21.15.156:0/126776371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1160004fb0 0x7f116000ba10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f116c00e1f0 tx=0x7f1160005880 comp rx=0 tx=0).stop 2023-11-28T13:08:57.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.800+0000 7f117a59c700 1 -- 172.21.15.156:0/126776371 reap_dead start 2023-11-28T13:08:57.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.800+0000 7f117bfff700 1 -- 172.21.15.156:0/126776371 shutdown_connections 2023-11-28T13:08:57.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.800+0000 7f117bfff700 1 -- 172.21.15.156:0/126776371 >> 172.21.15.156:0/126776371 conn(0x7f1174072620 msgr2=0x7f11740cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:57.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.800+0000 7f117bfff700 1 -- 172.21.15.156:0/126776371 shutdown_connections 2023-11-28T13:08:57.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.800+0000 7f117bfff700 1 -- 172.21.15.156:0/126776371 wait complete. 2023-11-28T13:08:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.802+0000 7f117bfff700 1 Processor -- start 2023-11-28T13:08:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.802+0000 7f117bfff700 1 -- start start 2023-11-28T13:08:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.803+0000 7f117bfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1160004fb0 0x7f11740c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.803+0000 7f117bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f117414c160 0x7f11741f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.803+0000 7f117bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f117415bd00 0x7f117415c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.803+0000 7f117bfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117415c6b0 con 0x7f117414c160 2023-11-28T13:08:57.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.803+0000 7f117bfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11740c7310 con 0x7f117415bd00 2023-11-28T13:08:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.803+0000 7f117bfff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11740b6a00 con 0x7f1160004fb0 2023-11-28T13:08:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.803+0000 7f117959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f117414c160 0x7f11741f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.803+0000 7f117959a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f117414c160 0x7f11741f1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41394/0 (socket says 172.21.15.156:41394) 2023-11-28T13:08:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.803+0000 7f117959a700 1 -- 172.21.15.156:0/4040008960 learned_addr learned my addr 172.21.15.156:0/4040008960 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.804+0000 7f1179d9b700 1 --2- 172.21.15.156:0/4040008960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1160004fb0 0x7f11740c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.804+0000 7f117a59c700 1 --2- 172.21.15.156:0/4040008960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f117415bd00 0x7f117415c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.804+0000 7f117959a700 1 -- 172.21.15.156:0/4040008960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1160004fb0 msgr2=0x7f11740c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.804+0000 7f117959a700 1 --2- 172.21.15.156:0/4040008960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1160004fb0 0x7f11740c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.804+0000 7f117959a700 1 -- 172.21.15.156:0/4040008960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f117415bd00 msgr2=0x7f117415c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.804+0000 7f117959a700 1 --2- 172.21.15.156:0/4040008960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f117415bd00 0x7f117415c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.804+0000 7f117959a700 1 -- 172.21.15.156:0/4040008960 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11740b5670 con 0x7f117414c160 2023-11-28T13:08:57.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.804+0000 7f117959a700 1 --2- 172.21.15.156:0/4040008960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f117414c160 0x7f11741f1de0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1164007340 tx=0x7f1164007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:57.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.804+0000 7f116a7fc700 1 -- 172.21.15.156:0/4040008960 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164007ca0 con 0x7f117414c160 2023-11-28T13:08:57.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.804+0000 7f116a7fc700 1 -- 172.21.15.156:0/4040008960 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116400f870 con 0x7f117414c160 2023-11-28T13:08:57.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.804+0000 7f116a7fc700 1 -- 172.21.15.156:0/4040008960 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116400fc20 con 0x7f117414c160 2023-11-28T13:08:57.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.804+0000 7f117bfff700 1 -- 172.21.15.156:0/4040008960 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11740b5390 con 0x7f117414c160 2023-11-28T13:08:57.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.804+0000 7f117bfff700 1 -- 172.21.15.156:0/4040008960 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1174160720 con 0x7f117414c160 2023-11-28T13:08:57.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.806+0000 7f117bfff700 1 -- 172.21.15.156:0/4040008960 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117404e620 con 0x7f117414c160 2023-11-28T13:08:57.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.806+0000 7f116a7fc700 1 -- 172.21.15.156:0/4040008960 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f116400fd80 con 0x7f117414c160 2023-11-28T13:08:57.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.807+0000 7f116a7fc700 1 --2- 172.21.15.156:0/4040008960 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f11580682f0 0x7f115806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:57.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.807+0000 7f116a7fc700 1 -- 172.21.15.156:0/4040008960 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f1164085660 con 0x7f117414c160 2023-11-28T13:08:57.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.807+0000 7f1179d9b700 1 --2- 172.21.15.156:0/4040008960 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f11580682f0 0x7f115806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:57.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.808+0000 7f1179d9b700 1 --2- 172.21.15.156:0/4040008960 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f11580682f0 0x7f115806a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1170009d30 tx=0x7f1170008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:57.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:57.811+0000 7f116a7fc700 1 -- 172.21.15.156:0/4040008960 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1164051d40 con 0x7f117414c160 2023-11-28T13:08:58.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.001+0000 7f117bfff700 1 -- 172.21.15.156:0/4040008960 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f11740b6b40 con 0x7f117414c160 2023-11-28T13:08:58.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.574+0000 7f116a7fc700 1 -- 172.21.15.156:0/4040008960 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1164055360 con 0x7f117414c160 2023-11-28T13:08:58.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.577+0000 7f117bfff700 1 -- 172.21.15.156:0/4040008960 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f11580682f0 msgr2=0x7f115806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:58.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.577+0000 7f117bfff700 1 --2- 172.21.15.156:0/4040008960 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f11580682f0 0x7f115806a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1170009d30 tx=0x7f1170008040 comp rx=0 tx=0).stop 2023-11-28T13:08:58.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.577+0000 7f117bfff700 1 -- 172.21.15.156:0/4040008960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f117414c160 msgr2=0x7f11741f1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:58.579 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.577+0000 7f117bfff700 1 --2- 172.21.15.156:0/4040008960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f117414c160 0x7f11741f1de0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1164007340 tx=0x7f1164007650 comp rx=0 tx=0).stop 2023-11-28T13:08:58.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.577+0000 7f117bfff700 1 -- 172.21.15.156:0/4040008960 shutdown_connections 2023-11-28T13:08:58.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.577+0000 7f117bfff700 1 --2- 172.21.15.156:0/4040008960 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f11580682f0 0x7f115806a7a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:58.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.577+0000 7f117bfff700 1 --2- 172.21.15.156:0/4040008960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f117415bd00 0x7f117415c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:58.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.577+0000 7f117bfff700 1 --2- 172.21.15.156:0/4040008960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1160004fb0 0x7f11740c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:58.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.577+0000 7f117bfff700 1 --2- 172.21.15.156:0/4040008960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f117414c160 0x7f11741f1de0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:58.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.577+0000 7f117bfff700 1 -- 172.21.15.156:0/4040008960 >> 172.21.15.156:0/4040008960 conn(0x7f1174072620 msgr2=0x7f117415d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:58.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.577+0000 7f117bfff700 1 -- 172.21.15.156:0/4040008960 shutdown_connections 2023-11-28T13:08:58.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.578+0000 7f117bfff700 1 -- 172.21.15.156:0/4040008960 wait complete. 2023-11-28T13:08:58.581 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_metadata' removed 2023-11-28T13:08:58.599 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T13:08:58.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.799+0000 7f3d9b9ff700 1 Processor -- start 2023-11-28T13:08:58.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.799+0000 7f3d9b9ff700 1 -- start start 2023-11-28T13:08:58.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.800+0000 7f3d9b9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d940cded0 0x7f3d94161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:58.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.800+0000 7f3d9b9ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3d9415dc30 0x7f3d940b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:58.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.800+0000 7f3d9b9ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d9415e0a0 con 0x7f3d9415a760 2023-11-28T13:08:58.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.800+0000 7f3d9b9ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d9415ab30 con 0x7f3d9415dc30 2023-11-28T13:08:58.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.800+0000 7f3d9b9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d94160d70 con 0x7f3d940cded0 2023-11-28T13:08:58.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.800+0000 7f3d99f9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3d9415dc30 0x7f3d940b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:58.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.800+0000 7f3d99f9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3d9415dc30 0x7f3d940b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41398/0 (socket says 172.21.15.156:41398) 2023-11-28T13:08:58.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.800+0000 7f3d99f9c700 1 -- 172.21.15.156:0/2621058765 learned_addr learned my addr 172.21.15.156:0/2621058765 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:58.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.801+0000 7f3d99f9c700 1 -- 172.21.15.156:0/2621058765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d940cded0 msgr2=0x7f3d94161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:58.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.801+0000 7f3d98f9a700 1 --2- 172.21.15.156:0/2621058765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d940cded0 0x7f3d94161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:58.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.801+0000 7f3d99f9c700 1 --2- 172.21.15.156:0/2621058765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d940cded0 0x7f3d94161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:58.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.801+0000 7f3d99f9c700 1 -- 172.21.15.156:0/2621058765 >> v1:172.21.15.186:6789/0 conn(0x7f3d9415a760 legacy=0x7f3d940cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:58.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.801+0000 7f3d99f9c700 1 -- 172.21.15.156:0/2621058765 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d940b5130 con 0x7f3d9415dc30 2023-11-28T13:08:58.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.801+0000 7f3d98f9a700 1 --2- 172.21.15.156:0/2621058765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d940cded0 0x7f3d94161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:58.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.801+0000 7f3d99f9c700 1 --2- 172.21.15.156:0/2621058765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d9415dc30 0x7f3d940b4bf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c009e50 tx=0x7f3d8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b7b305dfb99bd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:58.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.801+0000 7f3d8bfff700 1 -- 172.21.15.156:0/2621058765 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c00e070 con 0x7f3d9415dc30 2023-11-28T13:08:58.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.801+0000 7f3d8bfff700 1 -- 172.21.15.156:0/2621058765 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8c003a20 con 0x7f3d9415dc30 2023-11-28T13:08:58.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.801+0000 7f3d8bfff700 1 -- 172.21.15.156:0/2621058765 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c003dd0 con 0x7f3d9415dc30 2023-11-28T13:08:58.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.802+0000 7f3d9b9ff700 1 -- 172.21.15.156:0/2621058765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d9415dc30 msgr2=0x7f3d940b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:58.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.802+0000 7f3d9b9ff700 1 --2- 172.21.15.156:0/2621058765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d9415dc30 0x7f3d940b4bf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c009e50 tx=0x7f3d8c00b330 comp rx=0 tx=0).stop 2023-11-28T13:08:58.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.802+0000 7f3d9b9ff700 1 -- 172.21.15.156:0/2621058765 shutdown_connections 2023-11-28T13:08:58.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.802+0000 7f3d9b9ff700 1 --2- 172.21.15.156:0/2621058765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d9415dc30 0x7f3d940b4bf0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:58.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.802+0000 7f3d9b9ff700 1 --2- 172.21.15.156:0/2621058765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d940cded0 0x7f3d94161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:58.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.802+0000 7f3d9b9ff700 1 -- 172.21.15.156:0/2621058765 >> 172.21.15.156:0/2621058765 conn(0x7f3d94072620 msgr2=0x7f3d940cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:58.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.802+0000 7f3d9b9ff700 1 -- 172.21.15.156:0/2621058765 shutdown_connections 2023-11-28T13:08:58.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.802+0000 7f3d9b9ff700 1 -- 172.21.15.156:0/2621058765 wait complete. 2023-11-28T13:08:58.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.804+0000 7f3d9b9ff700 1 Processor -- start 2023-11-28T13:08:58.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.804+0000 7f3d9b9ff700 1 -- start start 2023-11-28T13:08:58.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.805+0000 7f3d9b9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d940cded0 0x7f3d940ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:58.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.805+0000 7f3d9b9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d9415a760 0x7f3d941f17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:58.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.805+0000 7f3d9b9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d9415dc30 0x7f3d941f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:58.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.805+0000 7f3d9b9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d940b7060 con 0x7f3d940cded0 2023-11-28T13:08:58.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.805+0000 7f3d9b9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d940ca940 con 0x7f3d9415a760 2023-11-28T13:08:58.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.805+0000 7f3d9b9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d940cca60 con 0x7f3d9415dc30 2023-11-28T13:08:58.814 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.805+0000 7f3d9979b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d940cded0 0x7f3d940ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:58.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.805+0000 7f3d9979b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d940cded0 0x7f3d940ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41414/0 (socket says 172.21.15.156:41414) 2023-11-28T13:08:58.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.805+0000 7f3d9979b700 1 -- 172.21.15.156:0/2284681349 learned_addr learned my addr 172.21.15.156:0/2284681349 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:58.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.805+0000 7f3d98f9a700 1 --2- 172.21.15.156:0/2284681349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d9415a760 0x7f3d941f17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:58.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.805+0000 7f3d9979b700 1 -- 172.21.15.156:0/2284681349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d9415dc30 msgr2=0x7f3d941f1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:58.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.805+0000 7f3d9979b700 1 --2- 172.21.15.156:0/2284681349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d9415dc30 0x7f3d941f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:58.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.805+0000 7f3d9979b700 1 -- 172.21.15.156:0/2284681349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d9415a760 msgr2=0x7f3d941f17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:58.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.805+0000 7f3d9979b700 1 --2- 172.21.15.156:0/2284681349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d9415a760 0x7f3d941f17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:58.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.805+0000 7f3d9979b700 1 -- 172.21.15.156:0/2284681349 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c009b30 con 0x7f3d940cded0 2023-11-28T13:08:58.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.805+0000 7f3d98f9a700 1 --2- 172.21.15.156:0/2284681349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d9415a760 0x7f3d941f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:08:58.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.806+0000 7f3d9979b700 1 --2- 172.21.15.156:0/2284681349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d940cded0 0x7f3d940ca240 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000c4d0 tx=0x7f3d9000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:58.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.806+0000 7f3d89ffb700 1 -- 172.21.15.156:0/2284681349 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d90011790 con 0x7f3d940cded0 2023-11-28T13:08:58.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.806+0000 7f3d89ffb700 1 -- 172.21.15.156:0/2284681349 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d900118f0 con 0x7f3d940cded0 2023-11-28T13:08:58.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.806+0000 7f3d89ffb700 1 -- 172.21.15.156:0/2284681349 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d90011ca0 con 0x7f3d940cded0 2023-11-28T13:08:58.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.806+0000 7f3d9b9ff700 1 -- 172.21.15.156:0/2284681349 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d940b5670 con 0x7f3d940cded0 2023-11-28T13:08:58.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.806+0000 7f3d9b9ff700 1 -- 172.21.15.156:0/2284681349 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d940b5390 con 0x7f3d940cded0 2023-11-28T13:08:58.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.807+0000 7f3d9b9ff700 1 -- 172.21.15.156:0/2284681349 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d94160720 con 0x7f3d940cded0 2023-11-28T13:08:58.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.808+0000 7f3d89ffb700 1 -- 172.21.15.156:0/2284681349 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3d90010050 con 0x7f3d940cded0 2023-11-28T13:08:58.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.809+0000 7f3d89ffb700 1 --2- 172.21.15.156:0/2284681349 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d780682f0 0x7f3d7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:58.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.809+0000 7f3d89ffb700 1 -- 172.21.15.156:0/2284681349 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f3d90085940 con 0x7f3d940cded0 2023-11-28T13:08:58.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.809+0000 7f3d98f9a700 1 --2- 172.21.15.156:0/2284681349 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d780682f0 0x7f3d7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:58.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.810+0000 7f3d98f9a700 1 --2- 172.21.15.156:0/2284681349 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d780682f0 0x7f3d7806a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3d9414e880 tx=0x7f3d84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:58.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:58.813+0000 7f3d89ffb700 1 -- 172.21.15.156:0/2284681349 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3d900521d0 con 0x7f3d940cded0 2023-11-28T13:08:59.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.004+0000 7f3d9b9ff700 1 -- 172.21.15.156:0/2284681349 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3d940b71a0 con 0x7f3d940cded0 2023-11-28T13:08:59.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.582+0000 7f3d89ffb700 1 -- 172.21.15.156:0/2284681349 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3d940b71a0 con 0x7f3d940cded0 2023-11-28T13:08:59.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.585+0000 7f3d9b9ff700 1 -- 172.21.15.156:0/2284681349 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d780682f0 msgr2=0x7f3d7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:59.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.585+0000 7f3d9b9ff700 1 --2- 172.21.15.156:0/2284681349 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d780682f0 0x7f3d7806a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3d9414e880 tx=0x7f3d84008040 comp rx=0 tx=0).stop 2023-11-28T13:08:59.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.585+0000 7f3d9b9ff700 1 -- 172.21.15.156:0/2284681349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d940cded0 msgr2=0x7f3d940ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:59.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.585+0000 7f3d9b9ff700 1 --2- 172.21.15.156:0/2284681349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d940cded0 0x7f3d940ca240 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000c4d0 tx=0x7f3d9000c890 comp rx=0 tx=0).stop 2023-11-28T13:08:59.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.585+0000 7f3d9b9ff700 1 -- 172.21.15.156:0/2284681349 shutdown_connections 2023-11-28T13:08:59.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.585+0000 7f3d9b9ff700 1 --2- 172.21.15.156:0/2284681349 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d780682f0 0x7f3d7806a7a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:59.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.585+0000 7f3d9b9ff700 1 --2- 172.21.15.156:0/2284681349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d9415dc30 0x7f3d941f1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:59.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.585+0000 7f3d9b9ff700 1 --2- 172.21.15.156:0/2284681349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d940cded0 0x7f3d940ca240 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:59.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.585+0000 7f3d9b9ff700 1 --2- 172.21.15.156:0/2284681349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d9415a760 0x7f3d941f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:59.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.585+0000 7f3d9b9ff700 1 -- 172.21.15.156:0/2284681349 >> 172.21.15.156:0/2284681349 conn(0x7f3d94072620 msgr2=0x7f3d940b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:59.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.585+0000 7f3d9b9ff700 1 -- 172.21.15.156:0/2284681349 shutdown_connections 2023-11-28T13:08:59.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.585+0000 7f3d9b9ff700 1 -- 172.21.15.156:0/2284681349 wait complete. 2023-11-28T13:08:59.587 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_data' removed 2023-11-28T13:08:59.605 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T13:08:59.606 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T13:08:59.606 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T13:08:59.606 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:59.607 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T13:08:59.607 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T13:08:59.607 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T13:08:59.607 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:59.607 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T13:08:59.607 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T13:08:59.608 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T13:08:59.608 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:59.608 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T13:08:59.608 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T13:08:59.608 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T13:08:59.608 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:59.625 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T13:08:59.625 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T13:08:59.626 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T13:08:59.626 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T13:08:59.627 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T13:08:59.627 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T13:08:59.628 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T13:08:59.628 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T13:08:59.670 INFO:tasks.ceph.mds.a:Started 2023-11-28T13:08:59.671 DEBUG:teuthology.parallel:result is None 2023-11-28T13:08:59.672 INFO:tasks.ceph.mds.b:Started 2023-11-28T13:08:59.672 DEBUG:teuthology.parallel:result is None 2023-11-28T13:08:59.673 INFO:tasks.ceph.mds.c:Started 2023-11-28T13:08:59.674 DEBUG:teuthology.parallel:result is None 2023-11-28T13:08:59.675 INFO:tasks.ceph.mds.d:Started 2023-11-28T13:08:59.675 DEBUG:teuthology.parallel:result is None 2023-11-28T13:08:59.675 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T13:08:59.821 INFO:tasks.ceph.mds.a.smithi156.stdout:starting mds.a at 2023-11-28T13:08:59.822 INFO:tasks.ceph.mds.c.smithi156.stdout:starting mds.c at 2023-11-28T13:08:59.823 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2023-11-28T13:08:59.825 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2023-11-28T13:08:59.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.898+0000 7f00113ff700 1 Processor -- start 2023-11-28T13:08:59.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.898+0000 7f00113ff700 1 -- start start 2023-11-28T13:08:59.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.899+0000 7f00113ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f000c0cd7c0 0x7f000c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:59.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.899+0000 7f00113ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000c0d1950 con 0x7f000c15c9b0 2023-11-28T13:08:59.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.899+0000 7f00113ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000c0ce140 con 0x7f000c161340 2023-11-28T13:08:59.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.899+0000 7f00113ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0b5800 con 0x7f000c0cd7c0 2023-11-28T13:08:59.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.899+0000 7f000affd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f000c15c9b0 0x7f000c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:33388/0 (socket says 172.21.15.156:33388) 2023-11-28T13:08:59.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.899+0000 7f000affd700 1 -- 172.21.15.156:0/3593127816 learned_addr learned my addr 172.21.15.156:0/3593127816 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:08:59.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.900+0000 7f000b7fe700 1 --2- 172.21.15.156:0/3593127816 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f000c0cd7c0 0x7f000c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:59.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.900+0000 7f0009ffb700 1 -- 172.21.15.156:0/3593127816 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461687207 0 0) 0x7f000c0d1950 con 0x7f000c15c9b0 2023-11-28T13:08:59.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.900+0000 7f0009ffb700 1 -- 172.21.15.156:0/3593127816 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efff40035b0 con 0x7f000c15c9b0 2023-11-28T13:08:59.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.900+0000 7f0009ffb700 1 -- 172.21.15.156:0/3593127816 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2028466717 0 0) 0x7efff40035b0 con 0x7f000c15c9b0 2023-11-28T13:08:59.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.900+0000 7f0009ffb700 1 -- 172.21.15.156:0/3593127816 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f000c0cd7c0 msgr2=0x7f000c0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:59.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.900+0000 7f0009ffb700 1 --2- 172.21.15.156:0/3593127816 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f000c0cd7c0 0x7f000c0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:59.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.900+0000 7f0009ffb700 1 -- 172.21.15.156:0/3593127816 >> v1:172.21.15.186:6789/0 conn(0x7f000c161340 legacy=0x7f000c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:59.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.900+0000 7f0009ffb700 1 -- 172.21.15.156:0/3593127816 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c0d24f0 con 0x7f000c15c9b0 2023-11-28T13:08:59.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.900+0000 7f0009ffb700 1 -- 172.21.15.156:0/3593127816 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7efffc001670 con 0x7f000c15c9b0 2023-11-28T13:08:59.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.900+0000 7f0009ffb700 1 -- 172.21.15.156:0/3593127816 >> v1:172.21.15.156:6789/0 conn(0x7f000c15c9b0 legacy=0x7f000c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:59.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.900+0000 7f0009ffb700 1 --2- 172.21.15.156:0/3593127816 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f000c0cd7c0 0x7efff4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:59.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.900+0000 7f0009ffb700 1 --2- 172.21.15.156:0/3593127816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efff4003f70 0x7efff4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:59.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.900+0000 7f0009ffb700 1 --2- 172.21.15.156:0/3593127816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efff4004d90 0x7efff4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:59.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.900+0000 7f0009ffb700 1 -- 172.21.15.156:0/3593127816 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0b5800 con 0x7efff4004d90 2023-11-28T13:08:59.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.900+0000 7f0009ffb700 1 -- 172.21.15.156:0/3593127816 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff400bb50 con 0x7efff4003f70 2023-11-28T13:08:59.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.900+0000 7f0009ffb700 1 -- 172.21.15.156:0/3593127816 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff400bd10 con 0x7f000c0cd7c0 2023-11-28T13:08:59.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.900+0000 7f000affd700 1 --2- 172.21.15.156:0/3593127816 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f000c0cd7c0 0x7efff4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:59.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.901+0000 7f000a7fc700 1 --2- 172.21.15.156:0/3593127816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efff4003f70 0x7efff4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:59.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.901+0000 7f000b7fe700 1 --2- 172.21.15.156:0/3593127816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efff4004d90 0x7efff4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:59.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.901+0000 7f000affd700 1 -- 172.21.15.156:0/3593127816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efff4003f70 msgr2=0x7efff4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:59.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.901+0000 7f000affd700 1 --2- 172.21.15.156:0/3593127816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efff4003f70 0x7efff4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:59.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.901+0000 7f000affd700 1 -- 172.21.15.156:0/3593127816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efff4004d90 msgr2=0x7efff4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:59.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.901+0000 7f000affd700 1 --2- 172.21.15.156:0/3593127816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efff4004d90 0x7efff4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:59.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.901+0000 7f000affd700 1 -- 172.21.15.156:0/3593127816 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff40035b0 con 0x7f000c0cd7c0 2023-11-28T13:08:59.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.901+0000 7f000affd700 1 --2- 172.21.15.156:0/3593127816 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f000c0cd7c0 0x7efff4003a30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efffc00c1e0 tx=0x7efff4005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=531dc854c420c36d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:59.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.901+0000 7f0009ffb700 1 -- 172.21.15.156:0/3593127816 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc007b70 con 0x7f000c0cd7c0 2023-11-28T13:08:59.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.901+0000 7f0009ffb700 1 -- 172.21.15.156:0/3593127816 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efffc00ccc0 con 0x7f000c0cd7c0 2023-11-28T13:08:59.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.901+0000 7f0009ffb700 1 -- 172.21.15.156:0/3593127816 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc005ba0 con 0x7f000c0cd7c0 2023-11-28T13:08:59.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.902+0000 7f00113ff700 1 -- 172.21.15.156:0/3593127816 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f000c0cd7c0 msgr2=0x7efff4003a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:59.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.902+0000 7f00113ff700 1 --2- 172.21.15.156:0/3593127816 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f000c0cd7c0 0x7efff4003a30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efffc00c1e0 tx=0x7efff4005880 comp rx=0 tx=0).stop 2023-11-28T13:08:59.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.902+0000 7f000b7fe700 1 -- 172.21.15.156:0/3593127816 reap_dead start 2023-11-28T13:08:59.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.902+0000 7f00113ff700 1 -- 172.21.15.156:0/3593127816 shutdown_connections 2023-11-28T13:08:59.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.902+0000 7f00113ff700 1 -- 172.21.15.156:0/3593127816 >> 172.21.15.156:0/3593127816 conn(0x7f000c072600 msgr2=0x7f000c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:59.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.902+0000 7f00113ff700 1 -- 172.21.15.156:0/3593127816 shutdown_connections 2023-11-28T13:08:59.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.902+0000 7f00113ff700 1 -- 172.21.15.156:0/3593127816 wait complete. 2023-11-28T13:08:59.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.904+0000 7f00113ff700 1 Processor -- start 2023-11-28T13:08:59.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.904+0000 7f00113ff700 1 -- start start 2023-11-28T13:08:59.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.905+0000 7f00113ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f000c0cd7c0 0x7f000c1575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:59.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.905+0000 7f00113ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f000c0caaa0 0x7f000c1f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:59.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.905+0000 7f00113ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f000c1f2d40 0x7f000c0c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:59.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.905+0000 7f00113ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0b7120 con 0x7f000c1f2d40 2023-11-28T13:08:59.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.905+0000 7f00113ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c1f3650 con 0x7f000c0cd7c0 2023-11-28T13:08:59.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.905+0000 7f00113ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c1f6010 con 0x7f000c0caaa0 2023-11-28T13:08:59.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.905+0000 7f000b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f000c1f2d40 0x7f000c0c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:59.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.905+0000 7f000b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f000c1f2d40 0x7f000c0c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41454/0 (socket says 172.21.15.156:41454) 2023-11-28T13:08:59.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.905+0000 7f000b7fe700 1 -- 172.21.15.156:0/3785017748 learned_addr learned my addr 172.21.15.156:0/3785017748 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:08:59.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.905+0000 7f000a7fc700 1 --2- 172.21.15.156:0/3785017748 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f000c0caaa0 0x7f000c1f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:59.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.905+0000 7f000affd700 1 --2- 172.21.15.156:0/3785017748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f000c0cd7c0 0x7f000c1575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:59.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.906+0000 7f000b7fe700 1 -- 172.21.15.156:0/3785017748 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f000c0caaa0 msgr2=0x7f000c1f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:59.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.906+0000 7f000b7fe700 1 --2- 172.21.15.156:0/3785017748 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f000c0caaa0 0x7f000c1f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:59.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.906+0000 7f000b7fe700 1 -- 172.21.15.156:0/3785017748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f000c0cd7c0 msgr2=0x7f000c1575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:59.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.906+0000 7f000b7fe700 1 --2- 172.21.15.156:0/3785017748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f000c0cd7c0 0x7f000c1575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:59.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.906+0000 7f000b7fe700 1 -- 172.21.15.156:0/3785017748 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c0d1300 con 0x7f000c1f2d40 2023-11-28T13:08:59.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.906+0000 7f000b7fe700 1 --2- 172.21.15.156:0/3785017748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f000c1f2d40 0x7f000c0c6ca0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f000c06bbb0 tx=0x7f0004011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:59.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.906+0000 7efff37fe700 1 -- 172.21.15.156:0/3785017748 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000400e040 con 0x7f000c1f2d40 2023-11-28T13:08:59.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.906+0000 7efff37fe700 1 -- 172.21.15.156:0/3785017748 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000400f040 con 0x7f000c1f2d40 2023-11-28T13:08:59.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.906+0000 7efff37fe700 1 -- 172.21.15.156:0/3785017748 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000400f950 con 0x7f000c1f2d40 2023-11-28T13:08:59.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.906+0000 7f00113ff700 1 -- 172.21.15.156:0/3785017748 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f000c0b5520 con 0x7f000c1f2d40 2023-11-28T13:08:59.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.906+0000 7f00113ff700 1 -- 172.21.15.156:0/3785017748 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000c0d15e0 con 0x7f000c1f2d40 2023-11-28T13:08:59.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.907+0000 7f00113ff700 1 -- 172.21.15.156:0/3785017748 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000c0b7260 con 0x7f000c1f2d40 2023-11-28T13:08:59.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.908+0000 7efff37fe700 1 -- 172.21.15.156:0/3785017748 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0004025070 con 0x7f000c1f2d40 2023-11-28T13:08:59.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.909+0000 7efff37fe700 1 --2- 172.21.15.156:0/3785017748 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efff8068320 0x7efff806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:59.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.909+0000 7efff37fe700 1 -- 172.21.15.156:0/3785017748 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0004085e40 con 0x7f000c1f2d40 2023-11-28T13:08:59.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.909+0000 7f000affd700 1 --2- 172.21.15.156:0/3785017748 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efff8068320 0x7efff806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:59.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.910+0000 7f000affd700 1 --2- 172.21.15.156:0/3785017748 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efff8068320 0x7efff806a7d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efffc015040 tx=0x7efffc007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:59.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:08:59.913+0000 7efff37fe700 1 -- 172.21.15.156:0/3785017748 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f000c0b7260 con 0x7f000c1f2d40 2023-11-28T13:09:00.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:00.124+0000 7f00113ff700 1 -- 172.21.15.156:0/3785017748 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f000c0b7260 con 0x7f000c1f2d40 2023-11-28T13:09:00.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:00.863+0000 7efff37fe700 1 -- 172.21.15.156:0/3785017748 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7f000c0b7260 con 0x7f000c1f2d40 2023-11-28T13:09:00.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:00.866+0000 7f00113ff700 1 -- 172.21.15.156:0/3785017748 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efff8068320 msgr2=0x7efff806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:00.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:00.866+0000 7f00113ff700 1 --2- 172.21.15.156:0/3785017748 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efff8068320 0x7efff806a7d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efffc015040 tx=0x7efffc007000 comp rx=0 tx=0).stop 2023-11-28T13:09:00.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:00.866+0000 7f00113ff700 1 -- 172.21.15.156:0/3785017748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f000c1f2d40 msgr2=0x7f000c0c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:00.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:00.866+0000 7f00113ff700 1 --2- 172.21.15.156:0/3785017748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f000c1f2d40 0x7f000c0c6ca0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f000c06bbb0 tx=0x7f0004011f40 comp rx=0 tx=0).stop 2023-11-28T13:09:00.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:00.866+0000 7f00113ff700 1 -- 172.21.15.156:0/3785017748 shutdown_connections 2023-11-28T13:09:00.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:00.866+0000 7f00113ff700 1 --2- 172.21.15.156:0/3785017748 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efff8068320 0x7efff806a7d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:00.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:00.866+0000 7f00113ff700 1 --2- 172.21.15.156:0/3785017748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f000c1f2d40 0x7f000c0c6ca0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:00.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:00.866+0000 7f00113ff700 1 --2- 172.21.15.156:0/3785017748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f000c0cd7c0 0x7f000c1575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:00.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:00.866+0000 7f00113ff700 1 --2- 172.21.15.156:0/3785017748 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f000c0caaa0 0x7f000c1f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:00.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:00.866+0000 7f00113ff700 1 -- 172.21.15.156:0/3785017748 >> 172.21.15.156:0/3785017748 conn(0x7f000c072600 msgr2=0x7f000c0b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:00.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:00.866+0000 7f00113ff700 1 -- 172.21.15.156:0/3785017748 shutdown_connections 2023-11-28T13:09:00.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:00.866+0000 7f00113ff700 1 -- 172.21.15.156:0/3785017748 wait complete. 2023-11-28T13:09:00.868 INFO:teuthology.orchestra.run.smithi156.stderr: removed all blocklist entries 2023-11-28T13:09:00.885 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T13:09:01.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.081+0000 7fa81ddff700 1 Processor -- start 2023-11-28T13:09:01.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.081+0000 7fa81ddff700 1 -- start start 2023-11-28T13:09:01.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.082+0000 7fa81ddff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa818161340 0x7fa8180d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:01.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.082+0000 7fa81ddff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa8180d2320 0x7fa8180cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:01.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.083+0000 7fa81ddff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8180d1950 con 0x7fa81815c9b0 2023-11-28T13:09:01.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.083+0000 7fa81ddff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8180b5800 con 0x7fa8180d2320 2023-11-28T13:09:01.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.083+0000 7fa81ddff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180b5300 con 0x7fa818161340 2023-11-28T13:09:01.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.083+0000 7fa8177fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa81815c9b0 0x7fa8180b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:33394/0 (socket says 172.21.15.156:33394) 2023-11-28T13:09:01.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.083+0000 7fa8177fe700 1 -- 172.21.15.156:0/2203791588 learned_addr learned my addr 172.21.15.156:0/2203791588 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:01.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.083+0000 7fa817fff700 1 --2- 172.21.15.156:0/2203791588 >> v2:172.21.15.186:3300/0 conn(0x7fa8180d2320 0x7fa8180cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:01.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.083+0000 7fa816ffd700 1 --2- 172.21.15.156:0/2203791588 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa818161340 0x7fa8180d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:01.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.083+0000 7fa8167fc700 1 -- 172.21.15.156:0/2203791588 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147259508 0 0) 0x7fa8180d1950 con 0x7fa81815c9b0 2023-11-28T13:09:01.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.083+0000 7fa8167fc700 1 -- 172.21.15.156:0/2203791588 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8000035b0 con 0x7fa81815c9b0 2023-11-28T13:09:01.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.083+0000 7fa817fff700 1 -- 172.21.15.156:0/2203791588 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa818161340 msgr2=0x7fa8180d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:01.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.083+0000 7fa817fff700 1 --2- 172.21.15.156:0/2203791588 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa818161340 0x7fa8180d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.083+0000 7fa817fff700 1 -- 172.21.15.156:0/2203791588 >> v1:172.21.15.156:6789/0 conn(0x7fa81815c9b0 legacy=0x7fa8180b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:01.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.083+0000 7fa817fff700 1 -- 172.21.15.156:0/2203791588 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8180b5520 con 0x7fa8180d2320 2023-11-28T13:09:01.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.084+0000 7fa817fff700 1 --2- 172.21.15.156:0/2203791588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8180d2320 0x7fa8180cd7c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa810009dc0 tx=0x7fa81000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed071d8f2834493 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:01.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.084+0000 7fa8167fc700 1 -- 172.21.15.156:0/2203791588 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81000e070 con 0x7fa8180d2320 2023-11-28T13:09:01.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.084+0000 7fa8167fc700 1 -- 172.21.15.156:0/2203791588 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81000bae0 con 0x7fa8180d2320 2023-11-28T13:09:01.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.084+0000 7fa8167fc700 1 -- 172.21.15.156:0/2203791588 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81000be90 con 0x7fa8180d2320 2023-11-28T13:09:01.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.085+0000 7fa81ddff700 1 -- 172.21.15.156:0/2203791588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8180d2320 msgr2=0x7fa8180cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:01.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.085+0000 7fa81ddff700 1 --2- 172.21.15.156:0/2203791588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8180d2320 0x7fa8180cd7c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa810009dc0 tx=0x7fa81000b330 comp rx=0 tx=0).stop 2023-11-28T13:09:01.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.085+0000 7fa81ddff700 1 -- 172.21.15.156:0/2203791588 shutdown_connections 2023-11-28T13:09:01.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.085+0000 7fa81ddff700 1 --2- 172.21.15.156:0/2203791588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8180d2320 0x7fa8180cd7c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.085+0000 7fa81ddff700 1 --2- 172.21.15.156:0/2203791588 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa818161340 0x7fa8180d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.085+0000 7fa81ddff700 1 -- 172.21.15.156:0/2203791588 >> 172.21.15.156:0/2203791588 conn(0x7fa818072600 msgr2=0x7fa8180cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:01.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.085+0000 7fa81ddff700 1 -- 172.21.15.156:0/2203791588 shutdown_connections 2023-11-28T13:09:01.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.085+0000 7fa81ddff700 1 -- 172.21.15.156:0/2203791588 wait complete. 2023-11-28T13:09:01.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.086+0000 7fa81ddff700 1 Processor -- start 2023-11-28T13:09:01.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.087+0000 7fa81ddff700 1 -- start start 2023-11-28T13:09:01.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.087+0000 7fa81ddff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8180d2320 0x7fa818150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:01.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.087+0000 7fa81ddff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa81815c9b0 0x7fa8180c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:01.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.087+0000 7fa81ddff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa818161340 0x7fa8180b7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:01.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.087+0000 7fa81ddff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa818152870 con 0x7fa81815c9b0 2023-11-28T13:09:01.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.087+0000 7fa81ddff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180c6e20 con 0x7fa818161340 2023-11-28T13:09:01.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.087+0000 7fa81ddff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180c7100 con 0x7fa8180d2320 2023-11-28T13:09:01.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.088+0000 7fa816ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa81815c9b0 0x7fa8180c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:01.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.088+0000 7fa816ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa81815c9b0 0x7fa8180c7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41462/0 (socket says 172.21.15.156:41462) 2023-11-28T13:09:01.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.088+0000 7fa816ffd700 1 -- 172.21.15.156:0/1266554455 learned_addr learned my addr 172.21.15.156:0/1266554455 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:01.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.088+0000 7fa8177fe700 1 --2- 172.21.15.156:0/1266554455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8180d2320 0x7fa818150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:01.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.088+0000 7fa817fff700 1 --2- 172.21.15.156:0/1266554455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa818161340 0x7fa8180b7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:01.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.088+0000 7fa816ffd700 1 -- 172.21.15.156:0/1266554455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8180d2320 msgr2=0x7fa818150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:01.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.088+0000 7fa816ffd700 1 --2- 172.21.15.156:0/1266554455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8180d2320 0x7fa818150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.088+0000 7fa816ffd700 1 -- 172.21.15.156:0/1266554455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa818161340 msgr2=0x7fa8180b7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:01.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.088+0000 7fa816ffd700 1 --2- 172.21.15.156:0/1266554455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa818161340 0x7fa8180b7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.088+0000 7fa816ffd700 1 -- 172.21.15.156:0/1266554455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa810009a70 con 0x7fa81815c9b0 2023-11-28T13:09:01.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.088+0000 7fa8177fe700 1 --2- 172.21.15.156:0/1266554455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8180d2320 0x7fa818150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:01.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.088+0000 7fa816ffd700 1 --2- 172.21.15.156:0/1266554455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa81815c9b0 0x7fa8180c7680 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00e870 tx=0x7fa80c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:01.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.089+0000 7fa7fffff700 1 -- 172.21.15.156:0/1266554455 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c009590 con 0x7fa81815c9b0 2023-11-28T13:09:01.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.089+0000 7fa7fffff700 1 -- 172.21.15.156:0/1266554455 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c00ee70 con 0x7fa81815c9b0 2023-11-28T13:09:01.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.089+0000 7fa7fffff700 1 -- 172.21.15.156:0/1266554455 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c00ca50 con 0x7fa81815c9b0 2023-11-28T13:09:01.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.089+0000 7fa81ddff700 1 -- 172.21.15.156:0/1266554455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8180d15e0 con 0x7fa81815c9b0 2023-11-28T13:09:01.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.089+0000 7fa81ddff700 1 -- 172.21.15.156:0/1266554455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8180d1300 con 0x7fa81815c9b0 2023-11-28T13:09:01.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.090+0000 7fa81ddff700 1 -- 172.21.15.156:0/1266554455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81815cd80 con 0x7fa81815c9b0 2023-11-28T13:09:01.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.091+0000 7fa7fffff700 1 -- 172.21.15.156:0/1266554455 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa80c019030 con 0x7fa81815c9b0 2023-11-28T13:09:01.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.091+0000 7fa7fffff700 1 --2- 172.21.15.156:0/1266554455 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa8040682f0 0x7fa80406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:01.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.092+0000 7fa7fffff700 1 -- 172.21.15.156:0/1266554455 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa80c0858c0 con 0x7fa81815c9b0 2023-11-28T13:09:01.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.092+0000 7fa8177fe700 1 --2- 172.21.15.156:0/1266554455 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa8040682f0 0x7fa80406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:01.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.092+0000 7fa8177fe700 1 --2- 172.21.15.156:0/1266554455 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa8040682f0 0x7fa80406a7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa8080011e0 tx=0x7fa808009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:01.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.095+0000 7fa7fffff700 1 -- 172.21.15.156:0/1266554455 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa80c052380 con 0x7fa81815c9b0 2023-11-28T13:09:01.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.359+0000 7fa81ddff700 1 -- 172.21.15.156:0/1266554455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa8180745c0 con 0x7fa81815c9b0 2023-11-28T13:09:01.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.360+0000 7fa7fffff700 1 -- 172.21.15.156:0/1266554455 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fa80c0559a0 con 0x7fa81815c9b0 2023-11-28T13:09:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:01.361 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:09:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: "auth_dump": [ 2023-11-28T13:09:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.a", 2023-11-28T13:09:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVl34w0ERAA9H2i6sMS60YUKvNCit0idg==", 2023-11-28T13:09:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:09:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T13:09:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T13:09:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.b", 2023-11-28T13:09:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVlzVSSKRAA3APxxYZXNOOOnG1t7DIIDg==", 2023-11-28T13:09:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:09:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T13:09:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T13:09:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.c", 2023-11-28T13:09:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVl++RtHRAAsenmAb/Y3eRvWw22940lkw==", 2023-11-28T13:09:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:09:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T13:09:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T13:09:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.367 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.d", 2023-11-28T13:09:01.367 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVlFDP1NRAASlTZM2P0RL+lcFskrr08uQ==", 2023-11-28T13:09:01.367 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.368 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:09:01.368 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T13:09:01.368 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T13:09:01.368 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:01.368 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.369 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.369 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.369 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.0", 2023-11-28T13:09:01.369 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD95WVlF1g2BBAAM1AkiYFhmzcy6f2xOUHuSA==", 2023-11-28T13:09:01.369 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.369 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:09:01.369 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:09:01.369 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:01.369 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.369 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.370 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.370 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.1", 2023-11-28T13:09:01.370 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAA5mVlMP+VChAAvBK9dslAIznG9cJCnmarfg==", 2023-11-28T13:09:01.370 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.370 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:09:01.370 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:09:01.370 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:01.370 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.371 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.371 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.371 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.2", 2023-11-28T13:09:01.371 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAD5mVlY5TMERAAH2W59d+/mWQ7vVOFV0uH5g==", 2023-11-28T13:09:01.371 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.372 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:09:01.372 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:09:01.372 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:01.372 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.372 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.372 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.372 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.3", 2023-11-28T13:09:01.372 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAG5mVlKreoGBAAHwbSd1vt4sF/Ahmz97lv5A==", 2023-11-28T13:09:01.373 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.373 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:09:01.373 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:09:01.373 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:01.373 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.373 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.373 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.373 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.4", 2023-11-28T13:09:01.374 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAM5mVlVb7BORAA2/l64tOgJl6WRDwI1Ee+YA==", 2023-11-28T13:09:01.374 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.374 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:09:01.374 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:09:01.374 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:01.374 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.374 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.374 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.374 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.5", 2023-11-28T13:09:01.374 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAQ5mVlIl0jBRAAYrEb4PGL8amZxfsGUhE0BQ==", 2023-11-28T13:09:01.375 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.375 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:09:01.375 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:09:01.375 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:01.375 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.375 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.375 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.375 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.6", 2023-11-28T13:09:01.375 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAT5mVlicggDRAANowLXao2cG7JHDi+ytdREg==", 2023-11-28T13:09:01.375 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:09:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:09:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.7", 2023-11-28T13:09:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAW5mVlPTz9EhAAyyq2SF/TYYStii1bExbDew==", 2023-11-28T13:09:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:09:01.377 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:09:01.377 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:01.377 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.377 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.377 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.377 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.0", 2023-11-28T13:09:01.377 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD75WVlhwXEBBAAu/NSgck5Joi7bKQoBne7MA==", 2023-11-28T13:09:01.377 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.377 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:09:01.378 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T13:09:01.378 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T13:09:01.378 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T13:09:01.378 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.378 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.378 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.378 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.1", 2023-11-28T13:09:01.378 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD75WVlSLGyFBAATUac8v+U0A55jCLv6SCsAA==", 2023-11-28T13:09:01.378 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.378 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:09:01.379 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T13:09:01.379 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T13:09:01.379 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T13:09:01.379 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.379 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.379 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.379 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.2", 2023-11-28T13:09:01.379 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD75WVl49+oDBAA/rsj8joMrNDo2FOlvvLvWQ==", 2023-11-28T13:09:01.379 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.379 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:09:01.380 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T13:09:01.380 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T13:09:01.380 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T13:09:01.380 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.380 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.380 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.380 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.3", 2023-11-28T13:09:01.380 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD75WVlmfuWHBAAbf1HvrNxzm3zJwTF6De2Kg==", 2023-11-28T13:09:01.380 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.380 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:09:01.381 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T13:09:01.381 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T13:09:01.381 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T13:09:01.381 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.381 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.381 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.381 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.admin", 2023-11-28T13:09:01.381 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD55WVld9ddFBAA6s9ybzLqB7e7Uj9gOd65/Q==", 2023-11-28T13:09:01.381 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.381 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T13:09:01.382 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T13:09:01.382 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T13:09:01.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:01.382 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.382 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.382 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.382 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.bootstrap-mds", 2023-11-28T13:09:01.382 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAe5mVl7oAmJBAAnfId8mX4R9hDgZuk7XpFzg==", 2023-11-28T13:09:01.382 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.383 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile bootstrap-mds" 2023-11-28T13:09:01.383 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.383 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.383 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.383 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.bootstrap-mgr", 2023-11-28T13:09:01.383 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAe5mVl35QmJBAAuPDStAEF94TW1TemXiC92w==", 2023-11-28T13:09:01.383 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.383 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-28T13:09:01.383 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.383 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.384 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.384 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.bootstrap-osd", 2023-11-28T13:09:01.384 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAe5mVlBaUmJBAAMy7HiFKJpN0mXkgNKD6dbQ==", 2023-11-28T13:09:01.384 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.384 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile bootstrap-osd" 2023-11-28T13:09:01.384 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.384 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.384 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.384 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.bootstrap-rbd", 2023-11-28T13:09:01.384 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAe5mVlybMmJBAATOCGcBvCkrAioxF7yRmVqA==", 2023-11-28T13:09:01.385 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.385 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-28T13:09:01.385 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.385 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.385 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.385 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-28T13:09:01.385 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAe5mVlQcMmJBAAPSrJ8JFGRnT7MiQ/9paGVA==", 2023-11-28T13:09:01.385 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.385 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-28T13:09:01.385 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.386 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.386 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.386 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.bootstrap-rgw", 2023-11-28T13:09:01.386 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAe5mVlb9MmJBAAJvKnpenK+yw6SB/LldRbUA==", 2023-11-28T13:09:01.386 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.386 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-28T13:09:01.386 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.386 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.386 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.387 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mgr.x", 2023-11-28T13:09:01.387 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVl3oYaCRAALjx/GgP3kX/gyD4ML7Af7Q==", 2023-11-28T13:09:01.387 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.387 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T13:09:01.387 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile mgr", 2023-11-28T13:09:01.387 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:01.387 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.387 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:01.387 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:01.387 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mgr.y", 2023-11-28T13:09:01.388 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVlhmWPABAA5F0d66op3eUWwWPKJ6Mc6w==", 2023-11-28T13:09:01.388 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:01.388 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T13:09:01.388 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile mgr", 2023-11-28T13:09:01.388 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:01.388 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.388 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:01.388 INFO:teuthology.orchestra.run.smithi156.stdout: ] 2023-11-28T13:09:01.388 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:09:01.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.362+0000 7fa81ddff700 1 -- 172.21.15.156:0/1266554455 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa8040682f0 msgr2=0x7fa80406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:01.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.362+0000 7fa81ddff700 1 --2- 172.21.15.156:0/1266554455 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa8040682f0 0x7fa80406a7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa8080011e0 tx=0x7fa808009040 comp rx=0 tx=0).stop 2023-11-28T13:09:01.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.362+0000 7fa81ddff700 1 -- 172.21.15.156:0/1266554455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa81815c9b0 msgr2=0x7fa8180c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:01.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.362+0000 7fa81ddff700 1 --2- 172.21.15.156:0/1266554455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa81815c9b0 0x7fa8180c7680 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00e870 tx=0x7fa80c00ec30 comp rx=0 tx=0).stop 2023-11-28T13:09:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.362+0000 7fa81ddff700 1 -- 172.21.15.156:0/1266554455 shutdown_connections 2023-11-28T13:09:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.362+0000 7fa81ddff700 1 --2- 172.21.15.156:0/1266554455 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa8040682f0 0x7fa80406a7a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.362+0000 7fa81ddff700 1 --2- 172.21.15.156:0/1266554455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa818161340 0x7fa8180b7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.362+0000 7fa81ddff700 1 --2- 172.21.15.156:0/1266554455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8180d2320 0x7fa818150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.362+0000 7fa81ddff700 1 --2- 172.21.15.156:0/1266554455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa81815c9b0 0x7fa8180c7680 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.362+0000 7fa81ddff700 1 -- 172.21.15.156:0/1266554455 >> 172.21.15.156:0/1266554455 conn(0x7fa818072600 msgr2=0x7fa81806a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.362+0000 7fa81ddff700 1 -- 172.21.15.156:0/1266554455 shutdown_connections 2023-11-28T13:09:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.362+0000 7fa81ddff700 1 -- 172.21.15.156:0/1266554455 wait complete. 2023-11-28T13:09:01.392 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-28T13:09:01.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.590+0000 7f2672dff700 1 Processor -- start 2023-11-28T13:09:01.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.590+0000 7f2672dff700 1 -- start start 2023-11-28T13:09:01.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.591+0000 7f2672dff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f266c0d01b0 0x7f266c0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:01.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.591+0000 7f2672dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f266c0b20b0 0x7f266c0d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:01.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.591+0000 7f2672dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266c0b0230 con 0x7f266c0b29b0 2023-11-28T13:09:01.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.591+0000 7f2672dff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266c15cec0 con 0x7f266c0d01b0 2023-11-28T13:09:01.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.591+0000 7f2672dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266c0b4570 con 0x7f266c0b20b0 2023-11-28T13:09:01.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.591+0000 7f2670b9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f266c0d01b0 0x7f266c0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:01.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.591+0000 7f2670b9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f266c0d01b0 0x7f266c0d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41470/0 (socket says 172.21.15.156:41470) 2023-11-28T13:09:01.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.591+0000 7f2670b9b700 1 -- 172.21.15.156:0/3123758453 learned_addr learned my addr 172.21.15.156:0/3123758453 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:01.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.591+0000 7f266bfff700 1 --2- 172.21.15.156:0/3123758453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f266c0b20b0 0x7f266c0d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:01.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.592+0000 7f2670b9b700 1 -- 172.21.15.156:0/3123758453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f266c0b20b0 msgr2=0x7f266c0d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:01.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.592+0000 7f2670b9b700 1 --2- 172.21.15.156:0/3123758453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f266c0b20b0 0x7f266c0d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.592+0000 7f2670b9b700 1 -- 172.21.15.156:0/3123758453 >> v1:172.21.15.186:6789/0 conn(0x7f266c0b29b0 legacy=0x7f266c0b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:01.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.592+0000 7f2670b9b700 1 -- 172.21.15.156:0/3123758453 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c0b4850 con 0x7f266c0d01b0 2023-11-28T13:09:01.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.592+0000 7f2670b9b700 1 --2- 172.21.15.156:0/3123758453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266c0d01b0 0x7f266c0d0c20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f26600098f0 tx=0x7f2660009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c546b7a4b66d9de5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:01.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.592+0000 7f266b7fe700 1 -- 172.21.15.156:0/3123758453 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660011830 con 0x7f266c0d01b0 2023-11-28T13:09:01.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.592+0000 7f266b7fe700 1 -- 172.21.15.156:0/3123758453 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2660011990 con 0x7f266c0d01b0 2023-11-28T13:09:01.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.592+0000 7f266b7fe700 1 -- 172.21.15.156:0/3123758453 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660011d20 con 0x7f266c0d01b0 2023-11-28T13:09:01.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.593+0000 7f2672dff700 1 -- 172.21.15.156:0/3123758453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266c0d01b0 msgr2=0x7f266c0d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:01.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.593+0000 7f2672dff700 1 --2- 172.21.15.156:0/3123758453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266c0d01b0 0x7f266c0d0c20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f26600098f0 tx=0x7f2660009c00 comp rx=0 tx=0).stop 2023-11-28T13:09:01.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.593+0000 7f2672dff700 1 -- 172.21.15.156:0/3123758453 shutdown_connections 2023-11-28T13:09:01.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.593+0000 7f2672dff700 1 --2- 172.21.15.156:0/3123758453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266c0d01b0 0x7f266c0d0c20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.593+0000 7f2672dff700 1 --2- 172.21.15.156:0/3123758453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f266c0b20b0 0x7f266c0d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.593+0000 7f2672dff700 1 -- 172.21.15.156:0/3123758453 >> 172.21.15.156:0/3123758453 conn(0x7f266c09c3f0 msgr2=0x7f266c0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:01.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.593+0000 7f2672dff700 1 -- 172.21.15.156:0/3123758453 shutdown_connections 2023-11-28T13:09:01.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.593+0000 7f2672dff700 1 -- 172.21.15.156:0/3123758453 wait complete. 2023-11-28T13:09:01.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.595+0000 7f2672dff700 1 Processor -- start 2023-11-28T13:09:01.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.595+0000 7f2672dff700 1 -- start start 2023-11-28T13:09:01.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.596+0000 7f2672dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f266c0b20b0 0x7f266c0ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:01.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.596+0000 7f2672dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f266c0b29b0 0x7f266c15a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:01.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.596+0000 7f2672dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266c0d01b0 0x7f266c15a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:01.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.596+0000 7f2672dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c150d70 con 0x7f266c0d01b0 2023-11-28T13:09:01.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.596+0000 7f2672dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c0b7750 con 0x7f266c0b20b0 2023-11-28T13:09:01.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.596+0000 7f2672dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266c150fc0 con 0x7f266c0b29b0 2023-11-28T13:09:01.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.596+0000 7f267139c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266c0d01b0 0x7f266c15a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:01.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.596+0000 7f267139c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266c0d01b0 0x7f266c15a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41478/0 (socket says 172.21.15.156:41478) 2023-11-28T13:09:01.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.596+0000 7f2670b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f266c0b20b0 0x7f266c0ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:01.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.596+0000 7f267139c700 1 -- 172.21.15.156:0/2826308103 learned_addr learned my addr 172.21.15.156:0/2826308103 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:01.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.596+0000 7f266bfff700 1 --2- 172.21.15.156:0/2826308103 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f266c0b29b0 0x7f266c15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:01.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.597+0000 7f267139c700 1 -- 172.21.15.156:0/2826308103 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f266c0b29b0 msgr2=0x7f266c15a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:01.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.597+0000 7f267139c700 1 --2- 172.21.15.156:0/2826308103 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f266c0b29b0 0x7f266c15a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.597+0000 7f267139c700 1 -- 172.21.15.156:0/2826308103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f266c0b20b0 msgr2=0x7f266c0ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:01.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.597+0000 7f267139c700 1 --2- 172.21.15.156:0/2826308103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f266c0b20b0 0x7f266c0ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.597+0000 7f267139c700 1 -- 172.21.15.156:0/2826308103 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2660009550 con 0x7f266c0d01b0 2023-11-28T13:09:01.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.597+0000 7f267139c700 1 --2- 172.21.15.156:0/2826308103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266c0d01b0 0x7f266c15a890 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f266400cc50 tx=0x7f266400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:01.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.597+0000 7f26697fa700 1 -- 172.21.15.156:0/2826308103 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664003c00 con 0x7f266c0d01b0 2023-11-28T13:09:01.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.597+0000 7f26697fa700 1 -- 172.21.15.156:0/2826308103 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2664003d60 con 0x7f266c0d01b0 2023-11-28T13:09:01.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.597+0000 7f26697fa700 1 -- 172.21.15.156:0/2826308103 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266400aab0 con 0x7f266c0d01b0 2023-11-28T13:09:01.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.597+0000 7f2672dff700 1 -- 172.21.15.156:0/2826308103 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266c0afec0 con 0x7f266c0d01b0 2023-11-28T13:09:01.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.597+0000 7f2672dff700 1 -- 172.21.15.156:0/2826308103 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266c0afbe0 con 0x7f266c0d01b0 2023-11-28T13:09:01.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.598+0000 7f2672dff700 1 -- 172.21.15.156:0/2826308103 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266c0b2600 con 0x7f266c0d01b0 2023-11-28T13:09:01.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.599+0000 7f26697fa700 1 -- 172.21.15.156:0/2826308103 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2664010050 con 0x7f266c0d01b0 2023-11-28T13:09:01.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.600+0000 7f26697fa700 1 --2- 172.21.15.156:0/2826308103 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26500682f0 0x7f265006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:01.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.601+0000 7f26697fa700 1 -- 172.21.15.156:0/2826308103 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2664085870 con 0x7f266c0d01b0 2023-11-28T13:09:01.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.601+0000 7f2670b9b700 1 --2- 172.21.15.156:0/2826308103 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26500682f0 0x7f265006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:01.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.602+0000 7f2670b9b700 1 --2- 172.21.15.156:0/2826308103 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26500682f0 0x7f265006a7a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2660014040 tx=0x7f2660008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:01.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.604+0000 7f26697fa700 1 -- 172.21.15.156:0/2826308103 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2664052330 con 0x7f266c0d01b0 2023-11-28T13:09:01.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.870+0000 7f2672dff700 1 -- 172.21.15.156:0/2826308103 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f266c068690 con 0x7f266c0d01b0 2023-11-28T13:09:01.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.874+0000 7f26697fa700 1 -- 172.21.15.156:0/2826308103 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2664055950 con 0x7f266c0d01b0 2023-11-28T13:09:01.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.876+0000 7f2672dff700 1 -- 172.21.15.156:0/2826308103 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26500682f0 msgr2=0x7f265006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:01.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.876+0000 7f2672dff700 1 --2- 172.21.15.156:0/2826308103 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26500682f0 0x7f265006a7a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2660014040 tx=0x7f2660008030 comp rx=0 tx=0).stop 2023-11-28T13:09:01.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.876+0000 7f2672dff700 1 -- 172.21.15.156:0/2826308103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266c0d01b0 msgr2=0x7f266c15a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:01.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.876+0000 7f2672dff700 1 --2- 172.21.15.156:0/2826308103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266c0d01b0 0x7f266c15a890 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f266400cc50 tx=0x7f266400cf60 comp rx=0 tx=0).stop 2023-11-28T13:09:01.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.876+0000 7f2672dff700 1 -- 172.21.15.156:0/2826308103 shutdown_connections 2023-11-28T13:09:01.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.876+0000 7f2672dff700 1 --2- 172.21.15.156:0/2826308103 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26500682f0 0x7f265006a7a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.876+0000 7f2672dff700 1 --2- 172.21.15.156:0/2826308103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266c0d01b0 0x7f266c15a890 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.876+0000 7f2672dff700 1 --2- 172.21.15.156:0/2826308103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f266c0b20b0 0x7f266c0ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.876+0000 7f2672dff700 1 --2- 172.21.15.156:0/2826308103 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f266c0b29b0 0x7f266c15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:01.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.876+0000 7f2672dff700 1 -- 172.21.15.156:0/2826308103 >> 172.21.15.156:0/2826308103 conn(0x7f266c09c3f0 msgr2=0x7f266c06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:01.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.876+0000 7f2672dff700 1 -- 172.21.15.156:0/2826308103 shutdown_connections 2023-11-28T13:09:01.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:01.877+0000 7f2672dff700 1 -- 172.21.15.156:0/2826308103 wait complete. 2023-11-28T13:09:01.896 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-28T13:09:02.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.091+0000 7f2710fff700 1 Processor -- start 2023-11-28T13:09:02.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.091+0000 7f2710fff700 1 -- start start 2023-11-28T13:09:02.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.092+0000 7f2710fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f270c159490 0x7f270c0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:02.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.092+0000 7f2710fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f270c0ccc00 con 0x7f270c0cdc80 2023-11-28T13:09:02.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.092+0000 7f2710fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f270c0ccec0 con 0x7f270c159860 2023-11-28T13:09:02.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.092+0000 7f2710fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f270c0cd1a0 con 0x7f270c159490 2023-11-28T13:09:02.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.093+0000 7f270a59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f270c159490 0x7f270c0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:02.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.093+0000 7f270ad9d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f270c0cdc80 0x7f270c06fb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:33398/0 (socket says 172.21.15.156:33398) 2023-11-28T13:09:02.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.093+0000 7f270ad9d700 1 -- 172.21.15.156:0/53647172 learned_addr learned my addr 172.21.15.156:0/53647172 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:02.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.093+0000 7f270a59c700 1 -- 172.21.15.156:0/53647172 >> v1:172.21.15.186:6789/0 conn(0x7f270c159860 legacy=0x7f270c0c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:02.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.093+0000 7f270a59c700 1 -- 172.21.15.156:0/53647172 >> v1:172.21.15.156:6789/0 conn(0x7f270c0cdc80 legacy=0x7f270c06fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:02.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.093+0000 7f270a59c700 1 -- 172.21.15.156:0/53647172 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270c0cd480 con 0x7f270c159490 2023-11-28T13:09:02.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.093+0000 7f2709d9b700 1 -- 172.21.15.156:0/53647172 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277065061 0 0) 0x7f270c0ccc00 con 0x7f270c0cdc80 2023-11-28T13:09:02.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.093+0000 7f270a59c700 1 --2- 172.21.15.156:0/53647172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f270c159490 0x7f270c0b0670 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009840 tx=0x7f26fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c8d8b1caf4cf12 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:02.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.094+0000 7f2709d9b700 1 -- 172.21.15.156:0/53647172 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc011830 con 0x7f270c159490 2023-11-28T13:09:02.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.094+0000 7f2709d9b700 1 -- 172.21.15.156:0/53647172 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26fc011990 con 0x7f270c159490 2023-11-28T13:09:02.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.094+0000 7f2709d9b700 1 -- 172.21.15.156:0/53647172 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc011d20 con 0x7f270c159490 2023-11-28T13:09:02.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.094+0000 7f2710fff700 1 -- 172.21.15.156:0/53647172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f270c159490 msgr2=0x7f270c0b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:02.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.094+0000 7f2710fff700 1 --2- 172.21.15.156:0/53647172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f270c159490 0x7f270c0b0670 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009840 tx=0x7f26fc009b50 comp rx=0 tx=0).stop 2023-11-28T13:09:02.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.094+0000 7f2710fff700 1 -- 172.21.15.156:0/53647172 shutdown_connections 2023-11-28T13:09:02.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.094+0000 7f2710fff700 1 --2- 172.21.15.156:0/53647172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f270c159490 0x7f270c0b0670 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:02.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.094+0000 7f2710fff700 1 -- 172.21.15.156:0/53647172 >> 172.21.15.156:0/53647172 conn(0x7f270c0cb670 msgr2=0x7f270c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:02.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.094+0000 7f2710fff700 1 -- 172.21.15.156:0/53647172 shutdown_connections 2023-11-28T13:09:02.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.094+0000 7f2710fff700 1 -- 172.21.15.156:0/53647172 wait complete. 2023-11-28T13:09:02.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.096+0000 7f2710fff700 1 Processor -- start 2023-11-28T13:09:02.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.096+0000 7f2710fff700 1 -- start start 2023-11-28T13:09:02.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.097+0000 7f2710fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f270c0cdc80 0x7f270c0a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:02.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.097+0000 7f2710fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f270c159490 0x7f270c0c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:02.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.097+0000 7f2710fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f270c153170 0x7f270c15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:02.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.097+0000 7f2710fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c06c860 con 0x7f270c153170 2023-11-28T13:09:02.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.097+0000 7f2710fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c0a89f0 con 0x7f270c159490 2023-11-28T13:09:02.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.097+0000 7f2710fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270c06c230 con 0x7f270c0cdc80 2023-11-28T13:09:02.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.097+0000 7f270ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f270c153170 0x7f270c15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:02.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.097+0000 7f270ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f270c153170 0x7f270c15ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41492/0 (socket says 172.21.15.156:41492) 2023-11-28T13:09:02.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.097+0000 7f270ad9d700 1 -- 172.21.15.156:0/3059760585 learned_addr learned my addr 172.21.15.156:0/3059760585 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:02.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.097+0000 7f2703fff700 1 --2- 172.21.15.156:0/3059760585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f270c159490 0x7f270c0c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:02.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.097+0000 7f270a59c700 1 --2- 172.21.15.156:0/3059760585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f270c0cdc80 0x7f270c0a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:02.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.098+0000 7f270ad9d700 1 -- 172.21.15.156:0/3059760585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f270c0cdc80 msgr2=0x7f270c0a84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:02.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.098+0000 7f270ad9d700 1 --2- 172.21.15.156:0/3059760585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f270c0cdc80 0x7f270c0a84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:02.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.098+0000 7f270ad9d700 1 -- 172.21.15.156:0/3059760585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f270c159490 msgr2=0x7f270c0c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:02.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.098+0000 7f270ad9d700 1 --2- 172.21.15.156:0/3059760585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f270c159490 0x7f270c0c4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:02.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.098+0000 7f270ad9d700 1 -- 172.21.15.156:0/3059760585 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc0094a0 con 0x7f270c153170 2023-11-28T13:09:02.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.098+0000 7f270a59c700 1 --2- 172.21.15.156:0/3059760585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f270c0cdc80 0x7f270c0a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:02.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.098+0000 7f270ad9d700 1 --2- 172.21.15.156:0/3059760585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f270c153170 0x7f270c15ab80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f270400cc70 tx=0x7f270400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:02.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.098+0000 7f2702ffd700 1 -- 172.21.15.156:0/3059760585 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270400ab70 con 0x7f270c153170 2023-11-28T13:09:02.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.098+0000 7f2702ffd700 1 -- 172.21.15.156:0/3059760585 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270400acf0 con 0x7f270c153170 2023-11-28T13:09:02.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.098+0000 7f2710fff700 1 -- 172.21.15.156:0/3059760585 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270c0b0bb0 con 0x7f270c153170 2023-11-28T13:09:02.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.098+0000 7f2702ffd700 1 -- 172.21.15.156:0/3059760585 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704007310 con 0x7f270c153170 2023-11-28T13:09:02.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.098+0000 7f2710fff700 1 -- 172.21.15.156:0/3059760585 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f270c0cd760 con 0x7f270c153170 2023-11-28T13:09:02.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.099+0000 7f2710fff700 1 -- 172.21.15.156:0/3059760585 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f270c0b0d40 con 0x7f270c153170 2023-11-28T13:09:02.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.100+0000 7f2702ffd700 1 -- 172.21.15.156:0/3059760585 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2704010070 con 0x7f270c153170 2023-11-28T13:09:02.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.101+0000 7f2702ffd700 1 --2- 172.21.15.156:0/3059760585 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26ec068320 0x7f26ec06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:02.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.101+0000 7f2702ffd700 1 -- 172.21.15.156:0/3059760585 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2704085380 con 0x7f270c153170 2023-11-28T13:09:02.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.102+0000 7f270a59c700 1 --2- 172.21.15.156:0/3059760585 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26ec068320 0x7f26ec06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:02.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.102+0000 7f270a59c700 1 --2- 172.21.15.156:0/3059760585 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26ec068320 0x7f26ec06a7d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f26fc000c00 tx=0x7f26fc01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:02.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.105+0000 7f2702ffd700 1 -- 172.21.15.156:0/3059760585 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2704051e40 con 0x7f270c153170 2023-11-28T13:09:02.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.367+0000 7f2710fff700 1 -- 172.21.15.156:0/3059760585 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f270c06c9a0 con 0x7f270c153170 2023-11-28T13:09:02.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.371+0000 7f2702ffd700 1 -- 172.21.15.156:0/3059760585 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f270c06c9a0 con 0x7f270c153170 2023-11-28T13:09:02.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.373+0000 7f2710fff700 1 -- 172.21.15.156:0/3059760585 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26ec068320 msgr2=0x7f26ec06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:02.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.373+0000 7f2710fff700 1 --2- 172.21.15.156:0/3059760585 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26ec068320 0x7f26ec06a7d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f26fc000c00 tx=0x7f26fc01c040 comp rx=0 tx=0).stop 2023-11-28T13:09:02.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.373+0000 7f2710fff700 1 -- 172.21.15.156:0/3059760585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f270c153170 msgr2=0x7f270c15ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:02.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.373+0000 7f2710fff700 1 --2- 172.21.15.156:0/3059760585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f270c153170 0x7f270c15ab80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f270400cc70 tx=0x7f270400cf80 comp rx=0 tx=0).stop 2023-11-28T13:09:02.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.373+0000 7f2710fff700 1 -- 172.21.15.156:0/3059760585 shutdown_connections 2023-11-28T13:09:02.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.373+0000 7f2710fff700 1 --2- 172.21.15.156:0/3059760585 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26ec068320 0x7f26ec06a7d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:02.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.373+0000 7f2710fff700 1 --2- 172.21.15.156:0/3059760585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f270c153170 0x7f270c15ab80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:02.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.373+0000 7f2710fff700 1 --2- 172.21.15.156:0/3059760585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f270c0cdc80 0x7f270c0a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:02.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.373+0000 7f2710fff700 1 --2- 172.21.15.156:0/3059760585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f270c159490 0x7f270c0c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:02.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.373+0000 7f2710fff700 1 -- 172.21.15.156:0/3059760585 >> 172.21.15.156:0/3059760585 conn(0x7f270c0cb670 msgr2=0x7f270c159f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:02.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.374+0000 7f2710fff700 1 -- 172.21.15.156:0/3059760585 shutdown_connections 2023-11-28T13:09:02.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.374+0000 7f2710fff700 1 -- 172.21.15.156:0/3059760585 wait complete. 2023-11-28T13:09:02.392 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-28T13:09:02.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.590+0000 7f97857ff700 1 Processor -- start 2023-11-28T13:09:02.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.590+0000 7f97857ff700 1 -- start start 2023-11-28T13:09:02.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.591+0000 7f97857ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f978015c9b0 0x7f97800b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:02.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.591+0000 7f97857ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f97800cd7c0 0x7f97800cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:02.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.591+0000 7f97857ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97800d1950 con 0x7f9780161340 2023-11-28T13:09:02.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.591+0000 7f97857ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97800b5800 con 0x7f97800cd7c0 2023-11-28T13:09:02.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.591+0000 7f97857ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97800ce140 con 0x7f978015c9b0 2023-11-28T13:09:02.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.592+0000 7f977f7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f97800cd7c0 0x7f97800cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:02.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.592+0000 7f977f7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f97800cd7c0 0x7f97800cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41496/0 (socket says 172.21.15.156:41496) 2023-11-28T13:09:02.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.592+0000 7f977f7fe700 1 -- 172.21.15.156:0/1038986861 learned_addr learned my addr 172.21.15.156:0/1038986861 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:02.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.592+0000 7f977effd700 1 --2- 172.21.15.156:0/1038986861 >> v2:172.21.15.186:3300/0 conn(0x7f978015c9b0 0x7f97800b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:02.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.592+0000 7f977f7fe700 1 -- 172.21.15.156:0/1038986861 >> v2:172.21.15.186:3300/0 conn(0x7f978015c9b0 msgr2=0x7f97800b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:02.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.592+0000 7f977f7fe700 1 --2- 172.21.15.156:0/1038986861 >> v2:172.21.15.186:3300/0 conn(0x7f978015c9b0 0x7f97800b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:02.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.592+0000 7f977f7fe700 1 -- 172.21.15.156:0/1038986861 >> v1:172.21.15.186:6789/0 conn(0x7f9780161340 legacy=0x7f97800d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:02.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.592+0000 7f977f7fe700 1 -- 172.21.15.156:0/1038986861 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97800d24f0 con 0x7f97800cd7c0 2023-11-28T13:09:02.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.592+0000 7f977effd700 1 --2- 172.21.15.156:0/1038986861 >> v2:172.21.15.186:3300/0 conn(0x7f978015c9b0 0x7f97800b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:02.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.593+0000 7f977f7fe700 1 --2- 172.21.15.156:0/1038986861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97800cd7c0 0x7f97800cdc00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9780067460 tx=0x7f977800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9399d91a6494bac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:02.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.593+0000 7f977dffb700 1 -- 172.21.15.156:0/1038986861 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977800e070 con 0x7f97800cd7c0 2023-11-28T13:09:02.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.593+0000 7f977dffb700 1 -- 172.21.15.156:0/1038986861 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977800b9c0 con 0x7f97800cd7c0 2023-11-28T13:09:02.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.593+0000 7f977dffb700 1 -- 172.21.15.156:0/1038986861 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977800bd70 con 0x7f97800cd7c0 2023-11-28T13:09:02.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.593+0000 7f97857ff700 1 -- 172.21.15.156:0/1038986861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97800cd7c0 msgr2=0x7f97800cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:02.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.593+0000 7f97857ff700 1 --2- 172.21.15.156:0/1038986861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97800cd7c0 0x7f97800cdc00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9780067460 tx=0x7f977800b330 comp rx=0 tx=0).stop 2023-11-28T13:09:02.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.593+0000 7f97857ff700 1 -- 172.21.15.156:0/1038986861 shutdown_connections 2023-11-28T13:09:02.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.593+0000 7f97857ff700 1 --2- 172.21.15.156:0/1038986861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97800cd7c0 0x7f97800cdc00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:02.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.593+0000 7f97857ff700 1 --2- 172.21.15.156:0/1038986861 >> v2:172.21.15.186:3300/0 conn(0x7f978015c9b0 0x7f97800b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:02.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.593+0000 7f97857ff700 1 -- 172.21.15.156:0/1038986861 >> 172.21.15.156:0/1038986861 conn(0x7f9780072600 msgr2=0x7f97800cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:02.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.594+0000 7f97857ff700 1 -- 172.21.15.156:0/1038986861 shutdown_connections 2023-11-28T13:09:02.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.594+0000 7f97857ff700 1 -- 172.21.15.156:0/1038986861 wait complete. 2023-11-28T13:09:02.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.595+0000 7f97857ff700 1 Processor -- start 2023-11-28T13:09:02.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.595+0000 7f97857ff700 1 -- start start 2023-11-28T13:09:02.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.596+0000 7f97857ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97800cd7c0 0x7f9780154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:02.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.596+0000 7f97857ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f978015c9b0 0x7f9780155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:02.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.596+0000 7f97857ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9780161340 0x7f978015ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:02.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.596+0000 7f97857ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97801e7180 con 0x7f97800cd7c0 2023-11-28T13:09:02.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.596+0000 7f97857ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97801e7610 con 0x7f9780161340 2023-11-28T13:09:02.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.596+0000 7f97857ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9780154bd0 con 0x7f978015c9b0 2023-11-28T13:09:02.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.596+0000 7f977effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97800cd7c0 0x7f9780154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:02.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.597+0000 7f977effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97800cd7c0 0x7f9780154690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41502/0 (socket says 172.21.15.156:41502) 2023-11-28T13:09:02.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.597+0000 7f977effd700 1 -- 172.21.15.156:0/132850436 learned_addr learned my addr 172.21.15.156:0/132850436 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:02.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.597+0000 7f977e7fc700 1 --2- 172.21.15.156:0/132850436 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f978015c9b0 0x7f9780155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:02.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.597+0000 7f977f7fe700 1 --2- 172.21.15.156:0/132850436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9780161340 0x7f978015ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:02.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.597+0000 7f977effd700 1 -- 172.21.15.156:0/132850436 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f978015c9b0 msgr2=0x7f9780155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:02.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.597+0000 7f977effd700 1 --2- 172.21.15.156:0/132850436 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f978015c9b0 0x7f9780155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:02.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.597+0000 7f977effd700 1 -- 172.21.15.156:0/132850436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9780161340 msgr2=0x7f978015ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:02.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.597+0000 7f977effd700 1 --2- 172.21.15.156:0/132850436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9780161340 0x7f978015ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:02.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.597+0000 7f977effd700 1 -- 172.21.15.156:0/132850436 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9778009b30 con 0x7f97800cd7c0 2023-11-28T13:09:02.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.597+0000 7f977effd700 1 --2- 172.21.15.156:0/132850436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97800cd7c0 0x7f9780154690 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f977000e790 tx=0x7f977000eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:02.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.597+0000 7f97677fe700 1 -- 172.21.15.156:0/132850436 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977000c070 con 0x7f97800cd7c0 2023-11-28T13:09:02.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.597+0000 7f97677fe700 1 -- 172.21.15.156:0/132850436 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9770013610 con 0x7f97800cd7c0 2023-11-28T13:09:02.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.597+0000 7f97857ff700 1 -- 172.21.15.156:0/132850436 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97800d1300 con 0x7f97800cd7c0 2023-11-28T13:09:02.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.597+0000 7f97677fe700 1 -- 172.21.15.156:0/132850436 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97700139c0 con 0x7f97800cd7c0 2023-11-28T13:09:02.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.598+0000 7f97857ff700 1 -- 172.21.15.156:0/132850436 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97800b5520 con 0x7f97800cd7c0 2023-11-28T13:09:02.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.599+0000 7f97857ff700 1 -- 172.21.15.156:0/132850436 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97800d15e0 con 0x7f97800cd7c0 2023-11-28T13:09:02.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.599+0000 7f97677fe700 1 -- 172.21.15.156:0/132850436 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9770023070 con 0x7f97800cd7c0 2023-11-28T13:09:02.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.601+0000 7f97677fe700 1 --2- 172.21.15.156:0/132850436 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f976c0682f0 0x7f976c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:02.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.601+0000 7f97677fe700 1 -- 172.21.15.156:0/132850436 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9770085620 con 0x7f97800cd7c0 2023-11-28T13:09:02.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.601+0000 7f977e7fc700 1 --2- 172.21.15.156:0/132850436 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f976c0682f0 0x7f976c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:02.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.602+0000 7f977e7fc700 1 --2- 172.21.15.156:0/132850436 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f976c0682f0 0x7f976c06a7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f978015ea00 tx=0x7f9774009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:02.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.605+0000 7f97677fe700 1 -- 172.21.15.156:0/132850436 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f97700520e0 con 0x7f97800cd7c0 2023-11-28T13:09:02.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.878+0000 7f97857ff700 1 -- 172.21.15.156:0/132850436 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f97800745c0 con 0x7f97800cd7c0 2023-11-28T13:09:02.879 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:09:02.885 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:09:02.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.881+0000 7f97677fe700 1 -- 172.21.15.156:0/132850436 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9770055700 con 0x7f97800cd7c0 2023-11-28T13:09:02.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.884+0000 7f97857ff700 1 -- 172.21.15.156:0/132850436 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f976c0682f0 msgr2=0x7f976c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:02.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.884+0000 7f97857ff700 1 --2- 172.21.15.156:0/132850436 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f976c0682f0 0x7f976c06a7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f978015ea00 tx=0x7f9774009040 comp rx=0 tx=0).stop 2023-11-28T13:09:02.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.884+0000 7f97857ff700 1 -- 172.21.15.156:0/132850436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97800cd7c0 msgr2=0x7f9780154690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:02.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.884+0000 7f97857ff700 1 --2- 172.21.15.156:0/132850436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97800cd7c0 0x7f9780154690 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f977000e790 tx=0x7f977000eaa0 comp rx=0 tx=0).stop 2023-11-28T13:09:02.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.884+0000 7f97857ff700 1 -- 172.21.15.156:0/132850436 shutdown_connections 2023-11-28T13:09:02.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.884+0000 7f97857ff700 1 --2- 172.21.15.156:0/132850436 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f976c0682f0 0x7f976c06a7a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:02.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.884+0000 7f97857ff700 1 --2- 172.21.15.156:0/132850436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9780161340 0x7f978015ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:02.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.884+0000 7f97857ff700 1 --2- 172.21.15.156:0/132850436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97800cd7c0 0x7f9780154690 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:02.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.884+0000 7f97857ff700 1 --2- 172.21.15.156:0/132850436 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f978015c9b0 0x7f9780155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:02.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.884+0000 7f97857ff700 1 -- 172.21.15.156:0/132850436 >> 172.21.15.156:0/132850436 conn(0x7f9780072600 msgr2=0x7f978006a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:02.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.884+0000 7f97857ff700 1 -- 172.21.15.156:0/132850436 shutdown_connections 2023-11-28T13:09:02.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:02.884+0000 7f97857ff700 1 -- 172.21.15.156:0/132850436 wait complete. 2023-11-28T13:09:02.904 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-28T13:09:03.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.142+0000 7f38cf5ff700 1 Processor -- start 2023-11-28T13:09:03.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.143+0000 7f38cf5ff700 1 -- start start 2023-11-28T13:09:03.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.144+0000 7f38cf5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38c815c9b0 0x7f38c80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:03.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.144+0000 7f38cf5ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f38c8161340 0x7f38c80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:03.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.144+0000 7f38cf5ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c80d1950 con 0x7f38c80d2320 2023-11-28T13:09:03.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.144+0000 7f38cf5ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c80b5800 con 0x7f38c8161340 2023-11-28T13:09:03.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.144+0000 7f38cf5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c80b5520 con 0x7f38c815c9b0 2023-11-28T13:09:03.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.144+0000 7f38ccb9a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f38c8161340 0x7f38c80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:03.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.144+0000 7f38ccb9a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f38c8161340 0x7f38c80d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41504/0 (socket says 172.21.15.156:41504) 2023-11-28T13:09:03.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.144+0000 7f38ccb9a700 1 -- 172.21.15.156:0/696116134 learned_addr learned my addr 172.21.15.156:0/696116134 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:03.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.144+0000 7f38ccb9a700 1 -- 172.21.15.156:0/696116134 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38c815c9b0 msgr2=0x7f38c80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:03.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.145+0000 7f38cd39b700 1 --2- 172.21.15.156:0/696116134 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38c815c9b0 0x7f38c80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:03.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.145+0000 7f38ccb9a700 1 --2- 172.21.15.156:0/696116134 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38c815c9b0 0x7f38c80b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.145+0000 7f38ccb9a700 1 -- 172.21.15.156:0/696116134 >> v1:172.21.15.186:6789/0 conn(0x7f38c80d2320 legacy=0x7f38c80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:03.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.145+0000 7f38ccb9a700 1 -- 172.21.15.156:0/696116134 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c80d1300 con 0x7f38c8161340 2023-11-28T13:09:03.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.145+0000 7f38cd39b700 1 --2- 172.21.15.156:0/696116134 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38c815c9b0 0x7f38c80b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:03.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.145+0000 7f38ccb9a700 1 --2- 172.21.15.156:0/696116134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c8161340 0x7f38c80d1de0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f38b80098f0 tx=0x7f38b8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d9c7420debab46 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:03.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.145+0000 7f38bffff700 1 -- 172.21.15.156:0/696116134 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8004230 con 0x7f38c8161340 2023-11-28T13:09:03.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.145+0000 7f38bffff700 1 -- 172.21.15.156:0/696116134 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b8004390 con 0x7f38c8161340 2023-11-28T13:09:03.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.145+0000 7f38bffff700 1 -- 172.21.15.156:0/696116134 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b80036a0 con 0x7f38c8161340 2023-11-28T13:09:03.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.146+0000 7f38cf5ff700 1 -- 172.21.15.156:0/696116134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c8161340 msgr2=0x7f38c80d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:03.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.146+0000 7f38cf5ff700 1 --2- 172.21.15.156:0/696116134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c8161340 0x7f38c80d1de0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f38b80098f0 tx=0x7f38b8009c00 comp rx=0 tx=0).stop 2023-11-28T13:09:03.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.146+0000 7f38cf5ff700 1 -- 172.21.15.156:0/696116134 shutdown_connections 2023-11-28T13:09:03.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.146+0000 7f38cf5ff700 1 --2- 172.21.15.156:0/696116134 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38c815c9b0 0x7f38c80b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.146+0000 7f38cf5ff700 1 --2- 172.21.15.156:0/696116134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c8161340 0x7f38c80d1de0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.146+0000 7f38cf5ff700 1 -- 172.21.15.156:0/696116134 >> 172.21.15.156:0/696116134 conn(0x7f38c8072600 msgr2=0x7f38c80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:03.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.146+0000 7f38cf5ff700 1 -- 172.21.15.156:0/696116134 shutdown_connections 2023-11-28T13:09:03.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.146+0000 7f38cf5ff700 1 -- 172.21.15.156:0/696116134 wait complete. 2023-11-28T13:09:03.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.148+0000 7f38cf5ff700 1 Processor -- start 2023-11-28T13:09:03.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.148+0000 7f38cf5ff700 1 -- start start 2023-11-28T13:09:03.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.150+0000 7f38cf5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38c80d2320 0x7f38c81504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:03.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.150+0000 7f38cf5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38c815c9b0 0x7f38c80c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:03.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.150+0000 7f38cf5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c8161340 0x7f38c80b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:03.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.150+0000 7f38cf5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c80c8940 con 0x7f38c8161340 2023-11-28T13:09:03.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.150+0000 7f38cf5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c80c8af0 con 0x7f38c815c9b0 2023-11-28T13:09:03.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.150+0000 7f38cf5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c8154820 con 0x7f38c80d2320 2023-11-28T13:09:03.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.150+0000 7f38cdb9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c8161340 0x7f38c80b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:03.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.150+0000 7f38cdb9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c8161340 0x7f38c80b7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41518/0 (socket says 172.21.15.156:41518) 2023-11-28T13:09:03.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.150+0000 7f38cdb9c700 1 -- 172.21.15.156:0/929035744 learned_addr learned my addr 172.21.15.156:0/929035744 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:03.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.150+0000 7f38ccb9a700 1 --2- 172.21.15.156:0/929035744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38c815c9b0 0x7f38c80c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:03.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.150+0000 7f38cd39b700 1 --2- 172.21.15.156:0/929035744 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38c80d2320 0x7f38c81504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:03.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.150+0000 7f38cdb9c700 1 -- 172.21.15.156:0/929035744 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38c80d2320 msgr2=0x7f38c81504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:03.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.150+0000 7f38cdb9c700 1 --2- 172.21.15.156:0/929035744 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38c80d2320 0x7f38c81504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.150+0000 7f38cdb9c700 1 -- 172.21.15.156:0/929035744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38c815c9b0 msgr2=0x7f38c80c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:03.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.150+0000 7f38cdb9c700 1 --2- 172.21.15.156:0/929035744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38c815c9b0 0x7f38c80c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.150+0000 7f38cdb9c700 1 -- 172.21.15.156:0/929035744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b8009550 con 0x7f38c8161340 2023-11-28T13:09:03.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.151+0000 7f38cdb9c700 1 --2- 172.21.15.156:0/929035744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c8161340 0x7f38c80b7cc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f38c000b970 tx=0x7f38c000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:03.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.151+0000 7f38bdffb700 1 -- 172.21.15.156:0/929035744 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c000d860 con 0x7f38c8161340 2023-11-28T13:09:03.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.151+0000 7f38bdffb700 1 -- 172.21.15.156:0/929035744 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c000d9c0 con 0x7f38c8161340 2023-11-28T13:09:03.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.151+0000 7f38cf5ff700 1 -- 172.21.15.156:0/929035744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c80b5130 con 0x7f38c8161340 2023-11-28T13:09:03.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.151+0000 7f38bdffb700 1 -- 172.21.15.156:0/929035744 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c000dd70 con 0x7f38c8161340 2023-11-28T13:09:03.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.151+0000 7f38cf5ff700 1 -- 172.21.15.156:0/929035744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c80d15e0 con 0x7f38c8161340 2023-11-28T13:09:03.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.152+0000 7f38cf5ff700 1 -- 172.21.15.156:0/929035744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c80b52c0 con 0x7f38c8161340 2023-11-28T13:09:03.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.153+0000 7f38bdffb700 1 -- 172.21.15.156:0/929035744 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f38c000e430 con 0x7f38c8161340 2023-11-28T13:09:03.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.154+0000 7f38bdffb700 1 --2- 172.21.15.156:0/929035744 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f38ac0682f0 0x7f38ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:03.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.154+0000 7f38bdffb700 1 -- 172.21.15.156:0/929035744 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f38c0085d20 con 0x7f38c8161340 2023-11-28T13:09:03.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.154+0000 7f38cd39b700 1 --2- 172.21.15.156:0/929035744 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f38ac0682f0 0x7f38ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:03.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.155+0000 7f38cd39b700 1 --2- 172.21.15.156:0/929035744 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f38ac0682f0 0x7f38ac06a7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f38c4006fd0 tx=0x7f38c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:03.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.158+0000 7f38bdffb700 1 -- 172.21.15.156:0/929035744 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f38c0052780 con 0x7f38c8161340 2023-11-28T13:09:03.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.424+0000 7f38cf5ff700 1 -- 172.21.15.156:0/929035744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f38c8066320 con 0x7f38c8161340 2023-11-28T13:09:03.428 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.428+0000 7f38bdffb700 1 -- 172.21.15.156:0/929035744 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f38c0055da0 con 0x7f38c8161340 2023-11-28T13:09:03.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.430+0000 7f38cf5ff700 1 -- 172.21.15.156:0/929035744 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f38ac0682f0 msgr2=0x7f38ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:03.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.430+0000 7f38cf5ff700 1 --2- 172.21.15.156:0/929035744 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f38ac0682f0 0x7f38ac06a7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f38c4006fd0 tx=0x7f38c4008040 comp rx=0 tx=0).stop 2023-11-28T13:09:03.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.430+0000 7f38cf5ff700 1 -- 172.21.15.156:0/929035744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c8161340 msgr2=0x7f38c80b7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:03.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.430+0000 7f38cf5ff700 1 --2- 172.21.15.156:0/929035744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c8161340 0x7f38c80b7cc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f38c000b970 tx=0x7f38c000bc80 comp rx=0 tx=0).stop 2023-11-28T13:09:03.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.431+0000 7f38cf5ff700 1 -- 172.21.15.156:0/929035744 shutdown_connections 2023-11-28T13:09:03.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.431+0000 7f38cf5ff700 1 --2- 172.21.15.156:0/929035744 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f38ac0682f0 0x7f38ac06a7a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.431+0000 7f38cf5ff700 1 --2- 172.21.15.156:0/929035744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c8161340 0x7f38c80b7cc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.431+0000 7f38cf5ff700 1 --2- 172.21.15.156:0/929035744 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38c80d2320 0x7f38c81504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.431+0000 7f38cf5ff700 1 --2- 172.21.15.156:0/929035744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38c815c9b0 0x7f38c80c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.431+0000 7f38cf5ff700 1 -- 172.21.15.156:0/929035744 >> 172.21.15.156:0/929035744 conn(0x7f38c8072600 msgr2=0x7f38c815ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:03.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.431+0000 7f38cf5ff700 1 -- 172.21.15.156:0/929035744 shutdown_connections 2023-11-28T13:09:03.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.431+0000 7f38cf5ff700 1 -- 172.21.15.156:0/929035744 wait complete. 2023-11-28T13:09:03.450 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-28T13:09:03.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.648+0000 7f7d307ff700 1 Processor -- start 2023-11-28T13:09:03.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.648+0000 7f7d307ff700 1 -- start start 2023-11-28T13:09:03.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.649+0000 7f7d307ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7d2815c9b0 0x7f7d280b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:03.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.649+0000 7f7d307ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7d280cd7c0 0x7f7d280cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:03.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.649+0000 7f7d307ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d280d1950 con 0x7f7d28161340 2023-11-28T13:09:03.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.649+0000 7f7d307ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d280b5800 con 0x7f7d280cd7c0 2023-11-28T13:09:03.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.649+0000 7f7d307ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d280ce140 con 0x7f7d2815c9b0 2023-11-28T13:09:03.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.649+0000 7f7d2dd9a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f7d28161340 0x7f7d280d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:33406/0 (socket says 172.21.15.156:33406) 2023-11-28T13:09:03.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.649+0000 7f7d2dd9a700 1 -- 172.21.15.156:0/3052575226 learned_addr learned my addr 172.21.15.156:0/3052575226 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:03.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.649+0000 7f7d2ed9c700 1 --2- 172.21.15.156:0/3052575226 >> v2:172.21.15.156:3300/0 conn(0x7f7d280cd7c0 0x7f7d280cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:03.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.649+0000 7f7d2e59b700 1 --2- 172.21.15.156:0/3052575226 >> v2:172.21.15.186:3300/0 conn(0x7f7d2815c9b0 0x7f7d280b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:03.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.650+0000 7f7d2d599700 1 -- 172.21.15.156:0/3052575226 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3654619940 0 0) 0x7f7d280d1950 con 0x7f7d28161340 2023-11-28T13:09:03.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.650+0000 7f7d2ed9c700 1 -- 172.21.15.156:0/3052575226 >> v2:172.21.15.186:3300/0 conn(0x7f7d2815c9b0 msgr2=0x7f7d280b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:03.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.650+0000 7f7d2ed9c700 1 --2- 172.21.15.156:0/3052575226 >> v2:172.21.15.186:3300/0 conn(0x7f7d2815c9b0 0x7f7d280b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.650+0000 7f7d2ed9c700 1 -- 172.21.15.156:0/3052575226 >> v1:172.21.15.156:6789/0 conn(0x7f7d28161340 legacy=0x7f7d280d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:03.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.650+0000 7f7d2ed9c700 1 -- 172.21.15.156:0/3052575226 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d280d24f0 con 0x7f7d280cd7c0 2023-11-28T13:09:03.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.650+0000 7f7d2ed9c700 1 --2- 172.21.15.156:0/3052575226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d280cd7c0 0x7f7d280cdc00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7d28067460 tx=0x7f7d2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bead2dcbbca765c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:03.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.650+0000 7f7d2d599700 1 -- 172.21.15.156:0/3052575226 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2000e070 con 0x7f7d280cd7c0 2023-11-28T13:09:03.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.650+0000 7f7d2d599700 1 -- 172.21.15.156:0/3052575226 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2000bb20 con 0x7f7d280cd7c0 2023-11-28T13:09:03.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.650+0000 7f7d2d599700 1 -- 172.21.15.156:0/3052575226 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20012430 con 0x7f7d280cd7c0 2023-11-28T13:09:03.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.651+0000 7f7d307ff700 1 -- 172.21.15.156:0/3052575226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d280cd7c0 msgr2=0x7f7d280cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:03.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.651+0000 7f7d307ff700 1 --2- 172.21.15.156:0/3052575226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d280cd7c0 0x7f7d280cdc00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7d28067460 tx=0x7f7d2000b330 comp rx=0 tx=0).stop 2023-11-28T13:09:03.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.651+0000 7f7d307ff700 1 -- 172.21.15.156:0/3052575226 shutdown_connections 2023-11-28T13:09:03.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.651+0000 7f7d307ff700 1 --2- 172.21.15.156:0/3052575226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d280cd7c0 0x7f7d280cdc00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.651+0000 7f7d307ff700 1 --2- 172.21.15.156:0/3052575226 >> v2:172.21.15.186:3300/0 conn(0x7f7d2815c9b0 0x7f7d280b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.651+0000 7f7d307ff700 1 -- 172.21.15.156:0/3052575226 >> 172.21.15.156:0/3052575226 conn(0x7f7d28072600 msgr2=0x7f7d280cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:03.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.651+0000 7f7d307ff700 1 -- 172.21.15.156:0/3052575226 shutdown_connections 2023-11-28T13:09:03.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.651+0000 7f7d307ff700 1 -- 172.21.15.156:0/3052575226 wait complete. 2023-11-28T13:09:03.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.653+0000 7f7d307ff700 1 Processor -- start 2023-11-28T13:09:03.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.653+0000 7f7d307ff700 1 -- start start 2023-11-28T13:09:03.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.654+0000 7f7d307ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d280cd7c0 0x7f7d281570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:03.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.654+0000 7f7d307ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d2815c9b0 0x7f7d280c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:03.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.654+0000 7f7d307ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d28161340 0x7f7d281f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:03.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.654+0000 7f7d307ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d281e8280 con 0x7f7d280cd7c0 2023-11-28T13:09:03.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.654+0000 7f7d307ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d281e7670 con 0x7f7d2815c9b0 2023-11-28T13:09:03.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.654+0000 7f7d307ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d281e7950 con 0x7f7d28161340 2023-11-28T13:09:03.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.654+0000 7f7d2e59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d280cd7c0 0x7f7d281570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:03.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.654+0000 7f7d2e59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d280cd7c0 0x7f7d281570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41540/0 (socket says 172.21.15.156:41540) 2023-11-28T13:09:03.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.654+0000 7f7d2e59b700 1 -- 172.21.15.156:0/1883491590 learned_addr learned my addr 172.21.15.156:0/1883491590 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:03.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.654+0000 7f7d2dd9a700 1 --2- 172.21.15.156:0/1883491590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d2815c9b0 0x7f7d280c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:03.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.654+0000 7f7d2ed9c700 1 --2- 172.21.15.156:0/1883491590 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d28161340 0x7f7d281f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:03.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.655+0000 7f7d2e59b700 1 -- 172.21.15.156:0/1883491590 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d28161340 msgr2=0x7f7d281f25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:03.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.655+0000 7f7d2e59b700 1 --2- 172.21.15.156:0/1883491590 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d28161340 0x7f7d281f25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.655+0000 7f7d2e59b700 1 -- 172.21.15.156:0/1883491590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d2815c9b0 msgr2=0x7f7d280c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:03.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.655+0000 7f7d2e59b700 1 --2- 172.21.15.156:0/1883491590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d2815c9b0 0x7f7d280c9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.655+0000 7f7d2e59b700 1 -- 172.21.15.156:0/1883491590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d20009a70 con 0x7f7d280cd7c0 2023-11-28T13:09:03.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.655+0000 7f7d2e59b700 1 --2- 172.21.15.156:0/1883491590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d280cd7c0 0x7f7d281570e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400e870 tx=0x7f7d2400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:03.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.655+0000 7f7d1effd700 1 -- 172.21.15.156:0/1883491590 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24009590 con 0x7f7d280cd7c0 2023-11-28T13:09:03.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.655+0000 7f7d1effd700 1 -- 172.21.15.156:0/1883491590 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2400ee70 con 0x7f7d280cd7c0 2023-11-28T13:09:03.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.655+0000 7f7d307ff700 1 -- 172.21.15.156:0/1883491590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d280d1300 con 0x7f7d280cd7c0 2023-11-28T13:09:03.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.655+0000 7f7d1effd700 1 -- 172.21.15.156:0/1883491590 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2400ca50 con 0x7f7d280cd7c0 2023-11-28T13:09:03.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.655+0000 7f7d307ff700 1 -- 172.21.15.156:0/1883491590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d280b5520 con 0x7f7d280cd7c0 2023-11-28T13:09:03.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.656+0000 7f7d307ff700 1 -- 172.21.15.156:0/1883491590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d280d15e0 con 0x7f7d280cd7c0 2023-11-28T13:09:03.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.657+0000 7f7d1effd700 1 -- 172.21.15.156:0/1883491590 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7d24019030 con 0x7f7d280cd7c0 2023-11-28T13:09:03.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.658+0000 7f7d1effd700 1 --2- 172.21.15.156:0/1883491590 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7d0c0682f0 0x7f7d0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:03.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.658+0000 7f7d1effd700 1 -- 172.21.15.156:0/1883491590 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7d240858c0 con 0x7f7d280cd7c0 2023-11-28T13:09:03.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.658+0000 7f7d2dd9a700 1 --2- 172.21.15.156:0/1883491590 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7d0c0682f0 0x7f7d0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:03.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.659+0000 7f7d2dd9a700 1 --2- 172.21.15.156:0/1883491590 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7d0c0682f0 0x7f7d0c06a7a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7d18000f60 tx=0x7f7d18007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:03.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.662+0000 7f7d1effd700 1 -- 172.21.15.156:0/1883491590 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7d24052380 con 0x7f7d280cd7c0 2023-11-28T13:09:03.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.931+0000 7f7d307ff700 1 -- 172.21.15.156:0/1883491590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f7d2815cd80 con 0x7f7d280cd7c0 2023-11-28T13:09:03.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.935+0000 7f7d1effd700 1 -- 172.21.15.156:0/1883491590 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7d240559a0 con 0x7f7d280cd7c0 2023-11-28T13:09:03.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.937+0000 7f7d307ff700 1 -- 172.21.15.156:0/1883491590 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7d0c0682f0 msgr2=0x7f7d0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:03.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.937+0000 7f7d307ff700 1 --2- 172.21.15.156:0/1883491590 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7d0c0682f0 0x7f7d0c06a7a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7d18000f60 tx=0x7f7d18007c90 comp rx=0 tx=0).stop 2023-11-28T13:09:03.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.937+0000 7f7d307ff700 1 -- 172.21.15.156:0/1883491590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d280cd7c0 msgr2=0x7f7d281570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:03.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.937+0000 7f7d307ff700 1 --2- 172.21.15.156:0/1883491590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d280cd7c0 0x7f7d281570e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400e870 tx=0x7f7d2400ec30 comp rx=0 tx=0).stop 2023-11-28T13:09:03.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.937+0000 7f7d307ff700 1 -- 172.21.15.156:0/1883491590 shutdown_connections 2023-11-28T13:09:03.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.937+0000 7f7d307ff700 1 --2- 172.21.15.156:0/1883491590 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7d0c0682f0 0x7f7d0c06a7a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.937+0000 7f7d307ff700 1 --2- 172.21.15.156:0/1883491590 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d28161340 0x7f7d281f25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.937+0000 7f7d307ff700 1 --2- 172.21.15.156:0/1883491590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d280cd7c0 0x7f7d281570e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.937+0000 7f7d307ff700 1 --2- 172.21.15.156:0/1883491590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d2815c9b0 0x7f7d280c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:03.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.937+0000 7f7d307ff700 1 -- 172.21.15.156:0/1883491590 >> 172.21.15.156:0/1883491590 conn(0x7f7d28072600 msgr2=0x7f7d2815bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:03.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.937+0000 7f7d307ff700 1 -- 172.21.15.156:0/1883491590 shutdown_connections 2023-11-28T13:09:03.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:03.937+0000 7f7d307ff700 1 -- 172.21.15.156:0/1883491590 wait complete. 2023-11-28T13:09:03.956 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-28T13:09:04.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.154+0000 7f4dad7ff700 1 Processor -- start 2023-11-28T13:09:04.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.154+0000 7f4dad7ff700 1 -- start start 2023-11-28T13:09:04.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.156+0000 7f4dad7ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4da8159860 0x7f4da80c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:04.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.156+0000 7f4dad7ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4da80b0d80 0x7f4da80c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:04.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.156+0000 7f4dad7ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da80ccc00 con 0x7f4da8159490 2023-11-28T13:09:04.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.156+0000 7f4dad7ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da80ccec0 con 0x7f4da80b0d80 2023-11-28T13:09:04.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.156+0000 7f4dad7ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da80cd1a0 con 0x7f4da8159860 2023-11-28T13:09:04.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.156+0000 7f4da6ffd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4da8159490 0x7f4da80b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:33420/0 (socket says 172.21.15.156:33420) 2023-11-28T13:09:04.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.156+0000 7f4da6ffd700 1 -- 172.21.15.156:0/1526422714 learned_addr learned my addr 172.21.15.156:0/1526422714 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:04.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.156+0000 7f4da77fe700 1 --2- 172.21.15.156:0/1526422714 >> v2:172.21.15.156:3300/0 conn(0x7f4da80b0d80 0x7f4da80c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:04.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.156+0000 7f4da67fc700 1 -- 172.21.15.156:0/1526422714 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561556805 0 0) 0x7f4da80ccc00 con 0x7f4da8159490 2023-11-28T13:09:04.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.156+0000 7f4da77fe700 1 -- 172.21.15.156:0/1526422714 >> v2:172.21.15.186:3300/0 conn(0x7f4da8159860 msgr2=0x7f4da80c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:04.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.156+0000 7f4d9ffff700 1 --2- 172.21.15.156:0/1526422714 >> v2:172.21.15.186:3300/0 conn(0x7f4da8159860 0x7f4da80c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:04.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.156+0000 7f4da77fe700 1 --2- 172.21.15.156:0/1526422714 >> v2:172.21.15.186:3300/0 conn(0x7f4da8159860 0x7f4da80c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.157+0000 7f4da77fe700 1 -- 172.21.15.156:0/1526422714 >> v1:172.21.15.156:6789/0 conn(0x7f4da8159490 legacy=0x7f4da80b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:04.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.157+0000 7f4da77fe700 1 -- 172.21.15.156:0/1526422714 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da80cd480 con 0x7f4da80b0d80 2023-11-28T13:09:04.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.157+0000 7f4d9ffff700 1 --2- 172.21.15.156:0/1526422714 >> v2:172.21.15.186:3300/0 conn(0x7f4da8159860 0x7f4da80c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:04.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.157+0000 7f4da77fe700 1 --2- 172.21.15.156:0/1526422714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4da80b0d80 0x7f4da80c9a00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4da80cb5d0 tx=0x7f4da000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c529f9d9f90021bf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:04.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.157+0000 7f4da67fc700 1 -- 172.21.15.156:0/1526422714 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da000e070 con 0x7f4da80b0d80 2023-11-28T13:09:04.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.157+0000 7f4da67fc700 1 -- 172.21.15.156:0/1526422714 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da000bbd0 con 0x7f4da80b0d80 2023-11-28T13:09:04.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.157+0000 7f4da67fc700 1 -- 172.21.15.156:0/1526422714 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0012430 con 0x7f4da80b0d80 2023-11-28T13:09:04.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.157+0000 7f4dad7ff700 1 -- 172.21.15.156:0/1526422714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4da80b0d80 msgr2=0x7f4da80c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:04.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.157+0000 7f4dad7ff700 1 --2- 172.21.15.156:0/1526422714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4da80b0d80 0x7f4da80c9a00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4da80cb5d0 tx=0x7f4da000b330 comp rx=0 tx=0).stop 2023-11-28T13:09:04.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.158+0000 7f4dad7ff700 1 -- 172.21.15.156:0/1526422714 shutdown_connections 2023-11-28T13:09:04.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.158+0000 7f4dad7ff700 1 --2- 172.21.15.156:0/1526422714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4da80b0d80 0x7f4da80c9a00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.158+0000 7f4dad7ff700 1 --2- 172.21.15.156:0/1526422714 >> v2:172.21.15.186:3300/0 conn(0x7f4da8159860 0x7f4da80c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.158+0000 7f4dad7ff700 1 -- 172.21.15.156:0/1526422714 >> 172.21.15.156:0/1526422714 conn(0x7f4da80cb670 msgr2=0x7f4da80cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:04.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.158+0000 7f4dad7ff700 1 -- 172.21.15.156:0/1526422714 shutdown_connections 2023-11-28T13:09:04.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.158+0000 7f4dad7ff700 1 -- 172.21.15.156:0/1526422714 wait complete. 2023-11-28T13:09:04.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.159+0000 7f4dad7ff700 1 Processor -- start 2023-11-28T13:09:04.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.159+0000 7f4dad7ff700 1 -- start start 2023-11-28T13:09:04.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.160+0000 7f4dad7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4da80b0d80 0x7f4da8152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:04.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.160+0000 7f4dad7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4da8159490 0x7f4da8148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:04.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.160+0000 7f4dad7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4da815c300 0x7f4da80c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:04.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.160+0000 7f4dad7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da815be70 con 0x7f4da80b0d80 2023-11-28T13:09:04.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.160+0000 7f4dad7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da815b2a0 con 0x7f4da8159490 2023-11-28T13:09:04.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.160+0000 7f4dad7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da815b540 con 0x7f4da815c300 2023-11-28T13:09:04.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.161+0000 7f4da6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4da80b0d80 0x7f4da8152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:04.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.161+0000 7f4da6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4da80b0d80 0x7f4da8152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41550/0 (socket says 172.21.15.156:41550) 2023-11-28T13:09:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.161+0000 7f4da6ffd700 1 -- 172.21.15.156:0/753704677 learned_addr learned my addr 172.21.15.156:0/753704677 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.161+0000 7f4d9ffff700 1 --2- 172.21.15.156:0/753704677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4da8159490 0x7f4da8148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.161+0000 7f4da77fe700 1 --2- 172.21.15.156:0/753704677 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4da815c300 0x7f4da80c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.161+0000 7f4da6ffd700 1 -- 172.21.15.156:0/753704677 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4da815c300 msgr2=0x7f4da80c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.161+0000 7f4da6ffd700 1 --2- 172.21.15.156:0/753704677 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4da815c300 0x7f4da80c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.161+0000 7f4da6ffd700 1 -- 172.21.15.156:0/753704677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4da8159490 msgr2=0x7f4da8148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.161+0000 7f4da6ffd700 1 --2- 172.21.15.156:0/753704677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4da8159490 0x7f4da8148fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.161+0000 7f4da6ffd700 1 -- 172.21.15.156:0/753704677 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da0009a70 con 0x7f4da80b0d80 2023-11-28T13:09:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.161+0000 7f4da6ffd700 1 --2- 172.21.15.156:0/753704677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4da80b0d80 0x7f4da8152d40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4d980011e0 tx=0x7f4d9800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.162+0000 7f4d9f7fe700 1 -- 172.21.15.156:0/753704677 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98004c80 con 0x7f4da80b0d80 2023-11-28T13:09:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.162+0000 7f4d9f7fe700 1 -- 172.21.15.156:0/753704677 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d98004e00 con 0x7f4da80b0d80 2023-11-28T13:09:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.162+0000 7f4dad7ff700 1 -- 172.21.15.156:0/753704677 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da80c9f40 con 0x7f4da80b0d80 2023-11-28T13:09:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.162+0000 7f4d9f7fe700 1 -- 172.21.15.156:0/753704677 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98007310 con 0x7f4da80b0d80 2023-11-28T13:09:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.162+0000 7f4dad7ff700 1 -- 172.21.15.156:0/753704677 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da80cd760 con 0x7f4da80b0d80 2023-11-28T13:09:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.163+0000 7f4dad7ff700 1 -- 172.21.15.156:0/753704677 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da80cdc80 con 0x7f4da80b0d80 2023-11-28T13:09:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.164+0000 7f4d9f7fe700 1 -- 172.21.15.156:0/753704677 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4d9801a020 con 0x7f4da80b0d80 2023-11-28T13:09:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.164+0000 7f4d9f7fe700 1 --2- 172.21.15.156:0/753704677 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d880682f0 0x7f4d8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:04.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.164+0000 7f4d9f7fe700 1 -- 172.21.15.156:0/753704677 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4d98085510 con 0x7f4da80b0d80 2023-11-28T13:09:04.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.165+0000 7f4d9ffff700 1 --2- 172.21.15.156:0/753704677 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d880682f0 0x7f4d8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:04.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.166+0000 7f4d9ffff700 1 --2- 172.21.15.156:0/753704677 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d880682f0 0x7f4d8806a7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4d90006fd0 tx=0x7f4d90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:04.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.168+0000 7f4d9f7fe700 1 -- 172.21.15.156:0/753704677 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4d98051fd0 con 0x7f4da80b0d80 2023-11-28T13:09:04.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.433+0000 7f4dad7ff700 1 -- 172.21.15.156:0/753704677 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f4da80745c0 con 0x7f4da80b0d80 2023-11-28T13:09:04.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.437+0000 7f4d9f7fe700 1 -- 172.21.15.156:0/753704677 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4d980555f0 con 0x7f4da80b0d80 2023-11-28T13:09:04.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.439+0000 7f4dad7ff700 1 -- 172.21.15.156:0/753704677 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d880682f0 msgr2=0x7f4d8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:04.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.439+0000 7f4dad7ff700 1 --2- 172.21.15.156:0/753704677 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d880682f0 0x7f4d8806a7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4d90006fd0 tx=0x7f4d90008040 comp rx=0 tx=0).stop 2023-11-28T13:09:04.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.439+0000 7f4dad7ff700 1 -- 172.21.15.156:0/753704677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4da80b0d80 msgr2=0x7f4da8152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:04.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.439+0000 7f4dad7ff700 1 --2- 172.21.15.156:0/753704677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4da80b0d80 0x7f4da8152d40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4d980011e0 tx=0x7f4d9800ca30 comp rx=0 tx=0).stop 2023-11-28T13:09:04.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.439+0000 7f4dad7ff700 1 -- 172.21.15.156:0/753704677 shutdown_connections 2023-11-28T13:09:04.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.439+0000 7f4dad7ff700 1 --2- 172.21.15.156:0/753704677 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d880682f0 0x7f4d8806a7a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.439+0000 7f4dad7ff700 1 --2- 172.21.15.156:0/753704677 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4da815c300 0x7f4da80c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.439+0000 7f4dad7ff700 1 --2- 172.21.15.156:0/753704677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4da80b0d80 0x7f4da8152d40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.439+0000 7f4dad7ff700 1 --2- 172.21.15.156:0/753704677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4da8159490 0x7f4da8148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.439+0000 7f4dad7ff700 1 -- 172.21.15.156:0/753704677 >> 172.21.15.156:0/753704677 conn(0x7f4da80cb670 msgr2=0x7f4da8157fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:04.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.439+0000 7f4dad7ff700 1 -- 172.21.15.156:0/753704677 shutdown_connections 2023-11-28T13:09:04.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.440+0000 7f4dad7ff700 1 -- 172.21.15.156:0/753704677 wait complete. 2023-11-28T13:09:04.458 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:09:04.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.656+0000 7f4bec9ff700 1 Processor -- start 2023-11-28T13:09:04.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.657+0000 7f4bec9ff700 1 -- start start 2023-11-28T13:09:04.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.658+0000 7f4bec9ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4be815c9b0 0x7f4be80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:04.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.658+0000 7f4bec9ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4be80cd7c0 0x7f4be80cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:04.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.658+0000 7f4bec9ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be80d1950 con 0x7f4be8161340 2023-11-28T13:09:04.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.658+0000 7f4bec9ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be80b5800 con 0x7f4be80cd7c0 2023-11-28T13:09:04.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.658+0000 7f4bec9ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be80ce140 con 0x7f4be815c9b0 2023-11-28T13:09:04.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.658+0000 7f4be6d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4be80cd7c0 0x7f4be80cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:04.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.658+0000 7f4be6d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4be80cd7c0 0x7f4be80cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41566/0 (socket says 172.21.15.156:41566) 2023-11-28T13:09:04.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.658+0000 7f4be6d9d700 1 -- 172.21.15.156:0/3425553418 learned_addr learned my addr 172.21.15.156:0/3425553418 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:04.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.658+0000 7f4bdffff700 1 --1- 172.21.15.156:0/3425553418 >> v1:172.21.15.156:6789/0 conn(0x7f4be8161340 0x7f4be80d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:33428/0 (socket says 172.21.15.156:33428) 2023-11-28T13:09:04.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.658+0000 7f4be659c700 1 --2- 172.21.15.156:0/3425553418 >> v2:172.21.15.186:3300/0 conn(0x7f4be815c9b0 0x7f4be80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:04.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.658+0000 7f4be6d9d700 1 -- 172.21.15.156:0/3425553418 >> v2:172.21.15.186:3300/0 conn(0x7f4be815c9b0 msgr2=0x7f4be80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:04.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.658+0000 7f4be6d9d700 1 --2- 172.21.15.156:0/3425553418 >> v2:172.21.15.186:3300/0 conn(0x7f4be815c9b0 0x7f4be80b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.658+0000 7f4be6d9d700 1 -- 172.21.15.156:0/3425553418 >> v1:172.21.15.156:6789/0 conn(0x7f4be8161340 legacy=0x7f4be80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:04.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.658+0000 7f4be6d9d700 1 -- 172.21.15.156:0/3425553418 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be80d24f0 con 0x7f4be80cd7c0 2023-11-28T13:09:04.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.658+0000 7f4be5d9b700 1 -- 172.21.15.156:0/3425553418 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784568479 0 0) 0x7f4be80d1950 con 0x7f4be8161340 2023-11-28T13:09:04.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.659+0000 7f4be6d9d700 1 --2- 172.21.15.156:0/3425553418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be80cd7c0 0x7f4be80cdc00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4be8067460 tx=0x7f4be000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8801d2c3bf634e10 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:04.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.659+0000 7f4be5d9b700 1 -- 172.21.15.156:0/3425553418 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be000e070 con 0x7f4be80cd7c0 2023-11-28T13:09:04.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.659+0000 7f4be5d9b700 1 -- 172.21.15.156:0/3425553418 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be000b9c0 con 0x7f4be80cd7c0 2023-11-28T13:09:04.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.659+0000 7f4be5d9b700 1 -- 172.21.15.156:0/3425553418 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be000bd70 con 0x7f4be80cd7c0 2023-11-28T13:09:04.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.659+0000 7f4bec9ff700 1 -- 172.21.15.156:0/3425553418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be80cd7c0 msgr2=0x7f4be80cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:04.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.659+0000 7f4bec9ff700 1 --2- 172.21.15.156:0/3425553418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be80cd7c0 0x7f4be80cdc00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4be8067460 tx=0x7f4be000b330 comp rx=0 tx=0).stop 2023-11-28T13:09:04.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.660+0000 7f4bec9ff700 1 -- 172.21.15.156:0/3425553418 shutdown_connections 2023-11-28T13:09:04.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.660+0000 7f4bec9ff700 1 --2- 172.21.15.156:0/3425553418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be80cd7c0 0x7f4be80cdc00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.660+0000 7f4bec9ff700 1 --2- 172.21.15.156:0/3425553418 >> v2:172.21.15.186:3300/0 conn(0x7f4be815c9b0 0x7f4be80b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.660+0000 7f4bec9ff700 1 -- 172.21.15.156:0/3425553418 >> 172.21.15.156:0/3425553418 conn(0x7f4be8072600 msgr2=0x7f4be80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:04.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.660+0000 7f4bec9ff700 1 -- 172.21.15.156:0/3425553418 shutdown_connections 2023-11-28T13:09:04.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.660+0000 7f4bec9ff700 1 -- 172.21.15.156:0/3425553418 wait complete. 2023-11-28T13:09:04.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.662+0000 7f4bec9ff700 1 Processor -- start 2023-11-28T13:09:04.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.662+0000 7f4bec9ff700 1 -- start start 2023-11-28T13:09:04.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.663+0000 7f4bec9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4be80cd7c0 0x7f4be814a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:04.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.663+0000 7f4bec9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4be815c9b0 0x7f4be815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:04.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.663+0000 7f4bec9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be8161340 0x7f4be815e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:04.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.663+0000 7f4bec9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be814aac0 con 0x7f4be8161340 2023-11-28T13:09:04.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.664+0000 7f4bec9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be80cc610 con 0x7f4be815c9b0 2023-11-28T13:09:04.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.664+0000 7f4bec9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be80ca370 con 0x7f4be80cd7c0 2023-11-28T13:09:04.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.664+0000 7f4be6d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be8161340 0x7f4be815e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:04.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.664+0000 7f4be6d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be8161340 0x7f4be815e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41578/0 (socket says 172.21.15.156:41578) 2023-11-28T13:09:04.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.664+0000 7f4be6d9d700 1 -- 172.21.15.156:0/3372742047 learned_addr learned my addr 172.21.15.156:0/3372742047 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:04.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.664+0000 7f4be659c700 1 --2- 172.21.15.156:0/3372742047 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4be80cd7c0 0x7f4be814a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:04.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.664+0000 7f4bdffff700 1 --2- 172.21.15.156:0/3372742047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4be815c9b0 0x7f4be815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:04.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.664+0000 7f4be6d9d700 1 -- 172.21.15.156:0/3372742047 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4be80cd7c0 msgr2=0x7f4be814a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:04.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.664+0000 7f4be6d9d700 1 --2- 172.21.15.156:0/3372742047 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4be80cd7c0 0x7f4be814a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.664+0000 7f4be6d9d700 1 -- 172.21.15.156:0/3372742047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4be815c9b0 msgr2=0x7f4be815e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:04.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.664+0000 7f4be6d9d700 1 --2- 172.21.15.156:0/3372742047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4be815c9b0 0x7f4be815e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.664+0000 7f4be6d9d700 1 -- 172.21.15.156:0/3372742047 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be0009b30 con 0x7f4be8161340 2023-11-28T13:09:04.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.664+0000 7f4bdffff700 1 --2- 172.21.15.156:0/3372742047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4be815c9b0 0x7f4be815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:04.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.664+0000 7f4be6d9d700 1 --2- 172.21.15.156:0/3372742047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be8161340 0x7f4be815e930 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4be80675d0 tx=0x7f4be0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:04.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.665+0000 7f4bdeffd700 1 -- 172.21.15.156:0/3372742047 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be00099b0 con 0x7f4be8161340 2023-11-28T13:09:04.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.665+0000 7f4bdeffd700 1 -- 172.21.15.156:0/3372742047 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be00092e0 con 0x7f4be8161340 2023-11-28T13:09:04.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.665+0000 7f4bdeffd700 1 -- 172.21.15.156:0/3372742047 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be001b650 con 0x7f4be8161340 2023-11-28T13:09:04.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.665+0000 7f4bec9ff700 1 -- 172.21.15.156:0/3372742047 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be80d1300 con 0x7f4be8161340 2023-11-28T13:09:04.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.665+0000 7f4bec9ff700 1 -- 172.21.15.156:0/3372742047 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be80b5520 con 0x7f4be8161340 2023-11-28T13:09:04.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.666+0000 7f4bec9ff700 1 -- 172.21.15.156:0/3372742047 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be80d15e0 con 0x7f4be8161340 2023-11-28T13:09:04.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.667+0000 7f4bdeffd700 1 -- 172.21.15.156:0/3372742047 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4be002b070 con 0x7f4be8161340 2023-11-28T13:09:04.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.667+0000 7f4bdeffd700 1 --2- 172.21.15.156:0/3372742047 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4bc80682f0 0x7f4bc806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:04.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.668+0000 7f4bdeffd700 1 -- 172.21.15.156:0/3372742047 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4be00871f0 con 0x7f4be8161340 2023-11-28T13:09:04.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.668+0000 7f4be659c700 1 --2- 172.21.15.156:0/3372742047 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4bc80682f0 0x7f4bc806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:04.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.669+0000 7f4be659c700 1 --2- 172.21.15.156:0/3372742047 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4bc80682f0 0x7f4bc806a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800b9b0 tx=0x7f4bd8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:04.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.672+0000 7f4bdeffd700 1 -- 172.21.15.156:0/3372742047 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4be0053cb0 con 0x7f4be8161340 2023-11-28T13:09:04.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.863+0000 7f4bec9ff700 1 -- 172.21.15.156:0/3372742047 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4be80cf0d0 con 0x7f4be8161340 2023-11-28T13:09:04.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.864+0000 7f4bdeffd700 1 -- 172.21.15.156:0/3372742047 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+14257 (secure 0 0 0) 0x7f4be00572d0 con 0x7f4be8161340 2023-11-28T13:09:04.865 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:04.865 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":31,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:09:00.855747+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:09:04.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.867+0000 7f4bec9ff700 1 -- 172.21.15.156:0/3372742047 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4bc80682f0 msgr2=0x7f4bc806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:04.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.867+0000 7f4bec9ff700 1 --2- 172.21.15.156:0/3372742047 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4bc80682f0 0x7f4bc806a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800b9b0 tx=0x7f4bd8008040 comp rx=0 tx=0).stop 2023-11-28T13:09:04.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.867+0000 7f4bec9ff700 1 -- 172.21.15.156:0/3372742047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be8161340 msgr2=0x7f4be815e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:04.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.867+0000 7f4bec9ff700 1 --2- 172.21.15.156:0/3372742047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be8161340 0x7f4be815e930 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4be80675d0 tx=0x7f4be0004ad0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.867+0000 7f4bec9ff700 1 -- 172.21.15.156:0/3372742047 shutdown_connections 2023-11-28T13:09:04.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.867+0000 7f4bec9ff700 1 --2- 172.21.15.156:0/3372742047 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4bc80682f0 0x7f4bc806a7a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.867+0000 7f4bec9ff700 1 --2- 172.21.15.156:0/3372742047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be8161340 0x7f4be815e930 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.867+0000 7f4bec9ff700 1 --2- 172.21.15.156:0/3372742047 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4be80cd7c0 0x7f4be814a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.867+0000 7f4bec9ff700 1 --2- 172.21.15.156:0/3372742047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4be815c9b0 0x7f4be815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:04.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.867+0000 7f4bec9ff700 1 -- 172.21.15.156:0/3372742047 >> 172.21.15.156:0/3372742047 conn(0x7f4be8072600 msgr2=0x7f4be80b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:04.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.867+0000 7f4bec9ff700 1 -- 172.21.15.156:0/3372742047 shutdown_connections 2023-11-28T13:09:04.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:04.867+0000 7f4bec9ff700 1 -- 172.21.15.156:0/3372742047 wait complete. 2023-11-28T13:09:04.886 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:09:04.886 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:09:05.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.085+0000 7f4ed8bff700 1 Processor -- start 2023-11-28T13:09:05.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.086+0000 7f4ed8bff700 1 -- start start 2023-11-28T13:09:05.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.087+0000 7f4ed8bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ed4161340 0x7f4ed40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.087+0000 7f4ed8bff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4ed40d2320 0x7f4ed40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.087+0000 7f4ed8bff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed40d1950 con 0x7f4ed415c9b0 2023-11-28T13:09:05.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.087+0000 7f4ed8bff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed40b5800 con 0x7f4ed40d2320 2023-11-28T13:09:05.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.087+0000 7f4ed8bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed40b5300 con 0x7f4ed4161340 2023-11-28T13:09:05.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.087+0000 7f4ed259c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4ed415c9b0 0x7f4ed40b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:33436/0 (socket says 172.21.15.156:33436) 2023-11-28T13:09:05.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.087+0000 7f4ed259c700 1 -- 172.21.15.156:0/1861294368 learned_addr learned my addr 172.21.15.156:0/1861294368 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:05.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.087+0000 7f4ecbfff700 1 --2- 172.21.15.156:0/1861294368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ed4161340 0x7f4ed40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.087+0000 7f4ed2d9d700 1 --2- 172.21.15.156:0/1861294368 >> v2:172.21.15.186:3300/0 conn(0x7f4ed40d2320 0x7f4ed40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.087+0000 7f4ed1d9b700 1 -- 172.21.15.156:0/1861294368 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122014038 0 0) 0x7f4ed40d1950 con 0x7f4ed415c9b0 2023-11-28T13:09:05.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.087+0000 7f4ed1d9b700 1 -- 172.21.15.156:0/1861294368 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ec00035b0 con 0x7f4ed415c9b0 2023-11-28T13:09:05.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.088+0000 7f4ed1d9b700 1 -- 172.21.15.156:0/1861294368 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4128887138 0 0) 0x7f4ec00035b0 con 0x7f4ed415c9b0 2023-11-28T13:09:05.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.088+0000 7f4ed1d9b700 1 -- 172.21.15.156:0/1861294368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ed4161340 msgr2=0x7f4ed40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:05.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.088+0000 7f4ed1d9b700 1 --2- 172.21.15.156:0/1861294368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ed4161340 0x7f4ed40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.088+0000 7f4ed1d9b700 1 -- 172.21.15.156:0/1861294368 >> v2:172.21.15.186:3300/0 conn(0x7f4ed40d2320 msgr2=0x7f4ed40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:05.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.088+0000 7f4ed1d9b700 1 --2- 172.21.15.156:0/1861294368 >> v2:172.21.15.186:3300/0 conn(0x7f4ed40d2320 0x7f4ed40cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.088+0000 7f4ed1d9b700 1 -- 172.21.15.156:0/1861294368 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed40b5520 con 0x7f4ed415c9b0 2023-11-28T13:09:05.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.088+0000 7f4ed1d9b700 1 -- 172.21.15.156:0/1861294368 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f4ec4001670 con 0x7f4ed415c9b0 2023-11-28T13:09:05.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.088+0000 7f4ed1d9b700 1 -- 172.21.15.156:0/1861294368 >> v1:172.21.15.156:6789/0 conn(0x7f4ed415c9b0 legacy=0x7f4ed40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:05.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.088+0000 7f4ed1d9b700 1 --2- 172.21.15.156:0/1861294368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ec0003a30 0x7f4ec0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.088+0000 7f4ed1d9b700 1 --2- 172.21.15.156:0/1861294368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ed4161340 0x7f4ec0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.088+0000 7f4ed1d9b700 1 --2- 172.21.15.156:0/1861294368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ec0004d90 0x7f4ec0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.088+0000 7f4ed1d9b700 1 -- 172.21.15.156:0/1861294368 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed40b5800 con 0x7f4ec0003a30 2023-11-28T13:09:05.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.088+0000 7f4ed1d9b700 1 -- 172.21.15.156:0/1861294368 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed40b5300 con 0x7f4ec0004d90 2023-11-28T13:09:05.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.088+0000 7f4ed1d9b700 1 -- 172.21.15.156:0/1861294368 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec000ba10 con 0x7f4ed4161340 2023-11-28T13:09:05.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.088+0000 7f4ed2d9d700 1 --2- 172.21.15.156:0/1861294368 >> v2:172.21.15.186:3300/0 conn(0x7f4ed40d2320 0x7f4ed40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:09:05.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.088+0000 7f4ed259c700 1 --2- 172.21.15.156:0/1861294368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ec0003a30 0x7f4ec0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.088+0000 7f4ecbfff700 1 --2- 172.21.15.156:0/1861294368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ed4161340 0x7f4ec0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.089+0000 7f4ed2d9d700 1 --2- 172.21.15.156:0/1861294368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ec0004d90 0x7f4ec0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.089+0000 7f4ed259c700 1 -- 172.21.15.156:0/1861294368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ed4161340 msgr2=0x7f4ec0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:05.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.089+0000 7f4ed259c700 1 --2- 172.21.15.156:0/1861294368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ed4161340 0x7f4ec0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.089+0000 7f4ed259c700 1 -- 172.21.15.156:0/1861294368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ec0004d90 msgr2=0x7f4ec0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:05.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.089+0000 7f4ed259c700 1 --2- 172.21.15.156:0/1861294368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ec0004d90 0x7f4ec0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.089+0000 7f4ed259c700 1 -- 172.21.15.156:0/1861294368 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec00035b0 con 0x7f4ec0003a30 2023-11-28T13:09:05.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.089+0000 7f4ed259c700 1 --2- 172.21.15.156:0/1861294368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ec0003a30 0x7f4ec0003e40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400c240 tx=0x7f4ec0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cec0774e44474f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:05.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.089+0000 7f4ed1d9b700 1 -- 172.21.15.156:0/1861294368 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4007c10 con 0x7f4ec0003a30 2023-11-28T13:09:05.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.089+0000 7f4ed1d9b700 1 -- 172.21.15.156:0/1861294368 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec400ce00 con 0x7f4ec0003a30 2023-11-28T13:09:05.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.089+0000 7f4ed1d9b700 1 -- 172.21.15.156:0/1861294368 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4005ba0 con 0x7f4ec0003a30 2023-11-28T13:09:05.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.090+0000 7f4ed8bff700 1 -- 172.21.15.156:0/1861294368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ec0003a30 msgr2=0x7f4ec0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:05.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.090+0000 7f4ed8bff700 1 --2- 172.21.15.156:0/1861294368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ec0003a30 0x7f4ec0003e40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400c240 tx=0x7f4ec0005880 comp rx=0 tx=0).stop 2023-11-28T13:09:05.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.090+0000 7f4ed2d9d700 1 -- 172.21.15.156:0/1861294368 reap_dead start 2023-11-28T13:09:05.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.090+0000 7f4ed8bff700 1 -- 172.21.15.156:0/1861294368 shutdown_connections 2023-11-28T13:09:05.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.090+0000 7f4ed8bff700 1 -- 172.21.15.156:0/1861294368 >> 172.21.15.156:0/1861294368 conn(0x7f4ed4072600 msgr2=0x7f4ed40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:05.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.090+0000 7f4ed8bff700 1 -- 172.21.15.156:0/1861294368 shutdown_connections 2023-11-28T13:09:05.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.090+0000 7f4ed8bff700 1 -- 172.21.15.156:0/1861294368 wait complete. 2023-11-28T13:09:05.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.092+0000 7f4ed8bff700 1 Processor -- start 2023-11-28T13:09:05.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.092+0000 7f4ed8bff700 1 -- start start 2023-11-28T13:09:05.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ed8bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ec0003a30 0x7f4ed40cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ed8bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ed40c91e0 0x7f4ed40cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ed8bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ed41578e0 0x7f4ed40b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ed8bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed40cc1f0 con 0x7f4ec0003a30 2023-11-28T13:09:05.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ed8bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed40cbc30 con 0x7f4ed41578e0 2023-11-28T13:09:05.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ed8bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed40cbf10 con 0x7f4ed40c91e0 2023-11-28T13:09:05.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ed259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ec0003a30 0x7f4ed40cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ed259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ec0003a30 0x7f4ed40cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41594/0 (socket says 172.21.15.156:41594) 2023-11-28T13:09:05.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ed259c700 1 -- 172.21.15.156:0/2932656890 learned_addr learned my addr 172.21.15.156:0/2932656890 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:05.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ecbfff700 1 --2- 172.21.15.156:0/2932656890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ed40c91e0 0x7f4ed40cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ed2d9d700 1 --2- 172.21.15.156:0/2932656890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ed41578e0 0x7f4ed40b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ed259c700 1 -- 172.21.15.156:0/2932656890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ed40c91e0 msgr2=0x7f4ed40cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:05.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ed259c700 1 --2- 172.21.15.156:0/2932656890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ed40c91e0 0x7f4ed40cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ed259c700 1 -- 172.21.15.156:0/2932656890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ed41578e0 msgr2=0x7f4ed40b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:05.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ed259c700 1 --2- 172.21.15.156:0/2932656890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ed41578e0 0x7f4ed40b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ed259c700 1 -- 172.21.15.156:0/2932656890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed40d15e0 con 0x7f4ec0003a30 2023-11-28T13:09:05.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ecbfff700 1 --2- 172.21.15.156:0/2932656890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ed40c91e0 0x7f4ed40cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:05.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.093+0000 7f4ed259c700 1 --2- 172.21.15.156:0/2932656890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ec0003a30 0x7f4ed40cb3e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400c830 tx=0x7f4ec4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:05.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.094+0000 7f4ecaffd700 1 -- 172.21.15.156:0/2932656890 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4001630 con 0x7f4ec0003a30 2023-11-28T13:09:05.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.094+0000 7f4ecaffd700 1 -- 172.21.15.156:0/2932656890 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec40070d0 con 0x7f4ec0003a30 2023-11-28T13:09:05.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.094+0000 7f4ed8bff700 1 -- 172.21.15.156:0/2932656890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed40d1300 con 0x7f4ec0003a30 2023-11-28T13:09:05.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.094+0000 7f4ecaffd700 1 -- 172.21.15.156:0/2932656890 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40074a0 con 0x7f4ec0003a30 2023-11-28T13:09:05.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.094+0000 7f4ed8bff700 1 -- 172.21.15.156:0/2932656890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed415cd80 con 0x7f4ec0003a30 2023-11-28T13:09:05.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.095+0000 7f4ed8bff700 1 -- 172.21.15.156:0/2932656890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed40ca220 con 0x7f4ec0003a30 2023-11-28T13:09:05.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.096+0000 7f4ecaffd700 1 -- 172.21.15.156:0/2932656890 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4ec4010070 con 0x7f4ec0003a30 2023-11-28T13:09:05.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.097+0000 7f4ecaffd700 1 --2- 172.21.15.156:0/2932656890 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4eb40682f0 0x7f4eb406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.097+0000 7f4ecaffd700 1 -- 172.21.15.156:0/2932656890 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4ec4086cf0 con 0x7f4ec0003a30 2023-11-28T13:09:05.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.097+0000 7f4ecbfff700 1 --2- 172.21.15.156:0/2932656890 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4eb40682f0 0x7f4eb406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.098+0000 7f4ecbfff700 1 --2- 172.21.15.156:0/2932656890 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4eb40682f0 0x7f4eb406a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40c8c20 tx=0x7f4ebc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:05.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.101+0000 7f4ecaffd700 1 -- 172.21.15.156:0/2932656890 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4ec4053730 con 0x7f4ec0003a30 2023-11-28T13:09:05.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.289+0000 7f4ed8bff700 1 -- 172.21.15.156:0/2932656890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4ed40cc130 con 0x7f4ec0003a30 2023-11-28T13:09:05.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.289+0000 7f4ecaffd700 1 -- 172.21.15.156:0/2932656890 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v31) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4ec4056d50 con 0x7f4ec0003a30 2023-11-28T13:09:05.290 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:09:05.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.291+0000 7f4ed8bff700 1 -- 172.21.15.156:0/2932656890 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4eb40682f0 msgr2=0x7f4eb406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:05.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.291+0000 7f4ed8bff700 1 --2- 172.21.15.156:0/2932656890 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4eb40682f0 0x7f4eb406a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40c8c20 tx=0x7f4ebc008040 comp rx=0 tx=0).stop 2023-11-28T13:09:05.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.291+0000 7f4ed8bff700 1 -- 172.21.15.156:0/2932656890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ec0003a30 msgr2=0x7f4ed40cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:05.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.291+0000 7f4ed8bff700 1 --2- 172.21.15.156:0/2932656890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ec0003a30 0x7f4ed40cb3e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400c830 tx=0x7f4ec4001a00 comp rx=0 tx=0).stop 2023-11-28T13:09:05.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.291+0000 7f4ed8bff700 1 -- 172.21.15.156:0/2932656890 shutdown_connections 2023-11-28T13:09:05.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.291+0000 7f4ed8bff700 1 --2- 172.21.15.156:0/2932656890 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4eb40682f0 0x7f4eb406a7a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.291+0000 7f4ed8bff700 1 --2- 172.21.15.156:0/2932656890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ed41578e0 0x7f4ed40b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.291+0000 7f4ed8bff700 1 --2- 172.21.15.156:0/2932656890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ec0003a30 0x7f4ed40cb3e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.291+0000 7f4ed8bff700 1 --2- 172.21.15.156:0/2932656890 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ed40c91e0 0x7f4ed40cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.291+0000 7f4ed8bff700 1 -- 172.21.15.156:0/2932656890 >> 172.21.15.156:0/2932656890 conn(0x7f4ed4072600 msgr2=0x7f4ed40b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:05.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.291+0000 7f4ed8bff700 1 -- 172.21.15.156:0/2932656890 shutdown_connections 2023-11-28T13:09:05.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.292+0000 7f4ed8bff700 1 -- 172.21.15.156:0/2932656890 wait complete. 2023-11-28T13:09:05.311 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:09:05.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.510+0000 7f450cfff700 1 Processor -- start 2023-11-28T13:09:05.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.510+0000 7f450cfff700 1 -- start start 2023-11-28T13:09:05.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.511+0000 7f450cfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4508161340 0x7f45080d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.512 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.511+0000 7f450cfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f45080d2320 0x7f45080cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.511+0000 7f450cfff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45080d1950 con 0x7f450815c9b0 2023-11-28T13:09:05.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.511+0000 7f450cfff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45080b5800 con 0x7f45080d2320 2023-11-28T13:09:05.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.511+0000 7f450cfff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45080b5300 con 0x7f4508161340 2023-11-28T13:09:05.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.511+0000 7f4506d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f45080d2320 0x7f45080cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.512+0000 7f4506d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f45080d2320 0x7f45080cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41608/0 (socket says 172.21.15.156:41608) 2023-11-28T13:09:05.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.512+0000 7f450659c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f450815c9b0 0x7f45080b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:33444/0 (socket says 172.21.15.156:33444) 2023-11-28T13:09:05.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.512+0000 7f4506d9d700 1 -- 172.21.15.156:0/475791592 learned_addr learned my addr 172.21.15.156:0/475791592 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:05.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.512+0000 7f4505d9b700 1 --2- 172.21.15.156:0/475791592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4508161340 0x7f45080d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.512+0000 7f4506d9d700 1 -- 172.21.15.156:0/475791592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4508161340 msgr2=0x7f45080d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:05.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.512+0000 7f4506d9d700 1 --2- 172.21.15.156:0/475791592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4508161340 0x7f45080d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.512+0000 7f4506d9d700 1 -- 172.21.15.156:0/475791592 >> v1:172.21.15.156:6789/0 conn(0x7f450815c9b0 legacy=0x7f45080b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:05.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.512+0000 7f4506d9d700 1 -- 172.21.15.156:0/475791592 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45080b5520 con 0x7f45080d2320 2023-11-28T13:09:05.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.512+0000 7f4506d9d700 1 --2- 172.21.15.156:0/475791592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45080d2320 0x7f45080cd7c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4500009e80 tx=0x7f450000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae73a4511a85b28c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:05.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.512+0000 7f450559a700 1 -- 172.21.15.156:0/475791592 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450000e070 con 0x7f45080d2320 2023-11-28T13:09:05.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.513+0000 7f450559a700 1 -- 172.21.15.156:0/475791592 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450000b980 con 0x7f45080d2320 2023-11-28T13:09:05.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.513+0000 7f450559a700 1 -- 172.21.15.156:0/475791592 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450000bd30 con 0x7f45080d2320 2023-11-28T13:09:05.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.513+0000 7f450cfff700 1 -- 172.21.15.156:0/475791592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45080d2320 msgr2=0x7f45080cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:05.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.513+0000 7f450cfff700 1 --2- 172.21.15.156:0/475791592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45080d2320 0x7f45080cd7c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4500009e80 tx=0x7f450000b330 comp rx=0 tx=0).stop 2023-11-28T13:09:05.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.513+0000 7f450cfff700 1 -- 172.21.15.156:0/475791592 shutdown_connections 2023-11-28T13:09:05.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.513+0000 7f450cfff700 1 --2- 172.21.15.156:0/475791592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45080d2320 0x7f45080cd7c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.513+0000 7f450cfff700 1 --2- 172.21.15.156:0/475791592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4508161340 0x7f45080d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.513+0000 7f450cfff700 1 -- 172.21.15.156:0/475791592 >> 172.21.15.156:0/475791592 conn(0x7f4508072600 msgr2=0x7f45080cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:05.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.513+0000 7f450cfff700 1 -- 172.21.15.156:0/475791592 shutdown_connections 2023-11-28T13:09:05.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.513+0000 7f450cfff700 1 -- 172.21.15.156:0/475791592 wait complete. 2023-11-28T13:09:05.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.515+0000 7f450cfff700 1 Processor -- start 2023-11-28T13:09:05.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.515+0000 7f450cfff700 1 -- start start 2023-11-28T13:09:05.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.516+0000 7f450cfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45080d2320 0x7f45081f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.516+0000 7f450cfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f450815c9b0 0x7f4508155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.516+0000 7f450cfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4508161340 0x7f4508155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.516+0000 7f450cfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4508157a40 con 0x7f45080d2320 2023-11-28T13:09:05.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.516+0000 7f450cfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45080b9fd0 con 0x7f4508161340 2023-11-28T13:09:05.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.516+0000 7f450cfff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45080ba2b0 con 0x7f450815c9b0 2023-11-28T13:09:05.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.516+0000 7f450659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45080d2320 0x7f45081f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.516+0000 7f450659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45080d2320 0x7f45081f1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41616/0 (socket says 172.21.15.156:41616) 2023-11-28T13:09:05.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.516+0000 7f450659c700 1 -- 172.21.15.156:0/2991060715 learned_addr learned my addr 172.21.15.156:0/2991060715 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:05.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.516+0000 7f4505d9b700 1 --2- 172.21.15.156:0/2991060715 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f450815c9b0 0x7f4508155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.516+0000 7f4506d9d700 1 --2- 172.21.15.156:0/2991060715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4508161340 0x7f4508155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.517+0000 7f450659c700 1 -- 172.21.15.156:0/2991060715 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f450815c9b0 msgr2=0x7f4508155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:05.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.517+0000 7f450659c700 1 --2- 172.21.15.156:0/2991060715 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f450815c9b0 0x7f4508155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.517+0000 7f450659c700 1 -- 172.21.15.156:0/2991060715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4508161340 msgr2=0x7f4508155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:05.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.517+0000 7f450659c700 1 --2- 172.21.15.156:0/2991060715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4508161340 0x7f4508155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.517+0000 7f450659c700 1 -- 172.21.15.156:0/2991060715 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4500009b30 con 0x7f45080d2320 2023-11-28T13:09:05.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.517+0000 7f450659c700 1 --2- 172.21.15.156:0/2991060715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45080d2320 0x7f45081f1ba0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f44f800c500 tx=0x7f44f800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:05.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.517+0000 7f44f6ffd700 1 -- 172.21.15.156:0/2991060715 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f8005cb0 con 0x7f45080d2320 2023-11-28T13:09:05.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.517+0000 7f44f6ffd700 1 -- 172.21.15.156:0/2991060715 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f8005e10 con 0x7f45080d2320 2023-11-28T13:09:05.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.517+0000 7f450cfff700 1 -- 172.21.15.156:0/2991060715 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45080d15e0 con 0x7f45080d2320 2023-11-28T13:09:05.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.517+0000 7f44f6ffd700 1 -- 172.21.15.156:0/2991060715 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f80072d0 con 0x7f45080d2320 2023-11-28T13:09:05.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.517+0000 7f450cfff700 1 -- 172.21.15.156:0/2991060715 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45080d1300 con 0x7f45080d2320 2023-11-28T13:09:05.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.519+0000 7f450cfff700 1 -- 172.21.15.156:0/2991060715 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450815cd80 con 0x7f45080d2320 2023-11-28T13:09:05.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.519+0000 7f44f6ffd700 1 -- 172.21.15.156:0/2991060715 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f44f8019030 con 0x7f45080d2320 2023-11-28T13:09:05.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.520+0000 7f44f6ffd700 1 --2- 172.21.15.156:0/2991060715 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f44e8068320 0x7f44e806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.520+0000 7f44f6ffd700 1 -- 172.21.15.156:0/2991060715 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f44f80849e0 con 0x7f45080d2320 2023-11-28T13:09:05.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.520+0000 7f4505d9b700 1 --2- 172.21.15.156:0/2991060715 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f44e8068320 0x7f44e806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.521+0000 7f4505d9b700 1 --2- 172.21.15.156:0/2991060715 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f44e8068320 0x7f44e806a7d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f45080c6b80 tx=0x7f44fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:05.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.524+0000 7f44f6ffd700 1 -- 172.21.15.156:0/2991060715 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f44f804b5e0 con 0x7f45080d2320 2023-11-28T13:09:05.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.717+0000 7f450cfff700 1 -- 172.21.15.156:0/2991060715 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4508157b80 con 0x7f45080d2320 2023-11-28T13:09:05.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.718+0000 7f44f6ffd700 1 -- 172.21.15.156:0/2991060715 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4508157b80 con 0x7f45080d2320 2023-11-28T13:09:05.718 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:09:05.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.720+0000 7f450cfff700 1 -- 172.21.15.156:0/2991060715 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f44e8068320 msgr2=0x7f44e806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:05.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.720+0000 7f450cfff700 1 --2- 172.21.15.156:0/2991060715 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f44e8068320 0x7f44e806a7d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f45080c6b80 tx=0x7f44fc008040 comp rx=0 tx=0).stop 2023-11-28T13:09:05.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.720+0000 7f450cfff700 1 -- 172.21.15.156:0/2991060715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45080d2320 msgr2=0x7f45081f1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:05.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.720+0000 7f450cfff700 1 --2- 172.21.15.156:0/2991060715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45080d2320 0x7f45081f1ba0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f44f800c500 tx=0x7f44f800c810 comp rx=0 tx=0).stop 2023-11-28T13:09:05.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.720+0000 7f450cfff700 1 -- 172.21.15.156:0/2991060715 shutdown_connections 2023-11-28T13:09:05.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.720+0000 7f450cfff700 1 --2- 172.21.15.156:0/2991060715 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f44e8068320 0x7f44e806a7d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.720+0000 7f450cfff700 1 --2- 172.21.15.156:0/2991060715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4508161340 0x7f4508155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.720+0000 7f450cfff700 1 --2- 172.21.15.156:0/2991060715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45080d2320 0x7f45081f1ba0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.720+0000 7f450cfff700 1 --2- 172.21.15.156:0/2991060715 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f450815c9b0 0x7f4508155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.720+0000 7f450cfff700 1 -- 172.21.15.156:0/2991060715 >> 172.21.15.156:0/2991060715 conn(0x7f4508072600 msgr2=0x7f45080b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:05.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.720+0000 7f450cfff700 1 -- 172.21.15.156:0/2991060715 shutdown_connections 2023-11-28T13:09:05.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.720+0000 7f450cfff700 1 -- 172.21.15.156:0/2991060715 wait complete. 2023-11-28T13:09:05.739 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T13:09:05.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.936+0000 7f484b7ff700 1 Processor -- start 2023-11-28T13:09:05.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.936+0000 7f484b7ff700 1 -- start start 2023-11-28T13:09:05.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.937+0000 7f484b7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f48440d01b0 0x7f48440d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.937+0000 7f484b7ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f48440b16d0 0x7f48440d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.937+0000 7f484b7ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48440b0230 con 0x7f48440b20b0 2023-11-28T13:09:05.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.937+0000 7f484b7ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484415cec0 con 0x7f48440b16d0 2023-11-28T13:09:05.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.937+0000 7f484b7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48440b4570 con 0x7f48440d01b0 2023-11-28T13:09:05.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.937+0000 7f4848d9a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f48440b20b0 0x7f48440b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:43428/0 (socket says 172.21.15.156:43428) 2023-11-28T13:09:05.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.937+0000 7f4848d9a700 1 -- 172.21.15.156:0/479790102 learned_addr learned my addr 172.21.15.156:0/479790102 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:05.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.938+0000 7f484959b700 1 --2- 172.21.15.156:0/479790102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f48440d01b0 0x7f48440d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.938+0000 7f4849d9c700 1 --2- 172.21.15.156:0/479790102 >> v2:172.21.15.186:3300/0 conn(0x7f48440b16d0 0x7f48440d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.938+0000 7f483bfff700 1 -- 172.21.15.156:0/479790102 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2711540999 0 0) 0x7f48440b0230 con 0x7f48440b20b0 2023-11-28T13:09:05.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.938+0000 7f483bfff700 1 -- 172.21.15.156:0/479790102 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48300035b0 con 0x7f48440b20b0 2023-11-28T13:09:05.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.938+0000 7f483bfff700 1 -- 172.21.15.156:0/479790102 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1201115569 0 0) 0x7f48300035b0 con 0x7f48440b20b0 2023-11-28T13:09:05.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.938+0000 7f483bfff700 1 -- 172.21.15.156:0/479790102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f48440d01b0 msgr2=0x7f48440d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:05.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.938+0000 7f483bfff700 1 --2- 172.21.15.156:0/479790102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f48440d01b0 0x7f48440d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.938+0000 7f483bfff700 1 -- 172.21.15.156:0/479790102 >> v2:172.21.15.186:3300/0 conn(0x7f48440b16d0 msgr2=0x7f48440d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:05.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.938+0000 7f483bfff700 1 --2- 172.21.15.156:0/479790102 >> v2:172.21.15.186:3300/0 conn(0x7f48440b16d0 0x7f48440d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.938+0000 7f483bfff700 1 -- 172.21.15.156:0/479790102 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48440b4850 con 0x7f48440b20b0 2023-11-28T13:09:05.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.938+0000 7f483bfff700 1 -- 172.21.15.156:0/479790102 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f4834001670 con 0x7f48440b20b0 2023-11-28T13:09:05.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.938+0000 7f484959b700 1 --2- 172.21.15.156:0/479790102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f48440d01b0 0x7f48440d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:09:05.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.938+0000 7f483bfff700 1 -- 172.21.15.156:0/479790102 >> v1:172.21.15.156:6789/0 conn(0x7f48440b20b0 legacy=0x7f48440b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:05.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.938+0000 7f483bfff700 1 --2- 172.21.15.156:0/479790102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4830003a00 0x7f4830003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.938+0000 7f483bfff700 1 --2- 172.21.15.156:0/479790102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f48440d01b0 0x7f4830004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.938+0000 7f483bfff700 1 --2- 172.21.15.156:0/479790102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4830004d90 0x7f4830005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.939+0000 7f483bfff700 1 -- 172.21.15.156:0/479790102 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484415cec0 con 0x7f4830003a00 2023-11-28T13:09:05.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.939+0000 7f483bfff700 1 -- 172.21.15.156:0/479790102 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48440b4570 con 0x7f4830004d90 2023-11-28T13:09:05.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.939+0000 7f483bfff700 1 -- 172.21.15.156:0/479790102 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483000ba10 con 0x7f48440d01b0 2023-11-28T13:09:05.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.939+0000 7f4849d9c700 1 --2- 172.21.15.156:0/479790102 >> v2:172.21.15.186:3300/0 conn(0x7f48440b16d0 0x7f48440d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:09:05.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.939+0000 7f484959b700 1 --2- 172.21.15.156:0/479790102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4830003a00 0x7f4830003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.939+0000 7f4848d9a700 1 --2- 172.21.15.156:0/479790102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f48440d01b0 0x7f4830004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.939+0000 7f4849d9c700 1 --2- 172.21.15.156:0/479790102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4830004d90 0x7f4830005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.939+0000 7f484959b700 1 -- 172.21.15.156:0/479790102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f48440d01b0 msgr2=0x7f4830004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:05.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.939+0000 7f484959b700 1 --2- 172.21.15.156:0/479790102 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f48440d01b0 0x7f4830004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.939+0000 7f484959b700 1 -- 172.21.15.156:0/479790102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4830004d90 msgr2=0x7f4830005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:05.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.939+0000 7f484959b700 1 --2- 172.21.15.156:0/479790102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4830004d90 0x7f4830005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.939+0000 7f484959b700 1 -- 172.21.15.156:0/479790102 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48400094a0 con 0x7f4830003a00 2023-11-28T13:09:05.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.939+0000 7f484959b700 1 --2- 172.21.15.156:0/479790102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4830003a00 0x7f4830003e10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4840009d30 tx=0x7f4830005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6405efb9f4fc21 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:05.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.939+0000 7f483bfff700 1 -- 172.21.15.156:0/479790102 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4840009e80 con 0x7f4830003a00 2023-11-28T13:09:05.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.940+0000 7f483bfff700 1 -- 172.21.15.156:0/479790102 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484000e940 con 0x7f4830003a00 2023-11-28T13:09:05.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.940+0000 7f483bfff700 1 -- 172.21.15.156:0/479790102 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484000ecb0 con 0x7f4830003a00 2023-11-28T13:09:05.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.940+0000 7f484b7ff700 1 -- 172.21.15.156:0/479790102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4830003a00 msgr2=0x7f4830003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:05.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.940+0000 7f484b7ff700 1 --2- 172.21.15.156:0/479790102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4830003a00 0x7f4830003e10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4840009d30 tx=0x7f4830005880 comp rx=0 tx=0).stop 2023-11-28T13:09:05.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.940+0000 7f4849d9c700 1 -- 172.21.15.156:0/479790102 reap_dead start 2023-11-28T13:09:05.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.940+0000 7f484b7ff700 1 -- 172.21.15.156:0/479790102 shutdown_connections 2023-11-28T13:09:05.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.940+0000 7f484b7ff700 1 -- 172.21.15.156:0/479790102 >> 172.21.15.156:0/479790102 conn(0x7f484409c3f0 msgr2=0x7f48440d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:05.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.940+0000 7f484b7ff700 1 -- 172.21.15.156:0/479790102 shutdown_connections 2023-11-28T13:09:05.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.940+0000 7f484b7ff700 1 -- 172.21.15.156:0/479790102 wait complete. 2023-11-28T13:09:05.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.943+0000 7f484b7ff700 1 Processor -- start 2023-11-28T13:09:05.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.943+0000 7f484b7ff700 1 -- start start 2023-11-28T13:09:05.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.944+0000 7f484b7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4830003a00 0x7f48440b7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.944+0000 7f484b7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48441f35f0 0x7f48440b8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.944+0000 7f484b7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48441f4b00 0x7f48441f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.944+0000 7f484b7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48440b8c90 con 0x7f48441f4b00 2023-11-28T13:09:05.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.944+0000 7f484b7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48441e5310 con 0x7f48441f35f0 2023-11-28T13:09:05.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.944+0000 7f484b7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48441e55f0 con 0x7f4830003a00 2023-11-28T13:09:05.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.944+0000 7f4849d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48441f4b00 0x7f48441f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.944+0000 7f4849d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48441f4b00 0x7f48441f4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49532/0 (socket says 172.21.15.156:49532) 2023-11-28T13:09:05.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.944+0000 7f4849d9c700 1 -- 172.21.15.156:0/2112349666 learned_addr learned my addr 172.21.15.156:0/2112349666 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:05.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.944+0000 7f4848d9a700 1 --2- 172.21.15.156:0/2112349666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48441f35f0 0x7f48440b8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.944+0000 7f484959b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4830003a00 0x7f48440b7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.945+0000 7f4849d9c700 1 -- 172.21.15.156:0/2112349666 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4830003a00 msgr2=0x7f48440b7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:05.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.945+0000 7f4849d9c700 1 --2- 172.21.15.156:0/2112349666 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4830003a00 0x7f48440b7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.945+0000 7f4849d9c700 1 -- 172.21.15.156:0/2112349666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48441f35f0 msgr2=0x7f48440b8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:05.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.945+0000 7f4849d9c700 1 --2- 172.21.15.156:0/2112349666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48441f35f0 0x7f48440b8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:05.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.945+0000 7f4849d9c700 1 -- 172.21.15.156:0/2112349666 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48440afec0 con 0x7f48441f4b00 2023-11-28T13:09:05.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.945+0000 7f4849d9c700 1 --2- 172.21.15.156:0/2112349666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48441f4b00 0x7f48441f4f70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f48440b5bb0 tx=0x7f483c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:05.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.945+0000 7f4839ffb700 1 -- 172.21.15.156:0/2112349666 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c00e070 con 0x7f48441f4b00 2023-11-28T13:09:05.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.945+0000 7f4839ffb700 1 -- 172.21.15.156:0/2112349666 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c003c30 con 0x7f48441f4b00 2023-11-28T13:09:05.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.945+0000 7f4839ffb700 1 -- 172.21.15.156:0/2112349666 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c003dc0 con 0x7f48441f4b00 2023-11-28T13:09:05.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.945+0000 7f484b7ff700 1 -- 172.21.15.156:0/2112349666 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48440afbe0 con 0x7f48441f4b00 2023-11-28T13:09:05.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.945+0000 7f484b7ff700 1 -- 172.21.15.156:0/2112349666 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48440b2600 con 0x7f48441f4b00 2023-11-28T13:09:05.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.946+0000 7f484b7ff700 1 -- 172.21.15.156:0/2112349666 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4844068690 con 0x7f48441f4b00 2023-11-28T13:09:05.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.947+0000 7f4839ffb700 1 -- 172.21.15.156:0/2112349666 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f483c0119b0 con 0x7f48441f4b00 2023-11-28T13:09:05.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.948+0000 7f4839ffb700 1 --2- 172.21.15.156:0/2112349666 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f48280682f0 0x7f482806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:05.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.948+0000 7f4839ffb700 1 -- 172.21.15.156:0/2112349666 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f483c086130 con 0x7f48441f4b00 2023-11-28T13:09:05.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.949+0000 7f484959b700 1 --2- 172.21.15.156:0/2112349666 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f48280682f0 0x7f482806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:05.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.949+0000 7f484959b700 1 --2- 172.21.15.156:0/2112349666 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f48280682f0 0x7f482806a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f48400090b0 tx=0x7f484001f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:05.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:05.952+0000 7f4839ffb700 1 -- 172.21.15.156:0/2112349666 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f483c052bf0 con 0x7f48441f4b00 2023-11-28T13:09:06.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.145+0000 7f484b7ff700 1 -- 172.21.15.156:0/2112349666 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f48440cd360 con 0x7f48441f4b00 2023-11-28T13:09:06.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.146+0000 7f4839ffb700 1 -- 172.21.15.156:0/2112349666 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+126 (secure 0 0 0) 0x7f483c056210 con 0x7f48441f4b00 2023-11-28T13:09:06.146 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:06.146 INFO:teuthology.orchestra.run.smithi156.stdout:[ 2023-11-28T13:09:06.147 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:06.147 INFO:teuthology.orchestra.run.smithi156.stdout: "poolnum": 1, 2023-11-28T13:09:06.147 INFO:teuthology.orchestra.run.smithi156.stdout: "poolname": "rbd" 2023-11-28T13:09:06.147 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:06.147 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:06.147 INFO:teuthology.orchestra.run.smithi156.stdout: "poolnum": 2, 2023-11-28T13:09:06.147 INFO:teuthology.orchestra.run.smithi156.stdout: "poolname": ".mgr" 2023-11-28T13:09:06.147 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:06.147 INFO:teuthology.orchestra.run.smithi156.stdout:] 2023-11-28T13:09:06.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.148+0000 7f484b7ff700 1 -- 172.21.15.156:0/2112349666 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f48280682f0 msgr2=0x7f482806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:06.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.148+0000 7f484b7ff700 1 --2- 172.21.15.156:0/2112349666 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f48280682f0 0x7f482806a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f48400090b0 tx=0x7f484001f040 comp rx=0 tx=0).stop 2023-11-28T13:09:06.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.148+0000 7f484b7ff700 1 -- 172.21.15.156:0/2112349666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48441f4b00 msgr2=0x7f48441f4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:06.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.148+0000 7f484b7ff700 1 --2- 172.21.15.156:0/2112349666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48441f4b00 0x7f48441f4f70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f48440b5bb0 tx=0x7f483c009f90 comp rx=0 tx=0).stop 2023-11-28T13:09:06.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.148+0000 7f484b7ff700 1 -- 172.21.15.156:0/2112349666 shutdown_connections 2023-11-28T13:09:06.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.148+0000 7f484b7ff700 1 --2- 172.21.15.156:0/2112349666 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f48280682f0 0x7f482806a7a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:06.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.148+0000 7f484b7ff700 1 --2- 172.21.15.156:0/2112349666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48441f4b00 0x7f48441f4f70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:06.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.148+0000 7f484b7ff700 1 --2- 172.21.15.156:0/2112349666 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4830003a00 0x7f48440b7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:06.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.148+0000 7f484b7ff700 1 --2- 172.21.15.156:0/2112349666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48441f35f0 0x7f48440b8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:06.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.148+0000 7f484b7ff700 1 -- 172.21.15.156:0/2112349666 >> 172.21.15.156:0/2112349666 conn(0x7f484409c3f0 msgr2=0x7f48440b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:06.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.148+0000 7f484b7ff700 1 -- 172.21.15.156:0/2112349666 shutdown_connections 2023-11-28T13:09:06.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.148+0000 7f484b7ff700 1 -- 172.21.15.156:0/2112349666 wait complete. 2023-11-28T13:09:06.167 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T13:09:06.167 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T13:09:06.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.367+0000 7fe52e5ff700 1 Processor -- start 2023-11-28T13:09:06.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.367+0000 7fe52e5ff700 1 -- start start 2023-11-28T13:09:06.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.368+0000 7fe52e5ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe5280b29d0 0x7fe5280b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:06.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.368+0000 7fe52e5ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5280afbe0 con 0x7fe5280d1350 2023-11-28T13:09:06.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.368+0000 7fe52e5ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5280b2750 con 0x7fe52815ca60 2023-11-28T13:09:06.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.368+0000 7fe52e5ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe52815ce30 con 0x7fe5280b29d0 2023-11-28T13:09:06.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.368+0000 7fe5277fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fe5280d1350 0x7fe5280b0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:43434/0 (socket says 172.21.15.156:43434) 2023-11-28T13:09:06.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.368+0000 7fe52cb9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe5280b29d0 0x7fe5280b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:06.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.368+0000 7fe5277fe700 1 -- 172.21.15.156:0/751641351 learned_addr learned my addr 172.21.15.156:0/751641351 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:06.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.369+0000 7fe526ffd700 1 -- 172.21.15.156:0/751641351 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265099143 0 0) 0x7fe5280afbe0 con 0x7fe5280d1350 2023-11-28T13:09:06.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.369+0000 7fe526ffd700 1 -- 172.21.15.156:0/751641351 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5100035b0 con 0x7fe5280d1350 2023-11-28T13:09:06.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.369+0000 7fe52cb9c700 1 -- 172.21.15.156:0/751641351 >> v1:172.21.15.186:6789/0 conn(0x7fe52815ca60 legacy=0x7fe5280d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:06.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.369+0000 7fe526ffd700 1 -- 172.21.15.156:0/751641351 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237720219 0 0) 0x7fe5280b2750 con 0x7fe52815ca60 2023-11-28T13:09:06.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.369+0000 7fe52cb9c700 1 -- 172.21.15.156:0/751641351 >> v1:172.21.15.156:6789/0 conn(0x7fe5280d1350 legacy=0x7fe5280b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:06.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.369+0000 7fe52cb9c700 1 -- 172.21.15.156:0/751641351 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5280b3f20 con 0x7fe5280b29d0 2023-11-28T13:09:06.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.369+0000 7fe52cb9c700 1 --2- 172.21.15.156:0/751641351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5280b29d0 0x7fe5280b38b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe520009d90 tx=0x7fe52000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270c14ae7813f14b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:06.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.369+0000 7fe526ffd700 1 -- 172.21.15.156:0/751641351 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52000e070 con 0x7fe5280b29d0 2023-11-28T13:09:06.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.370+0000 7fe526ffd700 1 -- 172.21.15.156:0/751641351 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52000bb20 con 0x7fe5280b29d0 2023-11-28T13:09:06.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.370+0000 7fe526ffd700 1 -- 172.21.15.156:0/751641351 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52000bed0 con 0x7fe5280b29d0 2023-11-28T13:09:06.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.370+0000 7fe52e5ff700 1 -- 172.21.15.156:0/751641351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5280b29d0 msgr2=0x7fe5280b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:06.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.370+0000 7fe52e5ff700 1 --2- 172.21.15.156:0/751641351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5280b29d0 0x7fe5280b38b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe520009d90 tx=0x7fe52000b330 comp rx=0 tx=0).stop 2023-11-28T13:09:06.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.370+0000 7fe52e5ff700 1 -- 172.21.15.156:0/751641351 shutdown_connections 2023-11-28T13:09:06.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.370+0000 7fe52e5ff700 1 --2- 172.21.15.156:0/751641351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5280b29d0 0x7fe5280b38b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:06.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.370+0000 7fe52e5ff700 1 -- 172.21.15.156:0/751641351 >> 172.21.15.156:0/751641351 conn(0x7fe52809c3f0 msgr2=0x7fe5280d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:06.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.370+0000 7fe52e5ff700 1 -- 172.21.15.156:0/751641351 shutdown_connections 2023-11-28T13:09:06.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.370+0000 7fe52e5ff700 1 -- 172.21.15.156:0/751641351 wait complete. 2023-11-28T13:09:06.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.372+0000 7fe52e5ff700 1 Processor -- start 2023-11-28T13:09:06.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.372+0000 7fe52e5ff700 1 -- start start 2023-11-28T13:09:06.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.373+0000 7fe52e5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5280b29d0 0x7fe52814f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:06.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.373+0000 7fe52e5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5280d1350 0x7fe5281591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:06.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.373+0000 7fe52e5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe52815ca60 0x7fe5280c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:06.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.373+0000 7fe52e5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52815dff0 con 0x7fe5280d1350 2023-11-28T13:09:06.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.373+0000 7fe52e5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5280c6cb0 con 0x7fe5280b29d0 2023-11-28T13:09:06.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.373+0000 7fe52e5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5280c6f90 con 0x7fe52815ca60 2023-11-28T13:09:06.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.373+0000 7fe5277fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5280d1350 0x7fe5281591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:06.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.373+0000 7fe5277fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5280d1350 0x7fe5281591a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49564/0 (socket says 172.21.15.156:49564) 2023-11-28T13:09:06.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.373+0000 7fe5277fe700 1 -- 172.21.15.156:0/1403401561 learned_addr learned my addr 172.21.15.156:0/1403401561 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:06.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.374+0000 7fe527fff700 1 --2- 172.21.15.156:0/1403401561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5280b29d0 0x7fe52814f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:06.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.374+0000 7fe52cb9c700 1 --2- 172.21.15.156:0/1403401561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe52815ca60 0x7fe5280c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:06.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.374+0000 7fe5277fe700 1 -- 172.21.15.156:0/1403401561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe52815ca60 msgr2=0x7fe5280c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:06.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.374+0000 7fe5277fe700 1 --2- 172.21.15.156:0/1403401561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe52815ca60 0x7fe5280c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:06.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.374+0000 7fe5277fe700 1 -- 172.21.15.156:0/1403401561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5280b29d0 msgr2=0x7fe52814f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:06.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.374+0000 7fe5277fe700 1 --2- 172.21.15.156:0/1403401561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5280b29d0 0x7fe52814f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:06.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.374+0000 7fe5277fe700 1 -- 172.21.15.156:0/1403401561 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe520009a70 con 0x7fe5280d1350 2023-11-28T13:09:06.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.374+0000 7fe527fff700 1 --2- 172.21.15.156:0/1403401561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5280b29d0 0x7fe52814f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:06.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.374+0000 7fe5277fe700 1 --2- 172.21.15.156:0/1403401561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5280d1350 0x7fe5281591a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00c4d0 tx=0x7fe51c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:06.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.374+0000 7fe524ff9700 1 -- 172.21.15.156:0/1403401561 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c005d70 con 0x7fe5280d1350 2023-11-28T13:09:06.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.374+0000 7fe524ff9700 1 -- 172.21.15.156:0/1403401561 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51c0070d0 con 0x7fe5280d1350 2023-11-28T13:09:06.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.374+0000 7fe524ff9700 1 -- 172.21.15.156:0/1403401561 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c007440 con 0x7fe5280d1350 2023-11-28T13:09:06.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.374+0000 7fe52e5ff700 1 -- 172.21.15.156:0/1403401561 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5280b44e0 con 0x7fe5280d1350 2023-11-28T13:09:06.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.374+0000 7fe52e5ff700 1 -- 172.21.15.156:0/1403401561 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5280b4200 con 0x7fe5280d1350 2023-11-28T13:09:06.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.375+0000 7fe52e5ff700 1 -- 172.21.15.156:0/1403401561 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5280b47c0 con 0x7fe5280d1350 2023-11-28T13:09:06.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.376+0000 7fe524ff9700 1 -- 172.21.15.156:0/1403401561 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe51c019030 con 0x7fe5280d1350 2023-11-28T13:09:06.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.377+0000 7fe524ff9700 1 --2- 172.21.15.156:0/1403401561 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe5140682f0 0x7fe51406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:06.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.377+0000 7fe524ff9700 1 -- 172.21.15.156:0/1403401561 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe51c084be0 con 0x7fe5280d1350 2023-11-28T13:09:06.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.378+0000 7fe527fff700 1 --2- 172.21.15.156:0/1403401561 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe5140682f0 0x7fe51406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:06.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.378+0000 7fe527fff700 1 --2- 172.21.15.156:0/1403401561 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe5140682f0 0x7fe51406a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe518000fb0 tx=0x7fe518009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:06.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.381+0000 7fe524ff9700 1 -- 172.21.15.156:0/1403401561 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe51c04b5e0 con 0x7fe5280d1350 2023-11-28T13:09:06.578 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.577+0000 7fe52e5ff700 1 -- 172.21.15.156:0/1403401561 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe5280672a0 con 0x7fe5280d1350 2023-11-28T13:09:06.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.672+0000 7fe524ff9700 1 -- 172.21.15.156:0/1403401561 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe51c017020 con 0x7fe5280d1350 2023-11-28T13:09:06.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.677+0000 7fe52e5ff700 1 -- 172.21.15.156:0/1403401561 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe5140682f0 msgr2=0x7fe51406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:06.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.677+0000 7fe52e5ff700 1 --2- 172.21.15.156:0/1403401561 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe5140682f0 0x7fe51406a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe518000fb0 tx=0x7fe518009040 comp rx=0 tx=0).stop 2023-11-28T13:09:06.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.677+0000 7fe52e5ff700 1 -- 172.21.15.156:0/1403401561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5280d1350 msgr2=0x7fe5281591a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:06.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.677+0000 7fe52e5ff700 1 --2- 172.21.15.156:0/1403401561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5280d1350 0x7fe5281591a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00c4d0 tx=0x7fe51c00c890 comp rx=0 tx=0).stop 2023-11-28T13:09:06.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.677+0000 7fe52e5ff700 1 -- 172.21.15.156:0/1403401561 shutdown_connections 2023-11-28T13:09:06.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.677+0000 7fe52e5ff700 1 --2- 172.21.15.156:0/1403401561 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe5140682f0 0x7fe51406a7a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:06.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.677+0000 7fe52e5ff700 1 --2- 172.21.15.156:0/1403401561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe52815ca60 0x7fe5280c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:06.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.677+0000 7fe52e5ff700 1 --2- 172.21.15.156:0/1403401561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5280b29d0 0x7fe52814f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:06.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.677+0000 7fe52e5ff700 1 --2- 172.21.15.156:0/1403401561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5280d1350 0x7fe5281591a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:06.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.677+0000 7fe52e5ff700 1 -- 172.21.15.156:0/1403401561 >> 172.21.15.156:0/1403401561 conn(0x7fe52809c3f0 msgr2=0x7fe5280b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:06.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.677+0000 7fe52e5ff700 1 -- 172.21.15.156:0/1403401561 shutdown_connections 2023-11-28T13:09:06.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.677+0000 7fe52e5ff700 1 -- 172.21.15.156:0/1403401561 wait complete. 2023-11-28T13:09:06.681 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_metadata' created 2023-11-28T13:09:06.699 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T13:09:06.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.902+0000 7fd7a8dff700 1 Processor -- start 2023-11-28T13:09:06.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.902+0000 7fd7a8dff700 1 -- start start 2023-11-28T13:09:06.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.903+0000 7fd7a8dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a4161340 0x7fd7a40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:06.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.904+0000 7fd7a8dff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a40d1950 con 0x7fd7a415c9b0 2023-11-28T13:09:06.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.904+0000 7fd7a8dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a40b5520 con 0x7fd7a40d2320 2023-11-28T13:09:06.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.904+0000 7fd7a8dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a40b5800 con 0x7fd7a4161340 2023-11-28T13:09:06.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.904+0000 7fd7a259c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd7a415c9b0 0x7fd7a40b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:43442/0 (socket says 172.21.15.156:43442) 2023-11-28T13:09:06.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.904+0000 7fd7a259c700 1 -- 172.21.15.156:0/1713420066 learned_addr learned my addr 172.21.15.156:0/1713420066 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:06.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.904+0000 7fd79bfff700 1 --2- 172.21.15.156:0/1713420066 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a4161340 0x7fd7a40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:06.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.904+0000 7fd7a1d9b700 1 -- 172.21.15.156:0/1713420066 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861214210 0 0) 0x7fd7a40d1950 con 0x7fd7a415c9b0 2023-11-28T13:09:06.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.904+0000 7fd7a1d9b700 1 -- 172.21.15.156:0/1713420066 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7900035b0 con 0x7fd7a415c9b0 2023-11-28T13:09:06.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.905+0000 7fd7a1d9b700 1 -- 172.21.15.156:0/1713420066 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679368515 0 0) 0x7fd7a40b5520 con 0x7fd7a40d2320 2023-11-28T13:09:06.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.905+0000 7fd7a1d9b700 1 -- 172.21.15.156:0/1713420066 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a40d1950 con 0x7fd7a40d2320 2023-11-28T13:09:06.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.905+0000 7fd7a1d9b700 1 -- 172.21.15.156:0/1713420066 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1548167480 0 0) 0x7fd7900035b0 con 0x7fd7a415c9b0 2023-11-28T13:09:06.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.905+0000 7fd79bfff700 1 -- 172.21.15.156:0/1713420066 >> v1:172.21.15.186:6789/0 conn(0x7fd7a40d2320 legacy=0x7fd7a40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:06.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.905+0000 7fd79bfff700 1 -- 172.21.15.156:0/1713420066 >> v1:172.21.15.156:6789/0 conn(0x7fd7a415c9b0 legacy=0x7fd7a40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:06.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.905+0000 7fd79bfff700 1 -- 172.21.15.156:0/1713420066 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a40d1300 con 0x7fd7a4161340 2023-11-28T13:09:06.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.905+0000 7fd79bfff700 1 --2- 172.21.15.156:0/1713420066 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a4161340 0x7fd7a40d1de0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009870 tx=0x7fd78c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d13b5415d730f778 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:06.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.905+0000 7fd7a1d9b700 1 -- 172.21.15.156:0/1713420066 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd794001670 con 0x7fd7a4161340 2023-11-28T13:09:06.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.905+0000 7fd7a1d9b700 1 -- 172.21.15.156:0/1713420066 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c004030 con 0x7fd7a4161340 2023-11-28T13:09:06.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.905+0000 7fd7a1d9b700 1 -- 172.21.15.156:0/1713420066 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0043c0 con 0x7fd7a4161340 2023-11-28T13:09:06.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.906+0000 7fd7a8dff700 1 -- 172.21.15.156:0/1713420066 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a4161340 msgr2=0x7fd7a40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:06.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.906+0000 7fd7a8dff700 1 --2- 172.21.15.156:0/1713420066 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a4161340 0x7fd7a40d1de0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009870 tx=0x7fd78c009b80 comp rx=0 tx=0).stop 2023-11-28T13:09:06.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.906+0000 7fd7a8dff700 1 -- 172.21.15.156:0/1713420066 shutdown_connections 2023-11-28T13:09:06.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.906+0000 7fd7a8dff700 1 --2- 172.21.15.156:0/1713420066 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a4161340 0x7fd7a40d1de0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:06.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.906+0000 7fd7a8dff700 1 -- 172.21.15.156:0/1713420066 >> 172.21.15.156:0/1713420066 conn(0x7fd7a4072600 msgr2=0x7fd7a40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:06.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.906+0000 7fd7a8dff700 1 -- 172.21.15.156:0/1713420066 shutdown_connections 2023-11-28T13:09:06.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.906+0000 7fd7a8dff700 1 -- 172.21.15.156:0/1713420066 wait complete. 2023-11-28T13:09:06.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.908+0000 7fd7a8dff700 1 Processor -- start 2023-11-28T13:09:06.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.908+0000 7fd7a8dff700 1 -- start start 2023-11-28T13:09:06.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd7a8dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a40d2320 0x7fd7a4156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:06.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd7a8dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a415c9b0 0x7fd7a41f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd7a8dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a4161340 0x7fd7a415f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd7a8dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a4157a70 con 0x7fd7a415c9b0 2023-11-28T13:09:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd7a8dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a41f2d10 con 0x7fd7a4161340 2023-11-28T13:09:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd7a8dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a40c6e40 con 0x7fd7a40d2320 2023-11-28T13:09:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd79bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a415c9b0 0x7fd7a41f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd79bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a415c9b0 0x7fd7a41f27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49570/0 (socket says 172.21.15.156:49570) 2023-11-28T13:09:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd79bfff700 1 -- 172.21.15.156:0/3203472586 learned_addr learned my addr 172.21.15.156:0/3203472586 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd7a2d9d700 1 --2- 172.21.15.156:0/3203472586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a4161340 0x7fd7a415f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd7a259c700 1 --2- 172.21.15.156:0/3203472586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a40d2320 0x7fd7a4156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd79bfff700 1 -- 172.21.15.156:0/3203472586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a40d2320 msgr2=0x7fd7a4156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd79bfff700 1 --2- 172.21.15.156:0/3203472586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a40d2320 0x7fd7a4156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd79bfff700 1 -- 172.21.15.156:0/3203472586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a4161340 msgr2=0x7fd7a415f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd79bfff700 1 --2- 172.21.15.156:0/3203472586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a4161340 0x7fd7a415f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd79bfff700 1 -- 172.21.15.156:0/3203472586 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c0094a0 con 0x7fd7a415c9b0 2023-11-28T13:09:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd7a2d9d700 1 --2- 172.21.15.156:0/3203472586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a4161340 0x7fd7a415f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.909+0000 7fd7a259c700 1 --2- 172.21.15.156:0/3203472586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a40d2320 0x7fd7a4156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.910+0000 7fd79bfff700 1 --2- 172.21.15.156:0/3203472586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a415c9b0 0x7fd7a41f27d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd78c000c00 tx=0x7fd78c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.910+0000 7fd79affd700 1 -- 172.21.15.156:0/3203472586 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c011630 con 0x7fd7a415c9b0 2023-11-28T13:09:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.910+0000 7fd79affd700 1 -- 172.21.15.156:0/3203472586 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c01f070 con 0x7fd7a415c9b0 2023-11-28T13:09:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.910+0000 7fd79affd700 1 -- 172.21.15.156:0/3203472586 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0119f0 con 0x7fd7a415c9b0 2023-11-28T13:09:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.910+0000 7fd7a8dff700 1 -- 172.21.15.156:0/3203472586 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a415cd80 con 0x7fd7a415c9b0 2023-11-28T13:09:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.910+0000 7fd7a8dff700 1 -- 172.21.15.156:0/3203472586 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a40d15e0 con 0x7fd7a415c9b0 2023-11-28T13:09:06.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.911+0000 7fd7a8dff700 1 -- 172.21.15.156:0/3203472586 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a415cf10 con 0x7fd7a415c9b0 2023-11-28T13:09:06.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.912+0000 7fd79affd700 1 -- 172.21.15.156:0/3203472586 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd78c017020 con 0x7fd7a415c9b0 2023-11-28T13:09:06.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.913+0000 7fd79affd700 1 --2- 172.21.15.156:0/3203472586 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd7840682f0 0x7fd78406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:06.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.913+0000 7fd79affd700 1 -- 172.21.15.156:0/3203472586 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fd78c021070 con 0x7fd7a415c9b0 2023-11-28T13:09:06.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.913+0000 7fd7a259c700 1 --2- 172.21.15.156:0/3203472586 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd7840682f0 0x7fd78406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:06.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.914+0000 7fd7a259c700 1 --2- 172.21.15.156:0/3203472586 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd7840682f0 0x7fd78406a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd7940011e0 tx=0x7fd794009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:06.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:06.918+0000 7fd79affd700 1 -- 172.21.15.156:0/3203472586 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd78c053e40 con 0x7fd7a415c9b0 2023-11-28T13:09:07.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.111+0000 7fd7a8dff700 1 -- 172.21.15.156:0/3203472586 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd7a40745c0 con 0x7fd7a415c9b0 2023-11-28T13:09:07.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.676+0000 7fd79affd700 1 -- 172.21.15.156:0/3203472586 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd78c085e60 con 0x7fd7a415c9b0 2023-11-28T13:09:07.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.686+0000 7fd7a8dff700 1 -- 172.21.15.156:0/3203472586 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd7840682f0 msgr2=0x7fd78406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:07.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.686+0000 7fd7a8dff700 1 --2- 172.21.15.156:0/3203472586 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd7840682f0 0x7fd78406a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd7940011e0 tx=0x7fd794009040 comp rx=0 tx=0).stop 2023-11-28T13:09:07.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.686+0000 7fd7a8dff700 1 -- 172.21.15.156:0/3203472586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a415c9b0 msgr2=0x7fd7a41f27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:07.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.686+0000 7fd7a8dff700 1 --2- 172.21.15.156:0/3203472586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a415c9b0 0x7fd7a41f27d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd78c000c00 tx=0x7fd78c004750 comp rx=0 tx=0).stop 2023-11-28T13:09:07.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.689+0000 7fd7a8dff700 1 -- 172.21.15.156:0/3203472586 shutdown_connections 2023-11-28T13:09:07.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.689+0000 7fd7a8dff700 1 --2- 172.21.15.156:0/3203472586 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd7840682f0 0x7fd78406a7a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:07.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.689+0000 7fd7a8dff700 1 --2- 172.21.15.156:0/3203472586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a4161340 0x7fd7a415f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:07.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.689+0000 7fd7a8dff700 1 --2- 172.21.15.156:0/3203472586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7a40d2320 0x7fd7a4156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:07.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.689+0000 7fd7a8dff700 1 --2- 172.21.15.156:0/3203472586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7a415c9b0 0x7fd7a41f27d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:07.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.689+0000 7fd7a8dff700 1 -- 172.21.15.156:0/3203472586 >> 172.21.15.156:0/3203472586 conn(0x7fd7a4072600 msgr2=0x7fd7a4160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:07.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.690+0000 7fd7a8dff700 1 -- 172.21.15.156:0/3203472586 shutdown_connections 2023-11-28T13:09:07.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.690+0000 7fd7a8dff700 1 -- 172.21.15.156:0/3203472586 wait complete. 2023-11-28T13:09:07.694 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_data' created 2023-11-28T13:09:07.715 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T13:09:07.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.916+0000 7fe1a03ff700 1 Processor -- start 2023-11-28T13:09:07.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.916+0000 7fe1a03ff700 1 -- start start 2023-11-28T13:09:07.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.917+0000 7fe1a03ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe198161340 0x7fe1980d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:07.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.917+0000 7fe1a03ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1980d1950 con 0x7fe19815c9b0 2023-11-28T13:09:07.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.917+0000 7fe1a03ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1980b5520 con 0x7fe1980d2320 2023-11-28T13:09:07.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.917+0000 7fe1a03ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1980b5800 con 0x7fe198161340 2023-11-28T13:09:07.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.917+0000 7fe19d99a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe198161340 0x7fe1980d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:07.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.917+0000 7fe19e19b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fe19815c9b0 0x7fe1980b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:43450/0 (socket says 172.21.15.156:43450) 2023-11-28T13:09:07.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.917+0000 7fe19e19b700 1 -- 172.21.15.156:0/412084088 learned_addr learned my addr 172.21.15.156:0/412084088 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:07.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.918+0000 7fe19d99a700 1 -- 172.21.15.156:0/412084088 >> v1:172.21.15.186:6789/0 conn(0x7fe1980d2320 legacy=0x7fe1980cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:07.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.918+0000 7fe19d99a700 1 -- 172.21.15.156:0/412084088 >> v1:172.21.15.156:6789/0 conn(0x7fe19815c9b0 legacy=0x7fe1980b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:07.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.918+0000 7fe19d99a700 1 -- 172.21.15.156:0/412084088 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1980d1300 con 0x7fe198161340 2023-11-28T13:09:07.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.918+0000 7fe19d99a700 1 --2- 172.21.15.156:0/412084088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe198161340 0x7fe1980d1de0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe188009840 tx=0x7fe188009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7b0cb3a714eaac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:07.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.918+0000 7fe19d199700 1 -- 172.21.15.156:0/412084088 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe188011830 con 0x7fe198161340 2023-11-28T13:09:07.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.918+0000 7fe19d199700 1 -- 172.21.15.156:0/412084088 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe188011990 con 0x7fe198161340 2023-11-28T13:09:07.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.918+0000 7fe19d199700 1 -- 172.21.15.156:0/412084088 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe188011d20 con 0x7fe198161340 2023-11-28T13:09:07.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.919+0000 7fe1a03ff700 1 -- 172.21.15.156:0/412084088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe198161340 msgr2=0x7fe1980d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:07.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.919+0000 7fe1a03ff700 1 --2- 172.21.15.156:0/412084088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe198161340 0x7fe1980d1de0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe188009840 tx=0x7fe188009b50 comp rx=0 tx=0).stop 2023-11-28T13:09:07.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.919+0000 7fe1a03ff700 1 -- 172.21.15.156:0/412084088 shutdown_connections 2023-11-28T13:09:07.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.919+0000 7fe1a03ff700 1 --2- 172.21.15.156:0/412084088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe198161340 0x7fe1980d1de0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:07.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.919+0000 7fe1a03ff700 1 -- 172.21.15.156:0/412084088 >> 172.21.15.156:0/412084088 conn(0x7fe198072600 msgr2=0x7fe1980cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:07.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.919+0000 7fe1a03ff700 1 -- 172.21.15.156:0/412084088 shutdown_connections 2023-11-28T13:09:07.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.919+0000 7fe1a03ff700 1 -- 172.21.15.156:0/412084088 wait complete. 2023-11-28T13:09:07.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.921+0000 7fe1a03ff700 1 Processor -- start 2023-11-28T13:09:07.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.922+0000 7fe1a03ff700 1 -- start start 2023-11-28T13:09:07.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.923+0000 7fe1a03ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1980d2320 0x7fe19814a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:07.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.923+0000 7fe1a03ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe19815c9b0 0x7fe1981594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:07.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.923+0000 7fe1a03ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe198161340 0x7fe198159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:07.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.923+0000 7fe1a03ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1981e6460 con 0x7fe198161340 2023-11-28T13:09:07.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.923+0000 7fe1a03ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19814a920 con 0x7fe1980d2320 2023-11-28T13:09:07.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.923+0000 7fe1a03ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1981e66b0 con 0x7fe19815c9b0 2023-11-28T13:09:07.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.923+0000 7fe19e99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe198161340 0x7fe198159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:07.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.923+0000 7fe19e99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe198161340 0x7fe198159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49586/0 (socket says 172.21.15.156:49586) 2023-11-28T13:09:07.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.923+0000 7fe19e99c700 1 -- 172.21.15.156:0/658546950 learned_addr learned my addr 172.21.15.156:0/658546950 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:07.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.923+0000 7fe19e19b700 1 --2- 172.21.15.156:0/658546950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1980d2320 0x7fe19814a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:07.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.923+0000 7fe19d99a700 1 --2- 172.21.15.156:0/658546950 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe19815c9b0 0x7fe1981594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:07.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.924+0000 7fe19e99c700 1 -- 172.21.15.156:0/658546950 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe19815c9b0 msgr2=0x7fe1981594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:07.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.924+0000 7fe19e99c700 1 --2- 172.21.15.156:0/658546950 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe19815c9b0 0x7fe1981594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:07.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.924+0000 7fe19e99c700 1 -- 172.21.15.156:0/658546950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1980d2320 msgr2=0x7fe19814a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:07.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.924+0000 7fe19e99c700 1 --2- 172.21.15.156:0/658546950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1980d2320 0x7fe19814a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:07.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.924+0000 7fe19e99c700 1 -- 172.21.15.156:0/658546950 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1880094a0 con 0x7fe198161340 2023-11-28T13:09:07.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.924+0000 7fe19e99c700 1 --2- 172.21.15.156:0/658546950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe198161340 0x7fe198159a30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe19000cbc0 tx=0x7fe19000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:07.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.924+0000 7fe18effd700 1 -- 172.21.15.156:0/658546950 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe190003c00 con 0x7fe198161340 2023-11-28T13:09:07.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.924+0000 7fe18effd700 1 -- 172.21.15.156:0/658546950 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe190003d60 con 0x7fe198161340 2023-11-28T13:09:07.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.924+0000 7fe18effd700 1 -- 172.21.15.156:0/658546950 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19000aae0 con 0x7fe198161340 2023-11-28T13:09:07.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.924+0000 7fe1a03ff700 1 -- 172.21.15.156:0/658546950 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe19815cd80 con 0x7fe198161340 2023-11-28T13:09:07.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.924+0000 7fe1a03ff700 1 -- 172.21.15.156:0/658546950 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1980d15e0 con 0x7fe198161340 2023-11-28T13:09:07.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.925+0000 7fe1a03ff700 1 -- 172.21.15.156:0/658546950 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe19815cf10 con 0x7fe198161340 2023-11-28T13:09:07.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.927+0000 7fe18effd700 1 -- 172.21.15.156:0/658546950 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe190010050 con 0x7fe198161340 2023-11-28T13:09:07.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.928+0000 7fe18effd700 1 --2- 172.21.15.156:0/658546950 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe17c0682f0 0x7fe17c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:07.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.928+0000 7fe18effd700 1 -- 172.21.15.156:0/658546950 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fe1900864e0 con 0x7fe198161340 2023-11-28T13:09:07.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.928+0000 7fe19e19b700 1 --2- 172.21.15.156:0/658546950 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe17c0682f0 0x7fe17c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:07.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.929+0000 7fe19e19b700 1 --2- 172.21.15.156:0/658546950 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe17c0682f0 0x7fe17c06a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe194005910 tx=0x7fe194009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:07.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:07.931+0000 7fe18effd700 1 -- 172.21.15.156:0/658546950 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe190052e40 con 0x7fe198161340 2023-11-28T13:09:08.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.166+0000 7fe1a03ff700 1 -- 172.21.15.156:0/658546950 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe1980745c0 con 0x7fe198161340 2023-11-28T13:09:08.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.173+0000 7fe18effd700 1 -- 172.21.15.156:0/658546950 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe190056460 con 0x7fe198161340 2023-11-28T13:09:08.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.180+0000 7fe1a03ff700 1 -- 172.21.15.156:0/658546950 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe17c0682f0 msgr2=0x7fe17c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:08.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.180+0000 7fe1a03ff700 1 --2- 172.21.15.156:0/658546950 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe17c0682f0 0x7fe17c06a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe194005910 tx=0x7fe194009040 comp rx=0 tx=0).stop 2023-11-28T13:09:08.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.180+0000 7fe1a03ff700 1 -- 172.21.15.156:0/658546950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe198161340 msgr2=0x7fe198159a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:08.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.180+0000 7fe1a03ff700 1 --2- 172.21.15.156:0/658546950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe198161340 0x7fe198159a30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe19000cbc0 tx=0x7fe19000ced0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.183+0000 7fe1a03ff700 1 -- 172.21.15.156:0/658546950 shutdown_connections 2023-11-28T13:09:08.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.183+0000 7fe1a03ff700 1 --2- 172.21.15.156:0/658546950 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe17c0682f0 0x7fe17c06a7a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.183+0000 7fe1a03ff700 1 --2- 172.21.15.156:0/658546950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe198161340 0x7fe198159a30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.183+0000 7fe1a03ff700 1 --2- 172.21.15.156:0/658546950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1980d2320 0x7fe19814a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.183+0000 7fe1a03ff700 1 --2- 172.21.15.156:0/658546950 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe19815c9b0 0x7fe1981594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.183+0000 7fe1a03ff700 1 -- 172.21.15.156:0/658546950 >> 172.21.15.156:0/658546950 conn(0x7fe198072600 msgr2=0x7fe19815a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:08.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.186+0000 7fe1a03ff700 1 -- 172.21.15.156:0/658546950 shutdown_connections 2023-11-28T13:09:08.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.186+0000 7fe1a03ff700 1 -- 172.21.15.156:0/658546950 wait complete. 2023-11-28T13:09:08.194 INFO:teuthology.orchestra.run.smithi156.stderr:new fs with metadata pool 5 and data pool 6 2023-11-28T13:09:08.217 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:09:08.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.412+0000 7fb50f7ff700 1 Processor -- start 2023-11-28T13:09:08.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.413+0000 7fb50f7ff700 1 -- start start 2023-11-28T13:09:08.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.414+0000 7fb50f7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb50815c9b0 0x7fb5080b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:08.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.414+0000 7fb50f7ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb5080cd7c0 0x7fb5080cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:08.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.414+0000 7fb50f7ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5080ce160 con 0x7fb508161340 2023-11-28T13:09:08.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.414+0000 7fb50f7ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5080b5800 con 0x7fb5080cd7c0 2023-11-28T13:09:08.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.414+0000 7fb50f7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080d1950 con 0x7fb50815c9b0 2023-11-28T13:09:08.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.414+0000 7fb50cd9a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb508161340 0x7fb5080d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:43460/0 (socket says 172.21.15.156:43460) 2023-11-28T13:09:08.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.414+0000 7fb50cd9a700 1 -- 172.21.15.156:0/1767603116 learned_addr learned my addr 172.21.15.156:0/1767603116 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:08.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.414+0000 7fb50dd9c700 1 --2- 172.21.15.156:0/1767603116 >> v2:172.21.15.156:3300/0 conn(0x7fb5080cd7c0 0x7fb5080cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:08.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.414+0000 7fb50d59b700 1 --2- 172.21.15.156:0/1767603116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb50815c9b0 0x7fb5080b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:08.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.414+0000 7fb4fffff700 1 -- 172.21.15.156:0/1767603116 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452682147 0 0) 0x7fb5080ce160 con 0x7fb508161340 2023-11-28T13:09:08.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.414+0000 7fb4fffff700 1 -- 172.21.15.156:0/1767603116 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4f40035b0 con 0x7fb508161340 2023-11-28T13:09:08.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.414+0000 7fb50dd9c700 1 -- 172.21.15.156:0/1767603116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb50815c9b0 msgr2=0x7fb5080b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:08.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.414+0000 7fb50dd9c700 1 --2- 172.21.15.156:0/1767603116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb50815c9b0 0x7fb5080b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.414+0000 7fb50dd9c700 1 -- 172.21.15.156:0/1767603116 >> v1:172.21.15.156:6789/0 conn(0x7fb508161340 legacy=0x7fb5080d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:08.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.414+0000 7fb50dd9c700 1 -- 172.21.15.156:0/1767603116 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5080d24f0 con 0x7fb5080cd7c0 2023-11-28T13:09:08.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.414+0000 7fb50d59b700 1 --2- 172.21.15.156:0/1767603116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb50815c9b0 0x7fb5080b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:08.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.415+0000 7fb50dd9c700 1 --2- 172.21.15.156:0/1767603116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5080cd7c0 0x7fb5080cdc20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb500009d90 tx=0x7fb50000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc86e1850468149c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:08.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.415+0000 7fb4fffff700 1 -- 172.21.15.156:0/1767603116 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50000e070 con 0x7fb5080cd7c0 2023-11-28T13:09:08.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.415+0000 7fb4fffff700 1 -- 172.21.15.156:0/1767603116 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50000bb20 con 0x7fb5080cd7c0 2023-11-28T13:09:08.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.415+0000 7fb4fffff700 1 -- 172.21.15.156:0/1767603116 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb500012430 con 0x7fb5080cd7c0 2023-11-28T13:09:08.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.415+0000 7fb50f7ff700 1 -- 172.21.15.156:0/1767603116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5080cd7c0 msgr2=0x7fb5080cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:08.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.415+0000 7fb50f7ff700 1 --2- 172.21.15.156:0/1767603116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5080cd7c0 0x7fb5080cdc20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb500009d90 tx=0x7fb50000b330 comp rx=0 tx=0).stop 2023-11-28T13:09:08.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.416+0000 7fb50f7ff700 1 -- 172.21.15.156:0/1767603116 shutdown_connections 2023-11-28T13:09:08.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.416+0000 7fb50f7ff700 1 --2- 172.21.15.156:0/1767603116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5080cd7c0 0x7fb5080cdc20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.416+0000 7fb50f7ff700 1 --2- 172.21.15.156:0/1767603116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb50815c9b0 0x7fb5080b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.416+0000 7fb50f7ff700 1 -- 172.21.15.156:0/1767603116 >> 172.21.15.156:0/1767603116 conn(0x7fb508072600 msgr2=0x7fb5080cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:08.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.416+0000 7fb50f7ff700 1 -- 172.21.15.156:0/1767603116 shutdown_connections 2023-11-28T13:09:08.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.416+0000 7fb50f7ff700 1 -- 172.21.15.156:0/1767603116 wait complete. 2023-11-28T13:09:08.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.417+0000 7fb50f7ff700 1 Processor -- start 2023-11-28T13:09:08.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.417+0000 7fb50f7ff700 1 -- start start 2023-11-28T13:09:08.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.418+0000 7fb50f7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5080cd7c0 0x7fb508148b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:08.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.418+0000 7fb50f7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb50815c9b0 0x7fb50815ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:08.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.418+0000 7fb50f7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb508161340 0x7fb5080cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:08.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.418+0000 7fb50f7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb508149120 con 0x7fb5080cd7c0 2023-11-28T13:09:08.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.418+0000 7fb50f7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5081e7c20 con 0x7fb50815c9b0 2023-11-28T13:09:08.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.418+0000 7fb50f7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080c9600 con 0x7fb508161340 2023-11-28T13:09:08.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.419+0000 7fb50d59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5080cd7c0 0x7fb508148b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:08.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.419+0000 7fb50d59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5080cd7c0 0x7fb508148b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49602/0 (socket says 172.21.15.156:49602) 2023-11-28T13:09:08.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.419+0000 7fb50d59b700 1 -- 172.21.15.156:0/858242231 learned_addr learned my addr 172.21.15.156:0/858242231 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:08.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.419+0000 7fb50dd9c700 1 --2- 172.21.15.156:0/858242231 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb508161340 0x7fb5080cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:08.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.419+0000 7fb50cd9a700 1 --2- 172.21.15.156:0/858242231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb50815c9b0 0x7fb50815ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:08.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.419+0000 7fb50d59b700 1 -- 172.21.15.156:0/858242231 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb508161340 msgr2=0x7fb5080cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:08.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.419+0000 7fb50d59b700 1 --2- 172.21.15.156:0/858242231 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb508161340 0x7fb5080cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.419+0000 7fb50d59b700 1 -- 172.21.15.156:0/858242231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb50815c9b0 msgr2=0x7fb50815ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:08.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.419+0000 7fb50d59b700 1 --2- 172.21.15.156:0/858242231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb50815c9b0 0x7fb50815ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.419+0000 7fb50d59b700 1 -- 172.21.15.156:0/858242231 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb500009a70 con 0x7fb5080cd7c0 2023-11-28T13:09:08.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.419+0000 7fb50dd9c700 1 --2- 172.21.15.156:0/858242231 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb508161340 0x7fb5080cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:08.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.419+0000 7fb50cd9a700 1 --2- 172.21.15.156:0/858242231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb50815c9b0 0x7fb50815ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:08.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.419+0000 7fb50d59b700 1 --2- 172.21.15.156:0/858242231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5080cd7c0 0x7fb508148b20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb50400c7f0 tx=0x7fb50400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:08.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.419+0000 7fb4fdffb700 1 -- 172.21.15.156:0/858242231 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504011800 con 0x7fb5080cd7c0 2023-11-28T13:09:08.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.419+0000 7fb4fdffb700 1 -- 172.21.15.156:0/858242231 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb504011960 con 0x7fb5080cd7c0 2023-11-28T13:09:08.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.419+0000 7fb4fdffb700 1 -- 172.21.15.156:0/858242231 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504011d10 con 0x7fb5080cd7c0 2023-11-28T13:09:08.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.419+0000 7fb50f7ff700 1 -- 172.21.15.156:0/858242231 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5080d1300 con 0x7fb5080cd7c0 2023-11-28T13:09:08.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.420+0000 7fb50f7ff700 1 -- 172.21.15.156:0/858242231 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5080b5520 con 0x7fb5080cd7c0 2023-11-28T13:09:08.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.421+0000 7fb50f7ff700 1 -- 172.21.15.156:0/858242231 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5080d15e0 con 0x7fb5080cd7c0 2023-11-28T13:09:08.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.421+0000 7fb4fdffb700 1 -- 172.21.15.156:0/858242231 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb504021070 con 0x7fb5080cd7c0 2023-11-28T13:09:08.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.422+0000 7fb4fdffb700 1 --2- 172.21.15.156:0/858242231 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb4ec068320 0x7fb4ec06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:08.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.422+0000 7fb4fdffb700 1 -- 172.21.15.156:0/858242231 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb504085ba0 con 0x7fb5080cd7c0 2023-11-28T13:09:08.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.423+0000 7fb50cd9a700 1 --2- 172.21.15.156:0/858242231 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb4ec068320 0x7fb4ec06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:08.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.423+0000 7fb50cd9a700 1 --2- 172.21.15.156:0/858242231 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb4ec068320 0x7fb4ec06a7d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb5080cc880 tx=0x7fb4f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:08.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.426+0000 7fb4fdffb700 1 -- 172.21.15.156:0/858242231 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb504052310 con 0x7fb5080cd7c0 2023-11-28T13:09:08.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.618+0000 7fb50f7ff700 1 -- 172.21.15.156:0/858242231 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5080c9b20 con 0x7fb5080cd7c0 2023-11-28T13:09:08.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.619+0000 7fb4fdffb700 1 -- 172.21.15.156:0/858242231 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+17982 (secure 0 0 0) 0x7fb504055930 con 0x7fb5080cd7c0 2023-11-28T13:09:08.620 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:08.620 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":34,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:09:08.167596+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:09:06.618698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T13:09:07.151554+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:09:08.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.621+0000 7fb50f7ff700 1 -- 172.21.15.156:0/858242231 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb4ec068320 msgr2=0x7fb4ec06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:08.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.621+0000 7fb50f7ff700 1 --2- 172.21.15.156:0/858242231 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb4ec068320 0x7fb4ec06a7d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb5080cc880 tx=0x7fb4f8009040 comp rx=0 tx=0).stop 2023-11-28T13:09:08.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.621+0000 7fb50f7ff700 1 -- 172.21.15.156:0/858242231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5080cd7c0 msgr2=0x7fb508148b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:08.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.621+0000 7fb50f7ff700 1 --2- 172.21.15.156:0/858242231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5080cd7c0 0x7fb508148b20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb50400c7f0 tx=0x7fb50400cbb0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.621+0000 7fb50f7ff700 1 -- 172.21.15.156:0/858242231 shutdown_connections 2023-11-28T13:09:08.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.621+0000 7fb50f7ff700 1 --2- 172.21.15.156:0/858242231 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb4ec068320 0x7fb4ec06a7d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.621+0000 7fb50f7ff700 1 --2- 172.21.15.156:0/858242231 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb508161340 0x7fb5080cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.621+0000 7fb50f7ff700 1 --2- 172.21.15.156:0/858242231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5080cd7c0 0x7fb508148b20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.621+0000 7fb50f7ff700 1 --2- 172.21.15.156:0/858242231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb50815c9b0 0x7fb50815ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.622+0000 7fb50f7ff700 1 -- 172.21.15.156:0/858242231 >> 172.21.15.156:0/858242231 conn(0x7fb508072600 msgr2=0x7fb50815aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:08.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.622+0000 7fb50f7ff700 1 -- 172.21.15.156:0/858242231 shutdown_connections 2023-11-28T13:09:08.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.622+0000 7fb50f7ff700 1 -- 172.21.15.156:0/858242231 wait complete. 2023-11-28T13:09:08.641 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:09:08.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.839+0000 7f3a61dff700 1 Processor -- start 2023-11-28T13:09:08.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.839+0000 7f3a61dff700 1 -- start start 2023-11-28T13:09:08.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.840+0000 7f3a61dff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3a5c15c9b0 0x7f3a5c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:08.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.840+0000 7f3a61dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a5c0cd7c0 0x7f3a5c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:08.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.840+0000 7f3a61dff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a5c0d1950 con 0x7f3a5c161340 2023-11-28T13:09:08.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.840+0000 7f3a61dff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a5c0b5800 con 0x7f3a5c15c9b0 2023-11-28T13:09:08.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.840+0000 7f3a61dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c0ce140 con 0x7f3a5c0cd7c0 2023-11-28T13:09:08.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.841+0000 7f3a5affd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f3a5c161340 0x7f3a5c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:43464/0 (socket says 172.21.15.156:43464) 2023-11-28T13:09:08.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.841+0000 7f3a5affd700 1 -- 172.21.15.156:0/179843031 learned_addr learned my addr 172.21.15.156:0/179843031 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:08.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.841+0000 7f3a5bfff700 1 --2- 172.21.15.156:0/179843031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a5c0cd7c0 0x7f3a5c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:08.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.841+0000 7f3a5b7fe700 1 --2- 172.21.15.156:0/179843031 >> v2:172.21.15.186:3300/0 conn(0x7f3a5c15c9b0 0x7f3a5c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:08.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.841+0000 7f3a5a7fc700 1 -- 172.21.15.156:0/179843031 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120166772 0 0) 0x7f3a5c0d1950 con 0x7f3a5c161340 2023-11-28T13:09:08.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.841+0000 7f3a5a7fc700 1 -- 172.21.15.156:0/179843031 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a440035b0 con 0x7f3a5c161340 2023-11-28T13:09:08.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.841+0000 7f3a5a7fc700 1 -- 172.21.15.156:0/179843031 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 108727115 0 0) 0x7f3a440035b0 con 0x7f3a5c161340 2023-11-28T13:09:08.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.841+0000 7f3a5a7fc700 1 -- 172.21.15.156:0/179843031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a5c0cd7c0 msgr2=0x7f3a5c0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:08.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.841+0000 7f3a5a7fc700 1 --2- 172.21.15.156:0/179843031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a5c0cd7c0 0x7f3a5c0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.841+0000 7f3a5a7fc700 1 -- 172.21.15.156:0/179843031 >> v2:172.21.15.186:3300/0 conn(0x7f3a5c15c9b0 msgr2=0x7f3a5c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:08.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.841+0000 7f3a5a7fc700 1 --2- 172.21.15.156:0/179843031 >> v2:172.21.15.186:3300/0 conn(0x7f3a5c15c9b0 0x7f3a5c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.841+0000 7f3a5a7fc700 1 -- 172.21.15.156:0/179843031 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c0d24f0 con 0x7f3a5c161340 2023-11-28T13:09:08.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.841+0000 7f3a5a7fc700 1 -- 172.21.15.156:0/179843031 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f3a50001670 con 0x7f3a5c161340 2023-11-28T13:09:08.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.841+0000 7f3a5a7fc700 1 -- 172.21.15.156:0/179843031 >> v1:172.21.15.156:6789/0 conn(0x7f3a5c161340 legacy=0x7f3a5c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:08.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.842+0000 7f3a5a7fc700 1 --2- 172.21.15.156:0/179843031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a5c0cd7c0 0x7f3a44003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:08.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.842+0000 7f3a5a7fc700 1 --2- 172.21.15.156:0/179843031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a44003fd0 0x7f3a44004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:08.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.842+0000 7f3a5a7fc700 1 --2- 172.21.15.156:0/179843031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a44004d90 0x7f3a44005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:08.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.842+0000 7f3a5a7fc700 1 -- 172.21.15.156:0/179843031 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c0b5800 con 0x7f3a44003fd0 2023-11-28T13:09:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.842+0000 7f3a5a7fc700 1 -- 172.21.15.156:0/179843031 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c0ce140 con 0x7f3a44004d90 2023-11-28T13:09:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.842+0000 7f3a5a7fc700 1 -- 172.21.15.156:0/179843031 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4400ba90 con 0x7f3a5c0cd7c0 2023-11-28T13:09:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.842+0000 7f3a5affd700 1 --2- 172.21.15.156:0/179843031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a44003fd0 0x7f3a44004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.842+0000 7f3a5bfff700 1 --2- 172.21.15.156:0/179843031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a44004d90 0x7f3a44005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.842+0000 7f3a5b7fe700 1 --2- 172.21.15.156:0/179843031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a5c0cd7c0 0x7f3a44003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.842+0000 7f3a5affd700 1 -- 172.21.15.156:0/179843031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a5c0cd7c0 msgr2=0x7f3a44003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:08.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.842+0000 7f3a5affd700 1 --2- 172.21.15.156:0/179843031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a5c0cd7c0 0x7f3a44003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.842+0000 7f3a5affd700 1 -- 172.21.15.156:0/179843031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a44004d90 msgr2=0x7f3a44005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:08.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.842+0000 7f3a5affd700 1 --2- 172.21.15.156:0/179843031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a44004d90 0x7f3a44005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.842+0000 7f3a5affd700 1 -- 172.21.15.156:0/179843031 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a440035b0 con 0x7f3a44003fd0 2023-11-28T13:09:08.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.842+0000 7f3a5affd700 1 --2- 172.21.15.156:0/179843031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a44003fd0 0x7f3a44004850 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000c190 tx=0x7f3a44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37d8bdf00e0405a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:08.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.843+0000 7f3a5a7fc700 1 -- 172.21.15.156:0/179843031 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50007b70 con 0x7f3a44003fd0 2023-11-28T13:09:08.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.843+0000 7f3a5a7fc700 1 -- 172.21.15.156:0/179843031 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5000cd30 con 0x7f3a44003fd0 2023-11-28T13:09:08.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.843+0000 7f3a5a7fc700 1 -- 172.21.15.156:0/179843031 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5000cea0 con 0x7f3a44003fd0 2023-11-28T13:09:08.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.843+0000 7f3a61dff700 1 -- 172.21.15.156:0/179843031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a44003fd0 msgr2=0x7f3a44004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:08.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.843+0000 7f3a61dff700 1 --2- 172.21.15.156:0/179843031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a44003fd0 0x7f3a44004850 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000c190 tx=0x7f3a44005880 comp rx=0 tx=0).stop 2023-11-28T13:09:08.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.843+0000 7f3a5bfff700 1 -- 172.21.15.156:0/179843031 reap_dead start 2023-11-28T13:09:08.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.843+0000 7f3a61dff700 1 -- 172.21.15.156:0/179843031 shutdown_connections 2023-11-28T13:09:08.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.843+0000 7f3a61dff700 1 -- 172.21.15.156:0/179843031 >> 172.21.15.156:0/179843031 conn(0x7f3a5c072600 msgr2=0x7f3a5c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:08.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.843+0000 7f3a61dff700 1 -- 172.21.15.156:0/179843031 shutdown_connections 2023-11-28T13:09:08.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.843+0000 7f3a61dff700 1 -- 172.21.15.156:0/179843031 wait complete. 2023-11-28T13:09:08.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.846+0000 7f3a61dff700 1 Processor -- start 2023-11-28T13:09:08.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.846+0000 7f3a61dff700 1 -- start start 2023-11-28T13:09:08.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.847+0000 7f3a61dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a44003fd0 0x7f3a5c0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:08.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.847+0000 7f3a61dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a5c0c99d0 0x7f3a5c15e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:08.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.847+0000 7f3a61dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a5c157290 0x7f3a5c157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:08.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.847+0000 7f3a61dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c0c7410 con 0x7f3a5c0c99d0 2023-11-28T13:09:08.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.847+0000 7f3a61dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c0c76f0 con 0x7f3a5c157290 2023-11-28T13:09:08.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.847+0000 7f3a61dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c0ca3e0 con 0x7f3a44003fd0 2023-11-28T13:09:08.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.848+0000 7f3a5affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a5c0c99d0 0x7f3a5c15e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:08.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.848+0000 7f3a5affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a5c0c99d0 0x7f3a5c15e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49620/0 (socket says 172.21.15.156:49620) 2023-11-28T13:09:08.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.848+0000 7f3a5affd700 1 -- 172.21.15.156:0/1872768324 learned_addr learned my addr 172.21.15.156:0/1872768324 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:08.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.848+0000 7f3a5bfff700 1 --2- 172.21.15.156:0/1872768324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a5c157290 0x7f3a5c157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:08.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.848+0000 7f3a5b7fe700 1 --2- 172.21.15.156:0/1872768324 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a44003fd0 0x7f3a5c0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:08.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.848+0000 7f3a5affd700 1 -- 172.21.15.156:0/1872768324 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a44003fd0 msgr2=0x7f3a5c0c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:08.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.848+0000 7f3a5affd700 1 --2- 172.21.15.156:0/1872768324 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a44003fd0 0x7f3a5c0c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.848+0000 7f3a5affd700 1 -- 172.21.15.156:0/1872768324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a5c157290 msgr2=0x7f3a5c157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:08.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.848+0000 7f3a5affd700 1 --2- 172.21.15.156:0/1872768324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a5c157290 0x7f3a5c157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:08.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.848+0000 7f3a5affd700 1 -- 172.21.15.156:0/1872768324 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c0d1300 con 0x7f3a5c0c99d0 2023-11-28T13:09:08.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.848+0000 7f3a5bfff700 1 --2- 172.21.15.156:0/1872768324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a5c157290 0x7f3a5c157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:08.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.848+0000 7f3a5b7fe700 1 --2- 172.21.15.156:0/1872768324 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a44003fd0 0x7f3a5c0c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:08.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.848+0000 7f3a5affd700 1 --2- 172.21.15.156:0/1872768324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a5c0c99d0 0x7f3a5c15e370 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3a50005b30 tx=0x7f3a50001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:08.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.849+0000 7f3a43fff700 1 -- 172.21.15.156:0/1872768324 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5000cea0 con 0x7f3a5c0c99d0 2023-11-28T13:09:08.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.849+0000 7f3a43fff700 1 -- 172.21.15.156:0/1872768324 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a500070d0 con 0x7f3a5c0c99d0 2023-11-28T13:09:08.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.849+0000 7f3a43fff700 1 -- 172.21.15.156:0/1872768324 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50007460 con 0x7f3a5c0c99d0 2023-11-28T13:09:08.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.849+0000 7f3a61dff700 1 -- 172.21.15.156:0/1872768324 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5c0b5520 con 0x7f3a5c0c99d0 2023-11-28T13:09:08.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.849+0000 7f3a61dff700 1 -- 172.21.15.156:0/1872768324 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5c0d15e0 con 0x7f3a5c0c99d0 2023-11-28T13:09:08.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.850+0000 7f3a61dff700 1 -- 172.21.15.156:0/1872768324 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5c0c7910 con 0x7f3a5c0c99d0 2023-11-28T13:09:08.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.851+0000 7f3a43fff700 1 -- 172.21.15.156:0/1872768324 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3a50010070 con 0x7f3a5c0c99d0 2023-11-28T13:09:08.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.851+0000 7f3a43fff700 1 --2- 172.21.15.156:0/1872768324 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3a480682f0 0x7f3a4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:08.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.852+0000 7f3a43fff700 1 -- 172.21.15.156:0/1872768324 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3a50086f50 con 0x7f3a5c0c99d0 2023-11-28T13:09:08.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.852+0000 7f3a5b7fe700 1 --2- 172.21.15.156:0/1872768324 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3a480682f0 0x7f3a4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:08.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.853+0000 7f3a5b7fe700 1 --2- 172.21.15.156:0/1872768324 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3a480682f0 0x7f3a4806a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009b50 tx=0x7f3a4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:08.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:08.855+0000 7f3a43fff700 1 -- 172.21.15.156:0/1872768324 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3a500536c0 con 0x7f3a5c0c99d0 2023-11-28T13:09:09.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.052+0000 7f3a61dff700 1 -- 172.21.15.156:0/1872768324 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a5c0ca220 con 0x7f3a5c0c99d0 2023-11-28T13:09:09.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.053+0000 7f3a43fff700 1 -- 172.21.15.156:0/1872768324 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+17982 (secure 0 0 0) 0x7f3a50056ce0 con 0x7f3a5c0c99d0 2023-11-28T13:09:09.054 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:09.055 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":34,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:09:08.167596+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:09:06.618698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T13:09:07.151554+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:09:09.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.056+0000 7f3a61dff700 1 -- 172.21.15.156:0/1872768324 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3a480682f0 msgr2=0x7f3a4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:09.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.056+0000 7f3a61dff700 1 --2- 172.21.15.156:0/1872768324 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3a480682f0 0x7f3a4806a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009b50 tx=0x7f3a4c008040 comp rx=0 tx=0).stop 2023-11-28T13:09:09.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.056+0000 7f3a61dff700 1 -- 172.21.15.156:0/1872768324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a5c0c99d0 msgr2=0x7f3a5c15e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:09.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.056+0000 7f3a61dff700 1 --2- 172.21.15.156:0/1872768324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a5c0c99d0 0x7f3a5c15e370 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3a50005b30 tx=0x7f3a50001930 comp rx=0 tx=0).stop 2023-11-28T13:09:09.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.056+0000 7f3a61dff700 1 -- 172.21.15.156:0/1872768324 shutdown_connections 2023-11-28T13:09:09.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.056+0000 7f3a61dff700 1 --2- 172.21.15.156:0/1872768324 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3a480682f0 0x7f3a4806a7a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:09.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.056+0000 7f3a61dff700 1 --2- 172.21.15.156:0/1872768324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a5c157290 0x7f3a5c157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:09.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.056+0000 7f3a61dff700 1 --2- 172.21.15.156:0/1872768324 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3a44003fd0 0x7f3a5c0c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:09.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.056+0000 7f3a61dff700 1 --2- 172.21.15.156:0/1872768324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a5c0c99d0 0x7f3a5c15e370 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:09.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.056+0000 7f3a61dff700 1 -- 172.21.15.156:0/1872768324 >> 172.21.15.156:0/1872768324 conn(0x7f3a5c072600 msgr2=0x7f3a5c0b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:09.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.056+0000 7f3a61dff700 1 -- 172.21.15.156:0/1872768324 shutdown_connections 2023-11-28T13:09:09.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.057+0000 7f3a61dff700 1 -- 172.21.15.156:0/1872768324 wait complete. 2023-11-28T13:09:09.076 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T13:09:09.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.276+0000 7f18ef5ff700 1 Processor -- start 2023-11-28T13:09:09.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.277+0000 7f18ef5ff700 1 -- start start 2023-11-28T13:09:09.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.278+0000 7f18ef5ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f18e80d0ff0 0x7f18e80cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:09.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.278+0000 7f18ef5ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f18e80b4750 0x7f18e80b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:09.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.278+0000 7f18ef5ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e80cf220 con 0x7f18e80d13c0 2023-11-28T13:09:09.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.278+0000 7f18ef5ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e80b3e50 con 0x7f18e80d0ff0 2023-11-28T13:09:09.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.278+0000 7f18ef5ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e80cf470 con 0x7f18e80b4750 2023-11-28T13:09:09.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.278+0000 7f18ed39b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f18e80d0ff0 0x7f18e80cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:09.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.278+0000 7f18ed39b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f18e80d0ff0 0x7f18e80cff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49624/0 (socket says 172.21.15.156:49624) 2023-11-28T13:09:09.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.278+0000 7f18ed39b700 1 -- 172.21.15.156:0/1640263939 learned_addr learned my addr 172.21.15.156:0/1640263939 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:09.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.278+0000 7f18ecb9a700 1 --2- 172.21.15.156:0/1640263939 >> v2:172.21.15.186:3300/0 conn(0x7f18e80b4750 0x7f18e80b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:09.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.279+0000 7f18ed39b700 1 -- 172.21.15.156:0/1640263939 >> v2:172.21.15.186:3300/0 conn(0x7f18e80b4750 msgr2=0x7f18e80b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:09.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.279+0000 7f18ed39b700 1 --2- 172.21.15.156:0/1640263939 >> v2:172.21.15.186:3300/0 conn(0x7f18e80b4750 0x7f18e80b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:09.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.279+0000 7f18ed39b700 1 -- 172.21.15.156:0/1640263939 >> v1:172.21.15.186:6789/0 conn(0x7f18e80d13c0 legacy=0x7f18e80b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:09.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.279+0000 7f18ed39b700 1 -- 172.21.15.156:0/1640263939 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e80cf750 con 0x7f18e80d0ff0 2023-11-28T13:09:09.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.279+0000 7f18ed39b700 1 --2- 172.21.15.156:0/1640263939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18e80d0ff0 0x7f18e80cff50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f18e808d340 tx=0x7f18e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2e058b48250ee4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:09.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.279+0000 7f18dffff700 1 -- 172.21.15.156:0/1640263939 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e400e070 con 0x7f18e80d0ff0 2023-11-28T13:09:09.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.279+0000 7f18dffff700 1 -- 172.21.15.156:0/1640263939 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e400b910 con 0x7f18e80d0ff0 2023-11-28T13:09:09.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.279+0000 7f18dffff700 1 -- 172.21.15.156:0/1640263939 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e400bcc0 con 0x7f18e80d0ff0 2023-11-28T13:09:09.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.280+0000 7f18ef5ff700 1 -- 172.21.15.156:0/1640263939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18e80d0ff0 msgr2=0x7f18e80cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:09.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.280+0000 7f18ef5ff700 1 --2- 172.21.15.156:0/1640263939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18e80d0ff0 0x7f18e80cff50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f18e808d340 tx=0x7f18e400b330 comp rx=0 tx=0).stop 2023-11-28T13:09:09.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.280+0000 7f18ef5ff700 1 -- 172.21.15.156:0/1640263939 shutdown_connections 2023-11-28T13:09:09.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.280+0000 7f18ef5ff700 1 --2- 172.21.15.156:0/1640263939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18e80d0ff0 0x7f18e80cff50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:09.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.280+0000 7f18ef5ff700 1 --2- 172.21.15.156:0/1640263939 >> v2:172.21.15.186:3300/0 conn(0x7f18e80b4750 0x7f18e80b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:09.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.280+0000 7f18ef5ff700 1 -- 172.21.15.156:0/1640263939 >> 172.21.15.156:0/1640263939 conn(0x7f18e809c3c0 msgr2=0x7f18e80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:09.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.280+0000 7f18ef5ff700 1 -- 172.21.15.156:0/1640263939 shutdown_connections 2023-11-28T13:09:09.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.281+0000 7f18ef5ff700 1 -- 172.21.15.156:0/1640263939 wait complete. 2023-11-28T13:09:09.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.283+0000 7f18ef5ff700 1 Processor -- start 2023-11-28T13:09:09.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.284+0000 7f18ef5ff700 1 -- start start 2023-11-28T13:09:09.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.284+0000 7f18ef5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18e80b4750 0x7f18e8158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:09.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.284+0000 7f18ef5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18e80d0ff0 0x7f18e815c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:09.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.284+0000 7f18ef5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18e80d13c0 0x7f18e815ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:09.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.285+0000 7f18ef5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e8158b50 con 0x7f18e80d0ff0 2023-11-28T13:09:09.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.285+0000 7f18ef5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e8159cb0 con 0x7f18e80b4750 2023-11-28T13:09:09.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.285+0000 7f18ef5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e8159f90 con 0x7f18e80d13c0 2023-11-28T13:09:09.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.285+0000 7f18ecb9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18e80d0ff0 0x7f18e815c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:09.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.285+0000 7f18ecb9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18e80d0ff0 0x7f18e815c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49632/0 (socket says 172.21.15.156:49632) 2023-11-28T13:09:09.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.285+0000 7f18ecb9a700 1 -- 172.21.15.156:0/3820179060 learned_addr learned my addr 172.21.15.156:0/3820179060 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:09.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.285+0000 7f18ed39b700 1 --2- 172.21.15.156:0/3820179060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18e80b4750 0x7f18e8158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:09.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.285+0000 7f18edb9c700 1 --2- 172.21.15.156:0/3820179060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18e80d13c0 0x7f18e815ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:09.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.285+0000 7f18ecb9a700 1 -- 172.21.15.156:0/3820179060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18e80d13c0 msgr2=0x7f18e815ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:09.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.285+0000 7f18ecb9a700 1 --2- 172.21.15.156:0/3820179060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18e80d13c0 0x7f18e815ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:09.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.285+0000 7f18ecb9a700 1 -- 172.21.15.156:0/3820179060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18e80b4750 msgr2=0x7f18e8158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:09.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.285+0000 7f18ecb9a700 1 --2- 172.21.15.156:0/3820179060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18e80b4750 0x7f18e8158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:09.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.285+0000 7f18ecb9a700 1 -- 172.21.15.156:0/3820179060 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e4009b30 con 0x7f18e80d0ff0 2023-11-28T13:09:09.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.286+0000 7f18ecb9a700 1 --2- 172.21.15.156:0/3820179060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18e80d0ff0 0x7f18e815c350 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f18d800e980 tx=0x7f18d800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:09.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.286+0000 7f18ddffb700 1 -- 172.21.15.156:0/3820179060 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8009710 con 0x7f18e80d0ff0 2023-11-28T13:09:09.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.286+0000 7f18ddffb700 1 -- 172.21.15.156:0/3820179060 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d800c690 con 0x7f18e80d0ff0 2023-11-28T13:09:09.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.286+0000 7f18ddffb700 1 -- 172.21.15.156:0/3820179060 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d800ca40 con 0x7f18e80d0ff0 2023-11-28T13:09:09.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.286+0000 7f18ef5ff700 1 -- 172.21.15.156:0/3820179060 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e80d0a50 con 0x7f18e80d0ff0 2023-11-28T13:09:09.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.286+0000 7f18ef5ff700 1 -- 172.21.15.156:0/3820179060 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e80cfa30 con 0x7f18e80d0ff0 2023-11-28T13:09:09.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.287+0000 7f18ef5ff700 1 -- 172.21.15.156:0/3820179060 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e80d0490 con 0x7f18e80d0ff0 2023-11-28T13:09:09.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.288+0000 7f18ddffb700 1 -- 172.21.15.156:0/3820179060 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f18d8019030 con 0x7f18e80d0ff0 2023-11-28T13:09:09.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.289+0000 7f18ddffb700 1 --2- 172.21.15.156:0/3820179060 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f18cc0682f0 0x7f18cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:09.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.289+0000 7f18ddffb700 1 -- 172.21.15.156:0/3820179060 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f18d8085be0 con 0x7f18e80d0ff0 2023-11-28T13:09:09.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.289+0000 7f18ed39b700 1 --2- 172.21.15.156:0/3820179060 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f18cc0682f0 0x7f18cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:09.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.290+0000 7f18ed39b700 1 --2- 172.21.15.156:0/3820179060 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f18cc0682f0 0x7f18cc06a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18e4000f80 tx=0x7f18e4009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:09.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.293+0000 7f18ddffb700 1 -- 172.21.15.156:0/3820179060 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f18d8052350 con 0x7f18e80d0ff0 2023-11-28T13:09:09.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:09.529+0000 7f18ef5ff700 1 -- 172.21.15.156:0/3820179060 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f18e80cefb0 con 0x7f18e80d0ff0 2023-11-28T13:09:10.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.185+0000 7f18ddffb700 1 -- 172.21.15.156:0/3820179060 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f18d8055970 con 0x7f18e80d0ff0 2023-11-28T13:09:10.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.188+0000 7f18ef5ff700 1 -- 172.21.15.156:0/3820179060 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f18cc0682f0 msgr2=0x7f18cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:10.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.188+0000 7f18ef5ff700 1 --2- 172.21.15.156:0/3820179060 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f18cc0682f0 0x7f18cc06a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18e4000f80 tx=0x7f18e4009000 comp rx=0 tx=0).stop 2023-11-28T13:09:10.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.188+0000 7f18ef5ff700 1 -- 172.21.15.156:0/3820179060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18e80d0ff0 msgr2=0x7f18e815c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:10.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.188+0000 7f18ef5ff700 1 --2- 172.21.15.156:0/3820179060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18e80d0ff0 0x7f18e815c350 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f18d800e980 tx=0x7f18d800ec90 comp rx=0 tx=0).stop 2023-11-28T13:09:10.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.188+0000 7f18ef5ff700 1 -- 172.21.15.156:0/3820179060 shutdown_connections 2023-11-28T13:09:10.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.188+0000 7f18ef5ff700 1 --2- 172.21.15.156:0/3820179060 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f18cc0682f0 0x7f18cc06a7a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:10.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.188+0000 7f18ef5ff700 1 --2- 172.21.15.156:0/3820179060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18e80d13c0 0x7f18e815ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:10.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.188+0000 7f18ef5ff700 1 --2- 172.21.15.156:0/3820179060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18e80b4750 0x7f18e8158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:10.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.188+0000 7f18ef5ff700 1 --2- 172.21.15.156:0/3820179060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18e80d0ff0 0x7f18e815c350 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:10.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.188+0000 7f18ef5ff700 1 -- 172.21.15.156:0/3820179060 >> 172.21.15.156:0/3820179060 conn(0x7f18e809c3c0 msgr2=0x7f18e806c410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:10.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.188+0000 7f18ef5ff700 1 -- 172.21.15.156:0/3820179060 shutdown_connections 2023-11-28T13:09:10.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.188+0000 7f18ef5ff700 1 -- 172.21.15.156:0/3820179060 wait complete. 2023-11-28T13:09:10.209 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T13:09:10.209 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T13:09:10.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.411+0000 7fd400fff700 1 Processor -- start 2023-11-28T13:09:10.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.412+0000 7fd400fff700 1 -- start start 2023-11-28T13:09:10.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.413+0000 7fd400fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3fc0cd7c0 0x7fd3fc0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:10.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.413+0000 7fd400fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3fc0d1950 con 0x7fd3fc15c9b0 2023-11-28T13:09:10.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.413+0000 7fd400fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3fc0ce140 con 0x7fd3fc161340 2023-11-28T13:09:10.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.413+0000 7fd400fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc0b5800 con 0x7fd3fc0cd7c0 2023-11-28T13:09:10.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.413+0000 7fd3fa59c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd3fc15c9b0 0x7fd3fc0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:43466/0 (socket says 172.21.15.156:43466) 2023-11-28T13:09:10.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.413+0000 7fd3fa59c700 1 -- 172.21.15.156:0/916537972 learned_addr learned my addr 172.21.15.156:0/916537972 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:10.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.413+0000 7fd3fad9d700 1 --2- 172.21.15.156:0/916537972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3fc0cd7c0 0x7fd3fc0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:10.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.413+0000 7fd3f9d9b700 1 -- 172.21.15.156:0/916537972 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661768701 0 0) 0x7fd3fc0d1950 con 0x7fd3fc15c9b0 2023-11-28T13:09:10.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.413+0000 7fd3f9d9b700 1 -- 172.21.15.156:0/916537972 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3e80035b0 con 0x7fd3fc15c9b0 2023-11-28T13:09:10.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3f9d9b700 1 -- 172.21.15.156:0/916537972 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173023987 0 0) 0x7fd3fc0ce140 con 0x7fd3fc161340 2023-11-28T13:09:10.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3f9d9b700 1 -- 172.21.15.156:0/916537972 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3fc0d1950 con 0x7fd3fc161340 2023-11-28T13:09:10.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3f9d9b700 1 -- 172.21.15.156:0/916537972 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 970842028 0 0) 0x7fd3e80035b0 con 0x7fd3fc15c9b0 2023-11-28T13:09:10.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3f9d9b700 1 -- 172.21.15.156:0/916537972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3fc0cd7c0 msgr2=0x7fd3fc0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:10.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3f9d9b700 1 --2- 172.21.15.156:0/916537972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3fc0cd7c0 0x7fd3fc0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:10.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3f9d9b700 1 -- 172.21.15.156:0/916537972 >> v1:172.21.15.186:6789/0 conn(0x7fd3fc161340 legacy=0x7fd3fc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:10.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3f9d9b700 1 -- 172.21.15.156:0/916537972 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3fc0d24f0 con 0x7fd3fc15c9b0 2023-11-28T13:09:10.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3f9d9b700 1 -- 172.21.15.156:0/916537972 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7fd3ec001670 con 0x7fd3fc15c9b0 2023-11-28T13:09:10.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3f9d9b700 1 -- 172.21.15.156:0/916537972 >> v1:172.21.15.156:6789/0 conn(0x7fd3fc15c9b0 legacy=0x7fd3fc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:10.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3f9d9b700 1 --2- 172.21.15.156:0/916537972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3e8003bc0 0x7fd3e8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:10.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3f9d9b700 1 --2- 172.21.15.156:0/916537972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3fc0cd7c0 0x7fd3e8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:10.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3f9d9b700 1 --2- 172.21.15.156:0/916537972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3e8004d90 0x7fd3e8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:10.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3f9d9b700 1 -- 172.21.15.156:0/916537972 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc0b5800 con 0x7fd3e8004d90 2023-11-28T13:09:10.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3f9d9b700 1 -- 172.21.15.156:0/916537972 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e800bb40 con 0x7fd3e8003bc0 2023-11-28T13:09:10.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3f9d9b700 1 -- 172.21.15.156:0/916537972 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e800bd00 con 0x7fd3fc0cd7c0 2023-11-28T13:09:10.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3fad9d700 1 --2- 172.21.15.156:0/916537972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3e8004d90 0x7fd3e8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:10.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3f3fff700 1 --2- 172.21.15.156:0/916537972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3fc0cd7c0 0x7fd3e8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:10.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3fa59c700 1 --2- 172.21.15.156:0/916537972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3e8003bc0 0x7fd3e8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:10.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3fad9d700 1 -- 172.21.15.156:0/916537972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3fc0cd7c0 msgr2=0x7fd3e8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:10.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3fad9d700 1 --2- 172.21.15.156:0/916537972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3fc0cd7c0 0x7fd3e8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:10.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3fad9d700 1 -- 172.21.15.156:0/916537972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3e8003bc0 msgr2=0x7fd3e8003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:10.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3fad9d700 1 --2- 172.21.15.156:0/916537972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3e8003bc0 0x7fd3e8003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:10.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3fad9d700 1 -- 172.21.15.156:0/916537972 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e80035b0 con 0x7fd3e8004d90 2023-11-28T13:09:10.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.414+0000 7fd3f3fff700 1 --2- 172.21.15.156:0/916537972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3fc0cd7c0 0x7fd3e8004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:10.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.415+0000 7fd3fad9d700 1 --2- 172.21.15.156:0/916537972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3e8004d90 0x7fd3e8005200 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009fc0 tx=0x7fd3e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb6088d09020134 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:10.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.415+0000 7fd3f9d9b700 1 -- 172.21.15.156:0/916537972 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f400ec30 con 0x7fd3e8004d90 2023-11-28T13:09:10.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.415+0000 7fd3f9d9b700 1 -- 172.21.15.156:0/916537972 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f400f040 con 0x7fd3e8004d90 2023-11-28T13:09:10.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.415+0000 7fd3f9d9b700 1 -- 172.21.15.156:0/916537972 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4014480 con 0x7fd3e8004d90 2023-11-28T13:09:10.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.415+0000 7fd400fff700 1 -- 172.21.15.156:0/916537972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3e8004d90 msgr2=0x7fd3e8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:10.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.415+0000 7fd400fff700 1 --2- 172.21.15.156:0/916537972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3e8004d90 0x7fd3e8005200 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009fc0 tx=0x7fd3e8005880 comp rx=0 tx=0).stop 2023-11-28T13:09:10.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.415+0000 7fd3fad9d700 1 -- 172.21.15.156:0/916537972 reap_dead start 2023-11-28T13:09:10.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.415+0000 7fd400fff700 1 -- 172.21.15.156:0/916537972 shutdown_connections 2023-11-28T13:09:10.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.415+0000 7fd400fff700 1 -- 172.21.15.156:0/916537972 >> 172.21.15.156:0/916537972 conn(0x7fd3fc072600 msgr2=0x7fd3fc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:10.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.416+0000 7fd400fff700 1 -- 172.21.15.156:0/916537972 shutdown_connections 2023-11-28T13:09:10.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.416+0000 7fd400fff700 1 -- 172.21.15.156:0/916537972 wait complete. 2023-11-28T13:09:10.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.417+0000 7fd400fff700 1 Processor -- start 2023-11-28T13:09:10.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.417+0000 7fd400fff700 1 -- start start 2023-11-28T13:09:10.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.418+0000 7fd400fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3e8004d90 0x7fd3fc0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:10.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.418+0000 7fd400fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3fc1573c0 0x7fd3fc0cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:10.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.418+0000 7fd400fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3fc156c20 0x7fd3fc15e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:10.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.418+0000 7fd400fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc0cbb80 con 0x7fd3fc156c20 2023-11-28T13:09:10.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.418+0000 7fd400fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc0cbd90 con 0x7fd3e8004d90 2023-11-28T13:09:10.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.418+0000 7fd400fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc0cc070 con 0x7fd3fc1573c0 2023-11-28T13:09:10.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.419+0000 7fd3fad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3fc156c20 0x7fd3fc15e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:10.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.419+0000 7fd3fad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3fc156c20 0x7fd3fc15e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49650/0 (socket says 172.21.15.156:49650) 2023-11-28T13:09:10.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.419+0000 7fd3fad9d700 1 -- 172.21.15.156:0/2894389400 learned_addr learned my addr 172.21.15.156:0/2894389400 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:10.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.419+0000 7fd3fa59c700 1 --2- 172.21.15.156:0/2894389400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3e8004d90 0x7fd3fc0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:10.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.419+0000 7fd3f3fff700 1 --2- 172.21.15.156:0/2894389400 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3fc1573c0 0x7fd3fc0cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:10.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.419+0000 7fd3fad9d700 1 -- 172.21.15.156:0/2894389400 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3fc1573c0 msgr2=0x7fd3fc0cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:10.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.419+0000 7fd3fad9d700 1 --2- 172.21.15.156:0/2894389400 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3fc1573c0 0x7fd3fc0cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:10.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.419+0000 7fd3fad9d700 1 -- 172.21.15.156:0/2894389400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3e8004d90 msgr2=0x7fd3fc0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:10.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.419+0000 7fd3fad9d700 1 --2- 172.21.15.156:0/2894389400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3e8004d90 0x7fd3fc0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:10.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.419+0000 7fd3fad9d700 1 -- 172.21.15.156:0/2894389400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3fc0d1300 con 0x7fd3fc156c20 2023-11-28T13:09:10.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.419+0000 7fd3fa59c700 1 --2- 172.21.15.156:0/2894389400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3e8004d90 0x7fd3fc0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:10.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.419+0000 7fd3fad9d700 1 --2- 172.21.15.156:0/2894389400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3fc156c20 0x7fd3fc15e960 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4017040 tx=0x7fd3f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:10.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.419+0000 7fd3f2ffd700 1 -- 172.21.15.156:0/2894389400 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4007000 con 0x7fd3fc156c20 2023-11-28T13:09:10.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.419+0000 7fd3f2ffd700 1 -- 172.21.15.156:0/2894389400 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f4003e50 con 0x7fd3fc156c20 2023-11-28T13:09:10.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.419+0000 7fd3f2ffd700 1 -- 172.21.15.156:0/2894389400 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4014b50 con 0x7fd3fc156c20 2023-11-28T13:09:10.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.420+0000 7fd400fff700 1 -- 172.21.15.156:0/2894389400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3fc0b5520 con 0x7fd3fc156c20 2023-11-28T13:09:10.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.420+0000 7fd400fff700 1 -- 172.21.15.156:0/2894389400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3fc0d15e0 con 0x7fd3fc156c20 2023-11-28T13:09:10.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.421+0000 7fd400fff700 1 -- 172.21.15.156:0/2894389400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3fc0745c0 con 0x7fd3fc156c20 2023-11-28T13:09:10.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.421+0000 7fd3f2ffd700 1 -- 172.21.15.156:0/2894389400 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd3f4014cb0 con 0x7fd3fc156c20 2023-11-28T13:09:10.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.422+0000 7fd3f2ffd700 1 --2- 172.21.15.156:0/2894389400 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd3dc068320 0x7fd3dc06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:10.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.422+0000 7fd3f2ffd700 1 -- 172.21.15.156:0/2894389400 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd3f4087a30 con 0x7fd3fc156c20 2023-11-28T13:09:10.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.423+0000 7fd3fa59c700 1 --2- 172.21.15.156:0/2894389400 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd3dc068320 0x7fd3dc06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:10.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.423+0000 7fd3fa59c700 1 --2- 172.21.15.156:0/2894389400 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd3dc068320 0x7fd3dc06a7d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec001eb0 tx=0x7fd3ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:10.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.426+0000 7fd3f2ffd700 1 -- 172.21.15.156:0/2894389400 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd3f40541a0 con 0x7fd3fc156c20 2023-11-28T13:09:10.590 INFO:tasks.ceph.mds.c.smithi156.stderr:2023-11-28T13:09:10.590+0000 7f01f9ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:09:10.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:10.661+0000 7fd400fff700 1 -- 172.21.15.156:0/2894389400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd3fc0d08a0 con 0x7fd3fc156c20 2023-11-28T13:09:11.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.190+0000 7fd3f2ffd700 1 -- 172.21.15.156:0/2894389400 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd3f40180c0 con 0x7fd3fc156c20 2023-11-28T13:09:11.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.193+0000 7fd400fff700 1 -- 172.21.15.156:0/2894389400 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd3dc068320 msgr2=0x7fd3dc06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:11.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.193+0000 7fd400fff700 1 --2- 172.21.15.156:0/2894389400 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd3dc068320 0x7fd3dc06a7d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec001eb0 tx=0x7fd3ec009040 comp rx=0 tx=0).stop 2023-11-28T13:09:11.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.193+0000 7fd400fff700 1 -- 172.21.15.156:0/2894389400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3fc156c20 msgr2=0x7fd3fc15e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:11.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.193+0000 7fd400fff700 1 --2- 172.21.15.156:0/2894389400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3fc156c20 0x7fd3fc15e960 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4017040 tx=0x7fd3f4009750 comp rx=0 tx=0).stop 2023-11-28T13:09:11.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.193+0000 7fd400fff700 1 -- 172.21.15.156:0/2894389400 shutdown_connections 2023-11-28T13:09:11.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.193+0000 7fd400fff700 1 --2- 172.21.15.156:0/2894389400 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd3dc068320 0x7fd3dc06a7d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.193+0000 7fd400fff700 1 --2- 172.21.15.156:0/2894389400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3fc156c20 0x7fd3fc15e960 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.193+0000 7fd400fff700 1 --2- 172.21.15.156:0/2894389400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3e8004d90 0x7fd3fc0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.193+0000 7fd400fff700 1 --2- 172.21.15.156:0/2894389400 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3fc1573c0 0x7fd3fc0cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.193+0000 7fd400fff700 1 -- 172.21.15.156:0/2894389400 >> 172.21.15.156:0/2894389400 conn(0x7fd3fc072600 msgr2=0x7fd3fc160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:11.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.193+0000 7fd400fff700 1 -- 172.21.15.156:0/2894389400 shutdown_connections 2023-11-28T13:09:11.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.193+0000 7fd400fff700 1 -- 172.21.15.156:0/2894389400 wait complete. 2023-11-28T13:09:11.212 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T13:09:11.213 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:09:11.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.409+0000 7f3564fff700 1 Processor -- start 2023-11-28T13:09:11.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.409+0000 7f3564fff700 1 -- start start 2023-11-28T13:09:11.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.410+0000 7f3564fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f356015c9b0 0x7f35600b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:11.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.410+0000 7f3564fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3560161340 0x7f35600d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:11.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.410+0000 7f3564fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35600d2320 0x7f35600cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:11.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.410+0000 7f3564fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35600b5800 con 0x7f356015c9b0 2023-11-28T13:09:11.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.410+0000 7f3564fff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35600d1950 con 0x7f3560161340 2023-11-28T13:09:11.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.410+0000 7f3564fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35600b5520 con 0x7f35600d2320 2023-11-28T13:09:11.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.410+0000 7f355e59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f356015c9b0 0x7f35600b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:11.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.410+0000 7f355e59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f356015c9b0 0x7f35600b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49666/0 (socket says 172.21.15.156:49666) 2023-11-28T13:09:11.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.410+0000 7f355e59c700 1 -- 172.21.15.156:0/3643790878 learned_addr learned my addr 172.21.15.156:0/3643790878 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:11.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.411+0000 7f3557fff700 1 --2- 172.21.15.156:0/3643790878 >> v2:172.21.15.186:3300/0 conn(0x7f3560161340 0x7f35600d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:11.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.411+0000 7f355ed9d700 1 --2- 172.21.15.156:0/3643790878 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35600d2320 0x7f35600cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:11.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.411+0000 7f355e59c700 1 -- 172.21.15.156:0/3643790878 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35600d2320 msgr2=0x7f35600cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:11.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.411+0000 7f355e59c700 1 --2- 172.21.15.156:0/3643790878 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35600d2320 0x7f35600cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.411+0000 7f355e59c700 1 -- 172.21.15.156:0/3643790878 >> v2:172.21.15.186:3300/0 conn(0x7f3560161340 msgr2=0x7f35600d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:11.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.411+0000 7f355e59c700 1 --2- 172.21.15.156:0/3643790878 >> v2:172.21.15.186:3300/0 conn(0x7f3560161340 0x7f35600d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.411+0000 7f355e59c700 1 -- 172.21.15.156:0/3643790878 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35600d1300 con 0x7f356015c9b0 2023-11-28T13:09:11.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.411+0000 7f355ed9d700 1 --2- 172.21.15.156:0/3643790878 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35600d2320 0x7f35600cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:11.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.411+0000 7f3557fff700 1 --2- 172.21.15.156:0/3643790878 >> v2:172.21.15.186:3300/0 conn(0x7f3560161340 0x7f35600d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:11.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.411+0000 7f355e59c700 1 --2- 172.21.15.156:0/3643790878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f356015c9b0 0x7f35600b4bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f35600685c0 tx=0x7f3550009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5953b8a6a9b23780 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:11.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.411+0000 7f355dd9b700 1 -- 172.21.15.156:0/3643790878 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355000b870 con 0x7f356015c9b0 2023-11-28T13:09:11.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.411+0000 7f355dd9b700 1 -- 172.21.15.156:0/3643790878 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355000b9d0 con 0x7f356015c9b0 2023-11-28T13:09:11.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.411+0000 7f355dd9b700 1 -- 172.21.15.156:0/3643790878 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355000bd60 con 0x7f356015c9b0 2023-11-28T13:09:11.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.412+0000 7f3564fff700 1 -- 172.21.15.156:0/3643790878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f356015c9b0 msgr2=0x7f35600b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:11.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.412+0000 7f3564fff700 1 --2- 172.21.15.156:0/3643790878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f356015c9b0 0x7f35600b4bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f35600685c0 tx=0x7f3550009bd0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.412+0000 7f3564fff700 1 -- 172.21.15.156:0/3643790878 shutdown_connections 2023-11-28T13:09:11.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.412+0000 7f3564fff700 1 --2- 172.21.15.156:0/3643790878 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f35600d2320 0x7f35600cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.412+0000 7f3564fff700 1 --2- 172.21.15.156:0/3643790878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f356015c9b0 0x7f35600b4bf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.412+0000 7f3564fff700 1 --2- 172.21.15.156:0/3643790878 >> v2:172.21.15.186:3300/0 conn(0x7f3560161340 0x7f35600d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.412+0000 7f3564fff700 1 -- 172.21.15.156:0/3643790878 >> 172.21.15.156:0/3643790878 conn(0x7f3560072600 msgr2=0x7f35600cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:11.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.412+0000 7f3564fff700 1 -- 172.21.15.156:0/3643790878 shutdown_connections 2023-11-28T13:09:11.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.412+0000 7f3564fff700 1 -- 172.21.15.156:0/3643790878 wait complete. 2023-11-28T13:09:11.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.414+0000 7f3564fff700 1 Processor -- start 2023-11-28T13:09:11.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.415+0000 7f3564fff700 1 -- start start 2023-11-28T13:09:11.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.416+0000 7f3564fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35600d2320 0x7f35601e8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:11.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.416+0000 7f3564fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f356015c9b0 0x7f35601f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:11.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.416+0000 7f3564fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3560161340 0x7f3560151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:11.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.416+0000 7f3564fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35601533c0 con 0x7f356015c9b0 2023-11-28T13:09:11.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.416+0000 7f3564fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35601524c0 con 0x7f35600d2320 2023-11-28T13:09:11.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.416+0000 7f3564fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3560152740 con 0x7f3560161340 2023-11-28T13:09:11.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.416+0000 7f3557fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f356015c9b0 0x7f35601f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:11.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.416+0000 7f3557fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f356015c9b0 0x7f35601f5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49674/0 (socket says 172.21.15.156:49674) 2023-11-28T13:09:11.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.416+0000 7f3557fff700 1 -- 172.21.15.156:0/628843032 learned_addr learned my addr 172.21.15.156:0/628843032 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:11.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.416+0000 7f355ed9d700 1 --2- 172.21.15.156:0/628843032 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3560161340 0x7f3560151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:11.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.416+0000 7f355e59c700 1 --2- 172.21.15.156:0/628843032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35600d2320 0x7f35601e8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:11.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.417+0000 7f3557fff700 1 -- 172.21.15.156:0/628843032 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3560161340 msgr2=0x7f3560151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:11.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.417+0000 7f3557fff700 1 --2- 172.21.15.156:0/628843032 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3560161340 0x7f3560151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.417+0000 7f3557fff700 1 -- 172.21.15.156:0/628843032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35600d2320 msgr2=0x7f35601e8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:11.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.417+0000 7f3557fff700 1 --2- 172.21.15.156:0/628843032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35600d2320 0x7f35601e8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.417+0000 7f3557fff700 1 -- 172.21.15.156:0/628843032 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3550009550 con 0x7f356015c9b0 2023-11-28T13:09:11.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.417+0000 7f355e59c700 1 --2- 172.21.15.156:0/628843032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35600d2320 0x7f35601e8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:11.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.417+0000 7f3557fff700 1 --2- 172.21.15.156:0/628843032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f356015c9b0 0x7f35601f5980 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f354800e810 tx=0x7f354800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:11.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.417+0000 7f3556ffd700 1 -- 172.21.15.156:0/628843032 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548013610 con 0x7f356015c9b0 2023-11-28T13:09:11.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.417+0000 7f3556ffd700 1 -- 172.21.15.156:0/628843032 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3548013770 con 0x7f356015c9b0 2023-11-28T13:09:11.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.417+0000 7f3556ffd700 1 -- 172.21.15.156:0/628843032 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548013b20 con 0x7f356015c9b0 2023-11-28T13:09:11.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.417+0000 7f3564fff700 1 -- 172.21.15.156:0/628843032 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35600b5130 con 0x7f356015c9b0 2023-11-28T13:09:11.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.417+0000 7f3564fff700 1 -- 172.21.15.156:0/628843032 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35600d15e0 con 0x7f356015c9b0 2023-11-28T13:09:11.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.418+0000 7f3564fff700 1 -- 172.21.15.156:0/628843032 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35600b52c0 con 0x7f356015c9b0 2023-11-28T13:09:11.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.419+0000 7f3556ffd700 1 -- 172.21.15.156:0/628843032 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3548023070 con 0x7f356015c9b0 2023-11-28T13:09:11.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.420+0000 7f3556ffd700 1 --2- 172.21.15.156:0/628843032 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3540068320 0x7f354006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:11.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.420+0000 7f3556ffd700 1 -- 172.21.15.156:0/628843032 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3548085940 con 0x7f356015c9b0 2023-11-28T13:09:11.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.420+0000 7f355e59c700 1 --2- 172.21.15.156:0/628843032 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3540068320 0x7f354006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:11.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.421+0000 7f355e59c700 1 --2- 172.21.15.156:0/628843032 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3540068320 0x7f354006a7d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3550000c00 tx=0x7f3550008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:11.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.424+0000 7f3556ffd700 1 -- 172.21.15.156:0/628843032 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f35480520b0 con 0x7f356015c9b0 2023-11-28T13:09:11.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.657+0000 7f3564fff700 1 -- 172.21.15.156:0/628843032 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35600745c0 con 0x7f356015c9b0 2023-11-28T13:09:11.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.657+0000 7f3556ffd700 1 -- 172.21.15.156:0/628843032 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f35480556d0 con 0x7f356015c9b0 2023-11-28T13:09:11.658 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:11.658 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1128367125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1128367125},{"type":"v1","addr":"172.21.15.156:6835","nonce":1128367125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4942,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/4096182008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4096182008},{"type":"v1","addr":"172.21.15.186:6835","nonce":4096182008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4945,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/561812165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":561812165},{"type":"v1","addr":"172.21.15.186:6837","nonce":561812165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:09:08.167940+0000","modified":"2023-11-28T13:09:11.184420+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5013":{"gid":5013,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6834/3471743149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3471743149},{"type":"v1","addr":"172.21.15.156:6834","nonce":3471743149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:09:11.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.659+0000 7f3564fff700 1 -- 172.21.15.156:0/628843032 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3540068320 msgr2=0x7f354006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:11.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.659+0000 7f3564fff700 1 --2- 172.21.15.156:0/628843032 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3540068320 0x7f354006a7d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3550000c00 tx=0x7f3550008030 comp rx=0 tx=0).stop 2023-11-28T13:09:11.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.659+0000 7f3564fff700 1 -- 172.21.15.156:0/628843032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f356015c9b0 msgr2=0x7f35601f5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:11.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.659+0000 7f3564fff700 1 --2- 172.21.15.156:0/628843032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f356015c9b0 0x7f35601f5980 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f354800e810 tx=0x7f354800ebd0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.659+0000 7f3564fff700 1 -- 172.21.15.156:0/628843032 shutdown_connections 2023-11-28T13:09:11.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.659+0000 7f3564fff700 1 --2- 172.21.15.156:0/628843032 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3540068320 0x7f354006a7d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.659+0000 7f3564fff700 1 --2- 172.21.15.156:0/628843032 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3560161340 0x7f3560151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.659+0000 7f3564fff700 1 --2- 172.21.15.156:0/628843032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35600d2320 0x7f35601e8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.659+0000 7f3564fff700 1 --2- 172.21.15.156:0/628843032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f356015c9b0 0x7f35601f5980 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.659+0000 7f3564fff700 1 -- 172.21.15.156:0/628843032 >> 172.21.15.156:0/628843032 conn(0x7f3560072600 msgr2=0x7f356006a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:11.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.660+0000 7f3564fff700 1 -- 172.21.15.156:0/628843032 shutdown_connections 2023-11-28T13:09:11.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.660+0000 7f3564fff700 1 -- 172.21.15.156:0/628843032 wait complete. 2023-11-28T13:09:11.661 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2023-11-28T13:09:11.678 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:09:11.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.879+0000 7fbe5c9ff700 1 Processor -- start 2023-11-28T13:09:11.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.879+0000 7fbe5c9ff700 1 -- start start 2023-11-28T13:09:11.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.880+0000 7fbe5c9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe58161340 0x7fbe580d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:11.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.880+0000 7fbe5c9ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbe580d2320 0x7fbe580cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:11.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.880+0000 7fbe5c9ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe580d1950 con 0x7fbe5815c9b0 2023-11-28T13:09:11.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.881+0000 7fbe5c9ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe580b5800 con 0x7fbe580d2320 2023-11-28T13:09:11.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.881+0000 7fbe5c9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580b5300 con 0x7fbe58161340 2023-11-28T13:09:11.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.881+0000 7fbe5659c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fbe5815c9b0 0x7fbe580b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:43482/0 (socket says 172.21.15.156:43482) 2023-11-28T13:09:11.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.881+0000 7fbe5659c700 1 -- 172.21.15.156:0/3210246888 learned_addr learned my addr 172.21.15.156:0/3210246888 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:11.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.881+0000 7fbe4ffff700 1 --2- 172.21.15.156:0/3210246888 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe58161340 0x7fbe580d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:11.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.881+0000 7fbe56d9d700 1 --2- 172.21.15.156:0/3210246888 >> v2:172.21.15.186:3300/0 conn(0x7fbe580d2320 0x7fbe580cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:11.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.881+0000 7fbe55d9b700 1 -- 172.21.15.156:0/3210246888 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024443512 0 0) 0x7fbe580d1950 con 0x7fbe5815c9b0 2023-11-28T13:09:11.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.881+0000 7fbe55d9b700 1 -- 172.21.15.156:0/3210246888 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe440035b0 con 0x7fbe5815c9b0 2023-11-28T13:09:11.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.881+0000 7fbe4ffff700 1 -- 172.21.15.156:0/3210246888 >> v2:172.21.15.186:3300/0 conn(0x7fbe580d2320 msgr2=0x7fbe580cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:11.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.881+0000 7fbe4ffff700 1 --2- 172.21.15.156:0/3210246888 >> v2:172.21.15.186:3300/0 conn(0x7fbe580d2320 0x7fbe580cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.881+0000 7fbe55d9b700 1 -- 172.21.15.156:0/3210246888 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 515351409 0 0) 0x7fbe440035b0 con 0x7fbe5815c9b0 2023-11-28T13:09:11.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.881+0000 7fbe4ffff700 1 -- 172.21.15.156:0/3210246888 >> v1:172.21.15.156:6789/0 conn(0x7fbe5815c9b0 legacy=0x7fbe580b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:11.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.881+0000 7fbe4ffff700 1 -- 172.21.15.156:0/3210246888 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe580b5520 con 0x7fbe58161340 2023-11-28T13:09:11.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.882+0000 7fbe4ffff700 1 --2- 172.21.15.156:0/3210246888 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe58161340 0x7fbe580d1de0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009840 tx=0x7fbe40009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=537ed26a08587a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:11.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.882+0000 7fbe55d9b700 1 -- 172.21.15.156:0/3210246888 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48001670 con 0x7fbe58161340 2023-11-28T13:09:11.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.882+0000 7fbe55d9b700 1 -- 172.21.15.156:0/3210246888 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe40004030 con 0x7fbe58161340 2023-11-28T13:09:11.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.882+0000 7fbe55d9b700 1 -- 172.21.15.156:0/3210246888 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe400043a0 con 0x7fbe58161340 2023-11-28T13:09:11.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.883+0000 7fbe5c9ff700 1 -- 172.21.15.156:0/3210246888 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe58161340 msgr2=0x7fbe580d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:11.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.883+0000 7fbe5c9ff700 1 --2- 172.21.15.156:0/3210246888 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe58161340 0x7fbe580d1de0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009840 tx=0x7fbe40009b50 comp rx=0 tx=0).stop 2023-11-28T13:09:11.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.883+0000 7fbe5c9ff700 1 -- 172.21.15.156:0/3210246888 shutdown_connections 2023-11-28T13:09:11.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.883+0000 7fbe5c9ff700 1 --2- 172.21.15.156:0/3210246888 >> v2:172.21.15.186:3300/0 conn(0x7fbe580d2320 0x7fbe580cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.883+0000 7fbe5c9ff700 1 --2- 172.21.15.156:0/3210246888 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe58161340 0x7fbe580d1de0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.883+0000 7fbe5c9ff700 1 -- 172.21.15.156:0/3210246888 >> 172.21.15.156:0/3210246888 conn(0x7fbe58072600 msgr2=0x7fbe580cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:11.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.883+0000 7fbe5c9ff700 1 -- 172.21.15.156:0/3210246888 shutdown_connections 2023-11-28T13:09:11.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.883+0000 7fbe5c9ff700 1 -- 172.21.15.156:0/3210246888 wait complete. 2023-11-28T13:09:11.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.884+0000 7fbe5c9ff700 1 Processor -- start 2023-11-28T13:09:11.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.885+0000 7fbe5c9ff700 1 -- start start 2023-11-28T13:09:11.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.885+0000 7fbe5c9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe580d2320 0x7fbe58150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:11.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.886+0000 7fbe5c9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe5815c9b0 0x7fbe580c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:11.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.886+0000 7fbe5c9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe58161340 0x7fbe58153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:11.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.886+0000 7fbe5c9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580c7b70 con 0x7fbe5815c9b0 2023-11-28T13:09:11.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.886+0000 7fbe5c9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580c7e50 con 0x7fbe580d2320 2023-11-28T13:09:11.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.886+0000 7fbe5c9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580c8130 con 0x7fbe58161340 2023-11-28T13:09:11.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.886+0000 7fbe4ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe5815c9b0 0x7fbe580c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:11.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.886+0000 7fbe4ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe5815c9b0 0x7fbe580c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49676/0 (socket says 172.21.15.156:49676) 2023-11-28T13:09:11.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.886+0000 7fbe4ffff700 1 -- 172.21.15.156:0/188978934 learned_addr learned my addr 172.21.15.156:0/188978934 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:11.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.886+0000 7fbe56d9d700 1 --2- 172.21.15.156:0/188978934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe58161340 0x7fbe58153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:11.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.886+0000 7fbe5659c700 1 --2- 172.21.15.156:0/188978934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe580d2320 0x7fbe58150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:11.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.886+0000 7fbe4ffff700 1 -- 172.21.15.156:0/188978934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe58161340 msgr2=0x7fbe58153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:11.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.886+0000 7fbe4ffff700 1 --2- 172.21.15.156:0/188978934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe58161340 0x7fbe58153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.886+0000 7fbe4ffff700 1 -- 172.21.15.156:0/188978934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe580d2320 msgr2=0x7fbe58150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:11.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.886+0000 7fbe4ffff700 1 --2- 172.21.15.156:0/188978934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe580d2320 0x7fbe58150bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:11.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.886+0000 7fbe4ffff700 1 -- 172.21.15.156:0/188978934 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe400094a0 con 0x7fbe5815c9b0 2023-11-28T13:09:11.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.886+0000 7fbe5659c700 1 --2- 172.21.15.156:0/188978934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe580d2320 0x7fbe58150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:11.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.887+0000 7fbe4ffff700 1 --2- 172.21.15.156:0/188978934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe5815c9b0 0x7fbe580c94e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbe40000c00 tx=0x7fbe40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:11.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.887+0000 7fbe4effd700 1 -- 172.21.15.156:0/188978934 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe40003df0 con 0x7fbe5815c9b0 2023-11-28T13:09:11.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.887+0000 7fbe4effd700 1 -- 172.21.15.156:0/188978934 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4001f070 con 0x7fbe5815c9b0 2023-11-28T13:09:11.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.887+0000 7fbe5c9ff700 1 -- 172.21.15.156:0/188978934 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe580d15e0 con 0x7fbe5815c9b0 2023-11-28T13:09:11.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.887+0000 7fbe4effd700 1 -- 172.21.15.156:0/188978934 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe40011870 con 0x7fbe5815c9b0 2023-11-28T13:09:11.890 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.887+0000 7fbe5c9ff700 1 -- 172.21.15.156:0/188978934 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe580d1300 con 0x7fbe5815c9b0 2023-11-28T13:09:11.890 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.888+0000 7fbe5c9ff700 1 -- 172.21.15.156:0/188978934 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe5815cd80 con 0x7fbe5815c9b0 2023-11-28T13:09:11.890 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.889+0000 7fbe4effd700 1 -- 172.21.15.156:0/188978934 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbe4002a020 con 0x7fbe5815c9b0 2023-11-28T13:09:11.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.890+0000 7fbe4effd700 1 --2- 172.21.15.156:0/188978934 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbe380682f0 0x7fbe3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:11.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.890+0000 7fbe4effd700 1 -- 172.21.15.156:0/188978934 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbe40087260 con 0x7fbe5815c9b0 2023-11-28T13:09:11.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.890+0000 7fbe5659c700 1 --2- 172.21.15.156:0/188978934 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbe380682f0 0x7fbe3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:11.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.891+0000 7fbe5659c700 1 --2- 172.21.15.156:0/188978934 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbe380682f0 0x7fbe3806a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbe48000c00 tx=0x7fbe48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:11.894 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:11.894+0000 7fbe4effd700 1 -- 172.21.15.156:0/188978934 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbe400539d0 con 0x7fbe5815c9b0 2023-11-28T13:09:12.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.090+0000 7fbe5c9ff700 1 -- 172.21.15.156:0/188978934 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe58155a80 con 0x7fbe5815c9b0 2023-11-28T13:09:12.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.091+0000 7fbe4effd700 1 -- 172.21.15.156:0/188978934 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+17969 (secure 0 0 0) 0x7fbe40056ff0 con 0x7fbe5815c9b0 2023-11-28T13:09:12.092 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:12.092 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":35,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:09:09.170289+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:09:06.618698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T13:09:07.151554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:09:12.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.093+0000 7fbe5c9ff700 1 -- 172.21.15.156:0/188978934 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbe380682f0 msgr2=0x7fbe3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:12.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.093+0000 7fbe5c9ff700 1 --2- 172.21.15.156:0/188978934 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbe380682f0 0x7fbe3806a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbe48000c00 tx=0x7fbe48009040 comp rx=0 tx=0).stop 2023-11-28T13:09:12.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.093+0000 7fbe5c9ff700 1 -- 172.21.15.156:0/188978934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe5815c9b0 msgr2=0x7fbe580c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:12.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.093+0000 7fbe5c9ff700 1 --2- 172.21.15.156:0/188978934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe5815c9b0 0x7fbe580c94e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbe40000c00 tx=0x7fbe40004750 comp rx=0 tx=0).stop 2023-11-28T13:09:12.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.093+0000 7fbe5c9ff700 1 -- 172.21.15.156:0/188978934 shutdown_connections 2023-11-28T13:09:12.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.093+0000 7fbe5c9ff700 1 --2- 172.21.15.156:0/188978934 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbe380682f0 0x7fbe3806a7a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:12.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.094+0000 7fbe5c9ff700 1 --2- 172.21.15.156:0/188978934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe58161340 0x7fbe58153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:12.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.094+0000 7fbe5c9ff700 1 --2- 172.21.15.156:0/188978934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe580d2320 0x7fbe58150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:12.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.094+0000 7fbe5c9ff700 1 --2- 172.21.15.156:0/188978934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe5815c9b0 0x7fbe580c94e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:12.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.094+0000 7fbe5c9ff700 1 -- 172.21.15.156:0/188978934 >> 172.21.15.156:0/188978934 conn(0x7fbe58072600 msgr2=0x7fbe5815fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:12.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.094+0000 7fbe5c9ff700 1 -- 172.21.15.156:0/188978934 shutdown_connections 2023-11-28T13:09:12.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.094+0000 7fbe5c9ff700 1 -- 172.21.15.156:0/188978934 wait complete. 2023-11-28T13:09:12.113 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T13:09:12.113 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:09:12.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.311+0000 7efcbd9ff700 1 Processor -- start 2023-11-28T13:09:12.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.311+0000 7efcbd9ff700 1 -- start start 2023-11-28T13:09:12.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.312+0000 7efcbd9ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efcb815c9b0 0x7efcb80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:12.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.312+0000 7efcbd9ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcb80d1950 con 0x7efcb8161340 2023-11-28T13:09:12.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.312+0000 7efcbd9ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcb80d24f0 con 0x7efcb80cd7c0 2023-11-28T13:09:12.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.312+0000 7efcbd9ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb80b5800 con 0x7efcb815c9b0 2023-11-28T13:09:12.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.313+0000 7efcb6ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efcb815c9b0 0x7efcb80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:12.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.313+0000 7efcb6ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efcb815c9b0 0x7efcb80b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49680/0 (socket says 172.21.15.156:49680) 2023-11-28T13:09:12.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.313+0000 7efcaffff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7efcb8161340 0x7efcb80d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:43490/0 (socket says 172.21.15.156:43490) 2023-11-28T13:09:12.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.313+0000 7efcb6ffd700 1 -- 172.21.15.156:0/3638827390 learned_addr learned my addr 172.21.15.156:0/3638827390 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:12.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.313+0000 7efcb6ffd700 1 -- 172.21.15.156:0/3638827390 >> v1:172.21.15.186:6789/0 conn(0x7efcb80cd7c0 legacy=0x7efcb80b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:12.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.313+0000 7efcb6ffd700 1 -- 172.21.15.156:0/3638827390 >> v1:172.21.15.156:6789/0 conn(0x7efcb8161340 legacy=0x7efcb80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:12.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.313+0000 7efcb6ffd700 1 -- 172.21.15.156:0/3638827390 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb80b5520 con 0x7efcb815c9b0 2023-11-28T13:09:12.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.313+0000 7efcb6ffd700 1 --2- 172.21.15.156:0/3638827390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb815c9b0 0x7efcb80b4bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efca80098f0 tx=0x7efca8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ec33cc42c9f838 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:12.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.314+0000 7efcb67fc700 1 -- 172.21.15.156:0/3638827390 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca800b8a0 con 0x7efcb815c9b0 2023-11-28T13:09:12.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.314+0000 7efcb67fc700 1 -- 172.21.15.156:0/3638827390 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca800ba00 con 0x7efcb815c9b0 2023-11-28T13:09:12.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.314+0000 7efcb67fc700 1 -- 172.21.15.156:0/3638827390 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca800bd90 con 0x7efcb815c9b0 2023-11-28T13:09:12.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.314+0000 7efcbd9ff700 1 -- 172.21.15.156:0/3638827390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb815c9b0 msgr2=0x7efcb80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:12.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.314+0000 7efcbd9ff700 1 --2- 172.21.15.156:0/3638827390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb815c9b0 0x7efcb80b4bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efca80098f0 tx=0x7efca8009c00 comp rx=0 tx=0).stop 2023-11-28T13:09:12.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.314+0000 7efcbd9ff700 1 -- 172.21.15.156:0/3638827390 shutdown_connections 2023-11-28T13:09:12.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.314+0000 7efcbd9ff700 1 --2- 172.21.15.156:0/3638827390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb815c9b0 0x7efcb80b4bf0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:12.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.314+0000 7efcbd9ff700 1 -- 172.21.15.156:0/3638827390 >> 172.21.15.156:0/3638827390 conn(0x7efcb8072600 msgr2=0x7efcb80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:12.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.314+0000 7efcbd9ff700 1 -- 172.21.15.156:0/3638827390 shutdown_connections 2023-11-28T13:09:12.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.314+0000 7efcbd9ff700 1 -- 172.21.15.156:0/3638827390 wait complete. 2023-11-28T13:09:12.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.317+0000 7efcbd9ff700 1 Processor -- start 2023-11-28T13:09:12.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.317+0000 7efcbd9ff700 1 -- start start 2023-11-28T13:09:12.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.318+0000 7efcbd9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efcb80cd7c0 0x7efcb80b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:12.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.318+0000 7efcbd9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb815c9b0 0x7efcb80cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:12.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.318+0000 7efcbd9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efcb8161340 0x7efcb8143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:12.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.318+0000 7efcbd9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb8157a20 con 0x7efcb815c9b0 2023-11-28T13:09:12.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.318+0000 7efcbd9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb80c96c0 con 0x7efcb8161340 2023-11-28T13:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.318+0000 7efcbd9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb80c99a0 con 0x7efcb80cd7c0 2023-11-28T13:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.318+0000 7efcaffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb815c9b0 0x7efcb80cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.318+0000 7efcaffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb815c9b0 0x7efcb80cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49694/0 (socket says 172.21.15.156:49694) 2023-11-28T13:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.318+0000 7efcaffff700 1 -- 172.21.15.156:0/1000208305 learned_addr learned my addr 172.21.15.156:0/1000208305 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.319+0000 7efcb6ffd700 1 --2- 172.21.15.156:0/1000208305 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efcb80cd7c0 0x7efcb80b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.319+0000 7efcb77fe700 1 --2- 172.21.15.156:0/1000208305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efcb8161340 0x7efcb8143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.319+0000 7efcaffff700 1 -- 172.21.15.156:0/1000208305 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efcb80cd7c0 msgr2=0x7efcb80b7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.319+0000 7efcaffff700 1 --2- 172.21.15.156:0/1000208305 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efcb80cd7c0 0x7efcb80b7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.319+0000 7efcaffff700 1 -- 172.21.15.156:0/1000208305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efcb8161340 msgr2=0x7efcb8143070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:12.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.319+0000 7efcaffff700 1 --2- 172.21.15.156:0/1000208305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efcb8161340 0x7efcb8143070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:12.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.319+0000 7efcaffff700 1 -- 172.21.15.156:0/1000208305 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca8009550 con 0x7efcb815c9b0 2023-11-28T13:09:12.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.319+0000 7efcb77fe700 1 --2- 172.21.15.156:0/1000208305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efcb8161340 0x7efcb8143070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:12.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.319+0000 7efcaffff700 1 --2- 172.21.15.156:0/1000208305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb815c9b0 0x7efcb80cc840 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efca000c520 tx=0x7efca000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:12.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.319+0000 7efcb4ff9700 1 -- 172.21.15.156:0/1000208305 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca0005cb0 con 0x7efcb815c9b0 2023-11-28T13:09:12.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.319+0000 7efcb4ff9700 1 -- 172.21.15.156:0/1000208305 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca0005e10 con 0x7efcb815c9b0 2023-11-28T13:09:12.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.319+0000 7efcb4ff9700 1 -- 172.21.15.156:0/1000208305 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca00072d0 con 0x7efcb815c9b0 2023-11-28T13:09:12.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.319+0000 7efcbd9ff700 1 -- 172.21.15.156:0/1000208305 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb80d15e0 con 0x7efcb815c9b0 2023-11-28T13:09:12.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.320+0000 7efcbd9ff700 1 -- 172.21.15.156:0/1000208305 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb80d1300 con 0x7efcb815c9b0 2023-11-28T13:09:12.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.321+0000 7efcbd9ff700 1 -- 172.21.15.156:0/1000208305 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb80b5130 con 0x7efcb815c9b0 2023-11-28T13:09:12.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.321+0000 7efcb4ff9700 1 -- 172.21.15.156:0/1000208305 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efca0019030 con 0x7efcb815c9b0 2023-11-28T13:09:12.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.322+0000 7efcb4ff9700 1 --2- 172.21.15.156:0/1000208305 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efc980682f0 0x7efc9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:12.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.322+0000 7efcb4ff9700 1 -- 172.21.15.156:0/1000208305 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efca0084e80 con 0x7efcb815c9b0 2023-11-28T13:09:12.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.323+0000 7efcb6ffd700 1 --2- 172.21.15.156:0/1000208305 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efc980682f0 0x7efc9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:12.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.324+0000 7efcb6ffd700 1 --2- 172.21.15.156:0/1000208305 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efc980682f0 0x7efc9806a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efca8014040 tx=0x7efca8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:12.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.326+0000 7efcb4ff9700 1 -- 172.21.15.156:0/1000208305 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efca004b5e0 con 0x7efcb815c9b0 2023-11-28T13:09:12.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.513+0000 7efcbd9ff700 1 -- 172.21.15.156:0/1000208305 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efcb80c9bc0 con 0x7efc980682f0 2023-11-28T13:09:12.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.524+0000 7efcb4ff9700 1 -- 172.21.15.156:0/1000208305 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312116 (secure 0 0 0) 0x7efcb80c9bc0 con 0x7efc980682f0 2023-11-28T13:09:12.529 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:12.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.528+0000 7efcbd9ff700 1 -- 172.21.15.156:0/1000208305 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efc980682f0 msgr2=0x7efc9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:12.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.528+0000 7efcbd9ff700 1 --2- 172.21.15.156:0/1000208305 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efc980682f0 0x7efc9806a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efca8014040 tx=0x7efca8008030 comp rx=0 tx=0).stop 2023-11-28T13:09:12.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.528+0000 7efcbd9ff700 1 -- 172.21.15.156:0/1000208305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb815c9b0 msgr2=0x7efcb80cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:12.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.528+0000 7efcbd9ff700 1 --2- 172.21.15.156:0/1000208305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb815c9b0 0x7efcb80cc840 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efca000c520 tx=0x7efca000c830 comp rx=0 tx=0).stop 2023-11-28T13:09:12.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.528+0000 7efcbd9ff700 1 -- 172.21.15.156:0/1000208305 shutdown_connections 2023-11-28T13:09:12.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.528+0000 7efcbd9ff700 1 --2- 172.21.15.156:0/1000208305 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efc980682f0 0x7efc9806a7a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:12.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.528+0000 7efcbd9ff700 1 --2- 172.21.15.156:0/1000208305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efcb8161340 0x7efcb8143070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:12.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.528+0000 7efcbd9ff700 1 --2- 172.21.15.156:0/1000208305 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efcb80cd7c0 0x7efcb80b7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:12.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.528+0000 7efcbd9ff700 1 --2- 172.21.15.156:0/1000208305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb815c9b0 0x7efcb80cc840 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:12.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.528+0000 7efcbd9ff700 1 -- 172.21.15.156:0/1000208305 >> 172.21.15.156:0/1000208305 conn(0x7efcb8072600 msgr2=0x7efcb80d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:12.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.528+0000 7efcbd9ff700 1 -- 172.21.15.156:0/1000208305 shutdown_connections 2023-11-28T13:09:12.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.528+0000 7efcbd9ff700 1 -- 172.21.15.156:0/1000208305 wait complete. 2023-11-28T13:09:12.533 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T13:09:12.547 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2023-11-28T13:09:12.031135+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":86180,"kb_used_data":4324,"kb_used_omap":0,"kb_used_meta":81792,"kb_avail":754888540,"statfs":{"total":773094113280,"available":773005864960,"internally_reserved":0,"allocated":4427776,"data_stored":2506338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":83755008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.358509"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282831+0000","last_change":"2023-11-28T13:09:08.192164+0000","last_active":"2023-11-28T13:09:09.282831+0000","last_peered":"2023-11-28T13:09:09.282831+0000","last_clean":"2023-11-28T13:09:09.282831+0000","last_became_active":"2023-11-28T13:09:08.191924+0000","last_became_peered":"2023-11-28T13:09:08.191924+0000","last_unstale":"2023-11-28T13:09:09.282831+0000","last_undegraded":"2023-11-28T13:09:09.282831+0000","last_fullsized":"2023-11-28T13:09:09.282831+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:11.552323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283744+0000","last_change":"2023-11-28T13:09:08.196036+0000","last_active":"2023-11-28T13:09:09.283744+0000","last_peered":"2023-11-28T13:09:09.283744+0000","last_clean":"2023-11-28T13:09:09.283744+0000","last_became_active":"2023-11-28T13:09:08.195790+0000","last_became_peered":"2023-11-28T13:09:08.195790+0000","last_unstale":"2023-11-28T13:09:09.283744+0000","last_undegraded":"2023-11-28T13:09:09.283744+0000","last_fullsized":"2023-11-28T13:09:09.283744+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:25:26.519823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.284286+0000","last_change":"2023-11-28T13:09:08.193619+0000","last_active":"2023-11-28T13:09:09.284286+0000","last_peered":"2023-11-28T13:09:09.284286+0000","last_clean":"2023-11-28T13:09:09.284286+0000","last_became_active":"2023-11-28T13:09:08.193373+0000","last_became_peered":"2023-11-28T13:09:08.193373+0000","last_unstale":"2023-11-28T13:09:09.284286+0000","last_undegraded":"2023-11-28T13:09:09.284286+0000","last_fullsized":"2023-11-28T13:09:09.284286+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:23:04.592096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174562+0000","last_change":"2023-11-28T13:09:08.201954+0000","last_active":"2023-11-28T13:09:09.174562+0000","last_peered":"2023-11-28T13:09:09.174562+0000","last_clean":"2023-11-28T13:09:09.174562+0000","last_became_active":"2023-11-28T13:09:08.201660+0000","last_became_peered":"2023-11-28T13:09:08.201660+0000","last_unstale":"2023-11-28T13:09:09.174562+0000","last_undegraded":"2023-11-28T13:09:09.174562+0000","last_fullsized":"2023-11-28T13:09:09.174562+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:27.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174351+0000","last_change":"2023-11-28T13:09:08.201162+0000","last_active":"2023-11-28T13:09:09.174351+0000","last_peered":"2023-11-28T13:09:09.174351+0000","last_clean":"2023-11-28T13:09:09.174351+0000","last_became_active":"2023-11-28T13:09:08.200067+0000","last_became_peered":"2023-11-28T13:09:08.200067+0000","last_unstale":"2023-11-28T13:09:09.174351+0000","last_undegraded":"2023-11-28T13:09:09.174351+0000","last_fullsized":"2023-11-28T13:09:09.174351+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:24:22.825713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282518+0000","last_change":"2023-11-28T13:09:08.192290+0000","last_active":"2023-11-28T13:09:09.282518+0000","last_peered":"2023-11-28T13:09:09.282518+0000","last_clean":"2023-11-28T13:09:09.282518+0000","last_became_active":"2023-11-28T13:09:08.192004+0000","last_became_peered":"2023-11-28T13:09:08.192004+0000","last_unstale":"2023-11-28T13:09:09.282518+0000","last_undegraded":"2023-11-28T13:09:09.282518+0000","last_fullsized":"2023-11-28T13:09:09.282518+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:24:22.825713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176017+0000","last_change":"2023-11-28T13:09:08.197117+0000","last_active":"2023-11-28T13:09:09.176017+0000","last_peered":"2023-11-28T13:09:09.176017+0000","last_clean":"2023-11-28T13:09:09.176017+0000","last_became_active":"2023-11-28T13:09:08.196821+0000","last_became_peered":"2023-11-28T13:09:08.196821+0000","last_unstale":"2023-11-28T13:09:09.176017+0000","last_undegraded":"2023-11-28T13:09:09.176017+0000","last_fullsized":"2023-11-28T13:09:09.176017+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:50.154603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.281870+0000","last_change":"2023-11-28T13:09:08.205284+0000","last_active":"2023-11-28T13:09:09.281870+0000","last_peered":"2023-11-28T13:09:09.281870+0000","last_clean":"2023-11-28T13:09:09.281870+0000","last_became_active":"2023-11-28T13:09:08.204963+0000","last_became_peered":"2023-11-28T13:09:08.204963+0000","last_unstale":"2023-11-28T13:09:09.281870+0000","last_undegraded":"2023-11-28T13:09:09.281870+0000","last_fullsized":"2023-11-28T13:09:09.281870+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:49.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.284475+0000","last_change":"2023-11-28T13:09:08.193795+0000","last_active":"2023-11-28T13:09:09.284475+0000","last_peered":"2023-11-28T13:09:09.284475+0000","last_clean":"2023-11-28T13:09:09.284475+0000","last_became_active":"2023-11-28T13:09:08.193273+0000","last_became_peered":"2023-11-28T13:09:08.193273+0000","last_unstale":"2023-11-28T13:09:09.284475+0000","last_undegraded":"2023-11-28T13:09:09.284475+0000","last_fullsized":"2023-11-28T13:09:09.284475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:05:20.629341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.177251+0000","last_change":"2023-11-28T13:09:08.193657+0000","last_active":"2023-11-28T13:09:09.177251+0000","last_peered":"2023-11-28T13:09:09.177251+0000","last_clean":"2023-11-28T13:09:09.177251+0000","last_became_active":"2023-11-28T13:09:08.193345+0000","last_became_peered":"2023-11-28T13:09:08.193345+0000","last_unstale":"2023-11-28T13:09:09.177251+0000","last_undegraded":"2023-11-28T13:09:09.177251+0000","last_fullsized":"2023-11-28T13:09:09.177251+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:43.048873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176936+0000","last_change":"2023-11-28T13:09:08.202009+0000","last_active":"2023-11-28T13:09:09.176936+0000","last_peered":"2023-11-28T13:09:09.176936+0000","last_clean":"2023-11-28T13:09:09.176936+0000","last_became_active":"2023-11-28T13:09:08.201742+0000","last_became_peered":"2023-11-28T13:09:08.201742+0000","last_unstale":"2023-11-28T13:09:09.176936+0000","last_undegraded":"2023-11-28T13:09:09.176936+0000","last_fullsized":"2023-11-28T13:09:09.176936+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:32:58.037324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.284141+0000","last_change":"2023-11-28T13:09:08.185329+0000","last_active":"2023-11-28T13:09:09.284141+0000","last_peered":"2023-11-28T13:09:09.284141+0000","last_clean":"2023-11-28T13:09:09.284141+0000","last_became_active":"2023-11-28T13:09:08.183016+0000","last_became_peered":"2023-11-28T13:09:08.183016+0000","last_unstale":"2023-11-28T13:09:09.284141+0000","last_undegraded":"2023-11-28T13:09:09.284141+0000","last_fullsized":"2023-11-28T13:09:09.284141+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:45:00.774152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176387+0000","last_change":"2023-11-28T13:09:08.195072+0000","last_active":"2023-11-28T13:09:09.176387+0000","last_peered":"2023-11-28T13:09:09.176387+0000","last_clean":"2023-11-28T13:09:09.176387+0000","last_became_active":"2023-11-28T13:09:08.193658+0000","last_became_peered":"2023-11-28T13:09:08.193658+0000","last_unstale":"2023-11-28T13:09:09.176387+0000","last_undegraded":"2023-11-28T13:09:09.176387+0000","last_fullsized":"2023-11-28T13:09:09.176387+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:32:38.688473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282649+0000","last_change":"2023-11-28T13:09:08.205959+0000","last_active":"2023-11-28T13:09:09.282649+0000","last_peered":"2023-11-28T13:09:09.282649+0000","last_clean":"2023-11-28T13:09:09.282649+0000","last_became_active":"2023-11-28T13:09:08.205612+0000","last_became_peered":"2023-11-28T13:09:08.205612+0000","last_unstale":"2023-11-28T13:09:09.282649+0000","last_undegraded":"2023-11-28T13:09:09.282649+0000","last_fullsized":"2023-11-28T13:09:09.282649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:43.788204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.175918+0000","last_change":"2023-11-28T13:09:08.189438+0000","last_active":"2023-11-28T13:09:09.175918+0000","last_peered":"2023-11-28T13:09:09.175918+0000","last_clean":"2023-11-28T13:09:09.175918+0000","last_became_active":"2023-11-28T13:09:08.189154+0000","last_became_peered":"2023-11-28T13:09:08.189154+0000","last_unstale":"2023-11-28T13:09:09.175918+0000","last_undegraded":"2023-11-28T13:09:09.175918+0000","last_fullsized":"2023-11-28T13:09:09.175918+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:25:26.519823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.175750+0000","last_change":"2023-11-28T13:09:08.187284+0000","last_active":"2023-11-28T13:09:09.175750+0000","last_peered":"2023-11-28T13:09:09.175750+0000","last_clean":"2023-11-28T13:09:09.175750+0000","last_became_active":"2023-11-28T13:09:08.186938+0000","last_became_peered":"2023-11-28T13:09:08.186938+0000","last_unstale":"2023-11-28T13:09:09.175750+0000","last_undegraded":"2023-11-28T13:09:09.175750+0000","last_fullsized":"2023-11-28T13:09:09.175750+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:02.994480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.177483+0000","last_change":"2023-11-28T13:09:08.196887+0000","last_active":"2023-11-28T13:09:09.177483+0000","last_peered":"2023-11-28T13:09:09.177483+0000","last_clean":"2023-11-28T13:09:09.177483+0000","last_became_active":"2023-11-28T13:09:08.196569+0000","last_became_peered":"2023-11-28T13:09:08.196569+0000","last_unstale":"2023-11-28T13:09:09.177483+0000","last_undegraded":"2023-11-28T13:09:09.177483+0000","last_fullsized":"2023-11-28T13:09:09.177483+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:24:22.825713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283269+0000","last_change":"2023-11-28T13:09:08.201589+0000","last_active":"2023-11-28T13:09:09.283269+0000","last_peered":"2023-11-28T13:09:09.283269+0000","last_clean":"2023-11-28T13:09:09.283269+0000","last_became_active":"2023-11-28T13:09:08.201320+0000","last_became_peered":"2023-11-28T13:09:08.201320+0000","last_unstale":"2023-11-28T13:09:09.283269+0000","last_undegraded":"2023-11-28T13:09:09.283269+0000","last_fullsized":"2023-11-28T13:09:09.283269+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:44.429374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174589+0000","last_change":"2023-11-28T13:09:08.194851+0000","last_active":"2023-11-28T13:09:09.174589+0000","last_peered":"2023-11-28T13:09:09.174589+0000","last_clean":"2023-11-28T13:09:09.174589+0000","last_became_active":"2023-11-28T13:09:08.192726+0000","last_became_peered":"2023-11-28T13:09:08.192726+0000","last_unstale":"2023-11-28T13:09:09.174589+0000","last_undegraded":"2023-11-28T13:09:09.174589+0000","last_fullsized":"2023-11-28T13:09:09.174589+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:44.819001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282723+0000","last_change":"2023-11-28T13:09:08.202918+0000","last_active":"2023-11-28T13:09:09.282723+0000","last_peered":"2023-11-28T13:09:09.282723+0000","last_clean":"2023-11-28T13:09:09.282723+0000","last_became_active":"2023-11-28T13:09:08.202598+0000","last_became_peered":"2023-11-28T13:09:08.202598+0000","last_unstale":"2023-11-28T13:09:09.282723+0000","last_undegraded":"2023-11-28T13:09:09.282723+0000","last_fullsized":"2023-11-28T13:09:09.282723+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:25.486309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176992+0000","last_change":"2023-11-28T13:09:08.191608+0000","last_active":"2023-11-28T13:09:09.176992+0000","last_peered":"2023-11-28T13:09:09.176992+0000","last_clean":"2023-11-28T13:09:09.176992+0000","last_became_active":"2023-11-28T13:09:08.190346+0000","last_became_peered":"2023-11-28T13:09:08.190346+0000","last_unstale":"2023-11-28T13:09:09.176992+0000","last_undegraded":"2023-11-28T13:09:09.176992+0000","last_fullsized":"2023-11-28T13:09:09.176992+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:43.788204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282941+0000","last_change":"2023-11-28T13:09:08.202801+0000","last_active":"2023-11-28T13:09:09.282941+0000","last_peered":"2023-11-28T13:09:09.282941+0000","last_clean":"2023-11-28T13:09:09.282941+0000","last_became_active":"2023-11-28T13:09:08.200949+0000","last_became_peered":"2023-11-28T13:09:08.200949+0000","last_unstale":"2023-11-28T13:09:09.282941+0000","last_undegraded":"2023-11-28T13:09:09.282941+0000","last_fullsized":"2023-11-28T13:09:09.282941+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:32:58.037324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176309+0000","last_change":"2023-11-28T13:09:08.185577+0000","last_active":"2023-11-28T13:09:09.176309+0000","last_peered":"2023-11-28T13:09:09.176309+0000","last_clean":"2023-11-28T13:09:09.176309+0000","last_became_active":"2023-11-28T13:09:08.184938+0000","last_became_peered":"2023-11-28T13:09:08.184938+0000","last_unstale":"2023-11-28T13:09:09.176309+0000","last_undegraded":"2023-11-28T13:09:09.176309+0000","last_fullsized":"2023-11-28T13:09:09.176309+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:43:18.075555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174199+0000","last_change":"2023-11-28T13:09:08.197611+0000","last_active":"2023-11-28T13:09:09.174199+0000","last_peered":"2023-11-28T13:09:09.174199+0000","last_clean":"2023-11-28T13:09:09.174199+0000","last_became_active":"2023-11-28T13:09:08.197316+0000","last_became_peered":"2023-11-28T13:09:08.197316+0000","last_unstale":"2023-11-28T13:09:09.174199+0000","last_undegraded":"2023-11-28T13:09:09.174199+0000","last_fullsized":"2023-11-28T13:09:09.174199+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:56.746816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283299+0000","last_change":"2023-11-28T13:09:08.191567+0000","last_active":"2023-11-28T13:09:09.283299+0000","last_peered":"2023-11-28T13:09:09.283299+0000","last_clean":"2023-11-28T13:09:09.283299+0000","last_became_active":"2023-11-28T13:09:08.191315+0000","last_became_peered":"2023-11-28T13:09:08.191315+0000","last_unstale":"2023-11-28T13:09:09.283299+0000","last_undegraded":"2023-11-28T13:09:09.283299+0000","last_fullsized":"2023-11-28T13:09:09.283299+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:11.170479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282992+0000","last_change":"2023-11-28T13:09:08.193355+0000","last_active":"2023-11-28T13:09:09.282992+0000","last_peered":"2023-11-28T13:09:09.282992+0000","last_clean":"2023-11-28T13:09:09.282992+0000","last_became_active":"2023-11-28T13:09:08.192994+0000","last_became_peered":"2023-11-28T13:09:08.192994+0000","last_unstale":"2023-11-28T13:09:09.282992+0000","last_undegraded":"2023-11-28T13:09:09.282992+0000","last_fullsized":"2023-11-28T13:09:09.282992+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:27.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283213+0000","last_change":"2023-11-28T13:09:08.187749+0000","last_active":"2023-11-28T13:09:09.283213+0000","last_peered":"2023-11-28T13:09:09.283213+0000","last_clean":"2023-11-28T13:09:09.283213+0000","last_became_active":"2023-11-28T13:09:08.187498+0000","last_became_peered":"2023-11-28T13:09:08.187498+0000","last_unstale":"2023-11-28T13:09:09.283213+0000","last_undegraded":"2023-11-28T13:09:09.283213+0000","last_fullsized":"2023-11-28T13:09:09.283213+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:31:08.410350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176683+0000","last_change":"2023-11-28T13:09:08.192530+0000","last_active":"2023-11-28T13:09:09.176683+0000","last_peered":"2023-11-28T13:09:09.176683+0000","last_clean":"2023-11-28T13:09:09.176683+0000","last_became_active":"2023-11-28T13:09:08.192185+0000","last_became_peered":"2023-11-28T13:09:08.192185+0000","last_unstale":"2023-11-28T13:09:09.176683+0000","last_undegraded":"2023-11-28T13:09:09.176683+0000","last_fullsized":"2023-11-28T13:09:09.176683+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:54:01.916204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283054+0000","last_change":"2023-11-28T13:09:08.188986+0000","last_active":"2023-11-28T13:09:09.283054+0000","last_peered":"2023-11-28T13:09:09.283054+0000","last_clean":"2023-11-28T13:09:09.283054+0000","last_became_active":"2023-11-28T13:09:08.188401+0000","last_became_peered":"2023-11-28T13:09:08.188401+0000","last_unstale":"2023-11-28T13:09:09.283054+0000","last_undegraded":"2023-11-28T13:09:09.283054+0000","last_fullsized":"2023-11-28T13:09:09.283054+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:30.306522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282814+0000","last_change":"2023-11-28T13:09:08.185405+0000","last_active":"2023-11-28T13:09:09.282814+0000","last_peered":"2023-11-28T13:09:09.282814+0000","last_clean":"2023-11-28T13:09:09.282814+0000","last_became_active":"2023-11-28T13:09:08.185116+0000","last_became_peered":"2023-11-28T13:09:08.185116+0000","last_unstale":"2023-11-28T13:09:09.282814+0000","last_undegraded":"2023-11-28T13:09:09.282814+0000","last_fullsized":"2023-11-28T13:09:09.282814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:57.738449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176277+0000","last_change":"2023-11-28T13:09:08.185784+0000","last_active":"2023-11-28T13:09:09.176277+0000","last_peered":"2023-11-28T13:09:09.176277+0000","last_clean":"2023-11-28T13:09:09.176277+0000","last_became_active":"2023-11-28T13:09:08.185296+0000","last_became_peered":"2023-11-28T13:09:08.185296+0000","last_unstale":"2023-11-28T13:09:09.176277+0000","last_undegraded":"2023-11-28T13:09:09.176277+0000","last_fullsized":"2023-11-28T13:09:09.176277+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:23:04.592096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.281893+0000","last_change":"2023-11-28T13:09:08.187794+0000","last_active":"2023-11-28T13:09:09.281893+0000","last_peered":"2023-11-28T13:09:09.281893+0000","last_clean":"2023-11-28T13:09:09.281893+0000","last_became_active":"2023-11-28T13:09:08.187325+0000","last_became_peered":"2023-11-28T13:09:08.187325+0000","last_unstale":"2023-11-28T13:09:09.281893+0000","last_undegraded":"2023-11-28T13:09:09.281893+0000","last_fullsized":"2023-11-28T13:09:09.281893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:43:44.115527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.177115+0000","last_change":"2023-11-28T13:09:08.196478+0000","last_active":"2023-11-28T13:09:09.177115+0000","last_peered":"2023-11-28T13:09:09.177115+0000","last_clean":"2023-11-28T13:09:09.177115+0000","last_became_active":"2023-11-28T13:09:08.196213+0000","last_became_peered":"2023-11-28T13:09:08.196213+0000","last_unstale":"2023-11-28T13:09:09.177115+0000","last_undegraded":"2023-11-28T13:09:09.177115+0000","last_fullsized":"2023-11-28T13:09:09.177115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:56.746816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"34'4","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283338+0000","last_change":"2023-11-28T13:09:07.698821+0000","last_active":"2023-11-28T13:09:09.283338+0000","last_peered":"2023-11-28T13:09:09.283338+0000","last_clean":"2023-11-28T13:09:09.283338+0000","last_became_active":"2023-11-28T13:09:07.696916+0000","last_became_peered":"2023-11-28T13:09:07.696916+0000","last_unstale":"2023-11-28T13:09:09.283338+0000","last_undegraded":"2023-11-28T13:09:09.283338+0000","last_fullsized":"2023-11-28T13:09:09.283338+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:25:57.383797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282800+0000","last_change":"2023-11-28T13:09:08.198901+0000","last_active":"2023-11-28T13:09:09.282800+0000","last_peered":"2023-11-28T13:09:09.282800+0000","last_clean":"2023-11-28T13:09:09.282800+0000","last_became_active":"2023-11-28T13:09:08.196951+0000","last_became_peered":"2023-11-28T13:09:08.196951+0000","last_unstale":"2023-11-28T13:09:09.282800+0000","last_undegraded":"2023-11-28T13:09:09.282800+0000","last_fullsized":"2023-11-28T13:09:09.282800+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:00:48.443209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.284373+0000","last_change":"2023-11-28T13:09:07.697577+0000","last_active":"2023-11-28T13:09:09.284373+0000","last_peered":"2023-11-28T13:09:09.284373+0000","last_clean":"2023-11-28T13:09:09.284373+0000","last_became_active":"2023-11-28T13:09:07.697315+0000","last_became_peered":"2023-11-28T13:09:07.697315+0000","last_unstale":"2023-11-28T13:09:09.284373+0000","last_undegraded":"2023-11-28T13:09:09.284373+0000","last_fullsized":"2023-11-28T13:09:09.284373+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:29.304404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282746+0000","last_change":"2023-11-28T13:09:08.204449+0000","last_active":"2023-11-28T13:09:09.282746+0000","last_peered":"2023-11-28T13:09:09.282746+0000","last_clean":"2023-11-28T13:09:09.282746+0000","last_became_active":"2023-11-28T13:09:08.204143+0000","last_became_peered":"2023-11-28T13:09:08.204143+0000","last_unstale":"2023-11-28T13:09:09.282746+0000","last_undegraded":"2023-11-28T13:09:09.282746+0000","last_fullsized":"2023-11-28T13:09:09.282746+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:34:15.446808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282848+0000","last_change":"2023-11-28T13:09:07.696960+0000","last_active":"2023-11-28T13:09:09.282848+0000","last_peered":"2023-11-28T13:09:09.282848+0000","last_clean":"2023-11-28T13:09:09.282848+0000","last_became_active":"2023-11-28T13:09:07.696708+0000","last_became_peered":"2023-11-28T13:09:07.696708+0000","last_unstale":"2023-11-28T13:09:09.282848+0000","last_undegraded":"2023-11-28T13:09:09.282848+0000","last_fullsized":"2023-11-28T13:09:09.282848+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:56.736331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282600+0000","last_change":"2023-11-28T13:09:08.195895+0000","last_active":"2023-11-28T13:09:09.282600+0000","last_peered":"2023-11-28T13:09:09.282600+0000","last_clean":"2023-11-28T13:09:09.282600+0000","last_became_active":"2023-11-28T13:09:08.195687+0000","last_became_peered":"2023-11-28T13:09:08.195687+0000","last_unstale":"2023-11-28T13:09:09.282600+0000","last_undegraded":"2023-11-28T13:09:09.282600+0000","last_fullsized":"2023-11-28T13:09:09.282600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:50.154603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282808+0000","last_change":"2023-11-28T13:09:07.693161+0000","last_active":"2023-11-28T13:09:09.282808+0000","last_peered":"2023-11-28T13:09:09.282808+0000","last_clean":"2023-11-28T13:09:09.282808+0000","last_became_active":"2023-11-28T13:09:07.692603+0000","last_became_peered":"2023-11-28T13:09:07.692603+0000","last_unstale":"2023-11-28T13:09:09.282808+0000","last_undegraded":"2023-11-28T13:09:09.282808+0000","last_fullsized":"2023-11-28T13:09:09.282808+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:48.030424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.281596+0000","last_change":"2023-11-28T13:09:08.188733+0000","last_active":"2023-11-28T13:09:09.281596+0000","last_peered":"2023-11-28T13:09:09.281596+0000","last_clean":"2023-11-28T13:09:09.281596+0000","last_became_active":"2023-11-28T13:09:08.188223+0000","last_became_peered":"2023-11-28T13:09:08.188223+0000","last_unstale":"2023-11-28T13:09:09.281596+0000","last_undegraded":"2023-11-28T13:09:09.281596+0000","last_fullsized":"2023-11-28T13:09:09.281596+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:02.994480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176910+0000","last_change":"2023-11-28T13:09:07.690065+0000","last_active":"2023-11-28T13:09:09.176910+0000","last_peered":"2023-11-28T13:09:09.176910+0000","last_clean":"2023-11-28T13:09:09.176910+0000","last_became_active":"2023-11-28T13:09:07.688359+0000","last_became_peered":"2023-11-28T13:09:07.688359+0000","last_unstale":"2023-11-28T13:09:09.176910+0000","last_undegraded":"2023-11-28T13:09:09.176910+0000","last_fullsized":"2023-11-28T13:09:09.176910+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:23:48.559180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176194+0000","last_change":"2023-11-28T13:09:08.191703+0000","last_active":"2023-11-28T13:09:09.176194+0000","last_peered":"2023-11-28T13:09:09.176194+0000","last_clean":"2023-11-28T13:09:09.176194+0000","last_became_active":"2023-11-28T13:09:08.191416+0000","last_became_peered":"2023-11-28T13:09:08.191416+0000","last_unstale":"2023-11-28T13:09:09.176194+0000","last_undegraded":"2023-11-28T13:09:09.176194+0000","last_fullsized":"2023-11-28T13:09:09.176194+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:34:15.446808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176517+0000","last_change":"2023-11-28T13:09:07.703153+0000","last_active":"2023-11-28T13:09:09.176517+0000","last_peered":"2023-11-28T13:09:09.176517+0000","last_clean":"2023-11-28T13:09:09.176517+0000","last_became_active":"2023-11-28T13:09:07.702844+0000","last_became_peered":"2023-11-28T13:09:07.702844+0000","last_unstale":"2023-11-28T13:09:09.176517+0000","last_undegraded":"2023-11-28T13:09:09.176517+0000","last_fullsized":"2023-11-28T13:09:09.176517+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:33.805070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282932+0000","last_change":"2023-11-28T13:09:08.200277+0000","last_active":"2023-11-28T13:09:09.282932+0000","last_peered":"2023-11-28T13:09:09.282932+0000","last_clean":"2023-11-28T13:09:09.282932+0000","last_became_active":"2023-11-28T13:09:08.199942+0000","last_became_peered":"2023-11-28T13:09:08.199942+0000","last_unstale":"2023-11-28T13:09:09.282932+0000","last_undegraded":"2023-11-28T13:09:09.282932+0000","last_fullsized":"2023-11-28T13:09:09.282932+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:06.423688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282649+0000","last_change":"2023-11-28T13:09:07.698087+0000","last_active":"2023-11-28T13:09:09.282649+0000","last_peered":"2023-11-28T13:09:09.282649+0000","last_clean":"2023-11-28T13:09:09.282649+0000","last_became_active":"2023-11-28T13:09:07.697270+0000","last_became_peered":"2023-11-28T13:09:07.697270+0000","last_unstale":"2023-11-28T13:09:09.282649+0000","last_undegraded":"2023-11-28T13:09:09.282649+0000","last_fullsized":"2023-11-28T13:09:09.282649+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:08:30.303680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282325+0000","last_change":"2023-11-28T13:09:08.190541+0000","last_active":"2023-11-28T13:09:09.282325+0000","last_peered":"2023-11-28T13:09:09.282325+0000","last_clean":"2023-11-28T13:09:09.282325+0000","last_became_active":"2023-11-28T13:09:08.190280+0000","last_became_peered":"2023-11-28T13:09:08.190280+0000","last_unstale":"2023-11-28T13:09:09.282325+0000","last_undegraded":"2023-11-28T13:09:09.282325+0000","last_fullsized":"2023-11-28T13:09:09.282325+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:40.222013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282274+0000","last_change":"2023-11-28T13:09:07.699359+0000","last_active":"2023-11-28T13:09:09.282274+0000","last_peered":"2023-11-28T13:09:09.282274+0000","last_clean":"2023-11-28T13:09:09.282274+0000","last_became_active":"2023-11-28T13:09:07.698890+0000","last_became_peered":"2023-11-28T13:09:07.698890+0000","last_unstale":"2023-11-28T13:09:09.282274+0000","last_undegraded":"2023-11-28T13:09:09.282274+0000","last_fullsized":"2023-11-28T13:09:09.282274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:43:17.073437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174656+0000","last_change":"2023-11-28T13:09:08.202115+0000","last_active":"2023-11-28T13:09:09.174656+0000","last_peered":"2023-11-28T13:09:09.174656+0000","last_clean":"2023-11-28T13:09:09.174656+0000","last_became_active":"2023-11-28T13:09:08.201654+0000","last_became_peered":"2023-11-28T13:09:08.201654+0000","last_unstale":"2023-11-28T13:09:09.174656+0000","last_undegraded":"2023-11-28T13:09:09.174656+0000","last_fullsized":"2023-11-28T13:09:09.174656+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:43:44.115527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282741+0000","last_change":"2023-11-28T13:09:07.693162+0000","last_active":"2023-11-28T13:09:09.282741+0000","last_peered":"2023-11-28T13:09:09.282741+0000","last_clean":"2023-11-28T13:09:09.282741+0000","last_became_active":"2023-11-28T13:09:07.692898+0000","last_became_peered":"2023-11-28T13:09:07.692898+0000","last_unstale":"2023-11-28T13:09:09.282741+0000","last_undegraded":"2023-11-28T13:09:09.282741+0000","last_fullsized":"2023-11-28T13:09:09.282741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:43.816883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176515+0000","last_change":"2023-11-28T13:09:08.180868+0000","last_active":"2023-11-28T13:09:09.176515+0000","last_peered":"2023-11-28T13:09:09.176515+0000","last_clean":"2023-11-28T13:09:09.176515+0000","last_became_active":"2023-11-28T13:09:08.180176+0000","last_became_peered":"2023-11-28T13:09:08.180176+0000","last_unstale":"2023-11-28T13:09:09.176515+0000","last_undegraded":"2023-11-28T13:09:09.176515+0000","last_fullsized":"2023-11-28T13:09:09.176515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:54:01.916204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.177257+0000","last_change":"2023-11-28T13:09:07.690657+0000","last_active":"2023-11-28T13:09:09.177257+0000","last_peered":"2023-11-28T13:09:09.177257+0000","last_clean":"2023-11-28T13:09:09.177257+0000","last_became_active":"2023-11-28T13:09:07.688744+0000","last_became_peered":"2023-11-28T13:09:07.688744+0000","last_unstale":"2023-11-28T13:09:09.177257+0000","last_undegraded":"2023-11-28T13:09:09.177257+0000","last_fullsized":"2023-11-28T13:09:09.177257+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:18:37.564835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.284328+0000","last_change":"2023-11-28T13:09:08.181863+0000","last_active":"2023-11-28T13:09:09.284328+0000","last_peered":"2023-11-28T13:09:09.284328+0000","last_clean":"2023-11-28T13:09:09.284328+0000","last_became_active":"2023-11-28T13:09:08.181585+0000","last_became_peered":"2023-11-28T13:09:08.181585+0000","last_unstale":"2023-11-28T13:09:09.284328+0000","last_undegraded":"2023-11-28T13:09:09.284328+0000","last_fullsized":"2023-11-28T13:09:09.284328+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:49.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"34'3","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176943+0000","last_change":"2023-11-28T13:09:07.693630+0000","last_active":"2023-11-28T13:09:09.176943+0000","last_peered":"2023-11-28T13:09:09.176943+0000","last_clean":"2023-11-28T13:09:09.176943+0000","last_became_active":"2023-11-28T13:09:07.693320+0000","last_became_peered":"2023-11-28T13:09:07.693320+0000","last_unstale":"2023-11-28T13:09:09.176943+0000","last_undegraded":"2023-11-28T13:09:09.176943+0000","last_fullsized":"2023-11-28T13:09:09.176943+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:43.816883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283152+0000","last_change":"2023-11-28T13:09:08.199094+0000","last_active":"2023-11-28T13:09:09.283152+0000","last_peered":"2023-11-28T13:09:09.283152+0000","last_clean":"2023-11-28T13:09:09.283152+0000","last_became_active":"2023-11-28T13:09:08.198565+0000","last_became_peered":"2023-11-28T13:09:08.198565+0000","last_unstale":"2023-11-28T13:09:09.283152+0000","last_undegraded":"2023-11-28T13:09:09.283152+0000","last_fullsized":"2023-11-28T13:09:09.283152+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:43.048873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"34'6","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282285+0000","last_change":"2023-11-28T13:09:07.695162+0000","last_active":"2023-11-28T13:09:09.282285+0000","last_peered":"2023-11-28T13:09:09.282285+0000","last_clean":"2023-11-28T13:09:09.282285+0000","last_became_active":"2023-11-28T13:09:07.694524+0000","last_became_peered":"2023-11-28T13:09:07.694524+0000","last_unstale":"2023-11-28T13:09:09.282285+0000","last_undegraded":"2023-11-28T13:09:09.282285+0000","last_fullsized":"2023-11-28T13:09:09.282285+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:43:43.113409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282225+0000","last_change":"2023-11-28T13:09:08.198687+0000","last_active":"2023-11-28T13:09:09.282225+0000","last_peered":"2023-11-28T13:09:09.282225+0000","last_clean":"2023-11-28T13:09:09.282225+0000","last_became_active":"2023-11-28T13:09:08.198411+0000","last_became_peered":"2023-11-28T13:09:08.198411+0000","last_unstale":"2023-11-28T13:09:09.282225+0000","last_undegraded":"2023-11-28T13:09:09.282225+0000","last_fullsized":"2023-11-28T13:09:09.282225+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:43:18.075555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282420+0000","last_change":"2023-11-28T13:09:07.689123+0000","last_active":"2023-11-28T13:09:09.282420+0000","last_peered":"2023-11-28T13:09:09.282420+0000","last_clean":"2023-11-28T13:09:09.282420+0000","last_became_active":"2023-11-28T13:09:07.688680+0000","last_became_peered":"2023-11-28T13:09:07.688680+0000","last_unstale":"2023-11-28T13:09:09.282420+0000","last_undegraded":"2023-11-28T13:09:09.282420+0000","last_fullsized":"2023-11-28T13:09:09.282420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:26.464168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282026+0000","last_change":"2023-11-28T13:09:08.190905+0000","last_active":"2023-11-28T13:09:09.282026+0000","last_peered":"2023-11-28T13:09:09.282026+0000","last_clean":"2023-11-28T13:09:09.282026+0000","last_became_active":"2023-11-28T13:09:08.190634+0000","last_became_peered":"2023-11-28T13:09:08.190634+0000","last_unstale":"2023-11-28T13:09:09.282026+0000","last_undegraded":"2023-11-28T13:09:09.282026+0000","last_fullsized":"2023-11-28T13:09:09.282026+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:21:24.073987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282610+0000","last_change":"2023-11-28T13:09:07.697732+0000","last_active":"2023-11-28T13:09:09.282610+0000","last_peered":"2023-11-28T13:09:09.282610+0000","last_clean":"2023-11-28T13:09:09.282610+0000","last_became_active":"2023-11-28T13:09:07.697473+0000","last_became_peered":"2023-11-28T13:09:07.697473+0000","last_unstale":"2023-11-28T13:09:09.282610+0000","last_undegraded":"2023-11-28T13:09:09.282610+0000","last_fullsized":"2023-11-28T13:09:09.282610+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:52.373871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176327+0000","last_change":"2023-11-28T13:09:08.186593+0000","last_active":"2023-11-28T13:09:09.176327+0000","last_peered":"2023-11-28T13:09:09.176327+0000","last_clean":"2023-11-28T13:09:09.176327+0000","last_became_active":"2023-11-28T13:09:08.186304+0000","last_became_peered":"2023-11-28T13:09:08.186304+0000","last_unstale":"2023-11-28T13:09:09.176327+0000","last_undegraded":"2023-11-28T13:09:09.176327+0000","last_fullsized":"2023-11-28T13:09:09.176327+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:49.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282629+0000","last_change":"2023-11-28T13:09:07.691316+0000","last_active":"2023-11-28T13:09:09.282629+0000","last_peered":"2023-11-28T13:09:09.282629+0000","last_clean":"2023-11-28T13:09:09.282629+0000","last_became_active":"2023-11-28T13:09:07.690881+0000","last_became_peered":"2023-11-28T13:09:07.690881+0000","last_unstale":"2023-11-28T13:09:09.282629+0000","last_undegraded":"2023-11-28T13:09:09.282629+0000","last_fullsized":"2023-11-28T13:09:09.282629+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:29:26.524090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282129+0000","last_change":"2023-11-28T13:09:08.194913+0000","last_active":"2023-11-28T13:09:09.282129+0000","last_peered":"2023-11-28T13:09:09.282129+0000","last_clean":"2023-11-28T13:09:09.282129+0000","last_became_active":"2023-11-28T13:09:08.194643+0000","last_became_peered":"2023-11-28T13:09:08.194643+0000","last_unstale":"2023-11-28T13:09:09.282129+0000","last_undegraded":"2023-11-28T13:09:09.282129+0000","last_fullsized":"2023-11-28T13:09:09.282129+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:30:57.343923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"34'8","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.175972+0000","last_change":"2023-11-28T13:09:07.700577+0000","last_active":"2023-11-28T13:09:09.175972+0000","last_peered":"2023-11-28T13:09:09.175972+0000","last_clean":"2023-11-28T13:09:09.175972+0000","last_became_active":"2023-11-28T13:09:07.700266+0000","last_became_peered":"2023-11-28T13:09:07.700266+0000","last_unstale":"2023-11-28T13:09:09.175972+0000","last_undegraded":"2023-11-28T13:09:09.175972+0000","last_fullsized":"2023-11-28T13:09:09.175972+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:52.373871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282160+0000","last_change":"2023-11-28T13:09:08.198898+0000","last_active":"2023-11-28T13:09:09.282160+0000","last_peered":"2023-11-28T13:09:09.282160+0000","last_clean":"2023-11-28T13:09:09.282160+0000","last_became_active":"2023-11-28T13:09:08.198662+0000","last_became_peered":"2023-11-28T13:09:08.198662+0000","last_unstale":"2023-11-28T13:09:09.282160+0000","last_undegraded":"2023-11-28T13:09:09.282160+0000","last_fullsized":"2023-11-28T13:09:09.282160+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:43.788204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.177329+0000","last_change":"2023-11-28T13:09:07.697477+0000","last_active":"2023-11-28T13:09:09.177329+0000","last_peered":"2023-11-28T13:09:09.177329+0000","last_clean":"2023-11-28T13:09:09.177329+0000","last_became_active":"2023-11-28T13:09:07.697067+0000","last_became_peered":"2023-11-28T13:09:07.697067+0000","last_unstale":"2023-11-28T13:09:09.177329+0000","last_undegraded":"2023-11-28T13:09:09.177329+0000","last_fullsized":"2023-11-28T13:09:09.177329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:26.464168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176872+0000","last_change":"2023-11-28T13:09:08.197776+0000","last_active":"2023-11-28T13:09:09.176872+0000","last_peered":"2023-11-28T13:09:09.176872+0000","last_clean":"2023-11-28T13:09:09.176872+0000","last_became_active":"2023-11-28T13:09:08.197434+0000","last_became_peered":"2023-11-28T13:09:08.197434+0000","last_unstale":"2023-11-28T13:09:09.176872+0000","last_undegraded":"2023-11-28T13:09:09.176872+0000","last_fullsized":"2023-11-28T13:09:09.176872+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:40.222013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282786+0000","last_change":"2023-11-28T13:09:07.696885+0000","last_active":"2023-11-28T13:09:09.282786+0000","last_peered":"2023-11-28T13:09:09.282786+0000","last_clean":"2023-11-28T13:09:09.282786+0000","last_became_active":"2023-11-28T13:09:07.696599+0000","last_became_peered":"2023-11-28T13:09:07.696599+0000","last_unstale":"2023-11-28T13:09:09.282786+0000","last_undegraded":"2023-11-28T13:09:09.282786+0000","last_fullsized":"2023-11-28T13:09:09.282786+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:44:59.772034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282679+0000","last_change":"2023-11-28T13:09:08.199701+0000","last_active":"2023-11-28T13:09:09.282679+0000","last_peered":"2023-11-28T13:09:09.282679+0000","last_clean":"2023-11-28T13:09:09.282679+0000","last_became_active":"2023-11-28T13:09:08.199190+0000","last_became_peered":"2023-11-28T13:09:08.199190+0000","last_unstale":"2023-11-28T13:09:09.282679+0000","last_undegraded":"2023-11-28T13:09:09.282679+0000","last_fullsized":"2023-11-28T13:09:09.282679+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:40.222013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174712+0000","last_change":"2023-11-28T13:09:07.699255+0000","last_active":"2023-11-28T13:09:09.174712+0000","last_peered":"2023-11-28T13:09:09.174712+0000","last_clean":"2023-11-28T13:09:09.174712+0000","last_became_active":"2023-11-28T13:09:07.698965+0000","last_became_peered":"2023-11-28T13:09:07.698965+0000","last_unstale":"2023-11-28T13:09:09.174712+0000","last_undegraded":"2023-11-28T13:09:09.174712+0000","last_fullsized":"2023-11-28T13:09:09.174712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:56.736331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174658+0000","last_change":"2023-11-28T13:09:08.201272+0000","last_active":"2023-11-28T13:09:09.174658+0000","last_peered":"2023-11-28T13:09:09.174658+0000","last_clean":"2023-11-28T13:09:09.174658+0000","last_became_active":"2023-11-28T13:09:08.200690+0000","last_became_peered":"2023-11-28T13:09:08.200690+0000","last_unstale":"2023-11-28T13:09:09.174658+0000","last_undegraded":"2023-11-28T13:09:09.174658+0000","last_fullsized":"2023-11-28T13:09:09.174658+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:00:48.443209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282676+0000","last_change":"2023-11-28T13:09:07.696238+0000","last_active":"2023-11-28T13:09:09.282676+0000","last_peered":"2023-11-28T13:09:09.282676+0000","last_clean":"2023-11-28T13:09:09.282676+0000","last_became_active":"2023-11-28T13:09:07.695861+0000","last_became_peered":"2023-11-28T13:09:07.695861+0000","last_unstale":"2023-11-28T13:09:09.282676+0000","last_undegraded":"2023-11-28T13:09:09.282676+0000","last_fullsized":"2023-11-28T13:09:09.282676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:33.805070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.175875+0000","last_change":"2023-11-28T13:09:08.187892+0000","last_active":"2023-11-28T13:09:09.175875+0000","last_peered":"2023-11-28T13:09:09.175875+0000","last_clean":"2023-11-28T13:09:09.175875+0000","last_became_active":"2023-11-28T13:09:08.187568+0000","last_became_peered":"2023-11-28T13:09:08.187568+0000","last_unstale":"2023-11-28T13:09:09.175875+0000","last_undegraded":"2023-11-28T13:09:09.175875+0000","last_fullsized":"2023-11-28T13:09:09.175875+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:45:00.774152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174420+0000","last_change":"2023-11-28T13:09:07.689897+0000","last_active":"2023-11-28T13:09:09.174420+0000","last_peered":"2023-11-28T13:09:09.174420+0000","last_clean":"2023-11-28T13:09:09.174420+0000","last_became_active":"2023-11-28T13:09:07.689582+0000","last_became_peered":"2023-11-28T13:09:07.689582+0000","last_unstale":"2023-11-28T13:09:09.174420+0000","last_undegraded":"2023-11-28T13:09:09.174420+0000","last_fullsized":"2023-11-28T13:09:09.174420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:08:30.303680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283532+0000","last_change":"2023-11-28T13:09:08.179835+0000","last_active":"2023-11-28T13:09:09.283532+0000","last_peered":"2023-11-28T13:09:09.283532+0000","last_clean":"2023-11-28T13:09:09.283532+0000","last_became_active":"2023-11-28T13:09:08.179540+0000","last_became_peered":"2023-11-28T13:09:08.179540+0000","last_unstale":"2023-11-28T13:09:09.283532+0000","last_undegraded":"2023-11-28T13:09:09.283532+0000","last_fullsized":"2023-11-28T13:09:09.283532+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:54:01.916204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174318+0000","last_change":"2023-11-28T13:09:07.694428+0000","last_active":"2023-11-28T13:09:09.174318+0000","last_peered":"2023-11-28T13:09:09.174318+0000","last_clean":"2023-11-28T13:09:09.174318+0000","last_became_active":"2023-11-28T13:09:07.694043+0000","last_became_peered":"2023-11-28T13:09:07.694043+0000","last_unstale":"2023-11-28T13:09:09.174318+0000","last_undegraded":"2023-11-28T13:09:09.174318+0000","last_fullsized":"2023-11-28T13:09:09.174318+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:31:07.408232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176434+0000","last_change":"2023-11-28T13:09:08.190816+0000","last_active":"2023-11-28T13:09:09.176434+0000","last_peered":"2023-11-28T13:09:09.176434+0000","last_clean":"2023-11-28T13:09:09.176434+0000","last_became_active":"2023-11-28T13:09:08.189681+0000","last_became_peered":"2023-11-28T13:09:08.189681+0000","last_unstale":"2023-11-28T13:09:09.176434+0000","last_undegraded":"2023-11-28T13:09:09.176434+0000","last_fullsized":"2023-11-28T13:09:09.176434+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:30:57.343923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176378+0000","last_change":"2023-11-28T13:09:07.702213+0000","last_active":"2023-11-28T13:09:09.176378+0000","last_peered":"2023-11-28T13:09:09.176378+0000","last_clean":"2023-11-28T13:09:09.176378+0000","last_became_active":"2023-11-28T13:09:07.701948+0000","last_became_peered":"2023-11-28T13:09:07.701948+0000","last_unstale":"2023-11-28T13:09:09.176378+0000","last_undegraded":"2023-11-28T13:09:09.176378+0000","last_fullsized":"2023-11-28T13:09:09.176378+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:56.736331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282227+0000","last_change":"2023-11-28T13:09:08.191608+0000","last_active":"2023-11-28T13:09:09.282227+0000","last_peered":"2023-11-28T13:09:09.282227+0000","last_clean":"2023-11-28T13:09:09.282227+0000","last_became_active":"2023-11-28T13:09:08.191356+0000","last_became_peered":"2023-11-28T13:09:08.191356+0000","last_unstale":"2023-11-28T13:09:09.282227+0000","last_undegraded":"2023-11-28T13:09:09.282227+0000","last_fullsized":"2023-11-28T13:09:09.282227+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:32:38.688473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174148+0000","last_change":"2023-11-28T13:09:07.687614+0000","last_active":"2023-11-28T13:09:09.174148+0000","last_peered":"2023-11-28T13:09:09.174148+0000","last_clean":"2023-11-28T13:09:09.174148+0000","last_became_active":"2023-11-28T13:09:07.687282+0000","last_became_peered":"2023-11-28T13:09:07.687282+0000","last_unstale":"2023-11-28T13:09:09.174148+0000","last_undegraded":"2023-11-28T13:09:09.174148+0000","last_fullsized":"2023-11-28T13:09:09.174148+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:52.373871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282995+0000","last_change":"2023-11-28T13:09:08.199870+0000","last_active":"2023-11-28T13:09:09.282995+0000","last_peered":"2023-11-28T13:09:09.282995+0000","last_clean":"2023-11-28T13:09:09.282995+0000","last_became_active":"2023-11-28T13:09:08.199494+0000","last_became_peered":"2023-11-28T13:09:08.199494+0000","last_unstale":"2023-11-28T13:09:09.282995+0000","last_undegraded":"2023-11-28T13:09:09.282995+0000","last_fullsized":"2023-11-28T13:09:09.282995+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:32:38.688473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282543+0000","last_change":"2023-11-28T13:09:07.692987+0000","last_active":"2023-11-28T13:09:09.282543+0000","last_peered":"2023-11-28T13:09:09.282543+0000","last_clean":"2023-11-28T13:09:09.282543+0000","last_became_active":"2023-11-28T13:09:07.692327+0000","last_became_peered":"2023-11-28T13:09:07.692327+0000","last_unstale":"2023-11-28T13:09:09.282543+0000","last_undegraded":"2023-11-28T13:09:09.282543+0000","last_fullsized":"2023-11-28T13:09:09.282543+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:01.992362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.197349+0000","last_change":"2023-11-28T13:07:56.890378+0000","last_active":"2023-11-28T13:09:08.197349+0000","last_peered":"2023-11-28T13:09:08.197349+0000","last_clean":"2023-11-28T13:09:08.197349+0000","last_became_active":"2023-11-28T13:07:56.890058+0000","last_became_peered":"2023-11-28T13:07:56.890058+0000","last_unstale":"2023-11-28T13:09:08.197349+0000","last_undegraded":"2023-11-28T13:09:08.197349+0000","last_fullsized":"2023-11-28T13:09:08.197349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:05.344602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176994+0000","last_change":"2023-11-28T13:09:08.183775+0000","last_active":"2023-11-28T13:09:09.176994+0000","last_peered":"2023-11-28T13:09:09.176994+0000","last_clean":"2023-11-28T13:09:09.176994+0000","last_became_active":"2023-11-28T13:09:08.183542+0000","last_became_peered":"2023-11-28T13:09:08.183542+0000","last_unstale":"2023-11-28T13:09:09.176994+0000","last_undegraded":"2023-11-28T13:09:09.176994+0000","last_fullsized":"2023-11-28T13:09:09.176994+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:05:20.629341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282114+0000","last_change":"2023-11-28T13:09:07.692706+0000","last_active":"2023-11-28T13:09:09.282114+0000","last_peered":"2023-11-28T13:09:09.282114+0000","last_clean":"2023-11-28T13:09:09.282114+0000","last_became_active":"2023-11-28T13:09:07.692147+0000","last_became_peered":"2023-11-28T13:09:07.692147+0000","last_unstale":"2023-11-28T13:09:09.282114+0000","last_undegraded":"2023-11-28T13:09:09.282114+0000","last_fullsized":"2023-11-28T13:09:09.282114+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:00:47.441091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.192692+0000","last_change":"2023-11-28T13:07:56.844509+0000","last_active":"2023-11-28T13:09:08.192692+0000","last_peered":"2023-11-28T13:09:08.192692+0000","last_clean":"2023-11-28T13:09:08.192692+0000","last_became_active":"2023-11-28T13:07:56.844081+0000","last_became_peered":"2023-11-28T13:07:56.844081+0000","last_unstale":"2023-11-28T13:09:08.192692+0000","last_undegraded":"2023-11-28T13:09:08.192692+0000","last_fullsized":"2023-11-28T13:09:08.192692+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282578+0000","last_change":"2023-11-28T13:09:08.196928+0000","last_active":"2023-11-28T13:09:09.282578+0000","last_peered":"2023-11-28T13:09:09.282578+0000","last_clean":"2023-11-28T13:09:09.282578+0000","last_became_active":"2023-11-28T13:09:08.196662+0000","last_became_peered":"2023-11-28T13:09:08.196662+0000","last_unstale":"2023-11-28T13:09:09.282578+0000","last_undegraded":"2023-11-28T13:09:09.282578+0000","last_fullsized":"2023-11-28T13:09:09.282578+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:27.466286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.177419+0000","last_change":"2023-11-28T13:09:07.695058+0000","last_active":"2023-11-28T13:09:09.177419+0000","last_peered":"2023-11-28T13:09:09.177419+0000","last_clean":"2023-11-28T13:09:09.177419+0000","last_became_active":"2023-11-28T13:09:07.694756+0000","last_became_peered":"2023-11-28T13:09:07.694756+0000","last_unstale":"2023-11-28T13:09:09.177419+0000","last_undegraded":"2023-11-28T13:09:09.177419+0000","last_fullsized":"2023-11-28T13:09:09.177419+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:00:47.441091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.182702+0000","last_change":"2023-11-28T13:07:56.844699+0000","last_active":"2023-11-28T13:09:08.182702+0000","last_peered":"2023-11-28T13:09:08.182702+0000","last_clean":"2023-11-28T13:09:08.182702+0000","last_became_active":"2023-11-28T13:07:56.844391+0000","last_became_peered":"2023-11-28T13:07:56.844391+0000","last_unstale":"2023-11-28T13:09:08.182702+0000","last_undegraded":"2023-11-28T13:09:08.182702+0000","last_fullsized":"2023-11-28T13:09:08.182702+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":176,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.175559+0000","last_change":"2023-11-28T13:07:57.844020+0000","last_active":"2023-11-28T13:09:08.175559+0000","last_peered":"2023-11-28T13:09:08.175559+0000","last_clean":"2023-11-28T13:09:08.175559+0000","last_became_active":"2023-11-28T13:07:57.842981+0000","last_became_peered":"2023-11-28T13:07:57.842981+0000","last_unstale":"2023-11-28T13:09:08.175559+0000","last_undegraded":"2023-11-28T13:09:08.175559+0000","last_fullsized":"2023-11-28T13:09:08.175559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_clean_scrub_stamp":"2023-11-28T13:07:56.825711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:44.973218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176633+0000","last_change":"2023-11-28T13:09:08.184319+0000","last_active":"2023-11-28T13:09:09.176633+0000","last_peered":"2023-11-28T13:09:09.176633+0000","last_clean":"2023-11-28T13:09:09.176633+0000","last_became_active":"2023-11-28T13:09:08.183988+0000","last_became_peered":"2023-11-28T13:09:08.183988+0000","last_unstale":"2023-11-28T13:09:09.176633+0000","last_undegraded":"2023-11-28T13:09:09.176633+0000","last_fullsized":"2023-11-28T13:09:09.176633+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:00:08.790472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.830511+0000","last_change":"2023-11-28T13:09:07.690149+0000","last_active":"2023-11-28T13:09:09.830511+0000","last_peered":"2023-11-28T13:09:09.830511+0000","last_clean":"2023-11-28T13:09:09.830511+0000","last_became_active":"2023-11-28T13:09:07.688707+0000","last_became_peered":"2023-11-28T13:09:07.688707+0000","last_unstale":"2023-11-28T13:09:09.830511+0000","last_undegraded":"2023-11-28T13:09:09.830511+0000","last_fullsized":"2023-11-28T13:09:09.830511+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:33.805070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.175653+0000","last_change":"2023-11-28T13:07:57.306318+0000","last_active":"2023-11-28T13:09:08.175653+0000","last_peered":"2023-11-28T13:09:08.175653+0000","last_clean":"2023-11-28T13:09:08.175653+0000","last_became_active":"2023-11-28T13:07:57.305937+0000","last_became_peered":"2023-11-28T13:07:57.305937+0000","last_unstale":"2023-11-28T13:09:08.175653+0000","last_undegraded":"2023-11-28T13:09:08.175653+0000","last_fullsized":"2023-11-28T13:09:08.175653+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.177331+0000","last_change":"2023-11-28T13:09:08.196458+0000","last_active":"2023-11-28T13:09:09.177331+0000","last_peered":"2023-11-28T13:09:09.177331+0000","last_clean":"2023-11-28T13:09:09.177331+0000","last_became_active":"2023-11-28T13:09:08.196196+0000","last_became_peered":"2023-11-28T13:09:08.196196+0000","last_unstale":"2023-11-28T13:09:09.177331+0000","last_undegraded":"2023-11-28T13:09:09.177331+0000","last_fullsized":"2023-11-28T13:09:09.177331+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:11.552323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.177195+0000","last_change":"2023-11-28T13:09:07.701427+0000","last_active":"2023-11-28T13:09:09.177195+0000","last_peered":"2023-11-28T13:09:09.177195+0000","last_clean":"2023-11-28T13:09:09.177195+0000","last_became_active":"2023-11-28T13:09:07.701135+0000","last_became_peered":"2023-11-28T13:09:07.701135+0000","last_unstale":"2023-11-28T13:09:09.177195+0000","last_undegraded":"2023-11-28T13:09:09.177195+0000","last_fullsized":"2023-11-28T13:09:09.177195+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:31:07.408232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.175459+0000","last_change":"2023-11-28T13:07:57.306277+0000","last_active":"2023-11-28T13:09:08.175459+0000","last_peered":"2023-11-28T13:09:08.175459+0000","last_clean":"2023-11-28T13:09:08.175459+0000","last_became_active":"2023-11-28T13:07:57.305983+0000","last_became_peered":"2023-11-28T13:07:57.305983+0000","last_unstale":"2023-11-28T13:09:08.175459+0000","last_undegraded":"2023-11-28T13:09:08.175459+0000","last_fullsized":"2023-11-28T13:09:08.175459+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283854+0000","last_change":"2023-11-28T13:09:08.204228+0000","last_active":"2023-11-28T13:09:09.283854+0000","last_peered":"2023-11-28T13:09:09.283854+0000","last_clean":"2023-11-28T13:09:09.283854+0000","last_became_active":"2023-11-28T13:09:08.201755+0000","last_became_peered":"2023-11-28T13:09:08.201755+0000","last_unstale":"2023-11-28T13:09:09.283854+0000","last_undegraded":"2023-11-28T13:09:09.283854+0000","last_fullsized":"2023-11-28T13:09:09.283854+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:43.048873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.284311+0000","last_change":"2023-11-28T13:09:07.690279+0000","last_active":"2023-11-28T13:09:09.284311+0000","last_peered":"2023-11-28T13:09:09.284311+0000","last_clean":"2023-11-28T13:09:09.284311+0000","last_became_active":"2023-11-28T13:09:07.689970+0000","last_became_peered":"2023-11-28T13:09:07.689970+0000","last_unstale":"2023-11-28T13:09:09.284311+0000","last_undegraded":"2023-11-28T13:09:09.284311+0000","last_fullsized":"2023-11-28T13:09:09.284311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:24.484191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":29,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.181655+0000","last_change":"2023-11-28T13:07:57.306177+0000","last_active":"2023-11-28T13:09:08.181655+0000","last_peered":"2023-11-28T13:09:08.181655+0000","last_clean":"2023-11-28T13:09:08.181655+0000","last_became_active":"2023-11-28T13:07:57.305853+0000","last_became_peered":"2023-11-28T13:07:57.305853+0000","last_unstale":"2023-11-28T13:09:08.181655+0000","last_undegraded":"2023-11-28T13:09:08.181655+0000","last_fullsized":"2023-11-28T13:09:08.181655+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282562+0000","last_change":"2023-11-28T13:09:08.204667+0000","last_active":"2023-11-28T13:09:09.282562+0000","last_peered":"2023-11-28T13:09:09.282562+0000","last_clean":"2023-11-28T13:09:09.282562+0000","last_became_active":"2023-11-28T13:09:08.204072+0000","last_became_peered":"2023-11-28T13:09:08.204072+0000","last_unstale":"2023-11-28T13:09:09.282562+0000","last_undegraded":"2023-11-28T13:09:09.282562+0000","last_fullsized":"2023-11-28T13:09:09.282562+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:44.819001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282917+0000","last_change":"2023-11-28T13:09:07.691730+0000","last_active":"2023-11-28T13:09:09.282917+0000","last_peered":"2023-11-28T13:09:09.282917+0000","last_clean":"2023-11-28T13:09:09.282917+0000","last_became_active":"2023-11-28T13:09:07.691439+0000","last_became_peered":"2023-11-28T13:09:07.691439+0000","last_unstale":"2023-11-28T13:09:09.282917+0000","last_undegraded":"2023-11-28T13:09:09.282917+0000","last_fullsized":"2023-11-28T13:09:09.282917+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:22.228475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.176828+0000","last_change":"2023-11-28T13:07:57.306321+0000","last_active":"2023-11-28T13:09:08.176828+0000","last_peered":"2023-11-28T13:09:08.176828+0000","last_clean":"2023-11-28T13:09:08.176828+0000","last_became_active":"2023-11-28T13:07:57.305923+0000","last_became_peered":"2023-11-28T13:07:57.305923+0000","last_unstale":"2023-11-28T13:09:08.176828+0000","last_undegraded":"2023-11-28T13:09:08.176828+0000","last_fullsized":"2023-11-28T13:09:08.176828+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:41.104679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.281847+0000","last_change":"2023-11-28T13:09:08.199317+0000","last_active":"2023-11-28T13:09:09.281847+0000","last_peered":"2023-11-28T13:09:09.281847+0000","last_clean":"2023-11-28T13:09:09.281847+0000","last_became_active":"2023-11-28T13:09:08.198903+0000","last_became_peered":"2023-11-28T13:09:08.198903+0000","last_unstale":"2023-11-28T13:09:09.281847+0000","last_undegraded":"2023-11-28T13:09:09.281847+0000","last_fullsized":"2023-11-28T13:09:09.281847+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:27.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.284131+0000","last_change":"2023-11-28T13:09:07.697118+0000","last_active":"2023-11-28T13:09:09.284131+0000","last_peered":"2023-11-28T13:09:09.284131+0000","last_clean":"2023-11-28T13:09:09.284131+0000","last_became_active":"2023-11-28T13:09:07.696403+0000","last_became_peered":"2023-11-28T13:09:07.696403+0000","last_unstale":"2023-11-28T13:09:09.284131+0000","last_undegraded":"2023-11-28T13:09:09.284131+0000","last_fullsized":"2023-11-28T13:09:09.284131+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:10.168361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.176133+0000","last_change":"2023-11-28T13:07:56.844305+0000","last_active":"2023-11-28T13:09:08.176133+0000","last_peered":"2023-11-28T13:09:08.176133+0000","last_clean":"2023-11-28T13:09:08.176133+0000","last_became_active":"2023-11-28T13:07:56.843980+0000","last_became_peered":"2023-11-28T13:07:56.843980+0000","last_unstale":"2023-11-28T13:09:08.176133+0000","last_undegraded":"2023-11-28T13:09:08.176133+0000","last_fullsized":"2023-11-28T13:09:08.176133+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9568,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362272,"statfs":{"total":96636764160,"available":96626966528,"internally_reserved":0,"allocated":1007616,"data_stored":757227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.64800000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.41899999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.76600000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.996}]},{"osd":4,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.90100000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.044},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.85799999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.94899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8336,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363504,"statfs":{"total":96636764160,"available":96628228096,"internally_reserved":0,"allocated":401408,"data_stored":165807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.495},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68899999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59299999999999997}]},{"osd":2,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.626},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.77700000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.755}]},{"osd":5,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.877},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.65200000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.90600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13588,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":12608,"kb_avail":94358252,"statfs":{"total":96636764160,"available":96622850048,"internally_reserved":0,"allocated":995328,"data_stored":755699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12910592},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.35499999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.443},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.434}]},{"osd":3,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.56799999999999995}]},{"osd":4,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.58599999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.61099999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.66500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12556,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":12160,"kb_avail":94359284,"statfs":{"total":96636764160,"available":96623906816,"internally_reserved":0,"allocated":397312,"data_stored":164289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12451840},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.33400000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.40799999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.505},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.42899999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.626}]},{"osd":5,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65400000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.75},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.68000000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12432,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":12032,"kb_avail":94359408,"statfs":{"total":96636764160,"available":96624033792,"internally_reserved":0,"allocated":401408,"data_stored":165311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12320768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.441},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.58199999999999996}]},{"osd":1,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.52200000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.59199999999999997}]},{"osd":4,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.93300000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.871},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.94299999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12636,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":12224,"kb_avail":94359204,"statfs":{"total":96636764160,"available":96623824896,"internally_reserved":0,"allocated":413696,"data_stored":166823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12517376},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.32900000000000001}]},{"osd":1,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.52300000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.54800000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.61199999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.71199999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.74099999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.71999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8596,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363244,"statfs":{"total":96636764160,"available":96627961856,"internally_reserved":0,"allocated":405504,"data_stored":165829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.76300000000000001}]},{"osd":1,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.57499999999999996}]},{"osd":2,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.65400000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.61599999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.63100000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.79400000000000004}]},{"osd":7,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.72899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8468,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363372,"statfs":{"total":96636764160,"available":96628092928,"internally_reserved":0,"allocated":405504,"data_stored":165353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.68999999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57799999999999996}]},{"osd":2,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.52600000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.63},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.83899999999999997}]},{"osd":4,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.749},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.84799999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.81999999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.88800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:09:12.553 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:09:12.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.782+0000 7f3bafdff700 1 Processor -- start 2023-11-28T13:09:12.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.783+0000 7f3bafdff700 1 -- start start 2023-11-28T13:09:12.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.784+0000 7f3bafdff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3ba815c9b0 0x7f3ba80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:12.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.784+0000 7f3bafdff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba80d1950 con 0x7f3ba8161340 2023-11-28T13:09:12.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.784+0000 7f3bafdff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba80d24f0 con 0x7f3ba80cd7c0 2023-11-28T13:09:12.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.784+0000 7f3bafdff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba80b5800 con 0x7f3ba815c9b0 2023-11-28T13:09:12.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.784+0000 7f3badb9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3ba815c9b0 0x7f3ba80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:12.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.784+0000 7f3bad39a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f3ba8161340 0x7f3ba80d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:43496/0 (socket says 172.21.15.156:43496) 2023-11-28T13:09:12.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.784+0000 7f3bad39a700 1 -- 172.21.15.156:0/2846360382 learned_addr learned my addr 172.21.15.156:0/2846360382 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:12.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.784+0000 7f3badb9b700 1 --2- 172.21.15.156:0/2846360382 >> v2:172.21.15.156:3300/0 conn(0x7f3ba815c9b0 0x7f3ba80b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49702/0 (socket says 172.21.15.156:49702) 2023-11-28T13:09:12.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.784+0000 7f3badb9b700 1 -- 172.21.15.156:0/2846360382 >> v1:172.21.15.186:6789/0 conn(0x7f3ba80cd7c0 legacy=0x7f3ba80b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:12.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.784+0000 7f3badb9b700 1 -- 172.21.15.156:0/2846360382 >> v1:172.21.15.156:6789/0 conn(0x7f3ba8161340 legacy=0x7f3ba80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:12.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.785+0000 7f3badb9b700 1 -- 172.21.15.156:0/2846360382 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba80b5520 con 0x7f3ba815c9b0 2023-11-28T13:09:12.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.784+0000 7f3bacb99700 1 -- 172.21.15.156:0/2846360382 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102334235 0 0) 0x7f3ba80d1950 con 0x7f3ba8161340 2023-11-28T13:09:12.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.785+0000 7f3badb9b700 1 --2- 172.21.15.156:0/2846360382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba815c9b0 0x7f3ba80b4bf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009840 tx=0x7f3ba4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fefedb542006271 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:12.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.785+0000 7f3bacb99700 1 -- 172.21.15.156:0/2846360382 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4011830 con 0x7f3ba815c9b0 2023-11-28T13:09:12.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.785+0000 7f3bacb99700 1 -- 172.21.15.156:0/2846360382 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba4011990 con 0x7f3ba815c9b0 2023-11-28T13:09:12.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.785+0000 7f3bacb99700 1 -- 172.21.15.156:0/2846360382 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4011d20 con 0x7f3ba815c9b0 2023-11-28T13:09:12.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.786+0000 7f3bafdff700 1 -- 172.21.15.156:0/2846360382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba815c9b0 msgr2=0x7f3ba80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:12.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.786+0000 7f3bafdff700 1 --2- 172.21.15.156:0/2846360382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba815c9b0 0x7f3ba80b4bf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009840 tx=0x7f3ba4009b50 comp rx=0 tx=0).stop 2023-11-28T13:09:12.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.786+0000 7f3bafdff700 1 -- 172.21.15.156:0/2846360382 shutdown_connections 2023-11-28T13:09:12.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.786+0000 7f3bafdff700 1 --2- 172.21.15.156:0/2846360382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba815c9b0 0x7f3ba80b4bf0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:12.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.786+0000 7f3bafdff700 1 -- 172.21.15.156:0/2846360382 >> 172.21.15.156:0/2846360382 conn(0x7f3ba8072600 msgr2=0x7f3ba80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:12.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.786+0000 7f3bafdff700 1 -- 172.21.15.156:0/2846360382 shutdown_connections 2023-11-28T13:09:12.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.786+0000 7f3bafdff700 1 -- 172.21.15.156:0/2846360382 wait complete. 2023-11-28T13:09:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.787+0000 7f3bafdff700 1 Processor -- start 2023-11-28T13:09:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.788+0000 7f3bafdff700 1 -- start start 2023-11-28T13:09:12.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.788+0000 7f3bafdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ba80cd7c0 0x7f3ba81f3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.788+0000 7f3bafdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ba815c9b0 0x7f3ba81f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.788+0000 7f3bafdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba8161340 0x7f3ba81f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.788+0000 7f3bafdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba814e0b0 con 0x7f3ba8161340 2023-11-28T13:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.788+0000 7f3bafdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba81f3450 con 0x7f3ba815c9b0 2023-11-28T13:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.788+0000 7f3bafdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba815e280 con 0x7f3ba80cd7c0 2023-11-28T13:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.789+0000 7f3bae39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba8161340 0x7f3ba81f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.789+0000 7f3bae39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba8161340 0x7f3ba81f51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49714/0 (socket says 172.21.15.156:49714) 2023-11-28T13:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.789+0000 7f3bae39c700 1 -- 172.21.15.156:0/2031118704 learned_addr learned my addr 172.21.15.156:0/2031118704 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.789+0000 7f3bad39a700 1 --2- 172.21.15.156:0/2031118704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ba815c9b0 0x7f3ba81f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.789+0000 7f3badb9b700 1 --2- 172.21.15.156:0/2031118704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ba80cd7c0 0x7f3ba81f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.789+0000 7f3bae39c700 1 -- 172.21.15.156:0/2031118704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ba80cd7c0 msgr2=0x7f3ba81f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:12.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.789+0000 7f3bae39c700 1 --2- 172.21.15.156:0/2031118704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ba80cd7c0 0x7f3ba81f3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:12.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.789+0000 7f3bae39c700 1 -- 172.21.15.156:0/2031118704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ba815c9b0 msgr2=0x7f3ba81f2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:12.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.789+0000 7f3bae39c700 1 --2- 172.21.15.156:0/2031118704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ba815c9b0 0x7f3ba81f2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:12.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.789+0000 7f3bae39c700 1 -- 172.21.15.156:0/2031118704 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba40094a0 con 0x7f3ba8161340 2023-11-28T13:09:12.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.789+0000 7f3bad39a700 1 --2- 172.21.15.156:0/2031118704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ba815c9b0 0x7f3ba81f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:12.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.789+0000 7f3badb9b700 1 --2- 172.21.15.156:0/2031118704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ba80cd7c0 0x7f3ba81f3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:12.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.790+0000 7f3bae39c700 1 --2- 172.21.15.156:0/2031118704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba8161340 0x7f3ba81f51d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000cb50 tx=0x7f3ba000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:12.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.790+0000 7f3b9e7fc700 1 -- 172.21.15.156:0/2031118704 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0003c00 con 0x7f3ba8161340 2023-11-28T13:09:12.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.790+0000 7f3b9e7fc700 1 -- 172.21.15.156:0/2031118704 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba0003d60 con 0x7f3ba8161340 2023-11-28T13:09:12.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.790+0000 7f3b9e7fc700 1 -- 172.21.15.156:0/2031118704 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000ab70 con 0x7f3ba8161340 2023-11-28T13:09:12.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.790+0000 7f3bafdff700 1 -- 172.21.15.156:0/2031118704 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba80d15e0 con 0x7f3ba8161340 2023-11-28T13:09:12.792 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.790+0000 7f3bafdff700 1 -- 172.21.15.156:0/2031118704 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba80d1300 con 0x7f3ba8161340 2023-11-28T13:09:12.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.791+0000 7f3bafdff700 1 -- 172.21.15.156:0/2031118704 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba80b5130 con 0x7f3ba8161340 2023-11-28T13:09:12.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.792+0000 7f3b9e7fc700 1 -- 172.21.15.156:0/2031118704 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3ba0010050 con 0x7f3ba8161340 2023-11-28T13:09:12.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.793+0000 7f3b9e7fc700 1 --2- 172.21.15.156:0/2031118704 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3b8c0682f0 0x7f3b8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.793+0000 7f3b9e7fc700 1 -- 172.21.15.156:0/2031118704 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3ba0085c40 con 0x7f3ba8161340 2023-11-28T13:09:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.793+0000 7f3badb9b700 1 --2- 172.21.15.156:0/2031118704 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3b8c0682f0 0x7f3b8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:12.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.794+0000 7f3badb9b700 1 --2- 172.21.15.156:0/2031118704 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3b8c0682f0 0x7f3b8c06a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4000c00 tx=0x7f3ba401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:12.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.797+0000 7f3b9e7fc700 1 -- 172.21.15.156:0/2031118704 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3ba00523b0 con 0x7f3ba8161340 2023-11-28T13:09:12.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.984+0000 7f3bafdff700 1 -- 172.21.15.156:0/2031118704 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3ba80745c0 con 0x7f3b8c0682f0 2023-11-28T13:09:12.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.993+0000 7f3b9e7fc700 1 -- 172.21.15.156:0/2031118704 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312116 (secure 0 0 0) 0x7f3ba80745c0 con 0x7f3b8c0682f0 2023-11-28T13:09:12.997 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:13.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.997+0000 7f3bafdff700 1 -- 172.21.15.156:0/2031118704 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3b8c0682f0 msgr2=0x7f3b8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:13.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.997+0000 7f3bafdff700 1 --2- 172.21.15.156:0/2031118704 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3b8c0682f0 0x7f3b8c06a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4000c00 tx=0x7f3ba401c040 comp rx=0 tx=0).stop 2023-11-28T13:09:13.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.997+0000 7f3bafdff700 1 -- 172.21.15.156:0/2031118704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba8161340 msgr2=0x7f3ba81f51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:13.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.997+0000 7f3bafdff700 1 --2- 172.21.15.156:0/2031118704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba8161340 0x7f3ba81f51d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000cb50 tx=0x7f3ba000cf10 comp rx=0 tx=0).stop 2023-11-28T13:09:13.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.997+0000 7f3bafdff700 1 -- 172.21.15.156:0/2031118704 shutdown_connections 2023-11-28T13:09:13.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.997+0000 7f3bafdff700 1 --2- 172.21.15.156:0/2031118704 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3b8c0682f0 0x7f3b8c06a7a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.997+0000 7f3bafdff700 1 --2- 172.21.15.156:0/2031118704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba8161340 0x7f3ba81f51d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.997+0000 7f3bafdff700 1 --2- 172.21.15.156:0/2031118704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ba80cd7c0 0x7f3ba81f3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.997+0000 7f3bafdff700 1 --2- 172.21.15.156:0/2031118704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ba815c9b0 0x7f3ba81f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.997+0000 7f3bafdff700 1 -- 172.21.15.156:0/2031118704 >> 172.21.15.156:0/2031118704 conn(0x7f3ba8072600 msgr2=0x7f3ba815ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:13.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.997+0000 7f3bafdff700 1 -- 172.21.15.156:0/2031118704 shutdown_connections 2023-11-28T13:09:13.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:12.997+0000 7f3bafdff700 1 -- 172.21.15.156:0/2031118704 wait complete. 2023-11-28T13:09:13.003 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T13:09:13.016 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2023-11-28T13:09:12.031135+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":86180,"kb_used_data":4324,"kb_used_omap":0,"kb_used_meta":81792,"kb_avail":754888540,"statfs":{"total":773094113280,"available":773005864960,"internally_reserved":0,"allocated":4427776,"data_stored":2506338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":83755008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.358509"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282831+0000","last_change":"2023-11-28T13:09:08.192164+0000","last_active":"2023-11-28T13:09:09.282831+0000","last_peered":"2023-11-28T13:09:09.282831+0000","last_clean":"2023-11-28T13:09:09.282831+0000","last_became_active":"2023-11-28T13:09:08.191924+0000","last_became_peered":"2023-11-28T13:09:08.191924+0000","last_unstale":"2023-11-28T13:09:09.282831+0000","last_undegraded":"2023-11-28T13:09:09.282831+0000","last_fullsized":"2023-11-28T13:09:09.282831+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:11.552323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283744+0000","last_change":"2023-11-28T13:09:08.196036+0000","last_active":"2023-11-28T13:09:09.283744+0000","last_peered":"2023-11-28T13:09:09.283744+0000","last_clean":"2023-11-28T13:09:09.283744+0000","last_became_active":"2023-11-28T13:09:08.195790+0000","last_became_peered":"2023-11-28T13:09:08.195790+0000","last_unstale":"2023-11-28T13:09:09.283744+0000","last_undegraded":"2023-11-28T13:09:09.283744+0000","last_fullsized":"2023-11-28T13:09:09.283744+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:25:26.519823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.284286+0000","last_change":"2023-11-28T13:09:08.193619+0000","last_active":"2023-11-28T13:09:09.284286+0000","last_peered":"2023-11-28T13:09:09.284286+0000","last_clean":"2023-11-28T13:09:09.284286+0000","last_became_active":"2023-11-28T13:09:08.193373+0000","last_became_peered":"2023-11-28T13:09:08.193373+0000","last_unstale":"2023-11-28T13:09:09.284286+0000","last_undegraded":"2023-11-28T13:09:09.284286+0000","last_fullsized":"2023-11-28T13:09:09.284286+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:23:04.592096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174562+0000","last_change":"2023-11-28T13:09:08.201954+0000","last_active":"2023-11-28T13:09:09.174562+0000","last_peered":"2023-11-28T13:09:09.174562+0000","last_clean":"2023-11-28T13:09:09.174562+0000","last_became_active":"2023-11-28T13:09:08.201660+0000","last_became_peered":"2023-11-28T13:09:08.201660+0000","last_unstale":"2023-11-28T13:09:09.174562+0000","last_undegraded":"2023-11-28T13:09:09.174562+0000","last_fullsized":"2023-11-28T13:09:09.174562+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:27.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174351+0000","last_change":"2023-11-28T13:09:08.201162+0000","last_active":"2023-11-28T13:09:09.174351+0000","last_peered":"2023-11-28T13:09:09.174351+0000","last_clean":"2023-11-28T13:09:09.174351+0000","last_became_active":"2023-11-28T13:09:08.200067+0000","last_became_peered":"2023-11-28T13:09:08.200067+0000","last_unstale":"2023-11-28T13:09:09.174351+0000","last_undegraded":"2023-11-28T13:09:09.174351+0000","last_fullsized":"2023-11-28T13:09:09.174351+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:24:22.825713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282518+0000","last_change":"2023-11-28T13:09:08.192290+0000","last_active":"2023-11-28T13:09:09.282518+0000","last_peered":"2023-11-28T13:09:09.282518+0000","last_clean":"2023-11-28T13:09:09.282518+0000","last_became_active":"2023-11-28T13:09:08.192004+0000","last_became_peered":"2023-11-28T13:09:08.192004+0000","last_unstale":"2023-11-28T13:09:09.282518+0000","last_undegraded":"2023-11-28T13:09:09.282518+0000","last_fullsized":"2023-11-28T13:09:09.282518+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:24:22.825713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176017+0000","last_change":"2023-11-28T13:09:08.197117+0000","last_active":"2023-11-28T13:09:09.176017+0000","last_peered":"2023-11-28T13:09:09.176017+0000","last_clean":"2023-11-28T13:09:09.176017+0000","last_became_active":"2023-11-28T13:09:08.196821+0000","last_became_peered":"2023-11-28T13:09:08.196821+0000","last_unstale":"2023-11-28T13:09:09.176017+0000","last_undegraded":"2023-11-28T13:09:09.176017+0000","last_fullsized":"2023-11-28T13:09:09.176017+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:50.154603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.281870+0000","last_change":"2023-11-28T13:09:08.205284+0000","last_active":"2023-11-28T13:09:09.281870+0000","last_peered":"2023-11-28T13:09:09.281870+0000","last_clean":"2023-11-28T13:09:09.281870+0000","last_became_active":"2023-11-28T13:09:08.204963+0000","last_became_peered":"2023-11-28T13:09:08.204963+0000","last_unstale":"2023-11-28T13:09:09.281870+0000","last_undegraded":"2023-11-28T13:09:09.281870+0000","last_fullsized":"2023-11-28T13:09:09.281870+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:49.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.284475+0000","last_change":"2023-11-28T13:09:08.193795+0000","last_active":"2023-11-28T13:09:09.284475+0000","last_peered":"2023-11-28T13:09:09.284475+0000","last_clean":"2023-11-28T13:09:09.284475+0000","last_became_active":"2023-11-28T13:09:08.193273+0000","last_became_peered":"2023-11-28T13:09:08.193273+0000","last_unstale":"2023-11-28T13:09:09.284475+0000","last_undegraded":"2023-11-28T13:09:09.284475+0000","last_fullsized":"2023-11-28T13:09:09.284475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:05:20.629341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.177251+0000","last_change":"2023-11-28T13:09:08.193657+0000","last_active":"2023-11-28T13:09:09.177251+0000","last_peered":"2023-11-28T13:09:09.177251+0000","last_clean":"2023-11-28T13:09:09.177251+0000","last_became_active":"2023-11-28T13:09:08.193345+0000","last_became_peered":"2023-11-28T13:09:08.193345+0000","last_unstale":"2023-11-28T13:09:09.177251+0000","last_undegraded":"2023-11-28T13:09:09.177251+0000","last_fullsized":"2023-11-28T13:09:09.177251+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:43.048873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176936+0000","last_change":"2023-11-28T13:09:08.202009+0000","last_active":"2023-11-28T13:09:09.176936+0000","last_peered":"2023-11-28T13:09:09.176936+0000","last_clean":"2023-11-28T13:09:09.176936+0000","last_became_active":"2023-11-28T13:09:08.201742+0000","last_became_peered":"2023-11-28T13:09:08.201742+0000","last_unstale":"2023-11-28T13:09:09.176936+0000","last_undegraded":"2023-11-28T13:09:09.176936+0000","last_fullsized":"2023-11-28T13:09:09.176936+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:32:58.037324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.284141+0000","last_change":"2023-11-28T13:09:08.185329+0000","last_active":"2023-11-28T13:09:09.284141+0000","last_peered":"2023-11-28T13:09:09.284141+0000","last_clean":"2023-11-28T13:09:09.284141+0000","last_became_active":"2023-11-28T13:09:08.183016+0000","last_became_peered":"2023-11-28T13:09:08.183016+0000","last_unstale":"2023-11-28T13:09:09.284141+0000","last_undegraded":"2023-11-28T13:09:09.284141+0000","last_fullsized":"2023-11-28T13:09:09.284141+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:45:00.774152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176387+0000","last_change":"2023-11-28T13:09:08.195072+0000","last_active":"2023-11-28T13:09:09.176387+0000","last_peered":"2023-11-28T13:09:09.176387+0000","last_clean":"2023-11-28T13:09:09.176387+0000","last_became_active":"2023-11-28T13:09:08.193658+0000","last_became_peered":"2023-11-28T13:09:08.193658+0000","last_unstale":"2023-11-28T13:09:09.176387+0000","last_undegraded":"2023-11-28T13:09:09.176387+0000","last_fullsized":"2023-11-28T13:09:09.176387+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:32:38.688473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282649+0000","last_change":"2023-11-28T13:09:08.205959+0000","last_active":"2023-11-28T13:09:09.282649+0000","last_peered":"2023-11-28T13:09:09.282649+0000","last_clean":"2023-11-28T13:09:09.282649+0000","last_became_active":"2023-11-28T13:09:08.205612+0000","last_became_peered":"2023-11-28T13:09:08.205612+0000","last_unstale":"2023-11-28T13:09:09.282649+0000","last_undegraded":"2023-11-28T13:09:09.282649+0000","last_fullsized":"2023-11-28T13:09:09.282649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:43.788204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.175918+0000","last_change":"2023-11-28T13:09:08.189438+0000","last_active":"2023-11-28T13:09:09.175918+0000","last_peered":"2023-11-28T13:09:09.175918+0000","last_clean":"2023-11-28T13:09:09.175918+0000","last_became_active":"2023-11-28T13:09:08.189154+0000","last_became_peered":"2023-11-28T13:09:08.189154+0000","last_unstale":"2023-11-28T13:09:09.175918+0000","last_undegraded":"2023-11-28T13:09:09.175918+0000","last_fullsized":"2023-11-28T13:09:09.175918+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:25:26.519823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.175750+0000","last_change":"2023-11-28T13:09:08.187284+0000","last_active":"2023-11-28T13:09:09.175750+0000","last_peered":"2023-11-28T13:09:09.175750+0000","last_clean":"2023-11-28T13:09:09.175750+0000","last_became_active":"2023-11-28T13:09:08.186938+0000","last_became_peered":"2023-11-28T13:09:08.186938+0000","last_unstale":"2023-11-28T13:09:09.175750+0000","last_undegraded":"2023-11-28T13:09:09.175750+0000","last_fullsized":"2023-11-28T13:09:09.175750+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:02.994480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.177483+0000","last_change":"2023-11-28T13:09:08.196887+0000","last_active":"2023-11-28T13:09:09.177483+0000","last_peered":"2023-11-28T13:09:09.177483+0000","last_clean":"2023-11-28T13:09:09.177483+0000","last_became_active":"2023-11-28T13:09:08.196569+0000","last_became_peered":"2023-11-28T13:09:08.196569+0000","last_unstale":"2023-11-28T13:09:09.177483+0000","last_undegraded":"2023-11-28T13:09:09.177483+0000","last_fullsized":"2023-11-28T13:09:09.177483+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:24:22.825713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283269+0000","last_change":"2023-11-28T13:09:08.201589+0000","last_active":"2023-11-28T13:09:09.283269+0000","last_peered":"2023-11-28T13:09:09.283269+0000","last_clean":"2023-11-28T13:09:09.283269+0000","last_became_active":"2023-11-28T13:09:08.201320+0000","last_became_peered":"2023-11-28T13:09:08.201320+0000","last_unstale":"2023-11-28T13:09:09.283269+0000","last_undegraded":"2023-11-28T13:09:09.283269+0000","last_fullsized":"2023-11-28T13:09:09.283269+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:44.429374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174589+0000","last_change":"2023-11-28T13:09:08.194851+0000","last_active":"2023-11-28T13:09:09.174589+0000","last_peered":"2023-11-28T13:09:09.174589+0000","last_clean":"2023-11-28T13:09:09.174589+0000","last_became_active":"2023-11-28T13:09:08.192726+0000","last_became_peered":"2023-11-28T13:09:08.192726+0000","last_unstale":"2023-11-28T13:09:09.174589+0000","last_undegraded":"2023-11-28T13:09:09.174589+0000","last_fullsized":"2023-11-28T13:09:09.174589+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:44.819001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282723+0000","last_change":"2023-11-28T13:09:08.202918+0000","last_active":"2023-11-28T13:09:09.282723+0000","last_peered":"2023-11-28T13:09:09.282723+0000","last_clean":"2023-11-28T13:09:09.282723+0000","last_became_active":"2023-11-28T13:09:08.202598+0000","last_became_peered":"2023-11-28T13:09:08.202598+0000","last_unstale":"2023-11-28T13:09:09.282723+0000","last_undegraded":"2023-11-28T13:09:09.282723+0000","last_fullsized":"2023-11-28T13:09:09.282723+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:25.486309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176992+0000","last_change":"2023-11-28T13:09:08.191608+0000","last_active":"2023-11-28T13:09:09.176992+0000","last_peered":"2023-11-28T13:09:09.176992+0000","last_clean":"2023-11-28T13:09:09.176992+0000","last_became_active":"2023-11-28T13:09:08.190346+0000","last_became_peered":"2023-11-28T13:09:08.190346+0000","last_unstale":"2023-11-28T13:09:09.176992+0000","last_undegraded":"2023-11-28T13:09:09.176992+0000","last_fullsized":"2023-11-28T13:09:09.176992+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:43.788204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282941+0000","last_change":"2023-11-28T13:09:08.202801+0000","last_active":"2023-11-28T13:09:09.282941+0000","last_peered":"2023-11-28T13:09:09.282941+0000","last_clean":"2023-11-28T13:09:09.282941+0000","last_became_active":"2023-11-28T13:09:08.200949+0000","last_became_peered":"2023-11-28T13:09:08.200949+0000","last_unstale":"2023-11-28T13:09:09.282941+0000","last_undegraded":"2023-11-28T13:09:09.282941+0000","last_fullsized":"2023-11-28T13:09:09.282941+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:32:58.037324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176309+0000","last_change":"2023-11-28T13:09:08.185577+0000","last_active":"2023-11-28T13:09:09.176309+0000","last_peered":"2023-11-28T13:09:09.176309+0000","last_clean":"2023-11-28T13:09:09.176309+0000","last_became_active":"2023-11-28T13:09:08.184938+0000","last_became_peered":"2023-11-28T13:09:08.184938+0000","last_unstale":"2023-11-28T13:09:09.176309+0000","last_undegraded":"2023-11-28T13:09:09.176309+0000","last_fullsized":"2023-11-28T13:09:09.176309+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:43:18.075555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174199+0000","last_change":"2023-11-28T13:09:08.197611+0000","last_active":"2023-11-28T13:09:09.174199+0000","last_peered":"2023-11-28T13:09:09.174199+0000","last_clean":"2023-11-28T13:09:09.174199+0000","last_became_active":"2023-11-28T13:09:08.197316+0000","last_became_peered":"2023-11-28T13:09:08.197316+0000","last_unstale":"2023-11-28T13:09:09.174199+0000","last_undegraded":"2023-11-28T13:09:09.174199+0000","last_fullsized":"2023-11-28T13:09:09.174199+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:56.746816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283299+0000","last_change":"2023-11-28T13:09:08.191567+0000","last_active":"2023-11-28T13:09:09.283299+0000","last_peered":"2023-11-28T13:09:09.283299+0000","last_clean":"2023-11-28T13:09:09.283299+0000","last_became_active":"2023-11-28T13:09:08.191315+0000","last_became_peered":"2023-11-28T13:09:08.191315+0000","last_unstale":"2023-11-28T13:09:09.283299+0000","last_undegraded":"2023-11-28T13:09:09.283299+0000","last_fullsized":"2023-11-28T13:09:09.283299+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:11.170479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282992+0000","last_change":"2023-11-28T13:09:08.193355+0000","last_active":"2023-11-28T13:09:09.282992+0000","last_peered":"2023-11-28T13:09:09.282992+0000","last_clean":"2023-11-28T13:09:09.282992+0000","last_became_active":"2023-11-28T13:09:08.192994+0000","last_became_peered":"2023-11-28T13:09:08.192994+0000","last_unstale":"2023-11-28T13:09:09.282992+0000","last_undegraded":"2023-11-28T13:09:09.282992+0000","last_fullsized":"2023-11-28T13:09:09.282992+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:27.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283213+0000","last_change":"2023-11-28T13:09:08.187749+0000","last_active":"2023-11-28T13:09:09.283213+0000","last_peered":"2023-11-28T13:09:09.283213+0000","last_clean":"2023-11-28T13:09:09.283213+0000","last_became_active":"2023-11-28T13:09:08.187498+0000","last_became_peered":"2023-11-28T13:09:08.187498+0000","last_unstale":"2023-11-28T13:09:09.283213+0000","last_undegraded":"2023-11-28T13:09:09.283213+0000","last_fullsized":"2023-11-28T13:09:09.283213+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:31:08.410350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176683+0000","last_change":"2023-11-28T13:09:08.192530+0000","last_active":"2023-11-28T13:09:09.176683+0000","last_peered":"2023-11-28T13:09:09.176683+0000","last_clean":"2023-11-28T13:09:09.176683+0000","last_became_active":"2023-11-28T13:09:08.192185+0000","last_became_peered":"2023-11-28T13:09:08.192185+0000","last_unstale":"2023-11-28T13:09:09.176683+0000","last_undegraded":"2023-11-28T13:09:09.176683+0000","last_fullsized":"2023-11-28T13:09:09.176683+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:54:01.916204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283054+0000","last_change":"2023-11-28T13:09:08.188986+0000","last_active":"2023-11-28T13:09:09.283054+0000","last_peered":"2023-11-28T13:09:09.283054+0000","last_clean":"2023-11-28T13:09:09.283054+0000","last_became_active":"2023-11-28T13:09:08.188401+0000","last_became_peered":"2023-11-28T13:09:08.188401+0000","last_unstale":"2023-11-28T13:09:09.283054+0000","last_undegraded":"2023-11-28T13:09:09.283054+0000","last_fullsized":"2023-11-28T13:09:09.283054+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:30.306522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282814+0000","last_change":"2023-11-28T13:09:08.185405+0000","last_active":"2023-11-28T13:09:09.282814+0000","last_peered":"2023-11-28T13:09:09.282814+0000","last_clean":"2023-11-28T13:09:09.282814+0000","last_became_active":"2023-11-28T13:09:08.185116+0000","last_became_peered":"2023-11-28T13:09:08.185116+0000","last_unstale":"2023-11-28T13:09:09.282814+0000","last_undegraded":"2023-11-28T13:09:09.282814+0000","last_fullsized":"2023-11-28T13:09:09.282814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:57.738449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176277+0000","last_change":"2023-11-28T13:09:08.185784+0000","last_active":"2023-11-28T13:09:09.176277+0000","last_peered":"2023-11-28T13:09:09.176277+0000","last_clean":"2023-11-28T13:09:09.176277+0000","last_became_active":"2023-11-28T13:09:08.185296+0000","last_became_peered":"2023-11-28T13:09:08.185296+0000","last_unstale":"2023-11-28T13:09:09.176277+0000","last_undegraded":"2023-11-28T13:09:09.176277+0000","last_fullsized":"2023-11-28T13:09:09.176277+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:23:04.592096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.281893+0000","last_change":"2023-11-28T13:09:08.187794+0000","last_active":"2023-11-28T13:09:09.281893+0000","last_peered":"2023-11-28T13:09:09.281893+0000","last_clean":"2023-11-28T13:09:09.281893+0000","last_became_active":"2023-11-28T13:09:08.187325+0000","last_became_peered":"2023-11-28T13:09:08.187325+0000","last_unstale":"2023-11-28T13:09:09.281893+0000","last_undegraded":"2023-11-28T13:09:09.281893+0000","last_fullsized":"2023-11-28T13:09:09.281893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:43:44.115527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.177115+0000","last_change":"2023-11-28T13:09:08.196478+0000","last_active":"2023-11-28T13:09:09.177115+0000","last_peered":"2023-11-28T13:09:09.177115+0000","last_clean":"2023-11-28T13:09:09.177115+0000","last_became_active":"2023-11-28T13:09:08.196213+0000","last_became_peered":"2023-11-28T13:09:08.196213+0000","last_unstale":"2023-11-28T13:09:09.177115+0000","last_undegraded":"2023-11-28T13:09:09.177115+0000","last_fullsized":"2023-11-28T13:09:09.177115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:56.746816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"34'4","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283338+0000","last_change":"2023-11-28T13:09:07.698821+0000","last_active":"2023-11-28T13:09:09.283338+0000","last_peered":"2023-11-28T13:09:09.283338+0000","last_clean":"2023-11-28T13:09:09.283338+0000","last_became_active":"2023-11-28T13:09:07.696916+0000","last_became_peered":"2023-11-28T13:09:07.696916+0000","last_unstale":"2023-11-28T13:09:09.283338+0000","last_undegraded":"2023-11-28T13:09:09.283338+0000","last_fullsized":"2023-11-28T13:09:09.283338+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:25:57.383797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282800+0000","last_change":"2023-11-28T13:09:08.198901+0000","last_active":"2023-11-28T13:09:09.282800+0000","last_peered":"2023-11-28T13:09:09.282800+0000","last_clean":"2023-11-28T13:09:09.282800+0000","last_became_active":"2023-11-28T13:09:08.196951+0000","last_became_peered":"2023-11-28T13:09:08.196951+0000","last_unstale":"2023-11-28T13:09:09.282800+0000","last_undegraded":"2023-11-28T13:09:09.282800+0000","last_fullsized":"2023-11-28T13:09:09.282800+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:00:48.443209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.284373+0000","last_change":"2023-11-28T13:09:07.697577+0000","last_active":"2023-11-28T13:09:09.284373+0000","last_peered":"2023-11-28T13:09:09.284373+0000","last_clean":"2023-11-28T13:09:09.284373+0000","last_became_active":"2023-11-28T13:09:07.697315+0000","last_became_peered":"2023-11-28T13:09:07.697315+0000","last_unstale":"2023-11-28T13:09:09.284373+0000","last_undegraded":"2023-11-28T13:09:09.284373+0000","last_fullsized":"2023-11-28T13:09:09.284373+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:29.304404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282746+0000","last_change":"2023-11-28T13:09:08.204449+0000","last_active":"2023-11-28T13:09:09.282746+0000","last_peered":"2023-11-28T13:09:09.282746+0000","last_clean":"2023-11-28T13:09:09.282746+0000","last_became_active":"2023-11-28T13:09:08.204143+0000","last_became_peered":"2023-11-28T13:09:08.204143+0000","last_unstale":"2023-11-28T13:09:09.282746+0000","last_undegraded":"2023-11-28T13:09:09.282746+0000","last_fullsized":"2023-11-28T13:09:09.282746+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:34:15.446808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282848+0000","last_change":"2023-11-28T13:09:07.696960+0000","last_active":"2023-11-28T13:09:09.282848+0000","last_peered":"2023-11-28T13:09:09.282848+0000","last_clean":"2023-11-28T13:09:09.282848+0000","last_became_active":"2023-11-28T13:09:07.696708+0000","last_became_peered":"2023-11-28T13:09:07.696708+0000","last_unstale":"2023-11-28T13:09:09.282848+0000","last_undegraded":"2023-11-28T13:09:09.282848+0000","last_fullsized":"2023-11-28T13:09:09.282848+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:56.736331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282600+0000","last_change":"2023-11-28T13:09:08.195895+0000","last_active":"2023-11-28T13:09:09.282600+0000","last_peered":"2023-11-28T13:09:09.282600+0000","last_clean":"2023-11-28T13:09:09.282600+0000","last_became_active":"2023-11-28T13:09:08.195687+0000","last_became_peered":"2023-11-28T13:09:08.195687+0000","last_unstale":"2023-11-28T13:09:09.282600+0000","last_undegraded":"2023-11-28T13:09:09.282600+0000","last_fullsized":"2023-11-28T13:09:09.282600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:50.154603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282808+0000","last_change":"2023-11-28T13:09:07.693161+0000","last_active":"2023-11-28T13:09:09.282808+0000","last_peered":"2023-11-28T13:09:09.282808+0000","last_clean":"2023-11-28T13:09:09.282808+0000","last_became_active":"2023-11-28T13:09:07.692603+0000","last_became_peered":"2023-11-28T13:09:07.692603+0000","last_unstale":"2023-11-28T13:09:09.282808+0000","last_undegraded":"2023-11-28T13:09:09.282808+0000","last_fullsized":"2023-11-28T13:09:09.282808+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:48.030424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.281596+0000","last_change":"2023-11-28T13:09:08.188733+0000","last_active":"2023-11-28T13:09:09.281596+0000","last_peered":"2023-11-28T13:09:09.281596+0000","last_clean":"2023-11-28T13:09:09.281596+0000","last_became_active":"2023-11-28T13:09:08.188223+0000","last_became_peered":"2023-11-28T13:09:08.188223+0000","last_unstale":"2023-11-28T13:09:09.281596+0000","last_undegraded":"2023-11-28T13:09:09.281596+0000","last_fullsized":"2023-11-28T13:09:09.281596+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:02.994480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176910+0000","last_change":"2023-11-28T13:09:07.690065+0000","last_active":"2023-11-28T13:09:09.176910+0000","last_peered":"2023-11-28T13:09:09.176910+0000","last_clean":"2023-11-28T13:09:09.176910+0000","last_became_active":"2023-11-28T13:09:07.688359+0000","last_became_peered":"2023-11-28T13:09:07.688359+0000","last_unstale":"2023-11-28T13:09:09.176910+0000","last_undegraded":"2023-11-28T13:09:09.176910+0000","last_fullsized":"2023-11-28T13:09:09.176910+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:23:48.559180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176194+0000","last_change":"2023-11-28T13:09:08.191703+0000","last_active":"2023-11-28T13:09:09.176194+0000","last_peered":"2023-11-28T13:09:09.176194+0000","last_clean":"2023-11-28T13:09:09.176194+0000","last_became_active":"2023-11-28T13:09:08.191416+0000","last_became_peered":"2023-11-28T13:09:08.191416+0000","last_unstale":"2023-11-28T13:09:09.176194+0000","last_undegraded":"2023-11-28T13:09:09.176194+0000","last_fullsized":"2023-11-28T13:09:09.176194+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:34:15.446808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176517+0000","last_change":"2023-11-28T13:09:07.703153+0000","last_active":"2023-11-28T13:09:09.176517+0000","last_peered":"2023-11-28T13:09:09.176517+0000","last_clean":"2023-11-28T13:09:09.176517+0000","last_became_active":"2023-11-28T13:09:07.702844+0000","last_became_peered":"2023-11-28T13:09:07.702844+0000","last_unstale":"2023-11-28T13:09:09.176517+0000","last_undegraded":"2023-11-28T13:09:09.176517+0000","last_fullsized":"2023-11-28T13:09:09.176517+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:33.805070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282932+0000","last_change":"2023-11-28T13:09:08.200277+0000","last_active":"2023-11-28T13:09:09.282932+0000","last_peered":"2023-11-28T13:09:09.282932+0000","last_clean":"2023-11-28T13:09:09.282932+0000","last_became_active":"2023-11-28T13:09:08.199942+0000","last_became_peered":"2023-11-28T13:09:08.199942+0000","last_unstale":"2023-11-28T13:09:09.282932+0000","last_undegraded":"2023-11-28T13:09:09.282932+0000","last_fullsized":"2023-11-28T13:09:09.282932+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:06.423688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282649+0000","last_change":"2023-11-28T13:09:07.698087+0000","last_active":"2023-11-28T13:09:09.282649+0000","last_peered":"2023-11-28T13:09:09.282649+0000","last_clean":"2023-11-28T13:09:09.282649+0000","last_became_active":"2023-11-28T13:09:07.697270+0000","last_became_peered":"2023-11-28T13:09:07.697270+0000","last_unstale":"2023-11-28T13:09:09.282649+0000","last_undegraded":"2023-11-28T13:09:09.282649+0000","last_fullsized":"2023-11-28T13:09:09.282649+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:08:30.303680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282325+0000","last_change":"2023-11-28T13:09:08.190541+0000","last_active":"2023-11-28T13:09:09.282325+0000","last_peered":"2023-11-28T13:09:09.282325+0000","last_clean":"2023-11-28T13:09:09.282325+0000","last_became_active":"2023-11-28T13:09:08.190280+0000","last_became_peered":"2023-11-28T13:09:08.190280+0000","last_unstale":"2023-11-28T13:09:09.282325+0000","last_undegraded":"2023-11-28T13:09:09.282325+0000","last_fullsized":"2023-11-28T13:09:09.282325+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:40.222013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282274+0000","last_change":"2023-11-28T13:09:07.699359+0000","last_active":"2023-11-28T13:09:09.282274+0000","last_peered":"2023-11-28T13:09:09.282274+0000","last_clean":"2023-11-28T13:09:09.282274+0000","last_became_active":"2023-11-28T13:09:07.698890+0000","last_became_peered":"2023-11-28T13:09:07.698890+0000","last_unstale":"2023-11-28T13:09:09.282274+0000","last_undegraded":"2023-11-28T13:09:09.282274+0000","last_fullsized":"2023-11-28T13:09:09.282274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:43:17.073437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174656+0000","last_change":"2023-11-28T13:09:08.202115+0000","last_active":"2023-11-28T13:09:09.174656+0000","last_peered":"2023-11-28T13:09:09.174656+0000","last_clean":"2023-11-28T13:09:09.174656+0000","last_became_active":"2023-11-28T13:09:08.201654+0000","last_became_peered":"2023-11-28T13:09:08.201654+0000","last_unstale":"2023-11-28T13:09:09.174656+0000","last_undegraded":"2023-11-28T13:09:09.174656+0000","last_fullsized":"2023-11-28T13:09:09.174656+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:43:44.115527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282741+0000","last_change":"2023-11-28T13:09:07.693162+0000","last_active":"2023-11-28T13:09:09.282741+0000","last_peered":"2023-11-28T13:09:09.282741+0000","last_clean":"2023-11-28T13:09:09.282741+0000","last_became_active":"2023-11-28T13:09:07.692898+0000","last_became_peered":"2023-11-28T13:09:07.692898+0000","last_unstale":"2023-11-28T13:09:09.282741+0000","last_undegraded":"2023-11-28T13:09:09.282741+0000","last_fullsized":"2023-11-28T13:09:09.282741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:43.816883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176515+0000","last_change":"2023-11-28T13:09:08.180868+0000","last_active":"2023-11-28T13:09:09.176515+0000","last_peered":"2023-11-28T13:09:09.176515+0000","last_clean":"2023-11-28T13:09:09.176515+0000","last_became_active":"2023-11-28T13:09:08.180176+0000","last_became_peered":"2023-11-28T13:09:08.180176+0000","last_unstale":"2023-11-28T13:09:09.176515+0000","last_undegraded":"2023-11-28T13:09:09.176515+0000","last_fullsized":"2023-11-28T13:09:09.176515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:54:01.916204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.177257+0000","last_change":"2023-11-28T13:09:07.690657+0000","last_active":"2023-11-28T13:09:09.177257+0000","last_peered":"2023-11-28T13:09:09.177257+0000","last_clean":"2023-11-28T13:09:09.177257+0000","last_became_active":"2023-11-28T13:09:07.688744+0000","last_became_peered":"2023-11-28T13:09:07.688744+0000","last_unstale":"2023-11-28T13:09:09.177257+0000","last_undegraded":"2023-11-28T13:09:09.177257+0000","last_fullsized":"2023-11-28T13:09:09.177257+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:18:37.564835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.284328+0000","last_change":"2023-11-28T13:09:08.181863+0000","last_active":"2023-11-28T13:09:09.284328+0000","last_peered":"2023-11-28T13:09:09.284328+0000","last_clean":"2023-11-28T13:09:09.284328+0000","last_became_active":"2023-11-28T13:09:08.181585+0000","last_became_peered":"2023-11-28T13:09:08.181585+0000","last_unstale":"2023-11-28T13:09:09.284328+0000","last_undegraded":"2023-11-28T13:09:09.284328+0000","last_fullsized":"2023-11-28T13:09:09.284328+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:49.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"34'3","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176943+0000","last_change":"2023-11-28T13:09:07.693630+0000","last_active":"2023-11-28T13:09:09.176943+0000","last_peered":"2023-11-28T13:09:09.176943+0000","last_clean":"2023-11-28T13:09:09.176943+0000","last_became_active":"2023-11-28T13:09:07.693320+0000","last_became_peered":"2023-11-28T13:09:07.693320+0000","last_unstale":"2023-11-28T13:09:09.176943+0000","last_undegraded":"2023-11-28T13:09:09.176943+0000","last_fullsized":"2023-11-28T13:09:09.176943+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:43.816883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283152+0000","last_change":"2023-11-28T13:09:08.199094+0000","last_active":"2023-11-28T13:09:09.283152+0000","last_peered":"2023-11-28T13:09:09.283152+0000","last_clean":"2023-11-28T13:09:09.283152+0000","last_became_active":"2023-11-28T13:09:08.198565+0000","last_became_peered":"2023-11-28T13:09:08.198565+0000","last_unstale":"2023-11-28T13:09:09.283152+0000","last_undegraded":"2023-11-28T13:09:09.283152+0000","last_fullsized":"2023-11-28T13:09:09.283152+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:43.048873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"34'6","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282285+0000","last_change":"2023-11-28T13:09:07.695162+0000","last_active":"2023-11-28T13:09:09.282285+0000","last_peered":"2023-11-28T13:09:09.282285+0000","last_clean":"2023-11-28T13:09:09.282285+0000","last_became_active":"2023-11-28T13:09:07.694524+0000","last_became_peered":"2023-11-28T13:09:07.694524+0000","last_unstale":"2023-11-28T13:09:09.282285+0000","last_undegraded":"2023-11-28T13:09:09.282285+0000","last_fullsized":"2023-11-28T13:09:09.282285+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:43:43.113409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282225+0000","last_change":"2023-11-28T13:09:08.198687+0000","last_active":"2023-11-28T13:09:09.282225+0000","last_peered":"2023-11-28T13:09:09.282225+0000","last_clean":"2023-11-28T13:09:09.282225+0000","last_became_active":"2023-11-28T13:09:08.198411+0000","last_became_peered":"2023-11-28T13:09:08.198411+0000","last_unstale":"2023-11-28T13:09:09.282225+0000","last_undegraded":"2023-11-28T13:09:09.282225+0000","last_fullsized":"2023-11-28T13:09:09.282225+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:43:18.075555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282420+0000","last_change":"2023-11-28T13:09:07.689123+0000","last_active":"2023-11-28T13:09:09.282420+0000","last_peered":"2023-11-28T13:09:09.282420+0000","last_clean":"2023-11-28T13:09:09.282420+0000","last_became_active":"2023-11-28T13:09:07.688680+0000","last_became_peered":"2023-11-28T13:09:07.688680+0000","last_unstale":"2023-11-28T13:09:09.282420+0000","last_undegraded":"2023-11-28T13:09:09.282420+0000","last_fullsized":"2023-11-28T13:09:09.282420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:26.464168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282026+0000","last_change":"2023-11-28T13:09:08.190905+0000","last_active":"2023-11-28T13:09:09.282026+0000","last_peered":"2023-11-28T13:09:09.282026+0000","last_clean":"2023-11-28T13:09:09.282026+0000","last_became_active":"2023-11-28T13:09:08.190634+0000","last_became_peered":"2023-11-28T13:09:08.190634+0000","last_unstale":"2023-11-28T13:09:09.282026+0000","last_undegraded":"2023-11-28T13:09:09.282026+0000","last_fullsized":"2023-11-28T13:09:09.282026+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:21:24.073987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282610+0000","last_change":"2023-11-28T13:09:07.697732+0000","last_active":"2023-11-28T13:09:09.282610+0000","last_peered":"2023-11-28T13:09:09.282610+0000","last_clean":"2023-11-28T13:09:09.282610+0000","last_became_active":"2023-11-28T13:09:07.697473+0000","last_became_peered":"2023-11-28T13:09:07.697473+0000","last_unstale":"2023-11-28T13:09:09.282610+0000","last_undegraded":"2023-11-28T13:09:09.282610+0000","last_fullsized":"2023-11-28T13:09:09.282610+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:52.373871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176327+0000","last_change":"2023-11-28T13:09:08.186593+0000","last_active":"2023-11-28T13:09:09.176327+0000","last_peered":"2023-11-28T13:09:09.176327+0000","last_clean":"2023-11-28T13:09:09.176327+0000","last_became_active":"2023-11-28T13:09:08.186304+0000","last_became_peered":"2023-11-28T13:09:08.186304+0000","last_unstale":"2023-11-28T13:09:09.176327+0000","last_undegraded":"2023-11-28T13:09:09.176327+0000","last_fullsized":"2023-11-28T13:09:09.176327+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:49.032542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282629+0000","last_change":"2023-11-28T13:09:07.691316+0000","last_active":"2023-11-28T13:09:09.282629+0000","last_peered":"2023-11-28T13:09:09.282629+0000","last_clean":"2023-11-28T13:09:09.282629+0000","last_became_active":"2023-11-28T13:09:07.690881+0000","last_became_peered":"2023-11-28T13:09:07.690881+0000","last_unstale":"2023-11-28T13:09:09.282629+0000","last_undegraded":"2023-11-28T13:09:09.282629+0000","last_fullsized":"2023-11-28T13:09:09.282629+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:29:26.524090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282129+0000","last_change":"2023-11-28T13:09:08.194913+0000","last_active":"2023-11-28T13:09:09.282129+0000","last_peered":"2023-11-28T13:09:09.282129+0000","last_clean":"2023-11-28T13:09:09.282129+0000","last_became_active":"2023-11-28T13:09:08.194643+0000","last_became_peered":"2023-11-28T13:09:08.194643+0000","last_unstale":"2023-11-28T13:09:09.282129+0000","last_undegraded":"2023-11-28T13:09:09.282129+0000","last_fullsized":"2023-11-28T13:09:09.282129+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:30:57.343923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"34'8","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.175972+0000","last_change":"2023-11-28T13:09:07.700577+0000","last_active":"2023-11-28T13:09:09.175972+0000","last_peered":"2023-11-28T13:09:09.175972+0000","last_clean":"2023-11-28T13:09:09.175972+0000","last_became_active":"2023-11-28T13:09:07.700266+0000","last_became_peered":"2023-11-28T13:09:07.700266+0000","last_unstale":"2023-11-28T13:09:09.175972+0000","last_undegraded":"2023-11-28T13:09:09.175972+0000","last_fullsized":"2023-11-28T13:09:09.175972+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:52.373871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282160+0000","last_change":"2023-11-28T13:09:08.198898+0000","last_active":"2023-11-28T13:09:09.282160+0000","last_peered":"2023-11-28T13:09:09.282160+0000","last_clean":"2023-11-28T13:09:09.282160+0000","last_became_active":"2023-11-28T13:09:08.198662+0000","last_became_peered":"2023-11-28T13:09:08.198662+0000","last_unstale":"2023-11-28T13:09:09.282160+0000","last_undegraded":"2023-11-28T13:09:09.282160+0000","last_fullsized":"2023-11-28T13:09:09.282160+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:43.788204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.177329+0000","last_change":"2023-11-28T13:09:07.697477+0000","last_active":"2023-11-28T13:09:09.177329+0000","last_peered":"2023-11-28T13:09:09.177329+0000","last_clean":"2023-11-28T13:09:09.177329+0000","last_became_active":"2023-11-28T13:09:07.697067+0000","last_became_peered":"2023-11-28T13:09:07.697067+0000","last_unstale":"2023-11-28T13:09:09.177329+0000","last_undegraded":"2023-11-28T13:09:09.177329+0000","last_fullsized":"2023-11-28T13:09:09.177329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:26.464168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176872+0000","last_change":"2023-11-28T13:09:08.197776+0000","last_active":"2023-11-28T13:09:09.176872+0000","last_peered":"2023-11-28T13:09:09.176872+0000","last_clean":"2023-11-28T13:09:09.176872+0000","last_became_active":"2023-11-28T13:09:08.197434+0000","last_became_peered":"2023-11-28T13:09:08.197434+0000","last_unstale":"2023-11-28T13:09:09.176872+0000","last_undegraded":"2023-11-28T13:09:09.176872+0000","last_fullsized":"2023-11-28T13:09:09.176872+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:40.222013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282786+0000","last_change":"2023-11-28T13:09:07.696885+0000","last_active":"2023-11-28T13:09:09.282786+0000","last_peered":"2023-11-28T13:09:09.282786+0000","last_clean":"2023-11-28T13:09:09.282786+0000","last_became_active":"2023-11-28T13:09:07.696599+0000","last_became_peered":"2023-11-28T13:09:07.696599+0000","last_unstale":"2023-11-28T13:09:09.282786+0000","last_undegraded":"2023-11-28T13:09:09.282786+0000","last_fullsized":"2023-11-28T13:09:09.282786+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:44:59.772034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282679+0000","last_change":"2023-11-28T13:09:08.199701+0000","last_active":"2023-11-28T13:09:09.282679+0000","last_peered":"2023-11-28T13:09:09.282679+0000","last_clean":"2023-11-28T13:09:09.282679+0000","last_became_active":"2023-11-28T13:09:08.199190+0000","last_became_peered":"2023-11-28T13:09:08.199190+0000","last_unstale":"2023-11-28T13:09:09.282679+0000","last_undegraded":"2023-11-28T13:09:09.282679+0000","last_fullsized":"2023-11-28T13:09:09.282679+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:40.222013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"34'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174712+0000","last_change":"2023-11-28T13:09:07.699255+0000","last_active":"2023-11-28T13:09:09.174712+0000","last_peered":"2023-11-28T13:09:09.174712+0000","last_clean":"2023-11-28T13:09:09.174712+0000","last_became_active":"2023-11-28T13:09:07.698965+0000","last_became_peered":"2023-11-28T13:09:07.698965+0000","last_unstale":"2023-11-28T13:09:09.174712+0000","last_undegraded":"2023-11-28T13:09:09.174712+0000","last_fullsized":"2023-11-28T13:09:09.174712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:56.736331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174658+0000","last_change":"2023-11-28T13:09:08.201272+0000","last_active":"2023-11-28T13:09:09.174658+0000","last_peered":"2023-11-28T13:09:09.174658+0000","last_clean":"2023-11-28T13:09:09.174658+0000","last_became_active":"2023-11-28T13:09:08.200690+0000","last_became_peered":"2023-11-28T13:09:08.200690+0000","last_unstale":"2023-11-28T13:09:09.174658+0000","last_undegraded":"2023-11-28T13:09:09.174658+0000","last_fullsized":"2023-11-28T13:09:09.174658+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:00:48.443209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282676+0000","last_change":"2023-11-28T13:09:07.696238+0000","last_active":"2023-11-28T13:09:09.282676+0000","last_peered":"2023-11-28T13:09:09.282676+0000","last_clean":"2023-11-28T13:09:09.282676+0000","last_became_active":"2023-11-28T13:09:07.695861+0000","last_became_peered":"2023-11-28T13:09:07.695861+0000","last_unstale":"2023-11-28T13:09:09.282676+0000","last_undegraded":"2023-11-28T13:09:09.282676+0000","last_fullsized":"2023-11-28T13:09:09.282676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:33.805070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.175875+0000","last_change":"2023-11-28T13:09:08.187892+0000","last_active":"2023-11-28T13:09:09.175875+0000","last_peered":"2023-11-28T13:09:09.175875+0000","last_clean":"2023-11-28T13:09:09.175875+0000","last_became_active":"2023-11-28T13:09:08.187568+0000","last_became_peered":"2023-11-28T13:09:08.187568+0000","last_unstale":"2023-11-28T13:09:09.175875+0000","last_undegraded":"2023-11-28T13:09:09.175875+0000","last_fullsized":"2023-11-28T13:09:09.175875+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:45:00.774152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174420+0000","last_change":"2023-11-28T13:09:07.689897+0000","last_active":"2023-11-28T13:09:09.174420+0000","last_peered":"2023-11-28T13:09:09.174420+0000","last_clean":"2023-11-28T13:09:09.174420+0000","last_became_active":"2023-11-28T13:09:07.689582+0000","last_became_peered":"2023-11-28T13:09:07.689582+0000","last_unstale":"2023-11-28T13:09:09.174420+0000","last_undegraded":"2023-11-28T13:09:09.174420+0000","last_fullsized":"2023-11-28T13:09:09.174420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:08:30.303680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283532+0000","last_change":"2023-11-28T13:09:08.179835+0000","last_active":"2023-11-28T13:09:09.283532+0000","last_peered":"2023-11-28T13:09:09.283532+0000","last_clean":"2023-11-28T13:09:09.283532+0000","last_became_active":"2023-11-28T13:09:08.179540+0000","last_became_peered":"2023-11-28T13:09:08.179540+0000","last_unstale":"2023-11-28T13:09:09.283532+0000","last_undegraded":"2023-11-28T13:09:09.283532+0000","last_fullsized":"2023-11-28T13:09:09.283532+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:54:01.916204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174318+0000","last_change":"2023-11-28T13:09:07.694428+0000","last_active":"2023-11-28T13:09:09.174318+0000","last_peered":"2023-11-28T13:09:09.174318+0000","last_clean":"2023-11-28T13:09:09.174318+0000","last_became_active":"2023-11-28T13:09:07.694043+0000","last_became_peered":"2023-11-28T13:09:07.694043+0000","last_unstale":"2023-11-28T13:09:09.174318+0000","last_undegraded":"2023-11-28T13:09:09.174318+0000","last_fullsized":"2023-11-28T13:09:09.174318+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:31:07.408232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176434+0000","last_change":"2023-11-28T13:09:08.190816+0000","last_active":"2023-11-28T13:09:09.176434+0000","last_peered":"2023-11-28T13:09:09.176434+0000","last_clean":"2023-11-28T13:09:09.176434+0000","last_became_active":"2023-11-28T13:09:08.189681+0000","last_became_peered":"2023-11-28T13:09:08.189681+0000","last_unstale":"2023-11-28T13:09:09.176434+0000","last_undegraded":"2023-11-28T13:09:09.176434+0000","last_fullsized":"2023-11-28T13:09:09.176434+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:30:57.343923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176378+0000","last_change":"2023-11-28T13:09:07.702213+0000","last_active":"2023-11-28T13:09:09.176378+0000","last_peered":"2023-11-28T13:09:09.176378+0000","last_clean":"2023-11-28T13:09:09.176378+0000","last_became_active":"2023-11-28T13:09:07.701948+0000","last_became_peered":"2023-11-28T13:09:07.701948+0000","last_unstale":"2023-11-28T13:09:09.176378+0000","last_undegraded":"2023-11-28T13:09:09.176378+0000","last_fullsized":"2023-11-28T13:09:09.176378+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:56.736331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282227+0000","last_change":"2023-11-28T13:09:08.191608+0000","last_active":"2023-11-28T13:09:09.282227+0000","last_peered":"2023-11-28T13:09:09.282227+0000","last_clean":"2023-11-28T13:09:09.282227+0000","last_became_active":"2023-11-28T13:09:08.191356+0000","last_became_peered":"2023-11-28T13:09:08.191356+0000","last_unstale":"2023-11-28T13:09:09.282227+0000","last_undegraded":"2023-11-28T13:09:09.282227+0000","last_fullsized":"2023-11-28T13:09:09.282227+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:32:38.688473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.174148+0000","last_change":"2023-11-28T13:09:07.687614+0000","last_active":"2023-11-28T13:09:09.174148+0000","last_peered":"2023-11-28T13:09:09.174148+0000","last_clean":"2023-11-28T13:09:09.174148+0000","last_became_active":"2023-11-28T13:09:07.687282+0000","last_became_peered":"2023-11-28T13:09:07.687282+0000","last_unstale":"2023-11-28T13:09:09.174148+0000","last_undegraded":"2023-11-28T13:09:09.174148+0000","last_fullsized":"2023-11-28T13:09:09.174148+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:52.373871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282995+0000","last_change":"2023-11-28T13:09:08.199870+0000","last_active":"2023-11-28T13:09:09.282995+0000","last_peered":"2023-11-28T13:09:09.282995+0000","last_clean":"2023-11-28T13:09:09.282995+0000","last_became_active":"2023-11-28T13:09:08.199494+0000","last_became_peered":"2023-11-28T13:09:08.199494+0000","last_unstale":"2023-11-28T13:09:09.282995+0000","last_undegraded":"2023-11-28T13:09:09.282995+0000","last_fullsized":"2023-11-28T13:09:09.282995+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:32:38.688473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282543+0000","last_change":"2023-11-28T13:09:07.692987+0000","last_active":"2023-11-28T13:09:09.282543+0000","last_peered":"2023-11-28T13:09:09.282543+0000","last_clean":"2023-11-28T13:09:09.282543+0000","last_became_active":"2023-11-28T13:09:07.692327+0000","last_became_peered":"2023-11-28T13:09:07.692327+0000","last_unstale":"2023-11-28T13:09:09.282543+0000","last_undegraded":"2023-11-28T13:09:09.282543+0000","last_fullsized":"2023-11-28T13:09:09.282543+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:01.992362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.197349+0000","last_change":"2023-11-28T13:07:56.890378+0000","last_active":"2023-11-28T13:09:08.197349+0000","last_peered":"2023-11-28T13:09:08.197349+0000","last_clean":"2023-11-28T13:09:08.197349+0000","last_became_active":"2023-11-28T13:07:56.890058+0000","last_became_peered":"2023-11-28T13:07:56.890058+0000","last_unstale":"2023-11-28T13:09:08.197349+0000","last_undegraded":"2023-11-28T13:09:08.197349+0000","last_fullsized":"2023-11-28T13:09:08.197349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:05.344602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176994+0000","last_change":"2023-11-28T13:09:08.183775+0000","last_active":"2023-11-28T13:09:09.176994+0000","last_peered":"2023-11-28T13:09:09.176994+0000","last_clean":"2023-11-28T13:09:09.176994+0000","last_became_active":"2023-11-28T13:09:08.183542+0000","last_became_peered":"2023-11-28T13:09:08.183542+0000","last_unstale":"2023-11-28T13:09:09.176994+0000","last_undegraded":"2023-11-28T13:09:09.176994+0000","last_fullsized":"2023-11-28T13:09:09.176994+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:05:20.629341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282114+0000","last_change":"2023-11-28T13:09:07.692706+0000","last_active":"2023-11-28T13:09:09.282114+0000","last_peered":"2023-11-28T13:09:09.282114+0000","last_clean":"2023-11-28T13:09:09.282114+0000","last_became_active":"2023-11-28T13:09:07.692147+0000","last_became_peered":"2023-11-28T13:09:07.692147+0000","last_unstale":"2023-11-28T13:09:09.282114+0000","last_undegraded":"2023-11-28T13:09:09.282114+0000","last_fullsized":"2023-11-28T13:09:09.282114+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:00:47.441091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.192692+0000","last_change":"2023-11-28T13:07:56.844509+0000","last_active":"2023-11-28T13:09:08.192692+0000","last_peered":"2023-11-28T13:09:08.192692+0000","last_clean":"2023-11-28T13:09:08.192692+0000","last_became_active":"2023-11-28T13:07:56.844081+0000","last_became_peered":"2023-11-28T13:07:56.844081+0000","last_unstale":"2023-11-28T13:09:08.192692+0000","last_undegraded":"2023-11-28T13:09:08.192692+0000","last_fullsized":"2023-11-28T13:09:08.192692+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282578+0000","last_change":"2023-11-28T13:09:08.196928+0000","last_active":"2023-11-28T13:09:09.282578+0000","last_peered":"2023-11-28T13:09:09.282578+0000","last_clean":"2023-11-28T13:09:09.282578+0000","last_became_active":"2023-11-28T13:09:08.196662+0000","last_became_peered":"2023-11-28T13:09:08.196662+0000","last_unstale":"2023-11-28T13:09:09.282578+0000","last_undegraded":"2023-11-28T13:09:09.282578+0000","last_fullsized":"2023-11-28T13:09:09.282578+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:02:27.466286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"34'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.177419+0000","last_change":"2023-11-28T13:09:07.695058+0000","last_active":"2023-11-28T13:09:09.177419+0000","last_peered":"2023-11-28T13:09:09.177419+0000","last_clean":"2023-11-28T13:09:09.177419+0000","last_became_active":"2023-11-28T13:09:07.694756+0000","last_became_peered":"2023-11-28T13:09:07.694756+0000","last_unstale":"2023-11-28T13:09:09.177419+0000","last_undegraded":"2023-11-28T13:09:09.177419+0000","last_fullsized":"2023-11-28T13:09:09.177419+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:00:47.441091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.182702+0000","last_change":"2023-11-28T13:07:56.844699+0000","last_active":"2023-11-28T13:09:08.182702+0000","last_peered":"2023-11-28T13:09:08.182702+0000","last_clean":"2023-11-28T13:09:08.182702+0000","last_became_active":"2023-11-28T13:07:56.844391+0000","last_became_peered":"2023-11-28T13:07:56.844391+0000","last_unstale":"2023-11-28T13:09:08.182702+0000","last_undegraded":"2023-11-28T13:09:08.182702+0000","last_fullsized":"2023-11-28T13:09:08.182702+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":176,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.175559+0000","last_change":"2023-11-28T13:07:57.844020+0000","last_active":"2023-11-28T13:09:08.175559+0000","last_peered":"2023-11-28T13:09:08.175559+0000","last_clean":"2023-11-28T13:09:08.175559+0000","last_became_active":"2023-11-28T13:07:57.842981+0000","last_became_peered":"2023-11-28T13:07:57.842981+0000","last_unstale":"2023-11-28T13:09:08.175559+0000","last_undegraded":"2023-11-28T13:09:08.175559+0000","last_fullsized":"2023-11-28T13:09:08.175559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_clean_scrub_stamp":"2023-11-28T13:07:56.825711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:44.973218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.176633+0000","last_change":"2023-11-28T13:09:08.184319+0000","last_active":"2023-11-28T13:09:09.176633+0000","last_peered":"2023-11-28T13:09:09.176633+0000","last_clean":"2023-11-28T13:09:09.176633+0000","last_became_active":"2023-11-28T13:09:08.183988+0000","last_became_peered":"2023-11-28T13:09:08.183988+0000","last_unstale":"2023-11-28T13:09:09.176633+0000","last_undegraded":"2023-11-28T13:09:09.176633+0000","last_fullsized":"2023-11-28T13:09:09.176633+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:00:08.790472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.830511+0000","last_change":"2023-11-28T13:09:07.690149+0000","last_active":"2023-11-28T13:09:09.830511+0000","last_peered":"2023-11-28T13:09:09.830511+0000","last_clean":"2023-11-28T13:09:09.830511+0000","last_became_active":"2023-11-28T13:09:07.688707+0000","last_became_peered":"2023-11-28T13:09:07.688707+0000","last_unstale":"2023-11-28T13:09:09.830511+0000","last_undegraded":"2023-11-28T13:09:09.830511+0000","last_fullsized":"2023-11-28T13:09:09.830511+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:33.805070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.175653+0000","last_change":"2023-11-28T13:07:57.306318+0000","last_active":"2023-11-28T13:09:08.175653+0000","last_peered":"2023-11-28T13:09:08.175653+0000","last_clean":"2023-11-28T13:09:08.175653+0000","last_became_active":"2023-11-28T13:07:57.305937+0000","last_became_peered":"2023-11-28T13:07:57.305937+0000","last_unstale":"2023-11-28T13:09:08.175653+0000","last_undegraded":"2023-11-28T13:09:08.175653+0000","last_fullsized":"2023-11-28T13:09:08.175653+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.177331+0000","last_change":"2023-11-28T13:09:08.196458+0000","last_active":"2023-11-28T13:09:09.177331+0000","last_peered":"2023-11-28T13:09:09.177331+0000","last_clean":"2023-11-28T13:09:09.177331+0000","last_became_active":"2023-11-28T13:09:08.196196+0000","last_became_peered":"2023-11-28T13:09:08.196196+0000","last_unstale":"2023-11-28T13:09:09.177331+0000","last_undegraded":"2023-11-28T13:09:09.177331+0000","last_fullsized":"2023-11-28T13:09:09.177331+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:11.552323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.177195+0000","last_change":"2023-11-28T13:09:07.701427+0000","last_active":"2023-11-28T13:09:09.177195+0000","last_peered":"2023-11-28T13:09:09.177195+0000","last_clean":"2023-11-28T13:09:09.177195+0000","last_became_active":"2023-11-28T13:09:07.701135+0000","last_became_peered":"2023-11-28T13:09:07.701135+0000","last_unstale":"2023-11-28T13:09:09.177195+0000","last_undegraded":"2023-11-28T13:09:09.177195+0000","last_fullsized":"2023-11-28T13:09:09.177195+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:31:07.408232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.175459+0000","last_change":"2023-11-28T13:07:57.306277+0000","last_active":"2023-11-28T13:09:08.175459+0000","last_peered":"2023-11-28T13:09:08.175459+0000","last_clean":"2023-11-28T13:09:08.175459+0000","last_became_active":"2023-11-28T13:07:57.305983+0000","last_became_peered":"2023-11-28T13:07:57.305983+0000","last_unstale":"2023-11-28T13:09:08.175459+0000","last_undegraded":"2023-11-28T13:09:08.175459+0000","last_fullsized":"2023-11-28T13:09:08.175459+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.283854+0000","last_change":"2023-11-28T13:09:08.204228+0000","last_active":"2023-11-28T13:09:09.283854+0000","last_peered":"2023-11-28T13:09:09.283854+0000","last_clean":"2023-11-28T13:09:09.283854+0000","last_became_active":"2023-11-28T13:09:08.201755+0000","last_became_peered":"2023-11-28T13:09:08.201755+0000","last_unstale":"2023-11-28T13:09:09.283854+0000","last_undegraded":"2023-11-28T13:09:09.283854+0000","last_fullsized":"2023-11-28T13:09:09.283854+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:43.048873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.284311+0000","last_change":"2023-11-28T13:09:07.690279+0000","last_active":"2023-11-28T13:09:09.284311+0000","last_peered":"2023-11-28T13:09:09.284311+0000","last_clean":"2023-11-28T13:09:09.284311+0000","last_became_active":"2023-11-28T13:09:07.689970+0000","last_became_peered":"2023-11-28T13:09:07.689970+0000","last_unstale":"2023-11-28T13:09:09.284311+0000","last_undegraded":"2023-11-28T13:09:09.284311+0000","last_fullsized":"2023-11-28T13:09:09.284311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:24.484191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":29,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.181655+0000","last_change":"2023-11-28T13:07:57.306177+0000","last_active":"2023-11-28T13:09:08.181655+0000","last_peered":"2023-11-28T13:09:08.181655+0000","last_clean":"2023-11-28T13:09:08.181655+0000","last_became_active":"2023-11-28T13:07:57.305853+0000","last_became_peered":"2023-11-28T13:07:57.305853+0000","last_unstale":"2023-11-28T13:09:08.181655+0000","last_undegraded":"2023-11-28T13:09:08.181655+0000","last_fullsized":"2023-11-28T13:09:08.181655+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282562+0000","last_change":"2023-11-28T13:09:08.204667+0000","last_active":"2023-11-28T13:09:09.282562+0000","last_peered":"2023-11-28T13:09:09.282562+0000","last_clean":"2023-11-28T13:09:09.282562+0000","last_became_active":"2023-11-28T13:09:08.204072+0000","last_became_peered":"2023-11-28T13:09:08.204072+0000","last_unstale":"2023-11-28T13:09:09.282562+0000","last_undegraded":"2023-11-28T13:09:09.282562+0000","last_fullsized":"2023-11-28T13:09:09.282562+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:44.819001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.282917+0000","last_change":"2023-11-28T13:09:07.691730+0000","last_active":"2023-11-28T13:09:09.282917+0000","last_peered":"2023-11-28T13:09:09.282917+0000","last_clean":"2023-11-28T13:09:09.282917+0000","last_became_active":"2023-11-28T13:09:07.691439+0000","last_became_peered":"2023-11-28T13:09:07.691439+0000","last_unstale":"2023-11-28T13:09:09.282917+0000","last_undegraded":"2023-11-28T13:09:09.282917+0000","last_fullsized":"2023-11-28T13:09:09.282917+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:22.228475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.176828+0000","last_change":"2023-11-28T13:07:57.306321+0000","last_active":"2023-11-28T13:09:08.176828+0000","last_peered":"2023-11-28T13:09:08.176828+0000","last_clean":"2023-11-28T13:09:08.176828+0000","last_became_active":"2023-11-28T13:07:57.305923+0000","last_became_peered":"2023-11-28T13:07:57.305923+0000","last_unstale":"2023-11-28T13:09:08.176828+0000","last_undegraded":"2023-11-28T13:09:08.176828+0000","last_fullsized":"2023-11-28T13:09:08.176828+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:41.104679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.281847+0000","last_change":"2023-11-28T13:09:08.199317+0000","last_active":"2023-11-28T13:09:09.281847+0000","last_peered":"2023-11-28T13:09:09.281847+0000","last_clean":"2023-11-28T13:09:09.281847+0000","last_became_active":"2023-11-28T13:09:08.198903+0000","last_became_peered":"2023-11-28T13:09:08.198903+0000","last_unstale":"2023-11-28T13:09:09.281847+0000","last_undegraded":"2023-11-28T13:09:09.281847+0000","last_fullsized":"2023-11-28T13:09:09.281847+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:07.671018+0000","last_clean_scrub_stamp":"2023-11-28T13:09:07.671018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:27.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T13:09:09.284131+0000","last_change":"2023-11-28T13:09:07.697118+0000","last_active":"2023-11-28T13:09:09.284131+0000","last_peered":"2023-11-28T13:09:09.284131+0000","last_clean":"2023-11-28T13:09:09.284131+0000","last_became_active":"2023-11-28T13:09:07.696403+0000","last_became_peered":"2023-11-28T13:09:07.696403+0000","last_unstale":"2023-11-28T13:09:09.284131+0000","last_undegraded":"2023-11-28T13:09:09.284131+0000","last_fullsized":"2023-11-28T13:09:09.284131+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:06.668900+0000","last_clean_scrub_stamp":"2023-11-28T13:09:06.668900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:10.168361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-28T13:09:08.176133+0000","last_change":"2023-11-28T13:07:56.844305+0000","last_active":"2023-11-28T13:09:08.176133+0000","last_peered":"2023-11-28T13:09:08.176133+0000","last_clean":"2023-11-28T13:09:08.176133+0000","last_became_active":"2023-11-28T13:07:56.843980+0000","last_became_peered":"2023-11-28T13:07:56.843980+0000","last_unstale":"2023-11-28T13:09:08.176133+0000","last_undegraded":"2023-11-28T13:09:08.176133+0000","last_fullsized":"2023-11-28T13:09:08.176133+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9568,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362272,"statfs":{"total":96636764160,"available":96626966528,"internally_reserved":0,"allocated":1007616,"data_stored":757227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.64800000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.41899999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.76600000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.996}]},{"osd":4,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.90100000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.044},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.85799999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.94899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8336,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363504,"statfs":{"total":96636764160,"available":96628228096,"internally_reserved":0,"allocated":401408,"data_stored":165807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.495},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68899999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59299999999999997}]},{"osd":2,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.626},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.77700000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.755}]},{"osd":5,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.877},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.65200000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.90600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13588,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":12608,"kb_avail":94358252,"statfs":{"total":96636764160,"available":96622850048,"internally_reserved":0,"allocated":995328,"data_stored":755699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12910592},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.35499999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.443},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.434}]},{"osd":3,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.56799999999999995}]},{"osd":4,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.58599999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.61099999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.66500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12556,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":12160,"kb_avail":94359284,"statfs":{"total":96636764160,"available":96623906816,"internally_reserved":0,"allocated":397312,"data_stored":164289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12451840},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.33400000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.40799999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.505},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.42899999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.626}]},{"osd":5,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65400000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.75},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.68000000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12432,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":12032,"kb_avail":94359408,"statfs":{"total":96636764160,"available":96624033792,"internally_reserved":0,"allocated":401408,"data_stored":165311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12320768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.441},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.58199999999999996}]},{"osd":1,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.52200000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.59199999999999997}]},{"osd":4,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.93300000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.871},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.94299999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12636,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":12224,"kb_avail":94359204,"statfs":{"total":96636764160,"available":96623824896,"internally_reserved":0,"allocated":413696,"data_stored":166823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12517376},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.32900000000000001}]},{"osd":1,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.52300000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.54800000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.61199999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.71199999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.74099999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.71999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8596,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363244,"statfs":{"total":96636764160,"available":96627961856,"internally_reserved":0,"allocated":405504,"data_stored":165829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.76300000000000001}]},{"osd":1,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.57499999999999996}]},{"osd":2,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.65400000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.61599999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.63100000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.79400000000000004}]},{"osd":7,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.72899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8468,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363372,"statfs":{"total":96636764160,"available":96628092928,"internally_reserved":0,"allocated":405504,"data_stored":165353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.68999999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57799999999999996}]},{"osd":2,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.52600000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.63},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.83899999999999997}]},{"osd":4,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.749},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.84799999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.81999999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.88800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:09:13.021 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T13:09:13.022 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T13:09:13.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.259+0000 7f1a003ff700 1 Processor -- start 2023-11-28T13:09:13.261 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.259+0000 7f1a003ff700 1 -- start start 2023-11-28T13:09:13.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.261+0000 7f1a003ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f19f815c9b0 0x7f19f80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:13.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.261+0000 7f1a003ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f19f80cd7c0 0x7f19f80cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:13.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.261+0000 7f1a003ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f80d1950 con 0x7f19f8161340 2023-11-28T13:09:13.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.261+0000 7f1a003ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f80b5800 con 0x7f19f815c9b0 2023-11-28T13:09:13.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.261+0000 7f1a003ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80ce140 con 0x7f19f80cd7c0 2023-11-28T13:09:13.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.261+0000 7f19fe19b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f19f815c9b0 0x7f19f80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:13.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.261+0000 7f19fe19b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f19f815c9b0 0x7f19f80b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49726/0 (socket says 172.21.15.156:49726) 2023-11-28T13:09:13.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.261+0000 7f19fe19b700 1 -- 172.21.15.156:0/4109689211 learned_addr learned my addr 172.21.15.156:0/4109689211 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:13.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.261+0000 7f19fe99c700 1 --2- 172.21.15.156:0/4109689211 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f19f80cd7c0 0x7f19f80cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:13.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.261+0000 7f19fe19b700 1 -- 172.21.15.156:0/4109689211 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f19f80cd7c0 msgr2=0x7f19f80cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:13.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.261+0000 7f19fe19b700 1 --2- 172.21.15.156:0/4109689211 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f19f80cd7c0 0x7f19f80cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.261+0000 7f19fe19b700 1 -- 172.21.15.156:0/4109689211 >> v1:172.21.15.186:6789/0 conn(0x7f19f8161340 legacy=0x7f19f80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:13.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.261+0000 7f19fe19b700 1 -- 172.21.15.156:0/4109689211 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f80d24f0 con 0x7f19f815c9b0 2023-11-28T13:09:13.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.262+0000 7f19fe19b700 1 --2- 172.21.15.156:0/4109689211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f815c9b0 0x7f19f80b4bf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f19f40098f0 tx=0x7f19f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5317c983b0ffe657 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:13.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.262+0000 7f19fd199700 1 -- 172.21.15.156:0/4109689211 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f400b8a0 con 0x7f19f815c9b0 2023-11-28T13:09:13.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.262+0000 7f19fd199700 1 -- 172.21.15.156:0/4109689211 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f400ba00 con 0x7f19f815c9b0 2023-11-28T13:09:13.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.262+0000 7f19fd199700 1 -- 172.21.15.156:0/4109689211 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f400bd90 con 0x7f19f815c9b0 2023-11-28T13:09:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.262+0000 7f1a003ff700 1 -- 172.21.15.156:0/4109689211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f815c9b0 msgr2=0x7f19f80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.262+0000 7f1a003ff700 1 --2- 172.21.15.156:0/4109689211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f815c9b0 0x7f19f80b4bf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f19f40098f0 tx=0x7f19f4009c00 comp rx=0 tx=0).stop 2023-11-28T13:09:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.263+0000 7f1a003ff700 1 -- 172.21.15.156:0/4109689211 shutdown_connections 2023-11-28T13:09:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.263+0000 7f1a003ff700 1 --2- 172.21.15.156:0/4109689211 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f19f80cd7c0 0x7f19f80cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.263+0000 7f1a003ff700 1 --2- 172.21.15.156:0/4109689211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f815c9b0 0x7f19f80b4bf0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.263+0000 7f1a003ff700 1 -- 172.21.15.156:0/4109689211 >> 172.21.15.156:0/4109689211 conn(0x7f19f8072600 msgr2=0x7f19f80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.263+0000 7f1a003ff700 1 -- 172.21.15.156:0/4109689211 shutdown_connections 2023-11-28T13:09:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.263+0000 7f1a003ff700 1 -- 172.21.15.156:0/4109689211 wait complete. 2023-11-28T13:09:13.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.264+0000 7f1a003ff700 1 Processor -- start 2023-11-28T13:09:13.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.264+0000 7f1a003ff700 1 -- start start 2023-11-28T13:09:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.265+0000 7f1a003ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f19f80cd7c0 0x7f19f81e6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.265+0000 7f1a003ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19f815c9b0 0x7f19f815ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.265+0000 7f1a003ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f8161340 0x7f19f8154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.265+0000 7f1a003ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f81e7770 con 0x7f19f8161340 2023-11-28T13:09:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.265+0000 7f1a003ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f81e7a50 con 0x7f19f815c9b0 2023-11-28T13:09:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.265+0000 7f1a003ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f81e7d30 con 0x7f19f80cd7c0 2023-11-28T13:09:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.266+0000 7f19fe99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f8161340 0x7f19f8154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.266+0000 7f19fe99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f8161340 0x7f19f8154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49732/0 (socket says 172.21.15.156:49732) 2023-11-28T13:09:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.266+0000 7f19fe99c700 1 -- 172.21.15.156:0/187793910 learned_addr learned my addr 172.21.15.156:0/187793910 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.266+0000 7f19fe19b700 1 --2- 172.21.15.156:0/187793910 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f19f80cd7c0 0x7f19f81e6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.266+0000 7f19fd99a700 1 --2- 172.21.15.156:0/187793910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19f815c9b0 0x7f19f815ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.266+0000 7f19fe99c700 1 -- 172.21.15.156:0/187793910 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f19f80cd7c0 msgr2=0x7f19f81e6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.266+0000 7f19fe99c700 1 --2- 172.21.15.156:0/187793910 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f19f80cd7c0 0x7f19f81e6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.266+0000 7f19fe99c700 1 -- 172.21.15.156:0/187793910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19f815c9b0 msgr2=0x7f19f815ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.266+0000 7f19fe99c700 1 --2- 172.21.15.156:0/187793910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19f815c9b0 0x7f19f815ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.266+0000 7f19fe99c700 1 -- 172.21.15.156:0/187793910 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f4009550 con 0x7f19f8161340 2023-11-28T13:09:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.266+0000 7f19fe99c700 1 --2- 172.21.15.156:0/187793910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f8161340 0x7f19f8154080 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f19f0007e60 tx=0x7f19f000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.266+0000 7f19eeffd700 1 -- 172.21.15.156:0/187793910 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f000ede0 con 0x7f19f8161340 2023-11-28T13:09:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.266+0000 7f19eeffd700 1 -- 172.21.15.156:0/187793910 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f0014420 con 0x7f19f8161340 2023-11-28T13:09:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.266+0000 7f1a003ff700 1 -- 172.21.15.156:0/187793910 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f80d1300 con 0x7f19f8161340 2023-11-28T13:09:13.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.267+0000 7f19eeffd700 1 -- 172.21.15.156:0/187793910 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f00147d0 con 0x7f19f8161340 2023-11-28T13:09:13.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.267+0000 7f1a003ff700 1 -- 172.21.15.156:0/187793910 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f80b5520 con 0x7f19f8161340 2023-11-28T13:09:13.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.268+0000 7f1a003ff700 1 -- 172.21.15.156:0/187793910 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f80d15e0 con 0x7f19f8161340 2023-11-28T13:09:13.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.269+0000 7f19eeffd700 1 -- 172.21.15.156:0/187793910 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f19f0012050 con 0x7f19f8161340 2023-11-28T13:09:13.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.269+0000 7f19eeffd700 1 --2- 172.21.15.156:0/187793910 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f19dc0682f0 0x7f19dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:13.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.269+0000 7f19eeffd700 1 -- 172.21.15.156:0/187793910 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f19f00863b0 con 0x7f19f8161340 2023-11-28T13:09:13.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.270+0000 7f19fe19b700 1 --2- 172.21.15.156:0/187793910 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f19dc0682f0 0x7f19dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:13.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.271+0000 7f19fe19b700 1 --2- 172.21.15.156:0/187793910 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f19dc0682f0 0x7f19dc06a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f19f4014040 tx=0x7f19f4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:13.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.273+0000 7f19eeffd700 1 -- 172.21.15.156:0/187793910 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f19f0052b20 con 0x7f19f8161340 2023-11-28T13:09:13.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.540+0000 7f1a003ff700 1 -- 172.21.15.156:0/187793910 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f19f80745c0 con 0x7f19f8161340 2023-11-28T13:09:13.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.544+0000 7f19eeffd700 1 -- 172.21.15.156:0/187793910 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 184+0+0 (secure 0 0 0) 0x7f19f0056140 con 0x7f19f8161340 2023-11-28T13:09:13.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.547+0000 7f1a003ff700 1 -- 172.21.15.156:0/187793910 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f19dc0682f0 msgr2=0x7f19dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:13.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.547+0000 7f1a003ff700 1 --2- 172.21.15.156:0/187793910 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f19dc0682f0 0x7f19dc06a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f19f4014040 tx=0x7f19f4008030 comp rx=0 tx=0).stop 2023-11-28T13:09:13.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.547+0000 7f1a003ff700 1 -- 172.21.15.156:0/187793910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f8161340 msgr2=0x7f19f8154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:13.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.547+0000 7f1a003ff700 1 --2- 172.21.15.156:0/187793910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f8161340 0x7f19f8154080 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f19f0007e60 tx=0x7f19f000e3c0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.547+0000 7f1a003ff700 1 -- 172.21.15.156:0/187793910 shutdown_connections 2023-11-28T13:09:13.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.547+0000 7f1a003ff700 1 --2- 172.21.15.156:0/187793910 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f19dc0682f0 0x7f19dc06a7a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.547+0000 7f1a003ff700 1 --2- 172.21.15.156:0/187793910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f8161340 0x7f19f8154080 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.547+0000 7f1a003ff700 1 --2- 172.21.15.156:0/187793910 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f19f80cd7c0 0x7f19f81e6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.547+0000 7f1a003ff700 1 --2- 172.21.15.156:0/187793910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19f815c9b0 0x7f19f815ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.547+0000 7f1a003ff700 1 -- 172.21.15.156:0/187793910 >> 172.21.15.156:0/187793910 conn(0x7f19f8072600 msgr2=0x7f19f806a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:13.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.547+0000 7f1a003ff700 1 -- 172.21.15.156:0/187793910 shutdown_connections 2023-11-28T13:09:13.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.547+0000 7f1a003ff700 1 -- 172.21.15.156:0/187793910 wait complete. 2023-11-28T13:09:13.549 INFO:teuthology.orchestra.run.smithi156.stderr:updated caps for client.0 2023-11-28T13:09:13.566 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:09:13.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.761+0000 7faeeb3ff700 1 Processor -- start 2023-11-28T13:09:13.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.761+0000 7faeeb3ff700 1 -- start start 2023-11-28T13:09:13.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.762+0000 7faeeb3ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faee415c9b0 0x7faee40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:13.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.762+0000 7faeeb3ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7faee4161340 0x7faee40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:13.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.762+0000 7faeeb3ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee40d1950 con 0x7faee40d2320 2023-11-28T13:09:13.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.762+0000 7faeeb3ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee40b5800 con 0x7faee415c9b0 2023-11-28T13:09:13.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.762+0000 7faeeb3ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee40b5520 con 0x7faee4161340 2023-11-28T13:09:13.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.762+0000 7faee919b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faee415c9b0 0x7faee40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:13.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.762+0000 7faee919b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faee415c9b0 0x7faee40b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49746/0 (socket says 172.21.15.156:49746) 2023-11-28T13:09:13.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.762+0000 7faee919b700 1 -- 172.21.15.156:0/1502619935 learned_addr learned my addr 172.21.15.156:0/1502619935 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:13.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.763+0000 7faee899a700 1 --2- 172.21.15.156:0/1502619935 >> v2:172.21.15.186:3300/0 conn(0x7faee4161340 0x7faee40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:13.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.763+0000 7faee919b700 1 -- 172.21.15.156:0/1502619935 >> v2:172.21.15.186:3300/0 conn(0x7faee4161340 msgr2=0x7faee40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:13.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.763+0000 7faee919b700 1 --2- 172.21.15.156:0/1502619935 >> v2:172.21.15.186:3300/0 conn(0x7faee4161340 0x7faee40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.763+0000 7faee919b700 1 -- 172.21.15.156:0/1502619935 >> v1:172.21.15.186:6789/0 conn(0x7faee40d2320 legacy=0x7faee40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:13.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.763+0000 7faee919b700 1 -- 172.21.15.156:0/1502619935 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee40d1300 con 0x7faee415c9b0 2023-11-28T13:09:13.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.763+0000 7faee899a700 1 --2- 172.21.15.156:0/1502619935 >> v2:172.21.15.186:3300/0 conn(0x7faee4161340 0x7faee40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:13.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.763+0000 7faee919b700 1 --2- 172.21.15.156:0/1502619935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee415c9b0 0x7faee40b4bf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7faee40685c0 tx=0x7faee0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d085065fd8ba5e21 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:13.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.763+0000 7faedbfff700 1 -- 172.21.15.156:0/1502619935 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee0004230 con 0x7faee415c9b0 2023-11-28T13:09:13.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.763+0000 7faedbfff700 1 -- 172.21.15.156:0/1502619935 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee0004390 con 0x7faee415c9b0 2023-11-28T13:09:13.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.763+0000 7faedbfff700 1 -- 172.21.15.156:0/1502619935 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee00036a0 con 0x7faee415c9b0 2023-11-28T13:09:13.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.764+0000 7faeeb3ff700 1 -- 172.21.15.156:0/1502619935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee415c9b0 msgr2=0x7faee40b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:13.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.764+0000 7faeeb3ff700 1 --2- 172.21.15.156:0/1502619935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee415c9b0 0x7faee40b4bf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7faee40685c0 tx=0x7faee0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.764+0000 7faeeb3ff700 1 -- 172.21.15.156:0/1502619935 shutdown_connections 2023-11-28T13:09:13.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.764+0000 7faeeb3ff700 1 --2- 172.21.15.156:0/1502619935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee415c9b0 0x7faee40b4bf0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.764+0000 7faeeb3ff700 1 --2- 172.21.15.156:0/1502619935 >> v2:172.21.15.186:3300/0 conn(0x7faee4161340 0x7faee40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.764+0000 7faeeb3ff700 1 -- 172.21.15.156:0/1502619935 >> 172.21.15.156:0/1502619935 conn(0x7faee4072600 msgr2=0x7faee40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:13.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.764+0000 7faeeb3ff700 1 -- 172.21.15.156:0/1502619935 shutdown_connections 2023-11-28T13:09:13.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.764+0000 7faeeb3ff700 1 -- 172.21.15.156:0/1502619935 wait complete. 2023-11-28T13:09:13.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.766+0000 7faeeb3ff700 1 Processor -- start 2023-11-28T13:09:13.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.767+0000 7faeeb3ff700 1 -- start start 2023-11-28T13:09:13.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.768+0000 7faeeb3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee40d2320 0x7faee41f6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:13.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.768+0000 7faeeb3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faee415c9b0 0x7faee4154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:13.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.768+0000 7faeeb3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee4161340 0x7faee41f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:13.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.768+0000 7faeeb3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee41f61b0 con 0x7faee40d2320 2023-11-28T13:09:13.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.768+0000 7faeeb3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee41f5b80 con 0x7faee4161340 2023-11-28T13:09:13.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.768+0000 7faeeb3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee41f30c0 con 0x7faee415c9b0 2023-11-28T13:09:13.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.768+0000 7faee919b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee40d2320 0x7faee41f6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:13.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.768+0000 7faee919b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee40d2320 0x7faee41f6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49758/0 (socket says 172.21.15.156:49758) 2023-11-28T13:09:13.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.768+0000 7faee919b700 1 -- 172.21.15.156:0/3459905926 learned_addr learned my addr 172.21.15.156:0/3459905926 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:13.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.768+0000 7faee919b700 1 -- 172.21.15.156:0/3459905926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faee415c9b0 msgr2=0x7faee4154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:13.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.768+0000 7faee919b700 1 --2- 172.21.15.156:0/3459905926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faee415c9b0 0x7faee4154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.768+0000 7faee919b700 1 -- 172.21.15.156:0/3459905926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee4161340 msgr2=0x7faee41f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:13.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.768+0000 7faee919b700 1 --2- 172.21.15.156:0/3459905926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee4161340 0x7faee41f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:13.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.768+0000 7faee919b700 1 -- 172.21.15.156:0/3459905926 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee0009550 con 0x7faee40d2320 2023-11-28T13:09:13.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.769+0000 7faee919b700 1 --2- 172.21.15.156:0/3459905926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee40d2320 0x7faee41f6640 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faee0009c10 tx=0x7faee0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:13.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.769+0000 7faed9ffb700 1 -- 172.21.15.156:0/3459905926 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee0003d80 con 0x7faee40d2320 2023-11-28T13:09:13.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.769+0000 7faed9ffb700 1 -- 172.21.15.156:0/3459905926 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee001f070 con 0x7faee40d2320 2023-11-28T13:09:13.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.769+0000 7faeeb3ff700 1 -- 172.21.15.156:0/3459905926 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee40b5130 con 0x7faee40d2320 2023-11-28T13:09:13.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.769+0000 7faed9ffb700 1 -- 172.21.15.156:0/3459905926 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee0011870 con 0x7faee40d2320 2023-11-28T13:09:13.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.769+0000 7faeeb3ff700 1 -- 172.21.15.156:0/3459905926 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee40d15e0 con 0x7faee40d2320 2023-11-28T13:09:13.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.770+0000 7faeeb3ff700 1 -- 172.21.15.156:0/3459905926 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee40b52c0 con 0x7faee40d2320 2023-11-28T13:09:13.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.771+0000 7faed9ffb700 1 -- 172.21.15.156:0/3459905926 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faee0017020 con 0x7faee40d2320 2023-11-28T13:09:13.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.772+0000 7faed9ffb700 1 --2- 172.21.15.156:0/3459905926 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faec80682f0 0x7faec806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:13.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.772+0000 7faed9ffb700 1 -- 172.21.15.156:0/3459905926 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faee00874d0 con 0x7faee40d2320 2023-11-28T13:09:13.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.772+0000 7faee899a700 1 --2- 172.21.15.156:0/3459905926 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faec80682f0 0x7faec806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:13.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.773+0000 7faee899a700 1 --2- 172.21.15.156:0/3459905926 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faec80682f0 0x7faec806a7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faee4160190 tx=0x7faed4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:13.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:13.776+0000 7faed9ffb700 1 -- 172.21.15.156:0/3459905926 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faee0053c40 con 0x7faee40d2320 2023-11-28T13:09:14.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.006+0000 7faeeb3ff700 1 -- 172.21.15.156:0/3459905926 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faee40745c0 con 0x7faee40d2320 2023-11-28T13:09:14.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.007+0000 7faed9ffb700 1 -- 172.21.15.156:0/3459905926 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7faee0057260 con 0x7faee40d2320 2023-11-28T13:09:14.008 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:14.008 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1128367125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1128367125},{"type":"v1","addr":"172.21.15.156:6835","nonce":1128367125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4942,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/4096182008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4096182008},{"type":"v1","addr":"172.21.15.186:6835","nonce":4096182008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4945,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/561812165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":561812165},{"type":"v1","addr":"172.21.15.186:6837","nonce":561812165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:09:08.167940+0000","modified":"2023-11-28T13:09:11.184420+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5013":{"gid":5013,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6834/3471743149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3471743149},{"type":"v1","addr":"172.21.15.156:6834","nonce":3471743149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:09:14.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.009+0000 7faeeb3ff700 1 -- 172.21.15.156:0/3459905926 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faec80682f0 msgr2=0x7faec806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:14.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.009+0000 7faeeb3ff700 1 --2- 172.21.15.156:0/3459905926 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faec80682f0 0x7faec806a7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faee4160190 tx=0x7faed4008040 comp rx=0 tx=0).stop 2023-11-28T13:09:14.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.009+0000 7faeeb3ff700 1 -- 172.21.15.156:0/3459905926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee40d2320 msgr2=0x7faee41f6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:14.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.009+0000 7faeeb3ff700 1 --2- 172.21.15.156:0/3459905926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee40d2320 0x7faee41f6640 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faee0009c10 tx=0x7faee0004750 comp rx=0 tx=0).stop 2023-11-28T13:09:14.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.009+0000 7faeeb3ff700 1 -- 172.21.15.156:0/3459905926 shutdown_connections 2023-11-28T13:09:14.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.009+0000 7faeeb3ff700 1 --2- 172.21.15.156:0/3459905926 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faec80682f0 0x7faec806a7a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.009+0000 7faeeb3ff700 1 --2- 172.21.15.156:0/3459905926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee4161340 0x7faee41f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.009+0000 7faeeb3ff700 1 --2- 172.21.15.156:0/3459905926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faee40d2320 0x7faee41f6640 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.009+0000 7faeeb3ff700 1 --2- 172.21.15.156:0/3459905926 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faee415c9b0 0x7faee4154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.009+0000 7faeeb3ff700 1 -- 172.21.15.156:0/3459905926 >> 172.21.15.156:0/3459905926 conn(0x7faee4072600 msgr2=0x7faee415a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:14.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.009+0000 7faeeb3ff700 1 -- 172.21.15.156:0/3459905926 shutdown_connections 2023-11-28T13:09:14.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.010+0000 7faeeb3ff700 1 -- 172.21.15.156:0/3459905926 wait complete. 2023-11-28T13:09:14.011 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2023-11-28T13:09:14.431 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T13:09:08.167940+0000', 'modified': '2023-11-28T13:09:11.184420+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5013}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5013': {'gid': 5013, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.156:6834/3471743149', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6832', 'nonce': 3471743149}, {'type': 'v1', 'addr': '172.21.15.156:6834', 'nonce': 3471743149}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T13:09:14.431 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T13:09:14.431 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T13:09:14.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.631+0000 7fa2dfdff700 1 Processor -- start 2023-11-28T13:09:14.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.632+0000 7fa2dfdff700 1 -- start start 2023-11-28T13:09:14.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.633+0000 7fa2dfdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d8161340 0x7fa2d80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.633+0000 7fa2dfdff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa2d80d2320 0x7fa2d80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.633+0000 7fa2dfdff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d80d1950 con 0x7fa2d815c9b0 2023-11-28T13:09:14.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.633+0000 7fa2dfdff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d80b5800 con 0x7fa2d80d2320 2023-11-28T13:09:14.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.633+0000 7fa2dfdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d80b5300 con 0x7fa2d8161340 2023-11-28T13:09:14.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.633+0000 7fa2ddb9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa2d815c9b0 0x7fa2d80b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:43498/0 (socket says 172.21.15.156:43498) 2023-11-28T13:09:14.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.633+0000 7fa2ddb9b700 1 -- 172.21.15.156:0/2576922060 learned_addr learned my addr 172.21.15.156:0/2576922060 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:14.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.633+0000 7fa2de39c700 1 --2- 172.21.15.156:0/2576922060 >> v2:172.21.15.156:3300/0 conn(0x7fa2d80d2320 0x7fa2d80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.633+0000 7fa2dd39a700 1 --2- 172.21.15.156:0/2576922060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d8161340 0x7fa2d80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.633+0000 7fa2dcb99700 1 -- 172.21.15.156:0/2576922060 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475517944 0 0) 0x7fa2d80d1950 con 0x7fa2d815c9b0 2023-11-28T13:09:14.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.634+0000 7fa2dcb99700 1 -- 172.21.15.156:0/2576922060 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c40035b0 con 0x7fa2d815c9b0 2023-11-28T13:09:14.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.634+0000 7fa2de39c700 1 -- 172.21.15.156:0/2576922060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d8161340 msgr2=0x7fa2d80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:14.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.634+0000 7fa2de39c700 1 --2- 172.21.15.156:0/2576922060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d8161340 0x7fa2d80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.634+0000 7fa2de39c700 1 -- 172.21.15.156:0/2576922060 >> v1:172.21.15.156:6789/0 conn(0x7fa2d815c9b0 legacy=0x7fa2d80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:14.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.634+0000 7fa2de39c700 1 -- 172.21.15.156:0/2576922060 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d80b5520 con 0x7fa2d80d2320 2023-11-28T13:09:14.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.634+0000 7fa2de39c700 1 --2- 172.21.15.156:0/2576922060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d80d2320 0x7fa2d80cd7c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009dc0 tx=0x7fa2d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7469360b82edf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.634+0000 7fa2dcb99700 1 -- 172.21.15.156:0/2576922060 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d000e070 con 0x7fa2d80d2320 2023-11-28T13:09:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.634+0000 7fa2dcb99700 1 -- 172.21.15.156:0/2576922060 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d000bae0 con 0x7fa2d80d2320 2023-11-28T13:09:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.634+0000 7fa2dcb99700 1 -- 172.21.15.156:0/2576922060 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d000be90 con 0x7fa2d80d2320 2023-11-28T13:09:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.635+0000 7fa2dfdff700 1 -- 172.21.15.156:0/2576922060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d80d2320 msgr2=0x7fa2d80cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.635+0000 7fa2dfdff700 1 --2- 172.21.15.156:0/2576922060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d80d2320 0x7fa2d80cd7c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009dc0 tx=0x7fa2d000b330 comp rx=0 tx=0).stop 2023-11-28T13:09:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.635+0000 7fa2dfdff700 1 -- 172.21.15.156:0/2576922060 shutdown_connections 2023-11-28T13:09:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.635+0000 7fa2dfdff700 1 --2- 172.21.15.156:0/2576922060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d80d2320 0x7fa2d80cd7c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.635+0000 7fa2dfdff700 1 --2- 172.21.15.156:0/2576922060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d8161340 0x7fa2d80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.635+0000 7fa2dfdff700 1 -- 172.21.15.156:0/2576922060 >> 172.21.15.156:0/2576922060 conn(0x7fa2d8072600 msgr2=0x7fa2d80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:14.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.635+0000 7fa2dfdff700 1 -- 172.21.15.156:0/2576922060 shutdown_connections 2023-11-28T13:09:14.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.635+0000 7fa2dfdff700 1 -- 172.21.15.156:0/2576922060 wait complete. 2023-11-28T13:09:14.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.637+0000 7fa2dfdff700 1 Processor -- start 2023-11-28T13:09:14.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.637+0000 7fa2dfdff700 1 -- start start 2023-11-28T13:09:14.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.638+0000 7fa2dfdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d80d2320 0x7fa2d8152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.638+0000 7fa2dfdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2d815c9b0 0x7fa2d8143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.638+0000 7fa2dfdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d8161340 0x7fa2d81514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.638+0000 7fa2dfdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d81439c0 con 0x7fa2d8161340 2023-11-28T13:09:14.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.638+0000 7fa2dfdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d8151dc0 con 0x7fa2d815c9b0 2023-11-28T13:09:14.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.638+0000 7fa2dfdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d81520a0 con 0x7fa2d80d2320 2023-11-28T13:09:14.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.638+0000 7fa2de39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d8161340 0x7fa2d81514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.638+0000 7fa2de39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d8161340 0x7fa2d81514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49782/0 (socket says 172.21.15.156:49782) 2023-11-28T13:09:14.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.638+0000 7fa2de39c700 1 -- 172.21.15.156:0/3587453640 learned_addr learned my addr 172.21.15.156:0/3587453640 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:14.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.638+0000 7fa2dd39a700 1 --2- 172.21.15.156:0/3587453640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2d815c9b0 0x7fa2d8143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.638+0000 7fa2ddb9b700 1 --2- 172.21.15.156:0/3587453640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d80d2320 0x7fa2d8152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.638+0000 7fa2de39c700 1 -- 172.21.15.156:0/3587453640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d80d2320 msgr2=0x7fa2d8152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:14.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.638+0000 7fa2de39c700 1 --2- 172.21.15.156:0/3587453640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d80d2320 0x7fa2d8152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.638+0000 7fa2de39c700 1 -- 172.21.15.156:0/3587453640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2d815c9b0 msgr2=0x7fa2d8143480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:14.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.638+0000 7fa2de39c700 1 --2- 172.21.15.156:0/3587453640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2d815c9b0 0x7fa2d8143480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.638+0000 7fa2de39c700 1 -- 172.21.15.156:0/3587453640 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d0009a70 con 0x7fa2d8161340 2023-11-28T13:09:14.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.639+0000 7fa2de39c700 1 --2- 172.21.15.156:0/3587453640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d8161340 0x7fa2d81514e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000b370 tx=0x7fa2d0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:14.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.639+0000 7fa2ce7fc700 1 -- 172.21.15.156:0/3587453640 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00044d0 con 0x7fa2d8161340 2023-11-28T13:09:14.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.639+0000 7fa2ce7fc700 1 -- 172.21.15.156:0/3587453640 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d0020070 con 0x7fa2d8161340 2023-11-28T13:09:14.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.639+0000 7fa2ce7fc700 1 -- 172.21.15.156:0/3587453640 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00048b0 con 0x7fa2d8161340 2023-11-28T13:09:14.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.639+0000 7fa2dfdff700 1 -- 172.21.15.156:0/3587453640 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d80d15e0 con 0x7fa2d8161340 2023-11-28T13:09:14.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.639+0000 7fa2dfdff700 1 -- 172.21.15.156:0/3587453640 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d80d1300 con 0x7fa2d8161340 2023-11-28T13:09:14.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.641+0000 7fa2ce7fc700 1 -- 172.21.15.156:0/3587453640 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa2d0012430 con 0x7fa2d8161340 2023-11-28T13:09:14.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.642+0000 7fa2ce7fc700 1 --2- 172.21.15.156:0/3587453640 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa2bc0682f0 0x7fa2bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.642+0000 7fa2ce7fc700 1 -- 172.21.15.156:0/3587453640 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa2d0022070 con 0x7fa2d8161340 2023-11-28T13:09:14.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.642+0000 7fa2ddb9b700 1 --2- 172.21.15.156:0/3587453640 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa2bc0682f0 0x7fa2bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.643+0000 7fa2ddb9b700 1 --2- 172.21.15.156:0/3587453640 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa2bc0682f0 0x7fa2bc06a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4007fd0 tx=0x7fa2d4007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:14.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.644+0000 7fa2f3643b80 1 Processor -- start 2023-11-28T13:09:14.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.644+0000 7fa2f3643b80 1 -- start start 2023-11-28T13:09:14.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.645+0000 7fa2f3643b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46167810 0x55fd4613da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.645+0000 7fa2f3643b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46165130 0x55fd46167be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.645+0000 7fa2f3643b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd46183770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.645+0000 7fa2f3643b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd46027bc0 con 0x55fd46165130 2023-11-28T13:09:14.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.645+0000 7fa2f3643b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd46096350 con 0x55fd46168120 2023-11-28T13:09:14.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.645+0000 7fa2f3643b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd4608f880 con 0x55fd46167810 2023-11-28T13:09:14.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.645+0000 7fa2dd39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46165130 0x55fd46167be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.645+0000 7fa2dd39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46165130 0x55fd46167be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49792/0 (socket says 172.21.15.156:49792) 2023-11-28T13:09:14.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.645+0000 7fa2dd39a700 1 -- 172.21.15.156:0/3952675031 learned_addr learned my addr 172.21.15.156:0/3952675031 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:14.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.645+0000 7fa2de39c700 1 --2- 172.21.15.156:0/3952675031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd46183770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.645+0000 7fa2dd39a700 1 -- 172.21.15.156:0/3952675031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46167810 msgr2=0x55fd4613da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:14.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.645+0000 7fa2dd39a700 1 --2- 172.21.15.156:0/3952675031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46167810 0x55fd4613da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.645+0000 7fa2dd39a700 1 -- 172.21.15.156:0/3952675031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 msgr2=0x55fd46183770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:14.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.645+0000 7fa2dd39a700 1 --2- 172.21.15.156:0/3952675031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd46183770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.645+0000 7fa2dd39a700 1 -- 172.21.15.156:0/3952675031 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd45f115e0 con 0x55fd46165130 2023-11-28T13:09:14.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.645+0000 7fa2de39c700 1 --2- 172.21.15.156:0/3952675031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd46183770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:14.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.645+0000 7fa2dd39a700 1 --2- 172.21.15.156:0/3952675031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46165130 0x55fd46167be0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009c90 tx=0x7fa2c80112c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf928ca9fe9814e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:14.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.646+0000 7fa2bb7fe700 1 -- 172.21.15.156:0/3952675031 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8011c60 con 0x55fd46165130 2023-11-28T13:09:14.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.646+0000 7fa2bb7fe700 1 -- 172.21.15.156:0/3952675031 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c8011dc0 con 0x55fd46165130 2023-11-28T13:09:14.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.646+0000 7fa2bb7fe700 1 -- 172.21.15.156:0/3952675031 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8004250 con 0x55fd46165130 2023-11-28T13:09:14.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.646+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3952675031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46165130 msgr2=0x55fd46167be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:14.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.646+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/3952675031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46165130 0x55fd46167be0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009c90 tx=0x7fa2c80112c0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.646+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3952675031 shutdown_connections 2023-11-28T13:09:14.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.646+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/3952675031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd46183770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.646+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/3952675031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46167810 0x55fd4613da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.646+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/3952675031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46165130 0x55fd46167be0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.646+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3952675031 >> 172.21.15.156:0/3952675031 conn(0x55fd4613ad60 msgr2=0x55fd4613b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:14.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.647+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3952675031 shutdown_connections 2023-11-28T13:09:14.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.647+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3952675031 wait complete. 2023-11-28T13:09:14.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.647+0000 7fa2f3643b80 1 -- start start 2023-11-28T13:09:14.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.648+0000 7fa2f3643b80 1 Processor -- start 2023-11-28T13:09:14.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.648+0000 7fa2f3643b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46165130 0x55fd46160020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2f3643b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd46160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2f3643b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd461889f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2f3643b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd45f138d0 con 0x55fd46167810 2023-11-28T13:09:14.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2f3643b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd461615b0 con 0x55fd46168120 2023-11-28T13:09:14.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2f3643b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd46187700 con 0x55fd46165130 2023-11-28T13:09:14.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2f3643b80 10 client.0 _ll_register_callbacks cb 0x55fd45f35240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:09:14.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2dd39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd46160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2dd39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd46160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49798/0 (socket says 172.21.15.156:49798) 2023-11-28T13:09:14.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2dd39a700 1 -- 172.21.15.156:0/2762040585 learned_addr learned my addr 172.21.15.156:0/2762040585 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:14.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2ddb9b700 1 --2- 172.21.15.156:0/2762040585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46165130 0x55fd46160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2de39c700 1 --2- 172.21.15.156:0/2762040585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd461889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2dd39a700 1 -- 172.21.15.156:0/2762040585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46165130 msgr2=0x55fd46160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:14.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2dd39a700 1 --2- 172.21.15.156:0/2762040585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46165130 0x55fd46160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2dd39a700 1 -- 172.21.15.156:0/2762040585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 msgr2=0x55fd461889f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:14.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2dd39a700 1 --2- 172.21.15.156:0/2762040585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd461889f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2dd39a700 1 -- 172.21.15.156:0/2762040585 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c80098c0 con 0x55fd46167810 2023-11-28T13:09:14.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2ddb9b700 1 --2- 172.21.15.156:0/2762040585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46165130 0x55fd46160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:14.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.649+0000 7fa2dd39a700 1 --2- 172.21.15.156:0/2762040585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd46160b30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8018000 tx=0x7fa2c80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b07ce31cec8e35 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:14.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.650+0000 7fa2b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:09:14.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.650+0000 7fa2b8ff9700 1 -- 172.21.15.156:0/2762040585 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8024070 con 0x55fd46167810 2023-11-28T13:09:14.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.650+0000 7fa2b8ff9700 1 -- 172.21.15.156:0/2762040585 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa2c80036a0 con 0x55fd46167810 2023-11-28T13:09:14.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.650+0000 7fa2f3643b80 1 -- 172.21.15.156:0/2762040585 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd4605d140 con 0x55fd46167810 2023-11-28T13:09:14.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.650+0000 7fa2b8ff9700 1 -- 172.21.15.156:0/2762040585 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2c80045d0 con 0x55fd46167810 2023-11-28T13:09:14.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.650+0000 7fa2f3643b80 10 client.5211 fetch_fsmap learned FSMap version 16 2023-11-28T13:09:14.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.650+0000 7fa2f3643b80 1 -- 172.21.15.156:0/2762040585 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55fd45ebdd90 con 0x55fd46167810 2023-11-28T13:09:14.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.650+0000 7fa2b8ff9700 1 -- 172.21.15.156:0/2762040585 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fd4605d140 con 0x55fd46167810 2023-11-28T13:09:14.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.650+0000 7fa2b8ff9700 1 -- 172.21.15.156:0/2762040585 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa2c8011020 con 0x55fd46167810 2023-11-28T13:09:14.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.650+0000 7fa2f3643b80 10 client.5211 fetch_fsmap finished waiting for FSMap version 16 2023-11-28T13:09:14.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.650+0000 7fa2f3643b80 10 client.5211 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:09:14.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.650+0000 7fa2f3643b80 20 client.5211 populate_metadata read hostname 'smithi156' 2023-11-28T13:09:14.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.650+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/2762040585 >> [v2:172.21.15.156:6832/3471743149,v1:172.21.15.156:6834/3471743149] conn(0x55fd4617c4f0 0x55fd46186020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.650+0000 7fa2f3643b80 4 client.5211 mds_command: new command op to 5013 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:09:14.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.650+0000 7fa2f3643b80 1 -- 172.21.15.156:0/2762040585 --> [v2:172.21.15.156:6832/3471743149,v1:172.21.15.156:6834/3471743149] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd45f13f10 con 0x55fd4617c4f0 2023-11-28T13:09:14.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.651+0000 7fa2ddb9b700 1 --2- 172.21.15.156:0/2762040585 >> [v2:172.21.15.156:6832/3471743149,v1:172.21.15.156:6834/3471743149] conn(0x55fd4617c4f0 0x55fd46186020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.651+0000 7fa2ddb9b700 1 --2- 172.21.15.156:0/2762040585 >> [v2:172.21.15.156:6832/3471743149,v1:172.21.15.156:6834/3471743149] conn(0x55fd4617c4f0 0x55fd46186020 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f50f53e7ae747ef server_cookie=842e81ba81d6428a in_seq=0 out_seq=0 2023-11-28T13:09:14.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.651+0000 7fa2b8ff9700 10 client.5211 ms_handle_connect on v2:172.21.15.156:6832/3471743149 2023-11-28T13:09:14.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.652+0000 7fa2b8ff9700 1 -- 172.21.15.156:0/2762040585 <== mds.0 v2:172.21.15.156:6832/3471743149 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55fd45f13f10 con 0x55fd4617c4f0 2023-11-28T13:09:14.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.652+0000 7fa2b8ff9700 10 client.5211 handle_command_reply: tid=0 2023-11-28T13:09:14.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.652+0000 7fa2f3643b80 1 client.5211 shutdown 2023-11-28T13:09:14.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.652+0000 7fa2f3643b80 1 -- 172.21.15.156:0/2762040585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 msgr2=0x55fd46160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:14.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.652+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/2762040585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd46160b30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8018000 tx=0x7fa2c80047c0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.652+0000 7fa2f3643b80 1 -- 172.21.15.156:0/2762040585 shutdown_connections 2023-11-28T13:09:14.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.652+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/2762040585 >> [v2:172.21.15.156:6832/3471743149,v1:172.21.15.156:6834/3471743149] conn(0x55fd4617c4f0 0x55fd46186020 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.652+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/2762040585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd461889f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.652+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/2762040585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46165130 0x55fd46160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.652+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/2762040585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd46160b30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.652+0000 7fa2b8ff9700 0 client.5211 ms_handle_reset on v2:172.21.15.156:6832/3471743149 2023-11-28T13:09:14.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.652+0000 7fa2f3643b80 1 -- 172.21.15.156:0/2762040585 >> 172.21.15.156:0/2762040585 conn(0x55fd4613ad60 msgr2=0x55fd4613d990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:14.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.652+0000 7fa2f3643b80 1 -- 172.21.15.156:0/2762040585 shutdown_connections 2023-11-28T13:09:14.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.652+0000 7fa2f3643b80 1 -- 172.21.15.156:0/2762040585 wait complete. 2023-11-28T13:09:14.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.653+0000 7fa2f3643b80 20 client.5211 trim_cache size 0 max 16384 2023-11-28T13:09:14.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.671+0000 7fa2f3643b80 1 Processor -- start 2023-11-28T13:09:14.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.671+0000 7fa2f3643b80 1 -- start start 2023-11-28T13:09:14.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.671+0000 7fa2f3643b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46165130 0x55fd4615ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.672+0000 7fa2f3643b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd4613c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.672+0000 7fa2f3643b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd4613c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.672+0000 7fa2f3643b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd46160630 con 0x55fd46167810 2023-11-28T13:09:14.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.672+0000 7fa2f3643b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4613a550 con 0x55fd46168120 2023-11-28T13:09:14.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.672+0000 7fa2f3643b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd461365f0 con 0x55fd46165130 2023-11-28T13:09:14.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.672+0000 7fa2dd39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd4613c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.672+0000 7fa2dd39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd4613c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49808/0 (socket says 172.21.15.156:49808) 2023-11-28T13:09:14.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.672+0000 7fa2dd39a700 1 -- 172.21.15.156:0/3984539528 learned_addr learned my addr 172.21.15.156:0/3984539528 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:14.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.672+0000 7fa2de39c700 1 --2- 172.21.15.156:0/3984539528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd4613c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.672+0000 7fa2ddb9b700 1 --2- 172.21.15.156:0/3984539528 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46165130 0x55fd4615ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.672+0000 7fa2dd39a700 1 -- 172.21.15.156:0/3984539528 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46165130 msgr2=0x55fd4615ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:14.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.672+0000 7fa2dd39a700 1 --2- 172.21.15.156:0/3984539528 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46165130 0x55fd4615ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.672+0000 7fa2dd39a700 1 -- 172.21.15.156:0/3984539528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 msgr2=0x55fd4613c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:14.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.672+0000 7fa2dd39a700 1 --2- 172.21.15.156:0/3984539528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd4613c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.672+0000 7fa2dd39a700 1 -- 172.21.15.156:0/3984539528 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd46056140 con 0x55fd46167810 2023-11-28T13:09:14.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.672+0000 7fa2ddb9b700 1 --2- 172.21.15.156:0/3984539528 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46165130 0x55fd4615ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:14.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.673+0000 7fa2dd39a700 1 --2- 172.21.15.156:0/3984539528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd4613c230 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009bb0 tx=0x7fa2c8024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7559cf4ce148da9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:14.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.673+0000 7fa2bbfff700 1 -- 172.21.15.156:0/3984539528 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8003cc0 con 0x55fd46167810 2023-11-28T13:09:14.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.673+0000 7fa2bbfff700 1 -- 172.21.15.156:0/3984539528 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c8003900 con 0x55fd46167810 2023-11-28T13:09:14.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.673+0000 7fa2bbfff700 1 -- 172.21.15.156:0/3984539528 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c801e8e0 con 0x55fd46167810 2023-11-28T13:09:14.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.673+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3984539528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 msgr2=0x55fd4613c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:14.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.673+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/3984539528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd4613c230 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009bb0 tx=0x7fa2c8024000 comp rx=0 tx=0).stop 2023-11-28T13:09:14.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.673+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3984539528 shutdown_connections 2023-11-28T13:09:14.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.673+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/3984539528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd4613c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.673+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/3984539528 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46165130 0x55fd4615ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.673+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/3984539528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd4613c230 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.673+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3984539528 >> 172.21.15.156:0/3984539528 conn(0x55fd4613ad60 msgr2=0x55fd46137e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:14.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.674+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3984539528 shutdown_connections 2023-11-28T13:09:14.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.674+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3984539528 wait complete. 2023-11-28T13:09:14.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.674+0000 7fa2f3643b80 1 -- start start 2023-11-28T13:09:14.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.674+0000 7fa2f3643b80 1 Processor -- start 2023-11-28T13:09:14.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.675+0000 7fa2f3643b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46165130 0x55fd46136450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.675+0000 7fa2f3643b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd46136990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.675+0000 7fa2f3643b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd4613ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.675+0000 7fa2f3643b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd461a4480 con 0x55fd46167810 2023-11-28T13:09:14.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.675+0000 7fa2f3643b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd461e7670 con 0x55fd46168120 2023-11-28T13:09:14.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.675+0000 7fa2f3643b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd46139c10 con 0x55fd46165130 2023-11-28T13:09:14.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.675+0000 7fa2f3643b80 10 client.0 _ll_register_callbacks cb 0x55fd45f35240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:09:14.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.676+0000 7fa2dd39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd46136990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.676+0000 7fa2dd39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd46136990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49824/0 (socket says 172.21.15.156:49824) 2023-11-28T13:09:14.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.676+0000 7fa2dd39a700 1 -- 172.21.15.156:0/3358768299 learned_addr learned my addr 172.21.15.156:0/3358768299 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:14.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.676+0000 7fa2de39c700 1 --2- 172.21.15.156:0/3358768299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd4613ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.676+0000 7fa2dd39a700 1 -- 172.21.15.156:0/3358768299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46165130 msgr2=0x55fd46136450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:14.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.676+0000 7fa2dd39a700 1 --2- 172.21.15.156:0/3358768299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46165130 0x55fd46136450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.676+0000 7fa2dd39a700 1 -- 172.21.15.156:0/3358768299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 msgr2=0x55fd4613ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:14.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.676+0000 7fa2dd39a700 1 --2- 172.21.15.156:0/3358768299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd4613ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.676+0000 7fa2dd39a700 1 -- 172.21.15.156:0/3358768299 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd45ebdd90 con 0x55fd46167810 2023-11-28T13:09:14.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.676+0000 7fa2de39c700 1 --2- 172.21.15.156:0/3358768299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd4613ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:14.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.676+0000 7fa2dd39a700 1 --2- 172.21.15.156:0/3358768299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd46136990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55fd46184c30 tx=0x7fa2c8009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47326f6f63d64de server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:14.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.676+0000 7fa2b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:09:14.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.676+0000 7fa2b8ff9700 1 -- 172.21.15.156:0/3358768299 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8024040 con 0x55fd46167810 2023-11-28T13:09:14.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.676+0000 7fa2b8ff9700 1 -- 172.21.15.156:0/3358768299 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa2c8000e40 con 0x55fd46167810 2023-11-28T13:09:14.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.676+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3358768299 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd4604d160 con 0x55fd46167810 2023-11-28T13:09:14.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.677+0000 7fa2b8ff9700 1 -- 172.21.15.156:0/3358768299 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2c80036a0 con 0x55fd46167810 2023-11-28T13:09:14.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.677+0000 7fa2f3643b80 10 client.5217 fetch_fsmap learned FSMap version 16 2023-11-28T13:09:14.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.677+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3358768299 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55fd45ee4200 con 0x55fd46167810 2023-11-28T13:09:14.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.677+0000 7fa2b8ff9700 1 -- 172.21.15.156:0/3358768299 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fd4604d160 con 0x55fd46167810 2023-11-28T13:09:14.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.677+0000 7fa2b8ff9700 1 -- 172.21.15.156:0/3358768299 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa2c8009cc0 con 0x55fd46167810 2023-11-28T13:09:14.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.677+0000 7fa2f3643b80 10 client.5217 fetch_fsmap finished waiting for FSMap version 16 2023-11-28T13:09:14.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.677+0000 7fa2f3643b80 10 client.5217 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:09:14.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.677+0000 7fa2f3643b80 20 client.5217 populate_metadata read hostname 'smithi156' 2023-11-28T13:09:14.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.677+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/3358768299 >> [v2:172.21.15.156:6832/3471743149,v1:172.21.15.156:6834/3471743149] conn(0x55fd4618a990 0x55fd4613d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.677+0000 7fa2f3643b80 4 client.5217 mds_command: new command op to 5013 tid=0[{"prefix": "status"}] 2023-11-28T13:09:14.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.677+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3358768299 --> [v2:172.21.15.156:6832/3471743149,v1:172.21.15.156:6834/3471743149] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fd46037170 con 0x55fd4618a990 2023-11-28T13:09:14.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.677+0000 7fa2ddb9b700 1 --2- 172.21.15.156:0/3358768299 >> [v2:172.21.15.156:6832/3471743149,v1:172.21.15.156:6834/3471743149] conn(0x55fd4618a990 0x55fd4613d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.678+0000 7fa2ddb9b700 1 --2- 172.21.15.156:0/3358768299 >> [v2:172.21.15.156:6832/3471743149,v1:172.21.15.156:6834/3471743149] conn(0x55fd4618a990 0x55fd4613d3c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f988380ad611a646 server_cookie=ad4a30540e2fb339 in_seq=0 out_seq=0 2023-11-28T13:09:14.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.678+0000 7fa2b8ff9700 10 client.5217 ms_handle_connect on v2:172.21.15.156:6832/3471743149 2023-11-28T13:09:14.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.678+0000 7fa2b8ff9700 1 -- 172.21.15.156:0/3358768299 <== mds.0 v2:172.21.15.156:6832/3471743149 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55fd46037170 con 0x55fd4618a990 2023-11-28T13:09:14.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.678+0000 7fa2b8ff9700 10 client.5217 handle_command_reply: tid=0 2023-11-28T13:09:14.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.678+0000 7fa2f3643b80 1 client.5217 shutdown 2023-11-28T13:09:14.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.678+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3358768299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 msgr2=0x55fd46136990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:14.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.678+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/3358768299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd46136990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55fd46184c30 tx=0x7fa2c8009210 comp rx=0 tx=0).stop 2023-11-28T13:09:14.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.678+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3358768299 shutdown_connections 2023-11-28T13:09:14.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.678+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/3358768299 >> [v2:172.21.15.156:6832/3471743149,v1:172.21.15.156:6834/3471743149] conn(0x55fd4618a990 0x55fd4613d3c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.678+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/3358768299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd46168120 0x55fd4613ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.678+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/3358768299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fd46165130 0x55fd46136450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.678+0000 7fa2b8ff9700 0 client.5217 ms_handle_reset on v2:172.21.15.156:6832/3471743149 2023-11-28T13:09:14.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.678+0000 7fa2f3643b80 1 --2- 172.21.15.156:0/3358768299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fd46167810 0x55fd46136990 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.678+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3358768299 >> 172.21.15.156:0/3358768299 conn(0x55fd4613ad60 msgr2=0x55fd46137e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:14.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.678+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3358768299 shutdown_connections 2023-11-28T13:09:14.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.678+0000 7fa2f3643b80 1 -- 172.21.15.156:0/3358768299 wait complete. 2023-11-28T13:09:14.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.679+0000 7fa2f3643b80 20 client.5217 trim_cache size 0 max 16384 2023-11-28T13:09:14.686 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:09:14.686 INFO:teuthology.orchestra.run.smithi156.stdout: "cluster_fsid": "12d9e849-dc60-46f0-8e8e-9e291633f61b", 2023-11-28T13:09:14.686 INFO:teuthology.orchestra.run.smithi156.stdout: "whoami": 0, 2023-11-28T13:09:14.686 INFO:teuthology.orchestra.run.smithi156.stdout: "id": 5013, 2023-11-28T13:09:14.687 INFO:teuthology.orchestra.run.smithi156.stdout: "want_state": "up:active", 2023-11-28T13:09:14.687 INFO:teuthology.orchestra.run.smithi156.stdout: "state": "up:active", 2023-11-28T13:09:14.687 INFO:teuthology.orchestra.run.smithi156.stdout: "fs_name": "cephfs", 2023-11-28T13:09:14.687 INFO:teuthology.orchestra.run.smithi156.stdout: "rank_uptime": 6.5078295810000002, 2023-11-28T13:09:14.687 INFO:teuthology.orchestra.run.smithi156.stdout: "mdsmap_epoch": 16, 2023-11-28T13:09:14.687 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_epoch": 35, 2023-11-28T13:09:14.687 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_epoch_barrier": 35, 2023-11-28T13:09:14.687 INFO:teuthology.orchestra.run.smithi156.stdout: "uptime": 14.853614415999999 2023-11-28T13:09:14.687 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:09:14.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.679+0000 7fa2dfdff700 1 -- 172.21.15.156:0/3587453640 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa2bc0682f0 msgr2=0x7fa2bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:14.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.679+0000 7fa2dfdff700 1 --2- 172.21.15.156:0/3587453640 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa2bc0682f0 0x7fa2bc06a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4007fd0 tx=0x7fa2d4007dc0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.679+0000 7fa2dfdff700 1 -- 172.21.15.156:0/3587453640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d8161340 msgr2=0x7fa2d81514e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:14.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.679+0000 7fa2dfdff700 1 --2- 172.21.15.156:0/3587453640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d8161340 0x7fa2d81514e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000b370 tx=0x7fa2d0004ad0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.679+0000 7fa2dfdff700 1 -- 172.21.15.156:0/3587453640 shutdown_connections 2023-11-28T13:09:14.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.679+0000 7fa2dfdff700 1 --2- 172.21.15.156:0/3587453640 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa2bc0682f0 0x7fa2bc06a7a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.679+0000 7fa2dfdff700 1 --2- 172.21.15.156:0/3587453640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d8161340 0x7fa2d81514e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.679+0000 7fa2dfdff700 1 --2- 172.21.15.156:0/3587453640 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2d80d2320 0x7fa2d8152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.679+0000 7fa2dfdff700 1 --2- 172.21.15.156:0/3587453640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2d815c9b0 0x7fa2d8143480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.679+0000 7fa2dfdff700 1 -- 172.21.15.156:0/3587453640 >> 172.21.15.156:0/3587453640 conn(0x7fa2d8072600 msgr2=0x7fa2d8160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:14.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.679+0000 7fa2dfdff700 1 -- 172.21.15.156:0/3587453640 shutdown_connections 2023-11-28T13:09:14.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.679+0000 7fa2dfdff700 1 -- 172.21.15.156:0/3587453640 wait complete. 2023-11-28T13:09:14.698 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:09:14.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.895+0000 7f2fef5ff700 1 Processor -- start 2023-11-28T13:09:14.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.895+0000 7f2fef5ff700 1 -- start start 2023-11-28T13:09:14.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.896+0000 7f2fef5ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2fe8161340 0x7f2fe80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.896+0000 7f2fef5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fe80d2320 0x7f2fe80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.897+0000 7f2fef5ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe80d1950 con 0x7f2fe815c9b0 2023-11-28T13:09:14.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.897+0000 7f2fef5ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe80b5800 con 0x7f2fe8161340 2023-11-28T13:09:14.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.897+0000 7f2fef5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe80b5300 con 0x7f2fe80d2320 2023-11-28T13:09:14.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.897+0000 7f2fecb9a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2fe8161340 0x7f2fe80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.897+0000 7f2fed39b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f2fe815c9b0 0x7f2fe80b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:43500/0 (socket says 172.21.15.156:43500) 2023-11-28T13:09:14.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.897+0000 7f2fed39b700 1 -- 172.21.15.156:0/2034361827 learned_addr learned my addr 172.21.15.156:0/2034361827 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:14.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.897+0000 7f2fedb9c700 1 --2- 172.21.15.156:0/2034361827 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fe80d2320 0x7f2fe80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.897+0000 7f2fecb9a700 1 -- 172.21.15.156:0/2034361827 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fe80d2320 msgr2=0x7f2fe80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:14.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.897+0000 7f2fecb9a700 1 --2- 172.21.15.156:0/2034361827 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fe80d2320 0x7f2fe80cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.897+0000 7f2fecb9a700 1 -- 172.21.15.156:0/2034361827 >> v1:172.21.15.156:6789/0 conn(0x7f2fe815c9b0 legacy=0x7f2fe80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:14.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.897+0000 7f2fecb9a700 1 -- 172.21.15.156:0/2034361827 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe80b5520 con 0x7f2fe8161340 2023-11-28T13:09:14.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.897+0000 7f2fedb9c700 1 --2- 172.21.15.156:0/2034361827 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fe80d2320 0x7f2fe80cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:14.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.897+0000 7f2fecb9a700 1 --2- 172.21.15.156:0/2034361827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe8161340 0x7f2fe80d1de0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009840 tx=0x7f2fd8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85bb05d5dc0e44ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:14.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.898+0000 7f2fdffff700 1 -- 172.21.15.156:0/2034361827 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8011830 con 0x7f2fe8161340 2023-11-28T13:09:14.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.898+0000 7f2fdffff700 1 -- 172.21.15.156:0/2034361827 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd8011990 con 0x7f2fe8161340 2023-11-28T13:09:14.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.898+0000 7f2fdffff700 1 -- 172.21.15.156:0/2034361827 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8011d20 con 0x7f2fe8161340 2023-11-28T13:09:14.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.898+0000 7f2fef5ff700 1 -- 172.21.15.156:0/2034361827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe8161340 msgr2=0x7f2fe80d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:14.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.898+0000 7f2fef5ff700 1 --2- 172.21.15.156:0/2034361827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe8161340 0x7f2fe80d1de0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009840 tx=0x7f2fd8009b50 comp rx=0 tx=0).stop 2023-11-28T13:09:14.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.898+0000 7f2fef5ff700 1 -- 172.21.15.156:0/2034361827 shutdown_connections 2023-11-28T13:09:14.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.898+0000 7f2fef5ff700 1 --2- 172.21.15.156:0/2034361827 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fe80d2320 0x7f2fe80cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.898+0000 7f2fef5ff700 1 --2- 172.21.15.156:0/2034361827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe8161340 0x7f2fe80d1de0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.898+0000 7f2fef5ff700 1 -- 172.21.15.156:0/2034361827 >> 172.21.15.156:0/2034361827 conn(0x7f2fe8072600 msgr2=0x7f2fe80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:14.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.898+0000 7f2fef5ff700 1 -- 172.21.15.156:0/2034361827 shutdown_connections 2023-11-28T13:09:14.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.898+0000 7f2fef5ff700 1 -- 172.21.15.156:0/2034361827 wait complete. 2023-11-28T13:09:14.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.900+0000 7f2fef5ff700 1 Processor -- start 2023-11-28T13:09:14.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.900+0000 7f2fef5ff700 1 -- start start 2023-11-28T13:09:14.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.901+0000 7f2fef5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe80d2320 0x7f2fe80cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.901+0000 7f2fef5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fe815c9b0 0x7f2fe815d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.901+0000 7f2fef5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fe8161340 0x7f2fe81e6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.901+0000 7f2fef5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe80ce1e0 con 0x7f2fe80d2320 2023-11-28T13:09:14.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.901+0000 7f2fef5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe815e030 con 0x7f2fe815c9b0 2023-11-28T13:09:14.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.901+0000 7f2fef5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe815da70 con 0x7f2fe8161340 2023-11-28T13:09:14.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.901+0000 7f2fed39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe80d2320 0x7f2fe80cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.901+0000 7f2fed39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe80d2320 0x7f2fe80cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49842/0 (socket says 172.21.15.156:49842) 2023-11-28T13:09:14.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.901+0000 7f2fed39b700 1 -- 172.21.15.156:0/516256794 learned_addr learned my addr 172.21.15.156:0/516256794 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:14.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.901+0000 7f2fedb9c700 1 --2- 172.21.15.156:0/516256794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fe8161340 0x7f2fe81e6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.901+0000 7f2fecb9a700 1 --2- 172.21.15.156:0/516256794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fe815c9b0 0x7f2fe815d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.902+0000 7f2fed39b700 1 -- 172.21.15.156:0/516256794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fe8161340 msgr2=0x7f2fe81e6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:14.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.902+0000 7f2fed39b700 1 --2- 172.21.15.156:0/516256794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fe8161340 0x7f2fe81e6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.902+0000 7f2fed39b700 1 -- 172.21.15.156:0/516256794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fe815c9b0 msgr2=0x7f2fe815d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:14.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.902+0000 7f2fed39b700 1 --2- 172.21.15.156:0/516256794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fe815c9b0 0x7f2fe815d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:14.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.902+0000 7f2fed39b700 1 -- 172.21.15.156:0/516256794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd80094a0 con 0x7f2fe80d2320 2023-11-28T13:09:14.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.902+0000 7f2fecb9a700 1 --2- 172.21.15.156:0/516256794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fe815c9b0 0x7f2fe815d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:14.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.902+0000 7f2fedb9c700 1 --2- 172.21.15.156:0/516256794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fe8161340 0x7f2fe81e6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:14.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.902+0000 7f2fed39b700 1 --2- 172.21.15.156:0/516256794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe80d2320 0x7f2fe80cdca0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400c480 tx=0x7f2fe400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:14.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.902+0000 7f2fddffb700 1 -- 172.21.15.156:0/516256794 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4005d70 con 0x7f2fe80d2320 2023-11-28T13:09:14.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.902+0000 7f2fddffb700 1 -- 172.21.15.156:0/516256794 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe40070d0 con 0x7f2fe80d2320 2023-11-28T13:09:14.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.902+0000 7f2fef5ff700 1 -- 172.21.15.156:0/516256794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe80d15e0 con 0x7f2fe80d2320 2023-11-28T13:09:14.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.902+0000 7f2fddffb700 1 -- 172.21.15.156:0/516256794 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4007440 con 0x7f2fe80d2320 2023-11-28T13:09:14.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.902+0000 7f2fef5ff700 1 -- 172.21.15.156:0/516256794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe80d1300 con 0x7f2fe80d2320 2023-11-28T13:09:14.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.903+0000 7f2fef5ff700 1 -- 172.21.15.156:0/516256794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe815cd80 con 0x7f2fe80d2320 2023-11-28T13:09:14.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.904+0000 7f2fddffb700 1 -- 172.21.15.156:0/516256794 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2fe4019030 con 0x7f2fe80d2320 2023-11-28T13:09:14.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.905+0000 7f2fddffb700 1 --2- 172.21.15.156:0/516256794 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2fcc0682f0 0x7f2fcc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:14.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.905+0000 7f2fddffb700 1 -- 172.21.15.156:0/516256794 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2fe4084f70 con 0x7f2fe80d2320 2023-11-28T13:09:14.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.905+0000 7f2fecb9a700 1 --2- 172.21.15.156:0/516256794 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2fcc0682f0 0x7f2fcc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:14.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.906+0000 7f2fecb9a700 1 --2- 172.21.15.156:0/516256794 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2fcc0682f0 0x7f2fcc06a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80090b0 tx=0x7f2fd8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:14.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:14.909+0000 7f2fddffb700 1 -- 172.21.15.156:0/516256794 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2fe404b5e0 con 0x7f2fe80d2320 2023-11-28T13:09:15.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:15.141+0000 7f2fef5ff700 1 -- 172.21.15.156:0/516256794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2fe80745c0 con 0x7f2fe80d2320 2023-11-28T13:09:15.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:15.141+0000 7f2fddffb700 1 -- 172.21.15.156:0/516256794 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2fe4017060 con 0x7f2fe80d2320 2023-11-28T13:09:15.142 INFO:teuthology.orchestra.run.smithi156.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T13:09:15.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:15.143+0000 7f2fef5ff700 1 -- 172.21.15.156:0/516256794 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2fcc0682f0 msgr2=0x7f2fcc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:15.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:15.143+0000 7f2fef5ff700 1 --2- 172.21.15.156:0/516256794 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2fcc0682f0 0x7f2fcc06a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80090b0 tx=0x7f2fd8008030 comp rx=0 tx=0).stop 2023-11-28T13:09:15.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:15.143+0000 7f2fef5ff700 1 -- 172.21.15.156:0/516256794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe80d2320 msgr2=0x7f2fe80cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:15.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:15.143+0000 7f2fef5ff700 1 --2- 172.21.15.156:0/516256794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe80d2320 0x7f2fe80cdca0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400c480 tx=0x7f2fe400c840 comp rx=0 tx=0).stop 2023-11-28T13:09:15.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:15.144+0000 7f2fef5ff700 1 -- 172.21.15.156:0/516256794 shutdown_connections 2023-11-28T13:09:15.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:15.144+0000 7f2fef5ff700 1 --2- 172.21.15.156:0/516256794 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2fcc0682f0 0x7f2fcc06a7a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:15.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:15.144+0000 7f2fef5ff700 1 --2- 172.21.15.156:0/516256794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fe8161340 0x7f2fe81e6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:15.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:15.144+0000 7f2fef5ff700 1 --2- 172.21.15.156:0/516256794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fe80d2320 0x7f2fe80cdca0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:15.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:15.144+0000 7f2fef5ff700 1 --2- 172.21.15.156:0/516256794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fe815c9b0 0x7f2fe815d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:15.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:15.144+0000 7f2fef5ff700 1 -- 172.21.15.156:0/516256794 >> 172.21.15.156:0/516256794 conn(0x7f2fe8072600 msgr2=0x7f2fe80b6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:15.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:15.144+0000 7f2fef5ff700 1 -- 172.21.15.156:0/516256794 shutdown_connections 2023-11-28T13:09:15.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:15.144+0000 7f2fef5ff700 1 -- 172.21.15.156:0/516256794 wait complete. 2023-11-28T13:09:15.162 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T13:09:15.162 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:09:15.163 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-11-28T13:09:15.163 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T13:09:15.163 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T13:09:15.163 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T13:09:15.163 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T13:09:15.163 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:09:15.163 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:09:15.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:15.163 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-11-28T13:09:15.207 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:09:15.207 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:09:15.207 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:09:15.207 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:09:15.207 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-28T13:09:15.208 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:09:15.208 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:09:15.208 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:09:15.208 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f0 2023-11-28T13:09:15.208 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:09:15.208 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:09:15.208 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f1 2023-11-28T13:09:15.208 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:09:15.208 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-11-28T13:09:15.208 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:09:15.208 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-11-28T13:09:15.208 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T13:09:15.209 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42109sec preferred_lft 42109sec 2023-11-28T13:09:15.209 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::d27c:fc8f:2674:dd68/64 scope link noprefixroute 2023-11-28T13:09:15.209 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:09:15.209 INFO:teuthology.orchestra.run.smithi156.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:09:15.209 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 3e:ff:33:55:aa:40 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:09:15.209 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T13:09:15.209 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:09:15.209 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::4497:b0ff:fe27:d641/64 scope link 2023-11-28T13:09:15.209 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:09:15.209 INFO:teuthology.orchestra.run.smithi156.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:09:15.209 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 96:cd:43:a7:29:10 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:09:15.209 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::94cd:43ff:fea7:2910/64 scope link 2023-11-28T13:09:15.209 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:09:15.210 INFO:teuthology.orchestra.run.smithi156.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:09:15.210 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 3e:ff:33:55:aa:40 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T13:09:15.210 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::3cff:33ff:fe55:aa40/64 scope link 2023-11-28T13:09:15.210 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:09:15.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:15.211 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:09:15.318 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-28T13:09:15.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:15.321 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-11-28T13:09:15.367 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-11-28T13:09:15.368 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:09:15.368 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T13:09:15.368 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T13:09:15.368 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:09:15.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:15.369 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T13:09:15.369 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T13:09:15.369 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T13:09:15.369 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T13:09:15.369 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T13:09:15.369 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T13:09:15.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:15.582 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T13:09:15.626 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T13:09:15.627 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T13:09:15.627 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:09:15.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:15.627 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.0 up 2023-11-28T13:09:15.737 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:09:15.737 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T13:09:15.782 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:09:15.782 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:09:15.783 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:09:15.865 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T13:09:15.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:15.866 DEBUG:teuthology.orchestra.run.smithi156:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:09:16.011 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:09:16.011 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:09:16.011 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T13:09:16.030 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:09:16.031 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:09:16.031 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T13:09:16.031 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T13:09:16.031 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:09:16.053 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:09:16.053 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:09:16.054 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T13:09:16.054 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T13:09:16.054 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:09:16.054 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790" 2023-11-28T13:09:16.054 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2023-11-28T13:09:16.054 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=12d9e849-dc60-46f0-8e8e-9e291633f61b" will pass to kernel 2023-11-28T13:09:16.058 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T13:09:16.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:16.059 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:09:16.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:16.128 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:09:16.228 INFO:teuthology.orchestra.run.smithi156.stdout:client4829 2023-11-28T13:09:16.229 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:09:16.229 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:09:16.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:16.273 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2023-11-28T13:09:16.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:16.365 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2023-11-28T13:09:16.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:16.411 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2023-11-28T13:09:16.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:16.498 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2023-11-28T13:09:16.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:16.752 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2023-11-28T13:09:16.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:16.803 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2023-11-28T13:09:16.896 INFO:teuthology.orchestra.run.smithi156.stdout:./ 2023-11-28T13:09:16.897 INFO:teuthology.orchestra.run.smithi156.stdout:./rfile 2023-11-28T13:09:16.897 INFO:teuthology.orchestra.run.smithi156.stdout:./parentdir 2023-11-28T13:09:16.897 INFO:teuthology.orchestra.run.smithi156.stdout:./parentdir/pfile 2023-11-28T13:09:16.897 INFO:teuthology.orchestra.run.smithi156.stdout:./parentdir/childdir 2023-11-28T13:09:16.898 INFO:teuthology.orchestra.run.smithi156.stdout:./parentdir/childdir/cfile 2023-11-28T13:09:16.902 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:09:16.902 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-11-28T13:09:16.902 DEBUG:teuthology.orchestra.run.smithi156:> import stat 2023-11-28T13:09:16.902 DEBUG:teuthology.orchestra.run.smithi156:> 2023-11-28T13:09:16.902 DEBUG:teuthology.orchestra.run.smithi156:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2023-11-28T13:09:16.902 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-11-28T13:09:17.016 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-28T13:09:17.200 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:09:17.200 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-11-28T13:09:17.200 DEBUG:teuthology.orchestra.run.smithi156:> import stat 2023-11-28T13:09:17.200 DEBUG:teuthology.orchestra.run.smithi156:> 2023-11-28T13:09:17.200 DEBUG:teuthology.orchestra.run.smithi156:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2023-11-28T13:09:17.200 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-11-28T13:09:17.352 INFO:teuthology.orchestra.run.smithi156.stdout:1099511627778 2023-11-28T13:09:17.538 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:09:17.538 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-11-28T13:09:17.538 DEBUG:teuthology.orchestra.run.smithi156:> import stat 2023-11-28T13:09:17.538 DEBUG:teuthology.orchestra.run.smithi156:> 2023-11-28T13:09:17.538 DEBUG:teuthology.orchestra.run.smithi156:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2023-11-28T13:09:17.539 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-11-28T13:09:17.691 INFO:teuthology.orchestra.run.smithi156.stdout:1099511627776 2023-11-28T13:09:17.875 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:09:17.875 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-11-28T13:09:17.875 DEBUG:teuthology.orchestra.run.smithi156:> import stat 2023-11-28T13:09:17.875 DEBUG:teuthology.orchestra.run.smithi156:> 2023-11-28T13:09:17.875 DEBUG:teuthology.orchestra.run.smithi156:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2023-11-28T13:09:17.875 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-11-28T13:09:18.027 INFO:teuthology.orchestra.run.smithi156.stdout:1099511628279 2023-11-28T13:09:18.212 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:09:18.212 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-11-28T13:09:18.212 DEBUG:teuthology.orchestra.run.smithi156:> import stat 2023-11-28T13:09:18.212 DEBUG:teuthology.orchestra.run.smithi156:> 2023-11-28T13:09:18.213 DEBUG:teuthology.orchestra.run.smithi156:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2023-11-28T13:09:18.213 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-11-28T13:09:18.365 INFO:teuthology.orchestra.run.smithi156.stdout:1099511627777 2023-11-28T13:09:18.555 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:09:18.556 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-11-28T13:09:18.556 DEBUG:teuthology.orchestra.run.smithi156:> import stat 2023-11-28T13:09:18.556 DEBUG:teuthology.orchestra.run.smithi156:> 2023-11-28T13:09:18.556 DEBUG:teuthology.orchestra.run.smithi156:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2023-11-28T13:09:18.557 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-11-28T13:09:18.707 INFO:teuthology.orchestra.run.smithi156.stdout:1099511628280 2023-11-28T13:09:18.892 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:09:18.893 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:09:18.977 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:09:18.978 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:09:19.063 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:09:19.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.297+0000 7faf577ff700 1 Processor -- start 2023-11-28T13:09:19.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.297+0000 7faf577ff700 1 -- start start 2023-11-28T13:09:19.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.298+0000 7faf577ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7faf5015c9b0 0x7faf500b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:19.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.298+0000 7faf577ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf500cd7c0 0x7faf500cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:19.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.298+0000 7faf577ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf500d1950 con 0x7faf50161340 2023-11-28T13:09:19.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.298+0000 7faf577ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf500b5800 con 0x7faf5015c9b0 2023-11-28T13:09:19.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.298+0000 7faf577ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf500ce140 con 0x7faf500cd7c0 2023-11-28T13:09:19.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.298+0000 7faf54d9a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7faf50161340 0x7faf500d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:34812/0 (socket says 172.21.15.156:34812) 2023-11-28T13:09:19.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.298+0000 7faf54d9a700 1 -- 172.21.15.156:0/706405219 learned_addr learned my addr 172.21.15.156:0/706405219 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:19.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf55d9c700 1 --2- 172.21.15.156:0/706405219 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf500cd7c0 0x7faf500cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:19.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf5559b700 1 --2- 172.21.15.156:0/706405219 >> v2:172.21.15.186:3300/0 conn(0x7faf5015c9b0 0x7faf500b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:19.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf47fff700 1 -- 172.21.15.156:0/706405219 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3037523399 0 0) 0x7faf500d1950 con 0x7faf50161340 2023-11-28T13:09:19.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf47fff700 1 -- 172.21.15.156:0/706405219 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf3c0035b0 con 0x7faf50161340 2023-11-28T13:09:19.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf47fff700 1 -- 172.21.15.156:0/706405219 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 155707471 0 0) 0x7faf3c0035b0 con 0x7faf50161340 2023-11-28T13:09:19.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf47fff700 1 -- 172.21.15.156:0/706405219 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf500cd7c0 msgr2=0x7faf500cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:19.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf47fff700 1 --2- 172.21.15.156:0/706405219 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf500cd7c0 0x7faf500cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf47fff700 1 -- 172.21.15.156:0/706405219 >> v2:172.21.15.186:3300/0 conn(0x7faf5015c9b0 msgr2=0x7faf500b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:19.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf47fff700 1 --2- 172.21.15.156:0/706405219 >> v2:172.21.15.186:3300/0 conn(0x7faf5015c9b0 0x7faf500b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf47fff700 1 -- 172.21.15.156:0/706405219 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf500d24f0 con 0x7faf50161340 2023-11-28T13:09:19.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf47fff700 1 -- 172.21.15.156:0/706405219 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7faf40001670 con 0x7faf50161340 2023-11-28T13:09:19.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf47fff700 1 -- 172.21.15.156:0/706405219 >> v1:172.21.15.156:6789/0 conn(0x7faf50161340 legacy=0x7faf500d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:19.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf47fff700 1 --2- 172.21.15.156:0/706405219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf3c003a00 0x7faf3c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:19.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf47fff700 1 --2- 172.21.15.156:0/706405219 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf500cd7c0 0x7faf3c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:19.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf47fff700 1 --2- 172.21.15.156:0/706405219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3c004d90 0x7faf3c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:19.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf47fff700 1 -- 172.21.15.156:0/706405219 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf500b5800 con 0x7faf3c004d90 2023-11-28T13:09:19.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf47fff700 1 -- 172.21.15.156:0/706405219 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf500ce140 con 0x7faf3c003a00 2023-11-28T13:09:19.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.299+0000 7faf47fff700 1 -- 172.21.15.156:0/706405219 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c00ba60 con 0x7faf500cd7c0 2023-11-28T13:09:19.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.300+0000 7faf55d9c700 1 --2- 172.21.15.156:0/706405219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3c004d90 0x7faf3c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:19.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.300+0000 7faf5559b700 1 --2- 172.21.15.156:0/706405219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf3c003a00 0x7faf3c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:19.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.300+0000 7faf54d9a700 1 --2- 172.21.15.156:0/706405219 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf500cd7c0 0x7faf3c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:19.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.300+0000 7faf55d9c700 1 -- 172.21.15.156:0/706405219 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf500cd7c0 msgr2=0x7faf3c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:19.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.300+0000 7faf55d9c700 1 --2- 172.21.15.156:0/706405219 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf500cd7c0 0x7faf3c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.300+0000 7faf55d9c700 1 -- 172.21.15.156:0/706405219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf3c003a00 msgr2=0x7faf3c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:19.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.300+0000 7faf55d9c700 1 --2- 172.21.15.156:0/706405219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf3c003a00 0x7faf3c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.300+0000 7faf55d9c700 1 -- 172.21.15.156:0/706405219 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0035b0 con 0x7faf3c004d90 2023-11-28T13:09:19.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.300+0000 7faf55d9c700 1 --2- 172.21.15.156:0/706405219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3c004d90 0x7faf3c005200 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7faf48009fc0 tx=0x7faf3c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45422270dcbed3a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:19.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.300+0000 7faf47fff700 1 -- 172.21.15.156:0/706405219 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4800ec30 con 0x7faf3c004d90 2023-11-28T13:09:19.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.300+0000 7faf47fff700 1 -- 172.21.15.156:0/706405219 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4800f040 con 0x7faf3c004d90 2023-11-28T13:09:19.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.300+0000 7faf47fff700 1 -- 172.21.15.156:0/706405219 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48014480 con 0x7faf3c004d90 2023-11-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.301+0000 7faf577ff700 1 -- 172.21.15.156:0/706405219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3c004d90 msgr2=0x7faf3c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.301+0000 7faf577ff700 1 --2- 172.21.15.156:0/706405219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3c004d90 0x7faf3c005200 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7faf48009fc0 tx=0x7faf3c005880 comp rx=0 tx=0).stop 2023-11-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.301+0000 7faf55d9c700 1 -- 172.21.15.156:0/706405219 reap_dead start 2023-11-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.301+0000 7faf577ff700 1 -- 172.21.15.156:0/706405219 shutdown_connections 2023-11-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.301+0000 7faf577ff700 1 -- 172.21.15.156:0/706405219 >> 172.21.15.156:0/706405219 conn(0x7faf50072600 msgr2=0x7faf500cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.301+0000 7faf577ff700 1 -- 172.21.15.156:0/706405219 shutdown_connections 2023-11-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.301+0000 7faf577ff700 1 -- 172.21.15.156:0/706405219 wait complete. 2023-11-28T13:09:19.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.303+0000 7faf577ff700 1 Processor -- start 2023-11-28T13:09:19.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.303+0000 7faf577ff700 1 -- start start 2023-11-28T13:09:19.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.304+0000 7faf577ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3c004d90 0x7faf500c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:19.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.304+0000 7faf577ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf500a9a20 0x7faf500a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:19.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.304+0000 7faf577ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf500cc130 0x7faf500cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:19.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.304+0000 7faf577ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf500cb090 con 0x7faf3c004d90 2023-11-28T13:09:19.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.304+0000 7faf577ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf500cb370 con 0x7faf500cc130 2023-11-28T13:09:19.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.304+0000 7faf577ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf500cb650 con 0x7faf500a9a20 2023-11-28T13:09:19.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.304+0000 7faf5559b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3c004d90 0x7faf500c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:19.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.304+0000 7faf5559b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3c004d90 0x7faf500c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39752/0 (socket says 172.21.15.156:39752) 2023-11-28T13:09:19.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.304+0000 7faf5559b700 1 -- 172.21.15.156:0/601035202 learned_addr learned my addr 172.21.15.156:0/601035202 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:19.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.304+0000 7faf55d9c700 1 --2- 172.21.15.156:0/601035202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf500cc130 0x7faf500cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:19.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.304+0000 7faf54d9a700 1 --2- 172.21.15.156:0/601035202 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf500a9a20 0x7faf500a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:19.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.304+0000 7faf5559b700 1 -- 172.21.15.156:0/601035202 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf500a9a20 msgr2=0x7faf500a9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:19.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.304+0000 7faf5559b700 1 --2- 172.21.15.156:0/601035202 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf500a9a20 0x7faf500a9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.304+0000 7faf5559b700 1 -- 172.21.15.156:0/601035202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf500cc130 msgr2=0x7faf500cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:19.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.304+0000 7faf5559b700 1 --2- 172.21.15.156:0/601035202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf500cc130 0x7faf500cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.304+0000 7faf5559b700 1 -- 172.21.15.156:0/601035202 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf500d1300 con 0x7faf3c004d90 2023-11-28T13:09:19.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.304+0000 7faf54d9a700 1 --2- 172.21.15.156:0/601035202 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf500a9a20 0x7faf500a9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:19.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.305+0000 7faf5559b700 1 --2- 172.21.15.156:0/601035202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3c004d90 0x7faf500c9620 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7faf4c011aa0 tx=0x7faf4c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:19.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.305+0000 7faf45ffb700 1 -- 172.21.15.156:0/601035202 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c00f980 con 0x7faf3c004d90 2023-11-28T13:09:19.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.305+0000 7faf45ffb700 1 -- 172.21.15.156:0/601035202 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4c00fb00 con 0x7faf3c004d90 2023-11-28T13:09:19.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.305+0000 7faf577ff700 1 -- 172.21.15.156:0/601035202 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf500b5520 con 0x7faf3c004d90 2023-11-28T13:09:19.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.305+0000 7faf45ffb700 1 -- 172.21.15.156:0/601035202 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c00feb0 con 0x7faf3c004d90 2023-11-28T13:09:19.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.305+0000 7faf577ff700 1 -- 172.21.15.156:0/601035202 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf500d15e0 con 0x7faf3c004d90 2023-11-28T13:09:19.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.306+0000 7faf577ff700 1 -- 172.21.15.156:0/601035202 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf500b9180 con 0x7faf3c004d90 2023-11-28T13:09:19.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.307+0000 7faf45ffb700 1 -- 172.21.15.156:0/601035202 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faf4c014440 con 0x7faf3c004d90 2023-11-28T13:09:19.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.308+0000 7faf45ffb700 1 --2- 172.21.15.156:0/601035202 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faf34068320 0x7faf3406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:19.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.308+0000 7faf45ffb700 1 -- 172.21.15.156:0/601035202 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faf4c086680 con 0x7faf3c004d90 2023-11-28T13:09:19.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.308+0000 7faf54d9a700 1 --2- 172.21.15.156:0/601035202 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faf34068320 0x7faf3406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:19.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.309+0000 7faf54d9a700 1 --2- 172.21.15.156:0/601035202 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faf34068320 0x7faf3406a7d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faf40001eb0 tx=0x7faf40000f50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:19.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.312+0000 7faf45ffb700 1 -- 172.21.15.156:0/601035202 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faf4c052f00 con 0x7faf3c004d90 2023-11-28T13:09:19.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.505+0000 7faf577ff700 1 -- 172.21.15.156:0/601035202 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf500745c0 con 0x7faf3c004d90 2023-11-28T13:09:19.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.507+0000 7faf45ffb700 1 -- 172.21.15.156:0/601035202 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+17969 (secure 0 0 0) 0x7faf4c056520 con 0x7faf3c004d90 2023-11-28T13:09:19.507 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:19.508 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":35,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:09:09.170289+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:09:06.618698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T13:09:07.151554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:09:19.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.509+0000 7faf577ff700 1 -- 172.21.15.156:0/601035202 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faf34068320 msgr2=0x7faf3406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:19.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.509+0000 7faf577ff700 1 --2- 172.21.15.156:0/601035202 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faf34068320 0x7faf3406a7d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faf40001eb0 tx=0x7faf40000f50 comp rx=0 tx=0).stop 2023-11-28T13:09:19.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.509+0000 7faf577ff700 1 -- 172.21.15.156:0/601035202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3c004d90 msgr2=0x7faf500c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:19.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.509+0000 7faf577ff700 1 --2- 172.21.15.156:0/601035202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3c004d90 0x7faf500c9620 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7faf4c011aa0 tx=0x7faf4c011e60 comp rx=0 tx=0).stop 2023-11-28T13:09:19.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.509+0000 7faf577ff700 1 -- 172.21.15.156:0/601035202 shutdown_connections 2023-11-28T13:09:19.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.509+0000 7faf577ff700 1 --2- 172.21.15.156:0/601035202 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faf34068320 0x7faf3406a7d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.509+0000 7faf577ff700 1 --2- 172.21.15.156:0/601035202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf500cc130 0x7faf500cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.509+0000 7faf577ff700 1 --2- 172.21.15.156:0/601035202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3c004d90 0x7faf500c9620 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.509+0000 7faf577ff700 1 --2- 172.21.15.156:0/601035202 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf500a9a20 0x7faf500a9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.509+0000 7faf577ff700 1 -- 172.21.15.156:0/601035202 >> 172.21.15.156:0/601035202 conn(0x7faf50072600 msgr2=0x7faf5015a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:19.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.509+0000 7faf577ff700 1 -- 172.21.15.156:0/601035202 shutdown_connections 2023-11-28T13:09:19.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.509+0000 7faf577ff700 1 -- 172.21.15.156:0/601035202 wait complete. 2023-11-28T13:09:19.528 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:09:06.618698+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:09:07.151554+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:09:19.529 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:09:19.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.728+0000 7f1a73bff700 1 Processor -- start 2023-11-28T13:09:19.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.728+0000 7f1a73bff700 1 -- start start 2023-11-28T13:09:19.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.729+0000 7f1a73bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a6c0d1330 0x7f1a6c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:19.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.729+0000 7f1a73bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1a6c0b20b0 0x7f1a6c0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:19.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.729+0000 7f1a73bff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6c0b0230 con 0x7f1a6c0d01b0 2023-11-28T13:09:19.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.729+0000 7f1a73bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6c15cec0 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.729+0000 7f1a73bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0b4570 con 0x7f1a6c0d1330 2023-11-28T13:09:19.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.729+0000 7f1a7219c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1a6c0b20b0 0x7f1a6c0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:19.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.729+0000 7f1a7219c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1a6c0b20b0 0x7f1a6c0b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39768/0 (socket says 172.21.15.156:39768) 2023-11-28T13:09:19.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.729+0000 7f1a7219c700 1 -- 172.21.15.156:0/1389087899 learned_addr learned my addr 172.21.15.156:0/1389087899 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:19.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.730+0000 7f1a7119a700 1 --2- 172.21.15.156:0/1389087899 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a6c0d1330 0x7f1a6c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:19.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.730+0000 7f1a7219c700 1 -- 172.21.15.156:0/1389087899 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a6c0d1330 msgr2=0x7f1a6c0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:19.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.730+0000 7f1a7219c700 1 --2- 172.21.15.156:0/1389087899 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a6c0d1330 0x7f1a6c0b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.730+0000 7f1a7219c700 1 -- 172.21.15.156:0/1389087899 >> v1:172.21.15.186:6789/0 conn(0x7f1a6c0d01b0 legacy=0x7f1a6c0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:19.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.730+0000 7f1a7219c700 1 -- 172.21.15.156:0/1389087899 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c0b4850 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.730+0000 7f1a7219c700 1 --2- 172.21.15.156:0/1389087899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6c0b20b0 0x7f1a6c0b0c00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009e50 tx=0x7f1a6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc43022935a428b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:19.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.730+0000 7f1a70999700 1 -- 172.21.15.156:0/1389087899 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6400e070 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.730+0000 7f1a70999700 1 -- 172.21.15.156:0/1389087899 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a64003a20 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.730+0000 7f1a70999700 1 -- 172.21.15.156:0/1389087899 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64003dd0 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.731+0000 7f1a73bff700 1 -- 172.21.15.156:0/1389087899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6c0b20b0 msgr2=0x7f1a6c0b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:19.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.731+0000 7f1a73bff700 1 --2- 172.21.15.156:0/1389087899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6c0b20b0 0x7f1a6c0b0c00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009e50 tx=0x7f1a6400b330 comp rx=0 tx=0).stop 2023-11-28T13:09:19.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.731+0000 7f1a73bff700 1 -- 172.21.15.156:0/1389087899 shutdown_connections 2023-11-28T13:09:19.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.731+0000 7f1a73bff700 1 --2- 172.21.15.156:0/1389087899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6c0b20b0 0x7f1a6c0b0c00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.731+0000 7f1a73bff700 1 --2- 172.21.15.156:0/1389087899 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a6c0d1330 0x7f1a6c0b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.731+0000 7f1a73bff700 1 -- 172.21.15.156:0/1389087899 >> 172.21.15.156:0/1389087899 conn(0x7f1a6c09c3f0 msgr2=0x7f1a6c0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:19.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.731+0000 7f1a73bff700 1 -- 172.21.15.156:0/1389087899 shutdown_connections 2023-11-28T13:09:19.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.731+0000 7f1a73bff700 1 -- 172.21.15.156:0/1389087899 wait complete. 2023-11-28T13:09:19.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.733+0000 7f1a73bff700 1 Processor -- start 2023-11-28T13:09:19.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.733+0000 7f1a73bff700 1 -- start start 2023-11-28T13:09:19.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.734+0000 7f1a73bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6c0b20b0 0x7f1a6c150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:19.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.734+0000 7f1a73bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a6c0d01b0 0x7f1a6c15e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:19.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.734+0000 7f1a73bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a6c0d1330 0x7f1a6c09e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:19.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.734+0000 7f1a73bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c159d90 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.734+0000 7f1a73bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c160060 con 0x7f1a6c0d01b0 2023-11-28T13:09:19.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.734+0000 7f1a73bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c15db40 con 0x7f1a6c0d1330 2023-11-28T13:09:19.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.734+0000 7f1a7199b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6c0b20b0 0x7f1a6c150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:19.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.734+0000 7f1a7199b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6c0b20b0 0x7f1a6c150530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39782/0 (socket says 172.21.15.156:39782) 2023-11-28T13:09:19.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.734+0000 7f1a7199b700 1 -- 172.21.15.156:0/1938409510 learned_addr learned my addr 172.21.15.156:0/1938409510 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:19.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.734+0000 7f1a7119a700 1 --2- 172.21.15.156:0/1938409510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a6c0d01b0 0x7f1a6c15e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:19.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.734+0000 7f1a7219c700 1 --2- 172.21.15.156:0/1938409510 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a6c0d1330 0x7f1a6c09e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:19.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.735+0000 7f1a7199b700 1 -- 172.21.15.156:0/1938409510 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a6c0d1330 msgr2=0x7f1a6c09e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:19.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.735+0000 7f1a7199b700 1 --2- 172.21.15.156:0/1938409510 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a6c0d1330 0x7f1a6c09e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.735+0000 7f1a7199b700 1 -- 172.21.15.156:0/1938409510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a6c0d01b0 msgr2=0x7f1a6c15e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:19.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.735+0000 7f1a7199b700 1 --2- 172.21.15.156:0/1938409510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a6c0d01b0 0x7f1a6c15e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.735+0000 7f1a7199b700 1 -- 172.21.15.156:0/1938409510 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a64009b30 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.735+0000 7f1a7199b700 1 --2- 172.21.15.156:0/1938409510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6c0b20b0 0x7f1a6c150530 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800c4d0 tx=0x7f1a6800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:19.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.735+0000 7f1a627fc700 1 -- 172.21.15.156:0/1938409510 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a68011790 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.735+0000 7f1a627fc700 1 -- 172.21.15.156:0/1938409510 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a680118f0 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.735+0000 7f1a627fc700 1 -- 172.21.15.156:0/1938409510 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a68011ca0 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.735+0000 7f1a73bff700 1 -- 172.21.15.156:0/1938409510 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6c0afec0 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.735+0000 7f1a73bff700 1 -- 172.21.15.156:0/1938409510 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c0afbe0 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.736+0000 7f1a73bff700 1 -- 172.21.15.156:0/1938409510 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6c0b2600 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.737+0000 7f1a627fc700 1 -- 172.21.15.156:0/1938409510 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1a68010050 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.738+0000 7f1a627fc700 1 --2- 172.21.15.156:0/1938409510 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a500682f0 0x7f1a5006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:19.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.738+0000 7f1a627fc700 1 -- 172.21.15.156:0/1938409510 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1a680858e0 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.738+0000 7f1a7119a700 1 --2- 172.21.15.156:0/1938409510 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a500682f0 0x7f1a5006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:19.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.739+0000 7f1a7119a700 1 --2- 172.21.15.156:0/1938409510 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a500682f0 0x7f1a5006a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0d40c0 tx=0x7f1a5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:19.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.742+0000 7f1a627fc700 1 -- 172.21.15.156:0/1938409510 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1a68052050 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.931+0000 7f1a73bff700 1 -- 172.21.15.156:0/1938409510 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1a6c068690 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.931+0000 7f1a627fc700 1 -- 172.21.15.156:0/1938409510 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1a68055670 con 0x7f1a6c0b20b0 2023-11-28T13:09:19.932 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:09:19.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.934+0000 7f1a73bff700 1 -- 172.21.15.156:0/1938409510 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a500682f0 msgr2=0x7f1a5006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:19.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.934+0000 7f1a73bff700 1 --2- 172.21.15.156:0/1938409510 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a500682f0 0x7f1a5006a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0d40c0 tx=0x7f1a5c008040 comp rx=0 tx=0).stop 2023-11-28T13:09:19.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.934+0000 7f1a73bff700 1 -- 172.21.15.156:0/1938409510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6c0b20b0 msgr2=0x7f1a6c150530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:19.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.934+0000 7f1a73bff700 1 --2- 172.21.15.156:0/1938409510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6c0b20b0 0x7f1a6c150530 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800c4d0 tx=0x7f1a6800c7e0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.934+0000 7f1a73bff700 1 -- 172.21.15.156:0/1938409510 shutdown_connections 2023-11-28T13:09:19.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.934+0000 7f1a73bff700 1 --2- 172.21.15.156:0/1938409510 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a500682f0 0x7f1a5006a7a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.934+0000 7f1a73bff700 1 --2- 172.21.15.156:0/1938409510 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1a6c0d1330 0x7f1a6c09e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.934+0000 7f1a73bff700 1 --2- 172.21.15.156:0/1938409510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6c0b20b0 0x7f1a6c150530 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.934+0000 7f1a73bff700 1 --2- 172.21.15.156:0/1938409510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a6c0d01b0 0x7f1a6c15e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:19.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.934+0000 7f1a73bff700 1 -- 172.21.15.156:0/1938409510 >> 172.21.15.156:0/1938409510 conn(0x7f1a6c09c3f0 msgr2=0x7f1a6c15c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:19.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.934+0000 7f1a73bff700 1 -- 172.21.15.156:0/1938409510 shutdown_connections 2023-11-28T13:09:19.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:19.934+0000 7f1a73bff700 1 -- 172.21.15.156:0/1938409510 wait complete. 2023-11-28T13:09:19.953 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:09:20.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.153+0000 7f3b985ff700 1 Processor -- start 2023-11-28T13:09:20.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.153+0000 7f3b985ff700 1 -- start start 2023-11-28T13:09:20.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.154+0000 7f3b985ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3b9015c9b0 0x7f3b900b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:20.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.154+0000 7f3b985ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3b90161340 0x7f3b900d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:20.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.154+0000 7f3b985ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3b900d2320 0x7f3b900cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:20.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.154+0000 7f3b985ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b900b5800 con 0x7f3b9015c9b0 2023-11-28T13:09:20.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.154+0000 7f3b985ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b900d1950 con 0x7f3b90161340 2023-11-28T13:09:20.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.154+0000 7f3b985ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b900b5520 con 0x7f3b900d2320 2023-11-28T13:09:20.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.154+0000 7f3b9639b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3b9015c9b0 0x7f3b900b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:20.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.154+0000 7f3b9639b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3b9015c9b0 0x7f3b900b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39788/0 (socket says 172.21.15.156:39788) 2023-11-28T13:09:20.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.154+0000 7f3b9639b700 1 -- 172.21.15.156:0/1444497647 learned_addr learned my addr 172.21.15.156:0/1444497647 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:20.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.155+0000 7f3b95b9a700 1 --2- 172.21.15.156:0/1444497647 >> v2:172.21.15.186:3300/0 conn(0x7f3b90161340 0x7f3b900d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:20.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.155+0000 7f3b96b9c700 1 --2- 172.21.15.156:0/1444497647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3b900d2320 0x7f3b900cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:20.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.155+0000 7f3b9639b700 1 -- 172.21.15.156:0/1444497647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3b900d2320 msgr2=0x7f3b900cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:20.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.155+0000 7f3b9639b700 1 --2- 172.21.15.156:0/1444497647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3b900d2320 0x7f3b900cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.155+0000 7f3b9639b700 1 -- 172.21.15.156:0/1444497647 >> v2:172.21.15.186:3300/0 conn(0x7f3b90161340 msgr2=0x7f3b900d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:20.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.155+0000 7f3b9639b700 1 --2- 172.21.15.156:0/1444497647 >> v2:172.21.15.186:3300/0 conn(0x7f3b90161340 0x7f3b900d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.155+0000 7f3b9639b700 1 -- 172.21.15.156:0/1444497647 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b900d1300 con 0x7f3b9015c9b0 2023-11-28T13:09:20.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.155+0000 7f3b96b9c700 1 --2- 172.21.15.156:0/1444497647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3b900d2320 0x7f3b900cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:20.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.155+0000 7f3b9639b700 1 --2- 172.21.15.156:0/1444497647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b9015c9b0 0x7f3b900b4bf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3b900685c0 tx=0x7f3b8c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d698700c0cca28 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:20.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.155+0000 7f3b95399700 1 -- 172.21.15.156:0/1444497647 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c00b870 con 0x7f3b9015c9b0 2023-11-28T13:09:20.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.155+0000 7f3b95399700 1 -- 172.21.15.156:0/1444497647 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8c00b9d0 con 0x7f3b9015c9b0 2023-11-28T13:09:20.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.155+0000 7f3b95399700 1 -- 172.21.15.156:0/1444497647 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c00bd60 con 0x7f3b9015c9b0 2023-11-28T13:09:20.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.156+0000 7f3b985ff700 1 -- 172.21.15.156:0/1444497647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b9015c9b0 msgr2=0x7f3b900b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:20.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.156+0000 7f3b985ff700 1 --2- 172.21.15.156:0/1444497647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b9015c9b0 0x7f3b900b4bf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3b900685c0 tx=0x7f3b8c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.156+0000 7f3b985ff700 1 -- 172.21.15.156:0/1444497647 shutdown_connections 2023-11-28T13:09:20.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.156+0000 7f3b985ff700 1 --2- 172.21.15.156:0/1444497647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3b900d2320 0x7f3b900cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.156+0000 7f3b985ff700 1 --2- 172.21.15.156:0/1444497647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b9015c9b0 0x7f3b900b4bf0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.156+0000 7f3b985ff700 1 --2- 172.21.15.156:0/1444497647 >> v2:172.21.15.186:3300/0 conn(0x7f3b90161340 0x7f3b900d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.156+0000 7f3b985ff700 1 -- 172.21.15.156:0/1444497647 >> 172.21.15.156:0/1444497647 conn(0x7f3b90072600 msgr2=0x7f3b900cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:20.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.156+0000 7f3b985ff700 1 -- 172.21.15.156:0/1444497647 shutdown_connections 2023-11-28T13:09:20.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.156+0000 7f3b985ff700 1 -- 172.21.15.156:0/1444497647 wait complete. 2023-11-28T13:09:20.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.158+0000 7f3b985ff700 1 Processor -- start 2023-11-28T13:09:20.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.158+0000 7f3b985ff700 1 -- start start 2023-11-28T13:09:20.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.159+0000 7f3b985ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b900d2320 0x7f3b9014e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:20.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.159+0000 7f3b985ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3b9015c9b0 0x7f3b901515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:20.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.159+0000 7f3b985ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b90161340 0x7f3b90151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:20.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.159+0000 7f3b985ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b901544c0 con 0x7f3b90161340 2023-11-28T13:09:20.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.159+0000 7f3b985ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b90160040 con 0x7f3b900d2320 2023-11-28T13:09:20.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.159+0000 7f3b985ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b90153940 con 0x7f3b9015c9b0 2023-11-28T13:09:20.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.159+0000 7f3b96b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b90161340 0x7f3b90151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:20.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.159+0000 7f3b9639b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b900d2320 0x7f3b9014e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:20.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.159+0000 7f3b96b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b90161340 0x7f3b90151ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39804/0 (socket says 172.21.15.156:39804) 2023-11-28T13:09:20.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.159+0000 7f3b95b9a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3b9015c9b0 0x7f3b901515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:20.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.159+0000 7f3b96b9c700 1 -- 172.21.15.156:0/318972595 learned_addr learned my addr 172.21.15.156:0/318972595 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:20.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.160+0000 7f3b96b9c700 1 -- 172.21.15.156:0/318972595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3b9015c9b0 msgr2=0x7f3b901515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:20.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.160+0000 7f3b96b9c700 1 --2- 172.21.15.156:0/318972595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3b9015c9b0 0x7f3b901515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.160+0000 7f3b96b9c700 1 -- 172.21.15.156:0/318972595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b900d2320 msgr2=0x7f3b9014e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:20.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.160+0000 7f3b96b9c700 1 --2- 172.21.15.156:0/318972595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b900d2320 0x7f3b9014e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.160+0000 7f3b96b9c700 1 -- 172.21.15.156:0/318972595 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8c009550 con 0x7f3b90161340 2023-11-28T13:09:20.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.160+0000 7f3b96b9c700 1 --2- 172.21.15.156:0/318972595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b90161340 0x7f3b90151ae0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800cd80 tx=0x7f3b8800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:20.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.160+0000 7f3b86ffd700 1 -- 172.21.15.156:0/318972595 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8800f480 con 0x7f3b90161340 2023-11-28T13:09:20.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.160+0000 7f3b86ffd700 1 -- 172.21.15.156:0/318972595 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8800f600 con 0x7f3b90161340 2023-11-28T13:09:20.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.160+0000 7f3b86ffd700 1 -- 172.21.15.156:0/318972595 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8800f9b0 con 0x7f3b90161340 2023-11-28T13:09:20.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.160+0000 7f3b985ff700 1 -- 172.21.15.156:0/318972595 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b900b5130 con 0x7f3b90161340 2023-11-28T13:09:20.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.160+0000 7f3b985ff700 1 -- 172.21.15.156:0/318972595 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b900d15e0 con 0x7f3b90161340 2023-11-28T13:09:20.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.161+0000 7f3b985ff700 1 -- 172.21.15.156:0/318972595 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b900b52c0 con 0x7f3b90161340 2023-11-28T13:09:20.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.162+0000 7f3b86ffd700 1 -- 172.21.15.156:0/318972595 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3b88009030 con 0x7f3b90161340 2023-11-28T13:09:20.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.163+0000 7f3b86ffd700 1 --2- 172.21.15.156:0/318972595 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3b740682f0 0x7f3b7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:20.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.163+0000 7f3b86ffd700 1 -- 172.21.15.156:0/318972595 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3b880865f0 con 0x7f3b90161340 2023-11-28T13:09:20.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.163+0000 7f3b9639b700 1 --2- 172.21.15.156:0/318972595 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3b740682f0 0x7f3b7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:20.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.164+0000 7f3b9639b700 1 --2- 172.21.15.156:0/318972595 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3b740682f0 0x7f3b7406a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c000c00 tx=0x7f3b8c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:20.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.167+0000 7f3b86ffd700 1 -- 172.21.15.156:0/318972595 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3b88052d60 con 0x7f3b90161340 2023-11-28T13:09:20.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.357+0000 7f3b985ff700 1 -- 172.21.15.156:0/318972595 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3b900745c0 con 0x7f3b90161340 2023-11-28T13:09:20.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.357+0000 7f3b86ffd700 1 -- 172.21.15.156:0/318972595 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3b8801d020 con 0x7f3b90161340 2023-11-28T13:09:20.361 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:09:20.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.359+0000 7f3b985ff700 1 -- 172.21.15.156:0/318972595 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3b740682f0 msgr2=0x7f3b7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:20.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.359+0000 7f3b985ff700 1 --2- 172.21.15.156:0/318972595 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3b740682f0 0x7f3b7406a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c000c00 tx=0x7f3b8c008030 comp rx=0 tx=0).stop 2023-11-28T13:09:20.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.359+0000 7f3b985ff700 1 -- 172.21.15.156:0/318972595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b90161340 msgr2=0x7f3b90151ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:20.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.359+0000 7f3b985ff700 1 --2- 172.21.15.156:0/318972595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b90161340 0x7f3b90151ae0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800cd80 tx=0x7f3b8800a560 comp rx=0 tx=0).stop 2023-11-28T13:09:20.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.359+0000 7f3b985ff700 1 -- 172.21.15.156:0/318972595 shutdown_connections 2023-11-28T13:09:20.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.359+0000 7f3b985ff700 1 --2- 172.21.15.156:0/318972595 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3b740682f0 0x7f3b7406a7a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.359+0000 7f3b985ff700 1 --2- 172.21.15.156:0/318972595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b90161340 0x7f3b90151ae0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.359+0000 7f3b985ff700 1 --2- 172.21.15.156:0/318972595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b900d2320 0x7f3b9014e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.359+0000 7f3b985ff700 1 --2- 172.21.15.156:0/318972595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3b9015c9b0 0x7f3b901515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.359+0000 7f3b985ff700 1 -- 172.21.15.156:0/318972595 >> 172.21.15.156:0/318972595 conn(0x7f3b90072600 msgr2=0x7f3b9006a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:20.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.360+0000 7f3b985ff700 1 -- 172.21.15.156:0/318972595 shutdown_connections 2023-11-28T13:09:20.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.360+0000 7f3b985ff700 1 -- 172.21.15.156:0/318972595 wait complete. 2023-11-28T13:09:20.380 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:09:20.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.581+0000 7f115d1ff700 1 Processor -- start 2023-11-28T13:09:20.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.582+0000 7f115d1ff700 1 -- start start 2023-11-28T13:09:20.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.583+0000 7f115d1ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f115815c9b0 0x7f11580b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:20.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.583+0000 7f115d1ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1158161340 0x7f11580d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:20.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.583+0000 7f115d1ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11580d1950 con 0x7f11580d2320 2023-11-28T13:09:20.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.583+0000 7f115d1ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11580b5800 con 0x7f115815c9b0 2023-11-28T13:09:20.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.583+0000 7f115d1ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11580b5520 con 0x7f1158161340 2023-11-28T13:09:20.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.583+0000 7f1156d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f115815c9b0 0x7f11580b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:20.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.583+0000 7f1156d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f115815c9b0 0x7f11580b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39808/0 (socket says 172.21.15.156:39808) 2023-11-28T13:09:20.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.583+0000 7f1156d9d700 1 -- 172.21.15.156:0/3636431853 learned_addr learned my addr 172.21.15.156:0/3636431853 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:20.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.583+0000 7f115659c700 1 --2- 172.21.15.156:0/3636431853 >> v2:172.21.15.186:3300/0 conn(0x7f1158161340 0x7f11580d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:20.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.583+0000 7f1156d9d700 1 -- 172.21.15.156:0/3636431853 >> v2:172.21.15.186:3300/0 conn(0x7f1158161340 msgr2=0x7f11580d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:20.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.583+0000 7f1156d9d700 1 --2- 172.21.15.156:0/3636431853 >> v2:172.21.15.186:3300/0 conn(0x7f1158161340 0x7f11580d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.583+0000 7f1156d9d700 1 -- 172.21.15.156:0/3636431853 >> v1:172.21.15.186:6789/0 conn(0x7f11580d2320 legacy=0x7f11580cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:20.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.583+0000 7f1156d9d700 1 -- 172.21.15.156:0/3636431853 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11580d1300 con 0x7f115815c9b0 2023-11-28T13:09:20.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.584+0000 7f1156d9d700 1 --2- 172.21.15.156:0/3636431853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f115815c9b0 0x7f11580b4bf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f11580685c0 tx=0x7f1148009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415d52d5265e3a37 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:20.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.584+0000 7f1155d9b700 1 -- 172.21.15.156:0/3636431853 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148004230 con 0x7f115815c9b0 2023-11-28T13:09:20.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.584+0000 7f1155d9b700 1 -- 172.21.15.156:0/3636431853 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1148004390 con 0x7f115815c9b0 2023-11-28T13:09:20.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.584+0000 7f1155d9b700 1 -- 172.21.15.156:0/3636431853 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11480036a0 con 0x7f115815c9b0 2023-11-28T13:09:20.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.584+0000 7f115d1ff700 1 -- 172.21.15.156:0/3636431853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f115815c9b0 msgr2=0x7f11580b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:20.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.584+0000 7f115d1ff700 1 --2- 172.21.15.156:0/3636431853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f115815c9b0 0x7f11580b4bf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f11580685c0 tx=0x7f1148009bd0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.584+0000 7f115d1ff700 1 -- 172.21.15.156:0/3636431853 shutdown_connections 2023-11-28T13:09:20.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.584+0000 7f115d1ff700 1 --2- 172.21.15.156:0/3636431853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f115815c9b0 0x7f11580b4bf0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.585+0000 7f115d1ff700 1 --2- 172.21.15.156:0/3636431853 >> v2:172.21.15.186:3300/0 conn(0x7f1158161340 0x7f11580d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.585+0000 7f115d1ff700 1 -- 172.21.15.156:0/3636431853 >> 172.21.15.156:0/3636431853 conn(0x7f1158072600 msgr2=0x7f11580cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:20.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.585+0000 7f115d1ff700 1 -- 172.21.15.156:0/3636431853 shutdown_connections 2023-11-28T13:09:20.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.585+0000 7f115d1ff700 1 -- 172.21.15.156:0/3636431853 wait complete. 2023-11-28T13:09:20.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.586+0000 7f115d1ff700 1 Processor -- start 2023-11-28T13:09:20.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.586+0000 7f115d1ff700 1 -- start start 2023-11-28T13:09:20.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.587+0000 7f115d1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11580d2320 0x7f115814b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:20.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.587+0000 7f115d1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f115815c9b0 0x7f1158158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:20.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.587+0000 7f115d1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1158161340 0x7f115814fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:20.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.587+0000 7f115d1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11581599b0 con 0x7f11580d2320 2023-11-28T13:09:20.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.587+0000 7f115d1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1158158d30 con 0x7f115815c9b0 2023-11-28T13:09:20.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.587+0000 7f115d1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1158159010 con 0x7f1158161340 2023-11-28T13:09:20.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.587+0000 7f1156d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11580d2320 0x7f115814b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:20.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.588+0000 7f1156d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11580d2320 0x7f115814b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39820/0 (socket says 172.21.15.156:39820) 2023-11-28T13:09:20.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.588+0000 7f1156d9d700 1 -- 172.21.15.156:0/3467516428 learned_addr learned my addr 172.21.15.156:0/3467516428 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:20.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.588+0000 7f115659c700 1 --2- 172.21.15.156:0/3467516428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f115815c9b0 0x7f1158158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:20.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.588+0000 7f115759e700 1 --2- 172.21.15.156:0/3467516428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1158161340 0x7f115814fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:20.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.588+0000 7f1156d9d700 1 -- 172.21.15.156:0/3467516428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1158161340 msgr2=0x7f115814fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:20.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.588+0000 7f1156d9d700 1 --2- 172.21.15.156:0/3467516428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1158161340 0x7f115814fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.588+0000 7f1156d9d700 1 -- 172.21.15.156:0/3467516428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f115815c9b0 msgr2=0x7f1158158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:20.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.588+0000 7f1156d9d700 1 --2- 172.21.15.156:0/3467516428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f115815c9b0 0x7f1158158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.588+0000 7f1156d9d700 1 -- 172.21.15.156:0/3467516428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1148009550 con 0x7f11580d2320 2023-11-28T13:09:20.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.588+0000 7f1156d9d700 1 --2- 172.21.15.156:0/3467516428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11580d2320 0x7f115814b9e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1148009c10 tx=0x7f1148004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:20.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.588+0000 7f113f7fe700 1 -- 172.21.15.156:0/3467516428 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148003e20 con 0x7f11580d2320 2023-11-28T13:09:20.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.589+0000 7f113f7fe700 1 -- 172.21.15.156:0/3467516428 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1148011630 con 0x7f11580d2320 2023-11-28T13:09:20.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.589+0000 7f115d1ff700 1 -- 172.21.15.156:0/3467516428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11580b5130 con 0x7f11580d2320 2023-11-28T13:09:20.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.589+0000 7f113f7fe700 1 -- 172.21.15.156:0/3467516428 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11480119e0 con 0x7f11580d2320 2023-11-28T13:09:20.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.589+0000 7f115d1ff700 1 -- 172.21.15.156:0/3467516428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11580d15e0 con 0x7f11580d2320 2023-11-28T13:09:20.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.590+0000 7f115d1ff700 1 -- 172.21.15.156:0/3467516428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11580b52c0 con 0x7f11580d2320 2023-11-28T13:09:20.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.591+0000 7f113f7fe700 1 -- 172.21.15.156:0/3467516428 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1148017020 con 0x7f11580d2320 2023-11-28T13:09:20.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.591+0000 7f113f7fe700 1 --2- 172.21.15.156:0/3467516428 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f11440682f0 0x7f114406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:20.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.592+0000 7f113f7fe700 1 -- 172.21.15.156:0/3467516428 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f11480874b0 con 0x7f11580d2320 2023-11-28T13:09:20.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.592+0000 7f115659c700 1 --2- 172.21.15.156:0/3467516428 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f11440682f0 0x7f114406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:20.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.592+0000 7f115659c700 1 --2- 172.21.15.156:0/3467516428 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f11440682f0 0x7f114406a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f11581f6260 tx=0x7f114c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:20.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.596+0000 7f113f7fe700 1 -- 172.21.15.156:0/3467516428 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1148053c90 con 0x7f11580d2320 2023-11-28T13:09:20.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.789+0000 7f115d1ff700 1 -- 172.21.15.156:0/3467516428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1158159af0 con 0x7f11580d2320 2023-11-28T13:09:20.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.790+0000 7f113f7fe700 1 -- 172.21.15.156:0/3467516428 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7f11480572b0 con 0x7f11580d2320 2023-11-28T13:09:20.791 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T13:09:20.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.792+0000 7f115d1ff700 1 -- 172.21.15.156:0/3467516428 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f11440682f0 msgr2=0x7f114406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:20.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.792+0000 7f115d1ff700 1 --2- 172.21.15.156:0/3467516428 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f11440682f0 0x7f114406a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f11581f6260 tx=0x7f114c008040 comp rx=0 tx=0).stop 2023-11-28T13:09:20.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.792+0000 7f115d1ff700 1 -- 172.21.15.156:0/3467516428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11580d2320 msgr2=0x7f115814b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:20.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.792+0000 7f115d1ff700 1 --2- 172.21.15.156:0/3467516428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11580d2320 0x7f115814b9e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1148009c10 tx=0x7f1148004750 comp rx=0 tx=0).stop 2023-11-28T13:09:20.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.792+0000 7f115d1ff700 1 -- 172.21.15.156:0/3467516428 shutdown_connections 2023-11-28T13:09:20.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.793+0000 7f115d1ff700 1 --2- 172.21.15.156:0/3467516428 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f11440682f0 0x7f114406a7a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.793+0000 7f115d1ff700 1 --2- 172.21.15.156:0/3467516428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1158161340 0x7f115814fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.793+0000 7f115d1ff700 1 --2- 172.21.15.156:0/3467516428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11580d2320 0x7f115814b9e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.793+0000 7f115d1ff700 1 --2- 172.21.15.156:0/3467516428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f115815c9b0 0x7f1158158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:20.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.793+0000 7f115d1ff700 1 -- 172.21.15.156:0/3467516428 >> 172.21.15.156:0/3467516428 conn(0x7f1158072600 msgr2=0x7f11580b7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:20.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.793+0000 7f115d1ff700 1 -- 172.21.15.156:0/3467516428 shutdown_connections 2023-11-28T13:09:20.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:20.793+0000 7f115d1ff700 1 -- 172.21.15.156:0/3467516428 wait complete. 2023-11-28T13:09:20.811 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:09:21.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.011+0000 7ff3e81ff700 1 Processor -- start 2023-11-28T13:09:21.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.011+0000 7ff3e81ff700 1 -- start start 2023-11-28T13:09:21.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.012+0000 7ff3e81ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3e015c9b0 0x7ff3e00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:21.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.012+0000 7ff3e81ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff3e00cd7c0 0x7ff3e00cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:21.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.012+0000 7ff3e81ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e00ce160 con 0x7ff3e0161340 2023-11-28T13:09:21.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.012+0000 7ff3e81ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e00b5800 con 0x7ff3e00cd7c0 2023-11-28T13:09:21.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.012+0000 7ff3e81ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e00d1950 con 0x7ff3e015c9b0 2023-11-28T13:09:21.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.013+0000 7ff3e5f9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3e015c9b0 0x7ff3e00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:21.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.013+0000 7ff3e5f9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3e015c9b0 0x7ff3e00b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:38188/0 (socket says 172.21.15.156:38188) 2023-11-28T13:09:21.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.013+0000 7ff3e5f9b700 1 -- 172.21.15.156:0/3484261656 learned_addr learned my addr 172.21.15.156:0/3484261656 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:21.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.013+0000 7ff3e679c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff3e00cd7c0 0x7ff3e00cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:21.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.013+0000 7ff3e5f9b700 1 -- 172.21.15.156:0/3484261656 >> v2:172.21.15.186:3300/0 conn(0x7ff3e00cd7c0 msgr2=0x7ff3e00cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:21.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.013+0000 7ff3e5f9b700 1 --2- 172.21.15.156:0/3484261656 >> v2:172.21.15.186:3300/0 conn(0x7ff3e00cd7c0 0x7ff3e00cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.013+0000 7ff3e4f99700 1 -- 172.21.15.156:0/3484261656 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2363666847 0 0) 0x7ff3e00ce160 con 0x7ff3e0161340 2023-11-28T13:09:21.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.013+0000 7ff3e5f9b700 1 -- 172.21.15.156:0/3484261656 >> v1:172.21.15.186:6789/0 conn(0x7ff3e0161340 legacy=0x7ff3e00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:21.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.013+0000 7ff3e5f9b700 1 -- 172.21.15.156:0/3484261656 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e00d24f0 con 0x7ff3e015c9b0 2023-11-28T13:09:21.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.013+0000 7ff3e5f9b700 1 --2- 172.21.15.156:0/3484261656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3e015c9b0 0x7ff3e00b4bf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0098f0 tx=0x7ff3dc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81ed269a69504754 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:21.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.014+0000 7ff3e4f99700 1 -- 172.21.15.156:0/3484261656 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc011830 con 0x7ff3e015c9b0 2023-11-28T13:09:21.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.014+0000 7ff3e4f99700 1 -- 172.21.15.156:0/3484261656 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc011990 con 0x7ff3e015c9b0 2023-11-28T13:09:21.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.014+0000 7ff3e4f99700 1 -- 172.21.15.156:0/3484261656 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc011d20 con 0x7ff3e015c9b0 2023-11-28T13:09:21.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.014+0000 7ff3e81ff700 1 -- 172.21.15.156:0/3484261656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3e015c9b0 msgr2=0x7ff3e00b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:21.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.014+0000 7ff3e81ff700 1 --2- 172.21.15.156:0/3484261656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3e015c9b0 0x7ff3e00b4bf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0098f0 tx=0x7ff3dc009c00 comp rx=0 tx=0).stop 2023-11-28T13:09:21.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.014+0000 7ff3e81ff700 1 -- 172.21.15.156:0/3484261656 shutdown_connections 2023-11-28T13:09:21.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.014+0000 7ff3e81ff700 1 --2- 172.21.15.156:0/3484261656 >> v2:172.21.15.186:3300/0 conn(0x7ff3e00cd7c0 0x7ff3e00cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.014+0000 7ff3e81ff700 1 --2- 172.21.15.156:0/3484261656 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3e015c9b0 0x7ff3e00b4bf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.014+0000 7ff3e81ff700 1 -- 172.21.15.156:0/3484261656 >> 172.21.15.156:0/3484261656 conn(0x7ff3e0072600 msgr2=0x7ff3e00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:21.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.014+0000 7ff3e81ff700 1 -- 172.21.15.156:0/3484261656 shutdown_connections 2023-11-28T13:09:21.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.014+0000 7ff3e81ff700 1 -- 172.21.15.156:0/3484261656 wait complete. 2023-11-28T13:09:21.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.016+0000 7ff3e81ff700 1 Processor -- start 2023-11-28T13:09:21.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.016+0000 7ff3e81ff700 1 -- start start 2023-11-28T13:09:21.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.017+0000 7ff3e81ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e00cd7c0 0x7ff3e014a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:21.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.017+0000 7ff3e81ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3e015c9b0 0x7ff3e015e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:21.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.017+0000 7ff3e81ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3e0161340 0x7ff3e015eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:21.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.017+0000 7ff3e81ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e01f4290 con 0x7ff3e015c9b0 2023-11-28T13:09:21.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.017+0000 7ff3e81ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e00ca2e0 con 0x7ff3e00cd7c0 2023-11-28T13:09:21.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.017+0000 7ff3e81ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e00ca580 con 0x7ff3e0161340 2023-11-28T13:09:21.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.017+0000 7ff3e579a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3e015c9b0 0x7ff3e015e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:21.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.017+0000 7ff3e579a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3e015c9b0 0x7ff3e015e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39836/0 (socket says 172.21.15.156:39836) 2023-11-28T13:09:21.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.017+0000 7ff3e579a700 1 -- 172.21.15.156:0/319625995 learned_addr learned my addr 172.21.15.156:0/319625995 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:21.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.017+0000 7ff3e5f9b700 1 --2- 172.21.15.156:0/319625995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e00cd7c0 0x7ff3e014a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:21.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.017+0000 7ff3e679c700 1 --2- 172.21.15.156:0/319625995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3e0161340 0x7ff3e015eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:21.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.018+0000 7ff3e579a700 1 -- 172.21.15.156:0/319625995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3e0161340 msgr2=0x7ff3e015eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:21.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.018+0000 7ff3e579a700 1 --2- 172.21.15.156:0/319625995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3e0161340 0x7ff3e015eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.018+0000 7ff3e579a700 1 -- 172.21.15.156:0/319625995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e00cd7c0 msgr2=0x7ff3e014a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:21.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.018+0000 7ff3e579a700 1 --2- 172.21.15.156:0/319625995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e00cd7c0 0x7ff3e014a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.018+0000 7ff3e579a700 1 -- 172.21.15.156:0/319625995 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc009550 con 0x7ff3e015c9b0 2023-11-28T13:09:21.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.018+0000 7ff3e579a700 1 --2- 172.21.15.156:0/319625995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3e015c9b0 0x7ff3e015e660 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000c550 tx=0x7ff3d000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:21.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.018+0000 7ff3d67fc700 1 -- 172.21.15.156:0/319625995 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00054e0 con 0x7ff3e015c9b0 2023-11-28T13:09:21.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.018+0000 7ff3d67fc700 1 -- 172.21.15.156:0/319625995 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d0003680 con 0x7ff3e015c9b0 2023-11-28T13:09:21.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.018+0000 7ff3e81ff700 1 -- 172.21.15.156:0/319625995 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e00d1300 con 0x7ff3e015c9b0 2023-11-28T13:09:21.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.018+0000 7ff3d67fc700 1 -- 172.21.15.156:0/319625995 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0003a30 con 0x7ff3e015c9b0 2023-11-28T13:09:21.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.018+0000 7ff3e81ff700 1 -- 172.21.15.156:0/319625995 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e00b5520 con 0x7ff3e015c9b0 2023-11-28T13:09:21.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.019+0000 7ff3e81ff700 1 -- 172.21.15.156:0/319625995 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e00d15e0 con 0x7ff3e015c9b0 2023-11-28T13:09:21.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.020+0000 7ff3d67fc700 1 -- 172.21.15.156:0/319625995 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff3d0019030 con 0x7ff3e015c9b0 2023-11-28T13:09:21.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.021+0000 7ff3d67fc700 1 --2- 172.21.15.156:0/319625995 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff3c40682f0 0x7ff3c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:21.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.021+0000 7ff3d67fc700 1 -- 172.21.15.156:0/319625995 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff3d0084fe0 con 0x7ff3e015c9b0 2023-11-28T13:09:21.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.021+0000 7ff3e5f9b700 1 --2- 172.21.15.156:0/319625995 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff3c40682f0 0x7ff3c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:21.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.022+0000 7ff3e5f9b700 1 --2- 172.21.15.156:0/319625995 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff3c40682f0 0x7ff3c406a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc014040 tx=0x7ff3dc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:21.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.025+0000 7ff3d67fc700 1 -- 172.21.15.156:0/319625995 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff3d0051710 con 0x7ff3e015c9b0 2023-11-28T13:09:21.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.214+0000 7ff3e81ff700 1 -- 172.21.15.156:0/319625995 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff3e00c9f00 con 0x7ff3e015c9b0 2023-11-28T13:09:21.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.214+0000 7ff3d67fc700 1 -- 172.21.15.156:0/319625995 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff3d0017070 con 0x7ff3e015c9b0 2023-11-28T13:09:21.216 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T13:09:21.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.217+0000 7ff3e81ff700 1 -- 172.21.15.156:0/319625995 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff3c40682f0 msgr2=0x7ff3c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:21.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.217+0000 7ff3e81ff700 1 --2- 172.21.15.156:0/319625995 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff3c40682f0 0x7ff3c406a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc014040 tx=0x7ff3dc008030 comp rx=0 tx=0).stop 2023-11-28T13:09:21.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.217+0000 7ff3e81ff700 1 -- 172.21.15.156:0/319625995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3e015c9b0 msgr2=0x7ff3e015e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:21.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.217+0000 7ff3e81ff700 1 --2- 172.21.15.156:0/319625995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3e015c9b0 0x7ff3e015e660 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000c550 tx=0x7ff3d000c860 comp rx=0 tx=0).stop 2023-11-28T13:09:21.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.217+0000 7ff3e81ff700 1 -- 172.21.15.156:0/319625995 shutdown_connections 2023-11-28T13:09:21.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.217+0000 7ff3e81ff700 1 --2- 172.21.15.156:0/319625995 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff3c40682f0 0x7ff3c406a7a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.217+0000 7ff3e81ff700 1 --2- 172.21.15.156:0/319625995 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3e0161340 0x7ff3e015eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.217+0000 7ff3e81ff700 1 --2- 172.21.15.156:0/319625995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e00cd7c0 0x7ff3e014a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.217+0000 7ff3e81ff700 1 --2- 172.21.15.156:0/319625995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3e015c9b0 0x7ff3e015e660 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.217+0000 7ff3e81ff700 1 -- 172.21.15.156:0/319625995 >> 172.21.15.156:0/319625995 conn(0x7ff3e0072600 msgr2=0x7ff3e006b280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:21.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.217+0000 7ff3e81ff700 1 -- 172.21.15.156:0/319625995 shutdown_connections 2023-11-28T13:09:21.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.217+0000 7ff3e81ff700 1 -- 172.21.15.156:0/319625995 wait complete. 2023-11-28T13:09:21.236 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:09:21.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.432+0000 7f2714fff700 1 Processor -- start 2023-11-28T13:09:21.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.432+0000 7f2714fff700 1 -- start start 2023-11-28T13:09:21.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.433+0000 7f2714fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f271015c9b0 0x7f27100b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:21.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.433+0000 7f2714fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2710161340 0x7f27100d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:21.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.433+0000 7f2714fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27100d1950 con 0x7f27100d2320 2023-11-28T13:09:21.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.433+0000 7f2714fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27100b5800 con 0x7f271015c9b0 2023-11-28T13:09:21.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.433+0000 7f2714fff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27100b5520 con 0x7f2710161340 2023-11-28T13:09:21.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.434+0000 7f270e59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f271015c9b0 0x7f27100b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:21.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.434+0000 7f270e59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f271015c9b0 0x7f27100b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39838/0 (socket says 172.21.15.156:39838) 2023-11-28T13:09:21.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.434+0000 7f270e59c700 1 -- 172.21.15.156:0/999708204 learned_addr learned my addr 172.21.15.156:0/999708204 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:21.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.434+0000 7f270e59c700 1 -- 172.21.15.156:0/999708204 >> v2:172.21.15.186:3300/0 conn(0x7f2710161340 msgr2=0x7f27100d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:21.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.434+0000 7f270e59c700 1 --2- 172.21.15.156:0/999708204 >> v2:172.21.15.186:3300/0 conn(0x7f2710161340 0x7f27100d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.434+0000 7f270e59c700 1 -- 172.21.15.156:0/999708204 >> v1:172.21.15.186:6789/0 conn(0x7f27100d2320 legacy=0x7f27100cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:21.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.434+0000 7f270e59c700 1 -- 172.21.15.156:0/999708204 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27100d1300 con 0x7f271015c9b0 2023-11-28T13:09:21.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.434+0000 7f270e59c700 1 --2- 172.21.15.156:0/999708204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f271015c9b0 0x7f27100b4bf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f27100685c0 tx=0x7f2700009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ce3c6bb48b2381 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:21.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.435+0000 7f270dd9b700 1 -- 172.21.15.156:0/999708204 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700004230 con 0x7f271015c9b0 2023-11-28T13:09:21.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.435+0000 7f270dd9b700 1 -- 172.21.15.156:0/999708204 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2700004390 con 0x7f271015c9b0 2023-11-28T13:09:21.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.435+0000 7f270dd9b700 1 -- 172.21.15.156:0/999708204 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27000036a0 con 0x7f271015c9b0 2023-11-28T13:09:21.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.435+0000 7f2714fff700 1 -- 172.21.15.156:0/999708204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f271015c9b0 msgr2=0x7f27100b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:21.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.435+0000 7f2714fff700 1 --2- 172.21.15.156:0/999708204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f271015c9b0 0x7f27100b4bf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f27100685c0 tx=0x7f2700009bd0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.435+0000 7f2714fff700 1 -- 172.21.15.156:0/999708204 shutdown_connections 2023-11-28T13:09:21.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.435+0000 7f2714fff700 1 --2- 172.21.15.156:0/999708204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f271015c9b0 0x7f27100b4bf0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.435+0000 7f2714fff700 1 --2- 172.21.15.156:0/999708204 >> v2:172.21.15.186:3300/0 conn(0x7f2710161340 0x7f27100d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.435+0000 7f2714fff700 1 -- 172.21.15.156:0/999708204 >> 172.21.15.156:0/999708204 conn(0x7f2710072600 msgr2=0x7f27100cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:21.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.435+0000 7f2714fff700 1 -- 172.21.15.156:0/999708204 shutdown_connections 2023-11-28T13:09:21.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.435+0000 7f2714fff700 1 -- 172.21.15.156:0/999708204 wait complete. 2023-11-28T13:09:21.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.437+0000 7f2714fff700 1 Processor -- start 2023-11-28T13:09:21.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.437+0000 7f2714fff700 1 -- start start 2023-11-28T13:09:21.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.438+0000 7f2714fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27100d2320 0x7f27101463b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:21.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.438+0000 7f2714fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f271015c9b0 0x7f27101543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:21.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.438+0000 7f2714fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2710161340 0x7f27101f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:21.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.438+0000 7f2714fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710157250 con 0x7f27100d2320 2023-11-28T13:09:21.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.438+0000 7f2714fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710156610 con 0x7f271015c9b0 2023-11-28T13:09:21.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.438+0000 7f2714fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27101568b0 con 0x7f2710161340 2023-11-28T13:09:21.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.438+0000 7f270e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27100d2320 0x7f27101463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:21.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.438+0000 7f270e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27100d2320 0x7f27101463b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39852/0 (socket says 172.21.15.156:39852) 2023-11-28T13:09:21.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.438+0000 7f270e59c700 1 -- 172.21.15.156:0/2256588866 learned_addr learned my addr 172.21.15.156:0/2256588866 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:21.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.438+0000 7f2707fff700 1 --2- 172.21.15.156:0/2256588866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f271015c9b0 0x7f27101543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:21.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.438+0000 7f270ed9d700 1 --2- 172.21.15.156:0/2256588866 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2710161340 0x7f27101f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:21.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.439+0000 7f270e59c700 1 -- 172.21.15.156:0/2256588866 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2710161340 msgr2=0x7f27101f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:21.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.439+0000 7f270e59c700 1 --2- 172.21.15.156:0/2256588866 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2710161340 0x7f27101f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.439+0000 7f270e59c700 1 -- 172.21.15.156:0/2256588866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f271015c9b0 msgr2=0x7f27101543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:21.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.439+0000 7f270e59c700 1 --2- 172.21.15.156:0/2256588866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f271015c9b0 0x7f27101543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.439+0000 7f270e59c700 1 -- 172.21.15.156:0/2256588866 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2700009550 con 0x7f27100d2320 2023-11-28T13:09:21.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.439+0000 7f270e59c700 1 --2- 172.21.15.156:0/2256588866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27100d2320 0x7f27101463b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f27100675d0 tx=0x7f2700004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:21.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.439+0000 7f2706ffd700 1 -- 172.21.15.156:0/2256588866 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700003c90 con 0x7f27100d2320 2023-11-28T13:09:21.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.439+0000 7f2706ffd700 1 -- 172.21.15.156:0/2256588866 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270001f070 con 0x7f27100d2320 2023-11-28T13:09:21.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.439+0000 7f2714fff700 1 -- 172.21.15.156:0/2256588866 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27100b5130 con 0x7f27100d2320 2023-11-28T13:09:21.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.439+0000 7f2706ffd700 1 -- 172.21.15.156:0/2256588866 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700011870 con 0x7f27100d2320 2023-11-28T13:09:21.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.439+0000 7f2714fff700 1 -- 172.21.15.156:0/2256588866 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27100d15e0 con 0x7f27100d2320 2023-11-28T13:09:21.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.440+0000 7f2714fff700 1 -- 172.21.15.156:0/2256588866 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27100b52c0 con 0x7f27100d2320 2023-11-28T13:09:21.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.441+0000 7f2706ffd700 1 -- 172.21.15.156:0/2256588866 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2700017020 con 0x7f27100d2320 2023-11-28T13:09:21.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.442+0000 7f2706ffd700 1 --2- 172.21.15.156:0/2256588866 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26f00682f0 0x7f26f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:21.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.442+0000 7f2706ffd700 1 -- 172.21.15.156:0/2256588866 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f27000874b0 con 0x7f27100d2320 2023-11-28T13:09:21.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.443+0000 7f2707fff700 1 --2- 172.21.15.156:0/2256588866 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26f00682f0 0x7f26f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:21.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.443+0000 7f2707fff700 1 --2- 172.21.15.156:0/2256588866 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26f00682f0 0x7f26f006a7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2710150b10 tx=0x7f26f8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:21.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.446+0000 7f2706ffd700 1 -- 172.21.15.156:0/2256588866 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2700053c20 con 0x7f27100d2320 2023-11-28T13:09:21.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.655+0000 7f2714fff700 1 -- 172.21.15.156:0/2256588866 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f27100745c0 con 0x7f27100d2320 2023-11-28T13:09:21.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.655+0000 7f2706ffd700 1 -- 172.21.15.156:0/2256588866 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2700057240 con 0x7f27100d2320 2023-11-28T13:09:21.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.657+0000 7f2714fff700 1 -- 172.21.15.156:0/2256588866 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26f00682f0 msgr2=0x7f26f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:21.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.657+0000 7f2714fff700 1 --2- 172.21.15.156:0/2256588866 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26f00682f0 0x7f26f006a7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2710150b10 tx=0x7f26f8009190 comp rx=0 tx=0).stop 2023-11-28T13:09:21.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.657+0000 7f2714fff700 1 -- 172.21.15.156:0/2256588866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27100d2320 msgr2=0x7f27101463b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:21.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.657+0000 7f2714fff700 1 --2- 172.21.15.156:0/2256588866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27100d2320 0x7f27101463b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f27100675d0 tx=0x7f2700004750 comp rx=0 tx=0).stop 2023-11-28T13:09:21.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.658+0000 7f2714fff700 1 -- 172.21.15.156:0/2256588866 shutdown_connections 2023-11-28T13:09:21.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.658+0000 7f2714fff700 1 --2- 172.21.15.156:0/2256588866 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f26f00682f0 0x7f26f006a7a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.658+0000 7f2714fff700 1 --2- 172.21.15.156:0/2256588866 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2710161340 0x7f27101f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.658+0000 7f2714fff700 1 --2- 172.21.15.156:0/2256588866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27100d2320 0x7f27101463b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.658+0000 7f2714fff700 1 --2- 172.21.15.156:0/2256588866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f271015c9b0 0x7f27101543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:21.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.658+0000 7f2714fff700 1 -- 172.21.15.156:0/2256588866 >> 172.21.15.156:0/2256588866 conn(0x7f2710072600 msgr2=0x7f271015a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:21.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.658+0000 7f2714fff700 1 -- 172.21.15.156:0/2256588866 shutdown_connections 2023-11-28T13:09:21.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:21.658+0000 7f2714fff700 1 -- 172.21.15.156:0/2256588866 wait complete. 2023-11-28T13:09:21.661 INFO:teuthology.orchestra.run.smithi156.stderr:listed 0 entries 2023-11-28T13:09:21.677 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T13:09:21.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:21.677 DEBUG:teuthology.orchestra.run.smithi156:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T13:09:21.792 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T13:09:21.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:21.792 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T13:09:21.842 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:09:22.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.078+0000 7fb65f1ff700 1 Processor -- start 2023-11-28T13:09:22.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.079+0000 7fb65f1ff700 1 -- start start 2023-11-28T13:09:22.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.080+0000 7fb65f1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6580cd7c0 0x7fb6580cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:22.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.080+0000 7fb65f1ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6580d1950 con 0x7fb65815c9b0 2023-11-28T13:09:22.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.080+0000 7fb65f1ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6580ce140 con 0x7fb658161340 2023-11-28T13:09:22.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.080+0000 7fb65f1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6580b5800 con 0x7fb6580cd7c0 2023-11-28T13:09:22.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.081+0000 7fb65cf9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb65815c9b0 0x7fb6580b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:34828/0 (socket says 172.21.15.156:34828) 2023-11-28T13:09:22.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.081+0000 7fb65cf9b700 1 -- 172.21.15.156:0/1406426144 learned_addr learned my addr 172.21.15.156:0/1406426144 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:22.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.081+0000 7fb65d79c700 1 --2- 172.21.15.156:0/1406426144 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6580cd7c0 0x7fb6580cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:22.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.081+0000 7fb6577fe700 1 -- 172.21.15.156:0/1406426144 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739396237 0 0) 0x7fb6580d1950 con 0x7fb65815c9b0 2023-11-28T13:09:22.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.081+0000 7fb6577fe700 1 -- 172.21.15.156:0/1406426144 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6480035b0 con 0x7fb65815c9b0 2023-11-28T13:09:22.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.081+0000 7fb6577fe700 1 -- 172.21.15.156:0/1406426144 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359290469 0 0) 0x7fb6580ce140 con 0x7fb658161340 2023-11-28T13:09:22.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.081+0000 7fb6577fe700 1 -- 172.21.15.156:0/1406426144 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6580d1950 con 0x7fb658161340 2023-11-28T13:09:22.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.081+0000 7fb6577fe700 1 -- 172.21.15.156:0/1406426144 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3647514257 0 0) 0x7fb6480035b0 con 0x7fb65815c9b0 2023-11-28T13:09:22.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.081+0000 7fb6577fe700 1 -- 172.21.15.156:0/1406426144 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6580cd7c0 msgr2=0x7fb6580cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:22.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.081+0000 7fb6577fe700 1 --2- 172.21.15.156:0/1406426144 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6580cd7c0 0x7fb6580cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:22.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.081+0000 7fb6577fe700 1 -- 172.21.15.156:0/1406426144 >> v1:172.21.15.186:6789/0 conn(0x7fb658161340 legacy=0x7fb6580d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:22.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.081+0000 7fb6577fe700 1 -- 172.21.15.156:0/1406426144 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6580d24f0 con 0x7fb65815c9b0 2023-11-28T13:09:22.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.081+0000 7fb6577fe700 1 -- 172.21.15.156:0/1406426144 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7fb64c001670 con 0x7fb65815c9b0 2023-11-28T13:09:22.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.081+0000 7fb6577fe700 1 -- 172.21.15.156:0/1406426144 >> v1:172.21.15.156:6789/0 conn(0x7fb65815c9b0 legacy=0x7fb6580b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:22.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.081+0000 7fb6577fe700 1 --2- 172.21.15.156:0/1406426144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb648003bc0 0x7fb648003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:22.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.082+0000 7fb6577fe700 1 --2- 172.21.15.156:0/1406426144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb648004850 0x7fb648004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:22.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.082+0000 7fb6577fe700 1 --2- 172.21.15.156:0/1406426144 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6580cd7c0 0x7fb648005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:22.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.082+0000 7fb6577fe700 1 -- 172.21.15.156:0/1406426144 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6580b5800 con 0x7fb648004850 2023-11-28T13:09:22.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.082+0000 7fb6577fe700 1 -- 172.21.15.156:0/1406426144 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64800bb20 con 0x7fb648003bc0 2023-11-28T13:09:22.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.082+0000 7fb6577fe700 1 -- 172.21.15.156:0/1406426144 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64800bce0 con 0x7fb6580cd7c0 2023-11-28T13:09:22.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.082+0000 7fb657fff700 1 --2- 172.21.15.156:0/1406426144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb648004850 0x7fb648004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:22.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.082+0000 7fb65cf9b700 1 --2- 172.21.15.156:0/1406426144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb648003bc0 0x7fb648003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:22.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.082+0000 7fb65d79c700 1 --2- 172.21.15.156:0/1406426144 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6580cd7c0 0x7fb648005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:22.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.082+0000 7fb657fff700 1 -- 172.21.15.156:0/1406426144 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6580cd7c0 msgr2=0x7fb648005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:22.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.082+0000 7fb657fff700 1 --2- 172.21.15.156:0/1406426144 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6580cd7c0 0x7fb648005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:22.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.082+0000 7fb657fff700 1 -- 172.21.15.156:0/1406426144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb648003bc0 msgr2=0x7fb648003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:22.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.082+0000 7fb657fff700 1 --2- 172.21.15.156:0/1406426144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb648003bc0 0x7fb648003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:22.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.082+0000 7fb657fff700 1 -- 172.21.15.156:0/1406426144 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6480035b0 con 0x7fb648004850 2023-11-28T13:09:22.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.082+0000 7fb657fff700 1 --2- 172.21.15.156:0/1406426144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb648004850 0x7fb648004cc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb644001940 tx=0x7fb648005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234aa60de68d8fe6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:22.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.083+0000 7fb6577fe700 1 -- 172.21.15.156:0/1406426144 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644001430 con 0x7fb648004850 2023-11-28T13:09:22.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.083+0000 7fb6577fe700 1 -- 172.21.15.156:0/1406426144 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb644001d70 con 0x7fb648004850 2023-11-28T13:09:22.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.083+0000 7fb6577fe700 1 -- 172.21.15.156:0/1406426144 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644005510 con 0x7fb648004850 2023-11-28T13:09:22.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.083+0000 7fb65f1ff700 1 -- 172.21.15.156:0/1406426144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb648004850 msgr2=0x7fb648004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:22.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.083+0000 7fb65f1ff700 1 --2- 172.21.15.156:0/1406426144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb648004850 0x7fb648004cc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb644001940 tx=0x7fb648005880 comp rx=0 tx=0).stop 2023-11-28T13:09:22.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.083+0000 7fb65d79c700 1 -- 172.21.15.156:0/1406426144 reap_dead start 2023-11-28T13:09:22.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.083+0000 7fb65f1ff700 1 -- 172.21.15.156:0/1406426144 shutdown_connections 2023-11-28T13:09:22.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.083+0000 7fb65f1ff700 1 -- 172.21.15.156:0/1406426144 >> 172.21.15.156:0/1406426144 conn(0x7fb658072600 msgr2=0x7fb6580cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:22.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.083+0000 7fb65f1ff700 1 -- 172.21.15.156:0/1406426144 shutdown_connections 2023-11-28T13:09:22.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.083+0000 7fb65f1ff700 1 -- 172.21.15.156:0/1406426144 wait complete. 2023-11-28T13:09:22.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.085+0000 7fb65f1ff700 1 Processor -- start 2023-11-28T13:09:22.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.085+0000 7fb65f1ff700 1 -- start start 2023-11-28T13:09:22.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.086+0000 7fb65f1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb648004850 0x7fb6581e7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:22.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.086+0000 7fb65f1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6580cac50 0x7fb65815e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:22.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.086+0000 7fb65f1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6581e6420 0x7fb6581e6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:22.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.086+0000 7fb65f1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6580cbd60 con 0x7fb6581e6420 2023-11-28T13:09:22.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.086+0000 7fb65f1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6580cb430 con 0x7fb6580cac50 2023-11-28T13:09:22.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.086+0000 7fb65f1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6580cb710 con 0x7fb648004850 2023-11-28T13:09:22.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.086+0000 7fb65d79c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6581e6420 0x7fb6581e6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:22.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.086+0000 7fb65d79c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6581e6420 0x7fb6581e6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39864/0 (socket says 172.21.15.156:39864) 2023-11-28T13:09:22.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.086+0000 7fb65d79c700 1 -- 172.21.15.156:0/2463177199 learned_addr learned my addr 172.21.15.156:0/2463177199 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:22.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.086+0000 7fb65cf9b700 1 --2- 172.21.15.156:0/2463177199 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb648004850 0x7fb6581e7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:22.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.086+0000 7fb657fff700 1 --2- 172.21.15.156:0/2463177199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6580cac50 0x7fb65815e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:22.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.087+0000 7fb65d79c700 1 -- 172.21.15.156:0/2463177199 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb648004850 msgr2=0x7fb6581e7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:22.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.087+0000 7fb65d79c700 1 --2- 172.21.15.156:0/2463177199 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb648004850 0x7fb6581e7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:22.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.087+0000 7fb65d79c700 1 -- 172.21.15.156:0/2463177199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6580cac50 msgr2=0x7fb65815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:22.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.087+0000 7fb65d79c700 1 --2- 172.21.15.156:0/2463177199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6580cac50 0x7fb65815e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:22.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.087+0000 7fb65d79c700 1 -- 172.21.15.156:0/2463177199 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6580d1300 con 0x7fb6581e6420 2023-11-28T13:09:22.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.087+0000 7fb65d79c700 1 --2- 172.21.15.156:0/2463177199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6581e6420 0x7fb6581e6890 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb65806bbb0 tx=0x7fb650011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:22.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.087+0000 7fb6557fa700 1 -- 172.21.15.156:0/2463177199 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65000e040 con 0x7fb6581e6420 2023-11-28T13:09:22.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.087+0000 7fb6557fa700 1 -- 172.21.15.156:0/2463177199 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65000f040 con 0x7fb6581e6420 2023-11-28T13:09:22.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.087+0000 7fb65f1ff700 1 -- 172.21.15.156:0/2463177199 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6580b5520 con 0x7fb6581e6420 2023-11-28T13:09:22.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.087+0000 7fb6557fa700 1 -- 172.21.15.156:0/2463177199 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65000fe40 con 0x7fb6581e6420 2023-11-28T13:09:22.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.087+0000 7fb65f1ff700 1 -- 172.21.15.156:0/2463177199 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6580d15e0 con 0x7fb6581e6420 2023-11-28T13:09:22.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.088+0000 7fb65f1ff700 1 -- 172.21.15.156:0/2463177199 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6580cbf10 con 0x7fb6581e6420 2023-11-28T13:09:22.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.089+0000 7fb6557fa700 1 -- 172.21.15.156:0/2463177199 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb650022460 con 0x7fb6581e6420 2023-11-28T13:09:22.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.090+0000 7fb6557fa700 1 --2- 172.21.15.156:0/2463177199 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb63c0682f0 0x7fb63c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:22.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.091+0000 7fb6557fa700 1 -- 172.21.15.156:0/2463177199 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb6500862a0 con 0x7fb6581e6420 2023-11-28T13:09:22.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.091+0000 7fb65cf9b700 1 --2- 172.21.15.156:0/2463177199 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb63c0682f0 0x7fb63c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:22.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.091+0000 7fb65cf9b700 1 --2- 172.21.15.156:0/2463177199 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb63c0682f0 0x7fb63c06a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb64c001eb0 tx=0x7fb64c001e00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:22.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.094+0000 7fb6557fa700 1 -- 172.21.15.156:0/2463177199 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb6500529b0 con 0x7fb6581e6420 2023-11-28T13:09:22.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.327+0000 7fb65f1ff700 1 -- 172.21.15.156:0/2463177199 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6580745c0 con 0x7fb6581e6420 2023-11-28T13:09:22.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.327+0000 7fb6557fa700 1 -- 172.21.15.156:0/2463177199 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fb650055fd0 con 0x7fb6581e6420 2023-11-28T13:09:22.328 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:22.328 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1128367125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1128367125},{"type":"v1","addr":"172.21.15.156:6835","nonce":1128367125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4942,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/4096182008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4096182008},{"type":"v1","addr":"172.21.15.186:6835","nonce":4096182008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4945,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/561812165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":561812165},{"type":"v1","addr":"172.21.15.186:6837","nonce":561812165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:09:08.167940+0000","modified":"2023-11-28T13:09:11.184420+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5013":{"gid":5013,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6834/3471743149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3471743149},{"type":"v1","addr":"172.21.15.156:6834","nonce":3471743149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:09:22.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.330+0000 7fb65f1ff700 1 -- 172.21.15.156:0/2463177199 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb63c0682f0 msgr2=0x7fb63c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:22.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.330+0000 7fb65f1ff700 1 --2- 172.21.15.156:0/2463177199 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb63c0682f0 0x7fb63c06a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb64c001eb0 tx=0x7fb64c001e00 comp rx=0 tx=0).stop 2023-11-28T13:09:22.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.330+0000 7fb65f1ff700 1 -- 172.21.15.156:0/2463177199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6581e6420 msgr2=0x7fb6581e6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:22.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.330+0000 7fb65f1ff700 1 --2- 172.21.15.156:0/2463177199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6581e6420 0x7fb6581e6890 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb65806bbb0 tx=0x7fb650011f60 comp rx=0 tx=0).stop 2023-11-28T13:09:22.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.330+0000 7fb65f1ff700 1 -- 172.21.15.156:0/2463177199 shutdown_connections 2023-11-28T13:09:22.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.330+0000 7fb65f1ff700 1 --2- 172.21.15.156:0/2463177199 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb63c0682f0 0x7fb63c06a7a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:22.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.330+0000 7fb65f1ff700 1 --2- 172.21.15.156:0/2463177199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6581e6420 0x7fb6581e6890 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:22.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.330+0000 7fb65f1ff700 1 --2- 172.21.15.156:0/2463177199 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb648004850 0x7fb6581e7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:22.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.330+0000 7fb65f1ff700 1 --2- 172.21.15.156:0/2463177199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6580cac50 0x7fb65815e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:22.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.330+0000 7fb65f1ff700 1 -- 172.21.15.156:0/2463177199 >> 172.21.15.156:0/2463177199 conn(0x7fb658072600 msgr2=0x7fb65815a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:22.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.330+0000 7fb65f1ff700 1 -- 172.21.15.156:0/2463177199 shutdown_connections 2023-11-28T13:09:22.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.330+0000 7fb65f1ff700 1 -- 172.21.15.156:0/2463177199 wait complete. 2023-11-28T13:09:22.332 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2023-11-28T13:09:22.349 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-11-28T13:09:22.581 INFO:teuthology.orchestra.run.smithi156.stdout:{"message":"","return_code":0} 2023-11-28T13:09:22.581 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T13:09:22.582 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:09:22.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.776+0000 7f3e6cbff700 1 Processor -- start 2023-11-28T13:09:22.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.776+0000 7f3e6cbff700 1 -- start start 2023-11-28T13:09:22.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.777+0000 7f3e6cbff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e6815c9b0 0x7f3e680b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:22.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.777+0000 7f3e6cbff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3e68161340 0x7f3e680d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:22.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.777+0000 7f3e6cbff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e680d1950 con 0x7f3e680d2320 2023-11-28T13:09:22.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.777+0000 7f3e6cbff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e680b5800 con 0x7f3e68161340 2023-11-28T13:09:22.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.777+0000 7f3e6cbff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e680b5520 con 0x7f3e6815c9b0 2023-11-28T13:09:22.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.778+0000 7f3e65d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3e68161340 0x7f3e680d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:22.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.778+0000 7f3e66d9d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f3e680d2320 0x7f3e680cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:34832/0 (socket says 172.21.15.156:34832) 2023-11-28T13:09:22.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.778+0000 7f3e66d9d700 1 -- 172.21.15.156:0/2397174333 learned_addr learned my addr 172.21.15.156:0/2397174333 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:22.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.778+0000 7f3e6659c700 1 --2- 172.21.15.156:0/2397174333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e6815c9b0 0x7f3e680b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:22.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.778+0000 7f3e65d9b700 1 -- 172.21.15.156:0/2397174333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e6815c9b0 msgr2=0x7f3e680b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:22.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.778+0000 7f3e65d9b700 1 --2- 172.21.15.156:0/2397174333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e6815c9b0 0x7f3e680b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:22.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.778+0000 7f3e65d9b700 1 -- 172.21.15.156:0/2397174333 >> v1:172.21.15.156:6789/0 conn(0x7f3e680d2320 legacy=0x7f3e680cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:22.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.778+0000 7f3e65d9b700 1 -- 172.21.15.156:0/2397174333 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e680d1300 con 0x7f3e68161340 2023-11-28T13:09:22.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.778+0000 7f3e6559a700 1 -- 172.21.15.156:0/2397174333 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2240790880 0 0) 0x7f3e680d1950 con 0x7f3e680d2320 2023-11-28T13:09:22.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.778+0000 7f3e65d9b700 1 --2- 172.21.15.156:0/2397174333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e68161340 0x7f3e680d1de0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009840 tx=0x7f3e5c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abcc08bd118b024 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:22.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.779+0000 7f3e6559a700 1 -- 172.21.15.156:0/2397174333 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c011830 con 0x7f3e68161340 2023-11-28T13:09:22.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.779+0000 7f3e6559a700 1 -- 172.21.15.156:0/2397174333 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5c011990 con 0x7f3e68161340 2023-11-28T13:09:22.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.779+0000 7f3e6559a700 1 -- 172.21.15.156:0/2397174333 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c011d20 con 0x7f3e68161340 2023-11-28T13:09:22.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.779+0000 7f3e6cbff700 1 -- 172.21.15.156:0/2397174333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e68161340 msgr2=0x7f3e680d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:22.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.779+0000 7f3e6cbff700 1 --2- 172.21.15.156:0/2397174333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e68161340 0x7f3e680d1de0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009840 tx=0x7f3e5c009b50 comp rx=0 tx=0).stop 2023-11-28T13:09:22.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.779+0000 7f3e6cbff700 1 -- 172.21.15.156:0/2397174333 shutdown_connections 2023-11-28T13:09:22.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.779+0000 7f3e6cbff700 1 --2- 172.21.15.156:0/2397174333 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e6815c9b0 0x7f3e680b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:22.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.779+0000 7f3e6cbff700 1 --2- 172.21.15.156:0/2397174333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e68161340 0x7f3e680d1de0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:22.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.779+0000 7f3e6cbff700 1 -- 172.21.15.156:0/2397174333 >> 172.21.15.156:0/2397174333 conn(0x7f3e68072600 msgr2=0x7f3e680cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:22.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.779+0000 7f3e6cbff700 1 -- 172.21.15.156:0/2397174333 shutdown_connections 2023-11-28T13:09:22.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.779+0000 7f3e6cbff700 1 -- 172.21.15.156:0/2397174333 wait complete. 2023-11-28T13:09:22.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.781+0000 7f3e6cbff700 1 Processor -- start 2023-11-28T13:09:22.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.781+0000 7f3e6cbff700 1 -- start start 2023-11-28T13:09:22.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.782+0000 7f3e6cbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e680d2320 0x7f3e681e7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:22.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.782+0000 7f3e6cbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e6815c9b0 0x7f3e680b7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:22.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.782+0000 7f3e6cbff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e68161340 0x7f3e680b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:22.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.782+0000 7f3e6cbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e68140e10 con 0x7f3e6815c9b0 2023-11-28T13:09:22.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.782+0000 7f3e6cbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e681e8430 con 0x7f3e680d2320 2023-11-28T13:09:22.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.782+0000 7f3e6cbff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e681e6460 con 0x7f3e68161340 2023-11-28T13:09:22.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.782+0000 7f3e65d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e6815c9b0 0x7f3e680b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:22.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.783+0000 7f3e65d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e6815c9b0 0x7f3e680b7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39872/0 (socket says 172.21.15.156:39872) 2023-11-28T13:09:22.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.783+0000 7f3e65d9b700 1 -- 172.21.15.156:0/1477631221 learned_addr learned my addr 172.21.15.156:0/1477631221 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:22.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.783+0000 7f3e6659c700 1 --2- 172.21.15.156:0/1477631221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e680d2320 0x7f3e681e7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:22.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.783+0000 7f3e66d9d700 1 --2- 172.21.15.156:0/1477631221 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e68161340 0x7f3e680b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:22.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.783+0000 7f3e65d9b700 1 -- 172.21.15.156:0/1477631221 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e68161340 msgr2=0x7f3e680b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:22.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.783+0000 7f3e65d9b700 1 --2- 172.21.15.156:0/1477631221 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e68161340 0x7f3e680b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:22.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.783+0000 7f3e65d9b700 1 -- 172.21.15.156:0/1477631221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e680d2320 msgr2=0x7f3e681e7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:22.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.783+0000 7f3e65d9b700 1 --2- 172.21.15.156:0/1477631221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e680d2320 0x7f3e681e7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:22.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.783+0000 7f3e65d9b700 1 -- 172.21.15.156:0/1477631221 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c0094a0 con 0x7f3e6815c9b0 2023-11-28T13:09:22.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.783+0000 7f3e65d9b700 1 --2- 172.21.15.156:0/1477631221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e6815c9b0 0x7f3e680b7270 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0090b0 tx=0x7f3e5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:22.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.784+0000 7f3e56ffd700 1 -- 172.21.15.156:0/1477631221 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c0036a0 con 0x7f3e6815c9b0 2023-11-28T13:09:22.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.784+0000 7f3e56ffd700 1 -- 172.21.15.156:0/1477631221 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5c01f070 con 0x7f3e6815c9b0 2023-11-28T13:09:22.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.784+0000 7f3e56ffd700 1 -- 172.21.15.156:0/1477631221 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c003a60 con 0x7f3e6815c9b0 2023-11-28T13:09:22.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.784+0000 7f3e6cbff700 1 -- 172.21.15.156:0/1477631221 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e680b5130 con 0x7f3e6815c9b0 2023-11-28T13:09:22.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.784+0000 7f3e6cbff700 1 -- 172.21.15.156:0/1477631221 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e680d15e0 con 0x7f3e6815c9b0 2023-11-28T13:09:22.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.785+0000 7f3e6cbff700 1 -- 172.21.15.156:0/1477631221 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e680b52c0 con 0x7f3e6815c9b0 2023-11-28T13:09:22.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.785+0000 7f3e56ffd700 1 -- 172.21.15.156:0/1477631221 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3e5c00f050 con 0x7f3e6815c9b0 2023-11-28T13:09:22.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.786+0000 7f3e56ffd700 1 --2- 172.21.15.156:0/1477631221 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3e480682f0 0x7f3e4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:22.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.786+0000 7f3e56ffd700 1 -- 172.21.15.156:0/1477631221 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3e5c021070 con 0x7f3e6815c9b0 2023-11-28T13:09:22.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.787+0000 7f3e6659c700 1 --2- 172.21.15.156:0/1477631221 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3e480682f0 0x7f3e4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:22.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.787+0000 7f3e6659c700 1 --2- 172.21.15.156:0/1477631221 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3e480682f0 0x7f3e4806a7a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009be0 tx=0x7f3e58009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:22.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:22.791+0000 7f3e56ffd700 1 -- 172.21.15.156:0/1477631221 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3e5c053d00 con 0x7f3e6815c9b0 2023-11-28T13:09:23.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:23.024+0000 7f3e6cbff700 1 -- 172.21.15.156:0/1477631221 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e680b9150 con 0x7f3e6815c9b0 2023-11-28T13:09:23.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:23.025+0000 7f3e56ffd700 1 -- 172.21.15.156:0/1477631221 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f3e5c057320 con 0x7f3e6815c9b0 2023-11-28T13:09:23.027 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:23.027 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1128367125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1128367125},{"type":"v1","addr":"172.21.15.156:6835","nonce":1128367125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4942,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/4096182008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4096182008},{"type":"v1","addr":"172.21.15.186:6835","nonce":4096182008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4945,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/561812165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":561812165},{"type":"v1","addr":"172.21.15.186:6837","nonce":561812165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:09:08.167940+0000","modified":"2023-11-28T13:09:11.184420+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5013":{"gid":5013,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6834/3471743149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3471743149},{"type":"v1","addr":"172.21.15.156:6834","nonce":3471743149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:09:23.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:23.027+0000 7f3e6cbff700 1 -- 172.21.15.156:0/1477631221 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3e480682f0 msgr2=0x7f3e4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:23.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:23.027+0000 7f3e6cbff700 1 --2- 172.21.15.156:0/1477631221 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3e480682f0 0x7f3e4806a7a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009be0 tx=0x7f3e58009210 comp rx=0 tx=0).stop 2023-11-28T13:09:23.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:23.027+0000 7f3e6cbff700 1 -- 172.21.15.156:0/1477631221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e6815c9b0 msgr2=0x7f3e680b7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:23.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:23.027+0000 7f3e6cbff700 1 --2- 172.21.15.156:0/1477631221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e6815c9b0 0x7f3e680b7270 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0090b0 tx=0x7f3e5c004750 comp rx=0 tx=0).stop 2023-11-28T13:09:23.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:23.028+0000 7f3e6cbff700 1 -- 172.21.15.156:0/1477631221 shutdown_connections 2023-11-28T13:09:23.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:23.028+0000 7f3e6cbff700 1 --2- 172.21.15.156:0/1477631221 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3e480682f0 0x7f3e4806a7a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:23.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:23.028+0000 7f3e6cbff700 1 --2- 172.21.15.156:0/1477631221 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e68161340 0x7f3e680b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:23.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:23.028+0000 7f3e6cbff700 1 --2- 172.21.15.156:0/1477631221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e680d2320 0x7f3e681e7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:23.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:23.028+0000 7f3e6cbff700 1 --2- 172.21.15.156:0/1477631221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e6815c9b0 0x7f3e680b7270 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:23.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:23.028+0000 7f3e6cbff700 1 -- 172.21.15.156:0/1477631221 >> 172.21.15.156:0/1477631221 conn(0x7f3e68072600 msgr2=0x7f3e68160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:23.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:23.028+0000 7f3e6cbff700 1 -- 172.21.15.156:0/1477631221 shutdown_connections 2023-11-28T13:09:23.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:23.028+0000 7f3e6cbff700 1 -- 172.21.15.156:0/1477631221 wait complete. 2023-11-28T13:09:23.031 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2023-11-28T13:09:23.048 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parentdir mytag 2023-11-28T13:09:23.270 INFO:teuthology.orchestra.run.smithi156.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2023-11-28T13:09:23.270 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2023-11-28T13:09:32.996 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:09:33.002 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:09:33.272 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2023-11-28T13:09:33.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.357+0000 7f441a184080 1 Processor -- start 2023-11-28T13:09:33.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.357+0000 7f441a184080 1 -- start start 2023-11-28T13:09:33.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.357+0000 7f441a184080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c23565920 0x558c235f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.357+0000 7f441a184080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c235f5250 0x558c235f7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.357+0000 7f441a184080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c235f5620 0x558c235f7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.357+0000 7f441a184080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c234c5000 con 0x558c235f5250 2023-11-28T13:09:33.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.357+0000 7f441a184080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c23457fe0 con 0x558c235f5620 2023-11-28T13:09:33.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.357+0000 7f441a184080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c235f4890 con 0x558c23565920 2023-11-28T13:09:33.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.358+0000 7f440af9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c235f5250 0x558c235f7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.358+0000 7f440af9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c235f5250 0x558c235f7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44468/0 (socket says 172.21.15.156:44468) 2023-11-28T13:09:33.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.358+0000 7f440af9a700 1 -- 172.21.15.156:0/2197183573 learned_addr learned my addr 172.21.15.156:0/2197183573 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:33.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.358+0000 7f440b79b700 1 --2- 172.21.15.156:0/2197183573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c23565920 0x558c235f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.358+0000 7f440bf9c700 1 --2- 172.21.15.156:0/2197183573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c235f5620 0x558c235f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.358+0000 7f440af9a700 1 -- 172.21.15.156:0/2197183573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c23565920 msgr2=0x558c235f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:33.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.358+0000 7f440af9a700 1 --2- 172.21.15.156:0/2197183573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c23565920 0x558c235f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.358+0000 7f440af9a700 1 -- 172.21.15.156:0/2197183573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c235f5620 msgr2=0x558c235f7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:33.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.358+0000 7f440af9a700 1 --2- 172.21.15.156:0/2197183573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c235f5620 0x558c235f7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.358+0000 7f440af9a700 1 -- 172.21.15.156:0/2197183573 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4400005d20 con 0x558c235f5250 2023-11-28T13:09:33.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.358+0000 7f440af9a700 1 --2- 172.21.15.156:0/2197183573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c235f5250 0x558c235f7e30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f440000a440 tx=0x7f440000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd29a4757d58e3d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.358+0000 7f4409f98700 1 -- 172.21.15.156:0/2197183573 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400008960 con 0x558c235f5250 2023-11-28T13:09:33.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.359+0000 7f4409f98700 1 -- 172.21.15.156:0/2197183573 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4400008ae0 con 0x558c235f5250 2023-11-28T13:09:33.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.359+0000 7f4409f98700 1 -- 172.21.15.156:0/2197183573 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400008e90 con 0x558c235f5250 2023-11-28T13:09:33.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.359+0000 7f441a184080 1 -- 172.21.15.156:0/2197183573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c235f5250 msgr2=0x558c235f7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.359+0000 7f441a184080 1 --2- 172.21.15.156:0/2197183573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c235f5250 0x558c235f7e30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f440000a440 tx=0x7f440000d000 comp rx=0 tx=0).stop 2023-11-28T13:09:33.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.359+0000 7f441a184080 1 -- 172.21.15.156:0/2197183573 shutdown_connections 2023-11-28T13:09:33.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.359+0000 7f441a184080 1 --2- 172.21.15.156:0/2197183573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c235f5620 0x558c235f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.359+0000 7f441a184080 1 --2- 172.21.15.156:0/2197183573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c23565920 0x558c235f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.359+0000 7f441a184080 1 --2- 172.21.15.156:0/2197183573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c235f5250 0x558c235f7e30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.359+0000 7f441a184080 1 -- 172.21.15.156:0/2197183573 >> 172.21.15.156:0/2197183573 conn(0x558c235699d0 msgr2=0x558c235f86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:33.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.359+0000 7f441a184080 1 -- 172.21.15.156:0/2197183573 shutdown_connections 2023-11-28T13:09:33.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.359+0000 7f441a184080 1 -- 172.21.15.156:0/2197183573 wait complete. 2023-11-28T13:09:33.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.360+0000 7f441a184080 1 Processor -- start 2023-11-28T13:09:33.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.360+0000 7f441a184080 1 -- start start 2023-11-28T13:09:33.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.361+0000 7f441a184080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c23565920 0x558c235ec350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.361+0000 7f441a184080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c235f5250 0x558c235f1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.361+0000 7f441a184080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c235f5620 0x558c2368baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.361+0000 7f441a184080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c235f18b0 con 0x558c235f5620 2023-11-28T13:09:33.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.361+0000 7f441a184080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c2368b300 con 0x558c235f5250 2023-11-28T13:09:33.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.361+0000 7f441a184080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c23569da0 con 0x558c23565920 2023-11-28T13:09:33.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.361+0000 7f440bf9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c235f5620 0x558c2368baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.361+0000 7f440bf9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c235f5620 0x558c2368baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44476/0 (socket says 172.21.15.156:44476) 2023-11-28T13:09:33.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.361+0000 7f440bf9c700 1 -- 172.21.15.156:0/2859338711 learned_addr learned my addr 172.21.15.156:0/2859338711 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:33.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.362+0000 7f440b79b700 1 --2- 172.21.15.156:0/2859338711 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c23565920 0x558c235ec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.362+0000 7f440af9a700 1 --2- 172.21.15.156:0/2859338711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c235f5250 0x558c235f1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.362+0000 7f440bf9c700 1 -- 172.21.15.156:0/2859338711 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c23565920 msgr2=0x558c235ec350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.362+0000 7f440bf9c700 1 --2- 172.21.15.156:0/2859338711 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x558c23565920 0x558c235ec350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.362+0000 7f440bf9c700 1 -- 172.21.15.156:0/2859338711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c235f5250 msgr2=0x558c235f1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.362+0000 7f440bf9c700 1 --2- 172.21.15.156:0/2859338711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c235f5250 0x558c235f1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.362+0000 7f440bf9c700 1 -- 172.21.15.156:0/2859338711 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c23470120 con 0x558c235f5620 2023-11-28T13:09:33.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.362+0000 7f440af9a700 1 --2- 172.21.15.156:0/2859338711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558c235f5250 0x558c235f1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:33.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.362+0000 7f440bf9c700 1 --2- 172.21.15.156:0/2859338711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c235f5620 0x558c2368baf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f44040041f0 tx=0x7f440400d4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.362+0000 7f4408f96700 1 -- 172.21.15.156:0/2859338711 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440400ddd0 con 0x558c235f5620 2023-11-28T13:09:33.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.362+0000 7f4408f96700 1 -- 172.21.15.156:0/2859338711 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4404013480 con 0x558c235f5620 2023-11-28T13:09:33.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.362+0000 7f4408f96700 1 -- 172.21.15.156:0/2859338711 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4404013830 con 0x558c235f5620 2023-11-28T13:09:33.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.362+0000 7f441a184080 1 -- 172.21.15.156:0/2859338711 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558c23463830 con 0x558c235f5620 2023-11-28T13:09:33.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.362+0000 7f441a184080 1 -- 172.21.15.156:0/2859338711 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c23470360 con 0x558c235f5620 2023-11-28T13:09:33.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.364+0000 7f4408f96700 1 -- 172.21.15.156:0/2859338711 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4404013990 con 0x558c235f5620 2023-11-28T13:09:33.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.365+0000 7f4408f96700 1 --2- 172.21.15.156:0/2859338711 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f43f8066c10 0x7f43f80690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.365+0000 7f4408f96700 1 -- 172.21.15.156:0/2859338711 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4404084bb0 con 0x558c235f5620 2023-11-28T13:09:33.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.366+0000 7f440b79b700 1 --2- 172.21.15.156:0/2859338711 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f43f8066c10 0x7f43f80690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.366+0000 7f441a184080 1 --2- 172.21.15.156:0/2859338711 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x558c2368c030 0x558c2354a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.366+0000 7f441a184080 1 -- 172.21.15.156:0/2859338711 --> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x558c235efa60 con 0x558c2368c030 2023-11-28T13:09:33.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.366+0000 7f440af9a700 1 --2- 172.21.15.156:0/2859338711 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x558c2368c030 0x558c2354a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.366+0000 7f440b79b700 1 --2- 172.21.15.156:0/2859338711 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f43f8066c10 0x7f43f80690c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x558c235effb0 tx=0x7f43fc0015e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.367+0000 7f440af9a700 1 --2- 172.21.15.156:0/2859338711 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x558c2368c030 0x558c2354a550 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.367+0000 7f440af9a700 1 -- 172.21.15.156:0/2859338711 <== osd.7 v2:172.21.15.186:6808/132936 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f440001d430 con 0x558c2368c030 2023-11-28T13:09:33.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.367+0000 7f441a184080 1 -- 172.21.15.156:0/2859338711 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x558c2368c030 msgr2=0x558c2354a550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.367+0000 7f441a184080 1 --2- 172.21.15.156:0/2859338711 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x558c2368c030 0x558c2354a550 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.367+0000 7f441a184080 1 -- 172.21.15.156:0/2859338711 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f43f8066c10 msgr2=0x7f43f80690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.367+0000 7f441a184080 1 --2- 172.21.15.156:0/2859338711 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f43f8066c10 0x7f43f80690c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x558c235effb0 tx=0x7f43fc0015e0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.367+0000 7f441a184080 1 -- 172.21.15.156:0/2859338711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c235f5620 msgr2=0x558c2368baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.373 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.367+0000 7f441a184080 1 --2- 172.21.15.156:0/2859338711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c235f5620 0x558c2368baf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f44040041f0 tx=0x7f440400d4f0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.367+0000 7f440bf9c700 1 -- 172.21.15.156:0/2859338711 reap_dead start 2023-11-28T13:09:33.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.367+0000 7f441a184080 1 -- 172.21.15.156:0/2859338711 shutdown_connections 2023-11-28T13:09:33.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.367+0000 7f441a184080 1 -- 172.21.15.156:0/2859338711 >> 172.21.15.156:0/2859338711 conn(0x558c235699d0 msgr2=0x558c235f86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:33.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.367+0000 7f441a184080 1 -- 172.21.15.156:0/2859338711 shutdown_connections 2023-11-28T13:09:33.374 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.367+0000 7f441a184080 1 -- 172.21.15.156:0/2859338711 wait complete. 2023-11-28T13:09:33.375 INFO:teuthology.orchestra.run.smithi156.stdout:mytag 2023-11-28T13:09:33.376 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2023-11-28T13:09:33.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.461+0000 7f6a280a8080 1 Processor -- start 2023-11-28T13:09:33.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.461+0000 7f6a280a8080 1 -- start start 2023-11-28T13:09:33.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.462+0000 7f6a280a8080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a040027d0 0x55776ca881d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.462+0000 7f6a280a8080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55776ca838c0 0x55776ca77e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.462+0000 7f6a280a8080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55776ca84ce0 0x55776ca78340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.462+0000 7f6a280a8080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55776c954000 con 0x55776ca84ce0 2023-11-28T13:09:33.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.462+0000 7f6a280a8080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55776c8e6fe0 con 0x55776ca838c0 2023-11-28T13:09:33.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.462+0000 7f6a280a8080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55776c9b0ca0 con 0x7f6a040027d0 2023-11-28T13:09:33.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.462+0000 7f6a19f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55776ca84ce0 0x55776ca78340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.462+0000 7f6a19f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55776ca84ce0 0x55776ca78340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44496/0 (socket says 172.21.15.156:44496) 2023-11-28T13:09:33.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.462+0000 7f6a19f9c700 1 -- 172.21.15.156:0/2899464063 learned_addr learned my addr 172.21.15.156:0/2899464063 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:33.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.462+0000 7f6a18f9a700 1 --2- 172.21.15.156:0/2899464063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55776ca838c0 0x55776ca77e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.462+0000 7f6a1979b700 1 --2- 172.21.15.156:0/2899464063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a040027d0 0x55776ca881d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.463+0000 7f6a19f9c700 1 -- 172.21.15.156:0/2899464063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a040027d0 msgr2=0x55776ca881d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:33.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.463+0000 7f6a19f9c700 1 --2- 172.21.15.156:0/2899464063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a040027d0 0x55776ca881d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.463+0000 7f6a19f9c700 1 -- 172.21.15.156:0/2899464063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55776ca838c0 msgr2=0x55776ca77e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:33.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.463+0000 7f6a19f9c700 1 --2- 172.21.15.156:0/2899464063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55776ca838c0 0x55776ca77e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.463+0000 7f6a19f9c700 1 -- 172.21.15.156:0/2899464063 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55776c8ff360 con 0x55776ca84ce0 2023-11-28T13:09:33.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.463+0000 7f6a19f9c700 1 --2- 172.21.15.156:0/2899464063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55776ca84ce0 0x55776ca78340 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55776c9f4da0 tx=0x7f6a14005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd949b91d894848 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.463+0000 7f6a0b7fe700 1 -- 172.21.15.156:0/2899464063 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1400ae90 con 0x55776ca84ce0 2023-11-28T13:09:33.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.463+0000 7f6a0b7fe700 1 -- 172.21.15.156:0/2899464063 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a14013650 con 0x55776ca84ce0 2023-11-28T13:09:33.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.463+0000 7f6a0b7fe700 1 -- 172.21.15.156:0/2899464063 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14013a00 con 0x55776ca84ce0 2023-11-28T13:09:33.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.464+0000 7f6a280a8080 1 -- 172.21.15.156:0/2899464063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55776ca84ce0 msgr2=0x55776ca78340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.464+0000 7f6a280a8080 1 --2- 172.21.15.156:0/2899464063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55776ca84ce0 0x55776ca78340 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55776c9f4da0 tx=0x7f6a14005d80 comp rx=0 tx=0).stop 2023-11-28T13:09:33.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.464+0000 7f6a280a8080 1 -- 172.21.15.156:0/2899464063 shutdown_connections 2023-11-28T13:09:33.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.464+0000 7f6a280a8080 1 --2- 172.21.15.156:0/2899464063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55776ca84ce0 0x55776ca78340 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.464+0000 7f6a280a8080 1 --2- 172.21.15.156:0/2899464063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a040027d0 0x55776ca881d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.464+0000 7f6a280a8080 1 --2- 172.21.15.156:0/2899464063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55776ca838c0 0x55776ca77e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.464+0000 7f6a280a8080 1 -- 172.21.15.156:0/2899464063 >> 172.21.15.156:0/2899464063 conn(0x55776c9f89d0 msgr2=0x55776c9f3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:33.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.464+0000 7f6a280a8080 1 -- 172.21.15.156:0/2899464063 shutdown_connections 2023-11-28T13:09:33.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.464+0000 7f6a280a8080 1 -- 172.21.15.156:0/2899464063 wait complete. 2023-11-28T13:09:33.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.465+0000 7f6a280a8080 1 Processor -- start 2023-11-28T13:09:33.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.465+0000 7f6a280a8080 1 -- start start 2023-11-28T13:09:33.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.466+0000 7f6a280a8080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55776ca838c0 0x55776cb1a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.466+0000 7f6a280a8080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55776ca84ce0 0x55776ca7bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.466+0000 7f6a280a8080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a040027d0 0x55776ca77bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.466+0000 7f6a280a8080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55776cb1ab60 con 0x55776ca838c0 2023-11-28T13:09:33.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.466+0000 7f6a280a8080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55776c9d92b0 con 0x55776ca84ce0 2023-11-28T13:09:33.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.466+0000 7f6a280a8080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55776ca7b610 con 0x7f6a040027d0 2023-11-28T13:09:33.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.466+0000 7f6a1979b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55776ca838c0 0x55776cb1a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.466+0000 7f6a1979b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55776ca838c0 0x55776cb1a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44504/0 (socket says 172.21.15.156:44504) 2023-11-28T13:09:33.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.466+0000 7f6a1979b700 1 -- 172.21.15.156:0/3207511291 learned_addr learned my addr 172.21.15.156:0/3207511291 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:33.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.466+0000 7f6a1979b700 1 -- 172.21.15.156:0/3207511291 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a040027d0 msgr2=0x55776ca77bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.466+0000 7f6a1979b700 1 --2- 172.21.15.156:0/3207511291 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a040027d0 0x55776ca77bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.466+0000 7f6a1979b700 1 -- 172.21.15.156:0/3207511291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55776ca84ce0 msgr2=0x55776ca7bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.466+0000 7f6a1979b700 1 --2- 172.21.15.156:0/3207511291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55776ca84ce0 0x55776ca7bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.466+0000 7f6a1979b700 1 -- 172.21.15.156:0/3207511291 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1400a380 con 0x55776ca838c0 2023-11-28T13:09:33.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.467+0000 7f6a1979b700 1 --2- 172.21.15.156:0/3207511291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55776ca838c0 0x55776cb1a620 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009fa0 tx=0x7f6a0c00f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.467+0000 7f6a0a7fc700 1 -- 172.21.15.156:0/3207511291 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c014070 con 0x55776ca838c0 2023-11-28T13:09:33.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.467+0000 7f6a0a7fc700 1 -- 172.21.15.156:0/3207511291 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c005040 con 0x55776ca838c0 2023-11-28T13:09:33.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.467+0000 7f6a280a8080 1 -- 172.21.15.156:0/3207511291 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55776c9f4440 con 0x55776ca838c0 2023-11-28T13:09:33.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.467+0000 7f6a0a7fc700 1 -- 172.21.15.156:0/3207511291 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00e860 con 0x55776ca838c0 2023-11-28T13:09:33.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.467+0000 7f6a280a8080 1 -- 172.21.15.156:0/3207511291 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55776c8f2830 con 0x55776ca838c0 2023-11-28T13:09:33.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.469+0000 7f6a0a7fc700 1 -- 172.21.15.156:0/3207511291 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6a0c00e9c0 con 0x55776ca838c0 2023-11-28T13:09:33.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.469+0000 7f6a0a7fc700 1 --2- 172.21.15.156:0/3207511291 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f69fc066c10 0x7f69fc0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.470+0000 7f6a0a7fc700 1 -- 172.21.15.156:0/3207511291 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6a0c085640 con 0x55776ca838c0 2023-11-28T13:09:33.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.470+0000 7f6a280a8080 1 --2- 172.21.15.156:0/3207511291 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x55776ca790c0 0x55776ca79530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.470+0000 7f6a280a8080 1 -- 172.21.15.156:0/3207511291 --> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55776ca79a70 con 0x55776ca790c0 2023-11-28T13:09:33.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.470+0000 7f6a18f9a700 1 --2- 172.21.15.156:0/3207511291 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f69fc066c10 0x7f69fc0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.470+0000 7f6a19f9c700 1 --2- 172.21.15.156:0/3207511291 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x55776ca790c0 0x55776ca79530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.471+0000 7f6a19f9c700 1 --2- 172.21.15.156:0/3207511291 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x55776ca790c0 0x55776ca79530 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.471+0000 7f6a18f9a700 1 --2- 172.21.15.156:0/3207511291 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f69fc066c10 0x7f69fc0690c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000b420 tx=0x7f6a1001d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.471+0000 7f6a19f9c700 1 -- 172.21.15.156:0/3207511291 <== osd.5 v2:172.21.15.186:6800/132935 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f6a1401b020 con 0x55776ca790c0 2023-11-28T13:09:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.471+0000 7f6a280a8080 1 -- 172.21.15.156:0/3207511291 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x55776ca790c0 msgr2=0x55776ca79530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.471+0000 7f6a280a8080 1 --2- 172.21.15.156:0/3207511291 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x55776ca790c0 0x55776ca79530 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.471+0000 7f6a280a8080 1 -- 172.21.15.156:0/3207511291 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f69fc066c10 msgr2=0x7f69fc0690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.471+0000 7f6a280a8080 1 --2- 172.21.15.156:0/3207511291 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f69fc066c10 0x7f69fc0690c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000b420 tx=0x7f6a1001d040 comp rx=0 tx=0).stop 2023-11-28T13:09:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.471+0000 7f6a280a8080 1 -- 172.21.15.156:0/3207511291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55776ca838c0 msgr2=0x55776cb1a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.471+0000 7f6a280a8080 1 --2- 172.21.15.156:0/3207511291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55776ca838c0 0x55776cb1a620 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009fa0 tx=0x7f6a0c00f7e0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.471+0000 7f6a19f9c700 1 -- 172.21.15.156:0/3207511291 reap_dead start 2023-11-28T13:09:33.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.472+0000 7f6a280a8080 1 -- 172.21.15.156:0/3207511291 shutdown_connections 2023-11-28T13:09:33.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.472+0000 7f6a280a8080 1 -- 172.21.15.156:0/3207511291 >> 172.21.15.156:0/3207511291 conn(0x55776c9f89d0 msgr2=0x55776c9f76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:33.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.472+0000 7f6a280a8080 1 -- 172.21.15.156:0/3207511291 shutdown_connections 2023-11-28T13:09:33.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.472+0000 7f6a280a8080 1 -- 172.21.15.156:0/3207511291 wait complete. 2023-11-28T13:09:33.477 INFO:teuthology.orchestra.run.smithi156.stdout:mytag 2023-11-28T13:09:33.477 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2023-11-28T13:09:33.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.560+0000 7faf63b01080 1 Processor -- start 2023-11-28T13:09:33.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.560+0000 7faf63b01080 1 -- start start 2023-11-28T13:09:33.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.561+0000 7faf63b01080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5585cf6bf0f0 0x5585cf6a4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.561+0000 7faf63b01080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585cf6bf4c0 0x5585cf6c3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.561+0000 7faf63b01080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585cf6bfb90 0x5585cf6c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.561+0000 7faf63b01080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585cf5b3fe0 con 0x5585cf6bfb90 2023-11-28T13:09:33.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.561+0000 7faf63b01080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585cf6aa160 con 0x5585cf6bf4c0 2023-11-28T13:09:33.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.561+0000 7faf63b01080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585cf6b9ff0 con 0x5585cf6bf0f0 2023-11-28T13:09:33.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.561+0000 7faf5599c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585cf6bfb90 0x5585cf6c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.561+0000 7faf5599c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585cf6bfb90 0x5585cf6c3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44522/0 (socket says 172.21.15.156:44522) 2023-11-28T13:09:33.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.561+0000 7faf5599c700 1 -- 172.21.15.156:0/921884783 learned_addr learned my addr 172.21.15.156:0/921884783 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:33.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.561+0000 7faf5499a700 1 --2- 172.21.15.156:0/921884783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585cf6bf4c0 0x5585cf6c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.561+0000 7faf5519b700 1 --2- 172.21.15.156:0/921884783 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5585cf6bf0f0 0x5585cf6a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.562+0000 7faf5599c700 1 -- 172.21.15.156:0/921884783 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5585cf6bf0f0 msgr2=0x5585cf6a4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:33.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.562+0000 7faf5599c700 1 --2- 172.21.15.156:0/921884783 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5585cf6bf0f0 0x5585cf6a4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.562+0000 7faf5599c700 1 -- 172.21.15.156:0/921884783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585cf6bf4c0 msgr2=0x5585cf6c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:33.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.562+0000 7faf5599c700 1 --2- 172.21.15.156:0/921884783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585cf6bf4c0 0x5585cf6c3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.562+0000 7faf5599c700 1 -- 172.21.15.156:0/921884783 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c005de0 con 0x5585cf6bfb90 2023-11-28T13:09:33.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.562+0000 7faf5519b700 1 --2- 172.21.15.156:0/921884783 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5585cf6bf0f0 0x5585cf6a4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:33.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.562+0000 7faf5499a700 1 --2- 172.21.15.156:0/921884783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585cf6bf4c0 0x5585cf6c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:33.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.562+0000 7faf5599c700 1 --2- 172.21.15.156:0/921884783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585cf6bfb90 0x5585cf6c3ea0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faf5000a4e0 tx=0x7faf5000a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8702afd1e6c19ebc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.562+0000 7faf477fe700 1 -- 172.21.15.156:0/921884783 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5000e710 con 0x5585cf6bfb90 2023-11-28T13:09:33.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.562+0000 7faf477fe700 1 -- 172.21.15.156:0/921884783 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5000e890 con 0x5585cf6bfb90 2023-11-28T13:09:33.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.562+0000 7faf477fe700 1 -- 172.21.15.156:0/921884783 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5000ec40 con 0x5585cf6bfb90 2023-11-28T13:09:33.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.563+0000 7faf63b01080 1 -- 172.21.15.156:0/921884783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585cf6bfb90 msgr2=0x5585cf6c3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.563+0000 7faf63b01080 1 --2- 172.21.15.156:0/921884783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585cf6bfb90 0x5585cf6c3ea0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faf5000a4e0 tx=0x7faf5000a7f0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.563+0000 7faf63b01080 1 -- 172.21.15.156:0/921884783 shutdown_connections 2023-11-28T13:09:33.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.563+0000 7faf63b01080 1 --2- 172.21.15.156:0/921884783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585cf6bfb90 0x5585cf6c3ea0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.563+0000 7faf63b01080 1 --2- 172.21.15.156:0/921884783 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5585cf6bf0f0 0x5585cf6a4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.563+0000 7faf63b01080 1 --2- 172.21.15.156:0/921884783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585cf6bf4c0 0x5585cf6c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.563+0000 7faf63b01080 1 -- 172.21.15.156:0/921884783 >> 172.21.15.156:0/921884783 conn(0x5585cf6c59d0 msgr2=0x5585cf752790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:33.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.563+0000 7faf63b01080 1 -- 172.21.15.156:0/921884783 shutdown_connections 2023-11-28T13:09:33.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.563+0000 7faf63b01080 1 -- 172.21.15.156:0/921884783 wait complete. 2023-11-28T13:09:33.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.564+0000 7faf63b01080 1 Processor -- start 2023-11-28T13:09:33.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.564+0000 7faf63b01080 1 -- start start 2023-11-28T13:09:33.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.565+0000 7faf63b01080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585cf6bf0f0 0x5585cf74d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.569 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.565+0000 7faf63b01080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585cf6bf4c0 0x5585cf6bb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.565+0000 7faf63b01080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5585cf6bfb90 0x5585cf6a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.565+0000 7faf63b01080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585cf6a66f0 con 0x5585cf6bf4c0 2023-11-28T13:09:33.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.565+0000 7faf63b01080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585cf6a3ca0 con 0x5585cf6bf0f0 2023-11-28T13:09:33.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.565+0000 7faf63b01080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585cf6a3810 con 0x5585cf6bfb90 2023-11-28T13:09:33.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.565+0000 7faf5499a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585cf6bf4c0 0x5585cf6bb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.565+0000 7faf5499a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585cf6bf4c0 0x5585cf6bb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44530/0 (socket says 172.21.15.156:44530) 2023-11-28T13:09:33.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.565+0000 7faf5499a700 1 -- 172.21.15.156:0/550447242 learned_addr learned my addr 172.21.15.156:0/550447242 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:33.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.565+0000 7faf5519b700 1 --2- 172.21.15.156:0/550447242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585cf6bf0f0 0x5585cf74d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.565+0000 7faf5599c700 1 --2- 172.21.15.156:0/550447242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5585cf6bfb90 0x5585cf6a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.565+0000 7faf5499a700 1 -- 172.21.15.156:0/550447242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5585cf6bfb90 msgr2=0x5585cf6a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.565+0000 7faf5499a700 1 --2- 172.21.15.156:0/550447242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5585cf6bfb90 0x5585cf6a59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.565+0000 7faf5499a700 1 -- 172.21.15.156:0/550447242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585cf6bf0f0 msgr2=0x5585cf74d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.565+0000 7faf5499a700 1 --2- 172.21.15.156:0/550447242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5585cf6bf0f0 0x5585cf74d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.565+0000 7faf5499a700 1 -- 172.21.15.156:0/550447242 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5000a0e0 con 0x5585cf6bf4c0 2023-11-28T13:09:33.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.565+0000 7faf5599c700 1 --2- 172.21.15.156:0/550447242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5585cf6bfb90 0x5585cf6a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:33.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.566+0000 7faf5499a700 1 --2- 172.21.15.156:0/550447242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585cf6bf4c0 0x5585cf6bb900 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00c010 tx=0x7faf4c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.566+0000 7faf467fc700 1 -- 172.21.15.156:0/550447242 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c0055c0 con 0x5585cf6bf4c0 2023-11-28T13:09:33.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.566+0000 7faf467fc700 1 -- 172.21.15.156:0/550447242 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4c01d880 con 0x5585cf6bf4c0 2023-11-28T13:09:33.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.566+0000 7faf467fc700 1 -- 172.21.15.156:0/550447242 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c01dc30 con 0x5585cf6bf4c0 2023-11-28T13:09:33.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.566+0000 7faf63b01080 1 -- 172.21.15.156:0/550447242 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5585cf5cc120 con 0x5585cf6bf4c0 2023-11-28T13:09:33.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.566+0000 7faf63b01080 1 -- 172.21.15.156:0/550447242 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585cf5bf830 con 0x5585cf6bf4c0 2023-11-28T13:09:33.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.568+0000 7faf467fc700 1 -- 172.21.15.156:0/550447242 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faf4c012450 con 0x5585cf6bf4c0 2023-11-28T13:09:33.575 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.568+0000 7faf467fc700 1 --2- 172.21.15.156:0/550447242 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faf38066c10 0x7faf380690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.569+0000 7faf467fc700 1 -- 172.21.15.156:0/550447242 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faf4c0867a0 con 0x5585cf6bf4c0 2023-11-28T13:09:33.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.569+0000 7faf63b01080 1 --2- 172.21.15.156:0/550447242 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x5585cf6bbe40 0x5585cf7ebaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.569+0000 7faf63b01080 1 -- 172.21.15.156:0/550447242 --> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5585cf67db70 con 0x5585cf6bbe40 2023-11-28T13:09:33.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.569+0000 7faf5519b700 1 --2- 172.21.15.156:0/550447242 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faf38066c10 0x7faf380690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.570+0000 7faf5599c700 1 --2- 172.21.15.156:0/550447242 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x5585cf6bbe40 0x5585cf7ebaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.570+0000 7faf5519b700 1 --2- 172.21.15.156:0/550447242 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faf38066c10 0x7faf380690c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x5585cf753940 tx=0x7faf48005b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.570+0000 7faf5599c700 1 --2- 172.21.15.156:0/550447242 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x5585cf6bbe40 0x5585cf7ebaa0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.570+0000 7faf5599c700 1 -- 172.21.15.156:0/550447242 <== osd.5 v2:172.21.15.186:6800/132935 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7faf5000ea00 con 0x5585cf6bbe40 2023-11-28T13:09:33.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.571+0000 7faf63b01080 1 -- 172.21.15.156:0/550447242 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x5585cf6bbe40 msgr2=0x5585cf7ebaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.571+0000 7faf63b01080 1 --2- 172.21.15.156:0/550447242 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x5585cf6bbe40 0x5585cf7ebaa0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.576 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.571+0000 7faf63b01080 1 -- 172.21.15.156:0/550447242 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faf38066c10 msgr2=0x7faf380690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.571+0000 7faf63b01080 1 --2- 172.21.15.156:0/550447242 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faf38066c10 0x7faf380690c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x5585cf753940 tx=0x7faf48005b60 comp rx=0 tx=0).stop 2023-11-28T13:09:33.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.571+0000 7faf63b01080 1 -- 172.21.15.156:0/550447242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585cf6bf4c0 msgr2=0x5585cf6bb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.571+0000 7faf63b01080 1 --2- 172.21.15.156:0/550447242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585cf6bf4c0 0x5585cf6bb900 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00c010 tx=0x7faf4c002fb0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.571+0000 7faf5599c700 1 -- 172.21.15.156:0/550447242 reap_dead start 2023-11-28T13:09:33.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.571+0000 7faf63b01080 1 -- 172.21.15.156:0/550447242 shutdown_connections 2023-11-28T13:09:33.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.571+0000 7faf63b01080 1 -- 172.21.15.156:0/550447242 >> 172.21.15.156:0/550447242 conn(0x5585cf6c59d0 msgr2=0x5585cf6c3960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:33.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.571+0000 7faf63b01080 1 -- 172.21.15.156:0/550447242 shutdown_connections 2023-11-28T13:09:33.577 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.571+0000 7faf63b01080 1 -- 172.21.15.156:0/550447242 wait complete. 2023-11-28T13:09:33.578 INFO:teuthology.orchestra.run.smithi156.stdout:mytag 2023-11-28T13:09:33.578 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2023-11-28T13:09:33.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.663+0000 7fde40733080 1 Processor -- start 2023-11-28T13:09:33.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.663+0000 7fde40733080 1 -- start start 2023-11-28T13:09:33.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.664+0000 7fde40733080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c306c1f920 0x55c306cadf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.664+0000 7fde40733080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c306caf250 0x55c306ca3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.664+0000 7fde40733080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c306caf620 0x55c306ca3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.664+0000 7fde40733080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c306b11fe0 con 0x55c306caf620 2023-11-28T13:09:33.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.664+0000 7fde40733080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c306c085a0 con 0x55c306caf250 2023-11-28T13:09:33.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.664+0000 7fde40733080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c306c08c60 con 0x55c306c1f920 2023-11-28T13:09:33.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.664+0000 7fde3259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c306caf620 0x55c306ca3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.664+0000 7fde3259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c306caf620 0x55c306ca3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44546/0 (socket says 172.21.15.156:44546) 2023-11-28T13:09:33.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.664+0000 7fde3259c700 1 -- 172.21.15.156:0/2913835035 learned_addr learned my addr 172.21.15.156:0/2913835035 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:33.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.665+0000 7fde3159a700 1 --2- 172.21.15.156:0/2913835035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c306caf250 0x55c306ca3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.665+0000 7fde31d9b700 1 --2- 172.21.15.156:0/2913835035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c306c1f920 0x55c306cadf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.665+0000 7fde3259c700 1 -- 172.21.15.156:0/2913835035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c306c1f920 msgr2=0x55c306cadf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:33.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.665+0000 7fde3259c700 1 --2- 172.21.15.156:0/2913835035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c306c1f920 0x55c306cadf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.665+0000 7fde3259c700 1 -- 172.21.15.156:0/2913835035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c306caf250 msgr2=0x55c306ca3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:33.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.665+0000 7fde3259c700 1 --2- 172.21.15.156:0/2913835035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c306caf250 0x55c306ca3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.665+0000 7fde3259c700 1 -- 172.21.15.156:0/2913835035 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde28005de0 con 0x55c306caf620 2023-11-28T13:09:33.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.665+0000 7fde3259c700 1 --2- 172.21.15.156:0/2913835035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c306caf620 0x55c306ca3660 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fde2c013040 tx=0x7fde2c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbf9d387ad9271d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.665+0000 7fde23fff700 1 -- 172.21.15.156:0/2913835035 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c00f790 con 0x55c306caf620 2023-11-28T13:09:33.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.665+0000 7fde23fff700 1 -- 172.21.15.156:0/2913835035 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c00fa60 con 0x55c306caf620 2023-11-28T13:09:33.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.665+0000 7fde23fff700 1 -- 172.21.15.156:0/2913835035 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c00fe10 con 0x55c306caf620 2023-11-28T13:09:33.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.666+0000 7fde40733080 1 -- 172.21.15.156:0/2913835035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c306caf620 msgr2=0x55c306ca3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.666+0000 7fde40733080 1 --2- 172.21.15.156:0/2913835035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c306caf620 0x55c306ca3660 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fde2c013040 tx=0x7fde2c00f000 comp rx=0 tx=0).stop 2023-11-28T13:09:33.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.666+0000 7fde40733080 1 -- 172.21.15.156:0/2913835035 shutdown_connections 2023-11-28T13:09:33.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.666+0000 7fde40733080 1 --2- 172.21.15.156:0/2913835035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c306caf620 0x55c306ca3660 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.666+0000 7fde40733080 1 --2- 172.21.15.156:0/2913835035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c306c1f920 0x55c306cadf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.666+0000 7fde40733080 1 --2- 172.21.15.156:0/2913835035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c306caf250 0x55c306ca3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.666+0000 7fde40733080 1 -- 172.21.15.156:0/2913835035 >> 172.21.15.156:0/2913835035 conn(0x55c306c239d0 msgr2=0x55c306cb2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:33.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.666+0000 7fde40733080 1 -- 172.21.15.156:0/2913835035 shutdown_connections 2023-11-28T13:09:33.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.666+0000 7fde40733080 1 -- 172.21.15.156:0/2913835035 wait complete. 2023-11-28T13:09:33.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.667+0000 7fde40733080 1 Processor -- start 2023-11-28T13:09:33.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.667+0000 7fde40733080 1 -- start start 2023-11-28T13:09:33.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.668+0000 7fde40733080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c306c1f920 0x55c306c9f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.668+0000 7fde40733080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c306caf250 0x55c306ca3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.668+0000 7fde40733080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c306caf620 0x55c306cadf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.668+0000 7fde40733080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c306c1d240 con 0x55c306caf250 2023-11-28T13:09:33.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.668+0000 7fde40733080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c306c1d4b0 con 0x55c306caf620 2023-11-28T13:09:33.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.668+0000 7fde40733080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c306c1d790 con 0x55c306c1f920 2023-11-28T13:09:33.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.668+0000 7fde3159a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c306caf250 0x55c306ca3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.668+0000 7fde3159a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c306caf250 0x55c306ca3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44562/0 (socket says 172.21.15.156:44562) 2023-11-28T13:09:33.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.668+0000 7fde3159a700 1 -- 172.21.15.156:0/2528737060 learned_addr learned my addr 172.21.15.156:0/2528737060 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:33.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.668+0000 7fde31d9b700 1 --2- 172.21.15.156:0/2528737060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c306c1f920 0x55c306c9f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.668+0000 7fde3259c700 1 --2- 172.21.15.156:0/2528737060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c306caf620 0x55c306cadf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.668+0000 7fde3159a700 1 -- 172.21.15.156:0/2528737060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c306c1f920 msgr2=0x55c306c9f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.668+0000 7fde3159a700 1 --2- 172.21.15.156:0/2528737060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55c306c1f920 0x55c306c9f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.669+0000 7fde3159a700 1 -- 172.21.15.156:0/2528737060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c306caf620 msgr2=0x55c306cadf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.669+0000 7fde3159a700 1 --2- 172.21.15.156:0/2528737060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c306caf620 0x55c306cadf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.669+0000 7fde3159a700 1 -- 172.21.15.156:0/2528737060 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c306b444f0 con 0x55c306caf250 2023-11-28T13:09:33.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.669+0000 7fde3159a700 1 --2- 172.21.15.156:0/2528737060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c306caf250 0x55c306ca3410 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fde28009fa0 tx=0x7fde280086e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.669+0000 7fde22ffd700 1 -- 172.21.15.156:0/2528737060 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2800f050 con 0x55c306caf250 2023-11-28T13:09:33.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.669+0000 7fde22ffd700 1 -- 172.21.15.156:0/2528737060 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde28010040 con 0x55c306caf250 2023-11-28T13:09:33.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.669+0000 7fde40733080 1 -- 172.21.15.156:0/2528737060 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c306b1d830 con 0x55c306caf250 2023-11-28T13:09:33.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.669+0000 7fde22ffd700 1 -- 172.21.15.156:0/2528737060 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28014820 con 0x55c306caf250 2023-11-28T13:09:33.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.669+0000 7fde40733080 1 -- 172.21.15.156:0/2528737060 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde2c00a100 con 0x55c306caf250 2023-11-28T13:09:33.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.671+0000 7fde22ffd700 1 -- 172.21.15.156:0/2528737060 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fde28014980 con 0x55c306caf250 2023-11-28T13:09:33.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.672+0000 7fde22ffd700 1 --2- 172.21.15.156:0/2528737060 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fde14066c10 0x7fde140690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.672+0000 7fde22ffd700 1 -- 172.21.15.156:0/2528737060 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fde28084a90 con 0x55c306caf250 2023-11-28T13:09:33.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.672+0000 7fde40733080 1 --2- 172.21.15.156:0/2528737060 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x55c306d450e0 0x55c306d454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.672+0000 7fde40733080 1 -- 172.21.15.156:0/2528737060 --> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c306d45a20 con 0x55c306d450e0 2023-11-28T13:09:33.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.672+0000 7fde31d9b700 1 --2- 172.21.15.156:0/2528737060 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fde14066c10 0x7fde140690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.673+0000 7fde3259c700 1 --2- 172.21.15.156:0/2528737060 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x55c306d450e0 0x55c306d454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.673+0000 7fde31d9b700 1 --2- 172.21.15.156:0/2528737060 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fde14066c10 0x7fde140690c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fde24000c00 tx=0x7fde240017f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.673+0000 7fde3259c700 1 --2- 172.21.15.156:0/2528737060 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x55c306d450e0 0x55c306d454e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.674+0000 7fde3259c700 1 -- 172.21.15.156:0/2528737060 <== osd.5 v2:172.21.15.186:6800/132935 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fde2c02b020 con 0x55c306d450e0 2023-11-28T13:09:33.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.674+0000 7fde40733080 1 -- 172.21.15.156:0/2528737060 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x55c306d450e0 msgr2=0x55c306d454e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.674+0000 7fde40733080 1 --2- 172.21.15.156:0/2528737060 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x55c306d450e0 0x55c306d454e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.674+0000 7fde40733080 1 -- 172.21.15.156:0/2528737060 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fde14066c10 msgr2=0x7fde140690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.674+0000 7fde40733080 1 --2- 172.21.15.156:0/2528737060 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fde14066c10 0x7fde140690c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fde24000c00 tx=0x7fde240017f0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.674+0000 7fde40733080 1 -- 172.21.15.156:0/2528737060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c306caf250 msgr2=0x55c306ca3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.674+0000 7fde40733080 1 --2- 172.21.15.156:0/2528737060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c306caf250 0x55c306ca3410 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fde28009fa0 tx=0x7fde280086e0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.674+0000 7fde3259c700 1 -- 172.21.15.156:0/2528737060 reap_dead start 2023-11-28T13:09:33.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.674+0000 7fde40733080 1 -- 172.21.15.156:0/2528737060 shutdown_connections 2023-11-28T13:09:33.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.674+0000 7fde40733080 1 -- 172.21.15.156:0/2528737060 >> 172.21.15.156:0/2528737060 conn(0x55c306c239d0 msgr2=0x55c306c1aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:33.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.674+0000 7fde40733080 1 -- 172.21.15.156:0/2528737060 shutdown_connections 2023-11-28T13:09:33.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.674+0000 7fde40733080 1 -- 172.21.15.156:0/2528737060 wait complete. 2023-11-28T13:09:33.680 INFO:teuthology.orchestra.run.smithi156.stdout:mytag 2023-11-28T13:09:33.681 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2023-11-28T13:09:33.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.765+0000 7f0a2198c080 1 Processor -- start 2023-11-28T13:09:33.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.765+0000 7f0a2198c080 1 -- start start 2023-11-28T13:09:33.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.766+0000 7f0a2198c080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590e3802920 0x5590e388e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.766+0000 7f0a2198c080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590e3892250 0x5590e388ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.766+0000 7f0a2198c080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590e3892620 0x5590e37e6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.766+0000 7f0a2198c080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590e37fa290 con 0x5590e3892250 2023-11-28T13:09:33.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.766+0000 7f0a2198c080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590e388efb0 con 0x5590e3892620 2023-11-28T13:09:33.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.766+0000 7f0a2198c080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590e37fa6d0 con 0x5590e3802920 2023-11-28T13:09:33.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.766+0000 7f0a1279a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590e3892250 0x5590e388ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.766+0000 7f0a1279a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590e3892250 0x5590e388ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44576/0 (socket says 172.21.15.156:44576) 2023-11-28T13:09:33.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.766+0000 7f0a1279a700 1 -- 172.21.15.156:0/2701046952 learned_addr learned my addr 172.21.15.156:0/2701046952 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:33.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.766+0000 7f0a12f9b700 1 --2- 172.21.15.156:0/2701046952 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590e3802920 0x5590e388e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.766+0000 7f0a1379c700 1 --2- 172.21.15.156:0/2701046952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590e3892620 0x5590e37e6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.766+0000 7f0a1279a700 1 -- 172.21.15.156:0/2701046952 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590e3802920 msgr2=0x5590e388e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:33.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.766+0000 7f0a1279a700 1 --2- 172.21.15.156:0/2701046952 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590e3802920 0x5590e388e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.766+0000 7f0a1279a700 1 -- 172.21.15.156:0/2701046952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590e3892620 msgr2=0x5590e37e6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:33.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.766+0000 7f0a1279a700 1 --2- 172.21.15.156:0/2701046952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590e3892620 0x5590e37e6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.766+0000 7f0a1279a700 1 -- 172.21.15.156:0/2701046952 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c00a1c0 con 0x5590e3892250 2023-11-28T13:09:33.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.766+0000 7f0a1279a700 1 --2- 172.21.15.156:0/2701046952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590e3892250 0x5590e388ea70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800a770 tx=0x7f0a0800aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3a1ea8107e2a60 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.767+0000 7f0a11798700 1 -- 172.21.15.156:0/2701046952 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0800f070 con 0x5590e3892250 2023-11-28T13:09:33.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.767+0000 7f0a11798700 1 -- 172.21.15.156:0/2701046952 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a080084c0 con 0x5590e3892250 2023-11-28T13:09:33.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.767+0000 7f0a11798700 1 -- 172.21.15.156:0/2701046952 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08008830 con 0x5590e3892250 2023-11-28T13:09:33.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.767+0000 7f0a2198c080 1 -- 172.21.15.156:0/2701046952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590e3892250 msgr2=0x5590e388ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.767+0000 7f0a2198c080 1 --2- 172.21.15.156:0/2701046952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590e3892250 0x5590e388ea70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800a770 tx=0x7f0a0800aa80 comp rx=0 tx=0).stop 2023-11-28T13:09:33.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.767+0000 7f0a2198c080 1 -- 172.21.15.156:0/2701046952 shutdown_connections 2023-11-28T13:09:33.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.767+0000 7f0a2198c080 1 --2- 172.21.15.156:0/2701046952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590e3892620 0x5590e37e6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.767+0000 7f0a2198c080 1 --2- 172.21.15.156:0/2701046952 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590e3802920 0x5590e388e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.767+0000 7f0a2198c080 1 --2- 172.21.15.156:0/2701046952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590e3892250 0x5590e388ea70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.767+0000 7f0a2198c080 1 -- 172.21.15.156:0/2701046952 >> 172.21.15.156:0/2701046952 conn(0x5590e38069d0 msgr2=0x5590e38914f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:33.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.767+0000 7f0a2198c080 1 -- 172.21.15.156:0/2701046952 shutdown_connections 2023-11-28T13:09:33.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.767+0000 7f0a2198c080 1 -- 172.21.15.156:0/2701046952 wait complete. 2023-11-28T13:09:33.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.768+0000 7f0a2198c080 1 Processor -- start 2023-11-28T13:09:33.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.768+0000 7f0a2198c080 1 -- start start 2023-11-28T13:09:33.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.769+0000 7f0a2198c080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590e3802920 0x5590e3895200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.769+0000 7f0a2198c080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590e3892250 0x5590e37fba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.769+0000 7f0a2198c080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590e3892620 0x5590e3892d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.769+0000 7f0a2198c080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590e38902a0 con 0x5590e3802920 2023-11-28T13:09:33.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.769+0000 7f0a2198c080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590e3804f90 con 0x5590e3892620 2023-11-28T13:09:33.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.769+0000 7f0a2198c080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590e3890540 con 0x5590e3892250 2023-11-28T13:09:33.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.769+0000 7f0a12f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590e3802920 0x5590e3895200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.769+0000 7f0a12f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590e3802920 0x5590e3895200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44590/0 (socket says 172.21.15.156:44590) 2023-11-28T13:09:33.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.769+0000 7f0a12f9b700 1 -- 172.21.15.156:0/874994711 learned_addr learned my addr 172.21.15.156:0/874994711 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:33.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.770+0000 7f0a1279a700 1 --2- 172.21.15.156:0/874994711 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590e3892250 0x5590e37fba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.770+0000 7f0a1379c700 1 --2- 172.21.15.156:0/874994711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590e3892620 0x5590e3892d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.770+0000 7f0a12f9b700 1 -- 172.21.15.156:0/874994711 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590e3892250 msgr2=0x5590e37fba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.770+0000 7f0a12f9b700 1 --2- 172.21.15.156:0/874994711 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5590e3892250 0x5590e37fba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.770+0000 7f0a12f9b700 1 -- 172.21.15.156:0/874994711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590e3892620 msgr2=0x5590e3892d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.770+0000 7f0a12f9b700 1 --2- 172.21.15.156:0/874994711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590e3892620 0x5590e3892d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.770+0000 7f0a12f9b700 1 -- 172.21.15.156:0/874994711 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0800a420 con 0x5590e3802920 2023-11-28T13:09:33.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.770+0000 7f0a1379c700 1 --2- 172.21.15.156:0/874994711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590e3892620 0x5590e3892d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:33.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.770+0000 7f0a12f9b700 1 --2- 172.21.15.156:0/874994711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590e3802920 0x5590e3895200 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009fa0 tx=0x7f0a04008790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.770+0000 7f09fbfff700 1 -- 172.21.15.156:0/874994711 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0400f050 con 0x5590e3802920 2023-11-28T13:09:33.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.770+0000 7f09fbfff700 1 -- 172.21.15.156:0/874994711 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a04010040 con 0x5590e3802920 2023-11-28T13:09:33.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.770+0000 7f09fbfff700 1 -- 172.21.15.156:0/874994711 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04014a10 con 0x5590e3802920 2023-11-28T13:09:33.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.770+0000 7f0a2198c080 1 -- 172.21.15.156:0/874994711 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5590e37274f0 con 0x5590e3802920 2023-11-28T13:09:33.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.770+0000 7f0a2198c080 1 -- 172.21.15.156:0/874994711 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590e370d120 con 0x5590e3802920 2023-11-28T13:09:33.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.772+0000 7f09fbfff700 1 -- 172.21.15.156:0/874994711 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0a04014b70 con 0x5590e3802920 2023-11-28T13:09:33.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.773+0000 7f09fbfff700 1 --2- 172.21.15.156:0/874994711 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0a00066c10 0x7f0a000690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.773+0000 7f09fbfff700 1 -- 172.21.15.156:0/874994711 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0a04084e60 con 0x5590e3802920 2023-11-28T13:09:33.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.773+0000 7f0a2198c080 1 --2- 172.21.15.156:0/874994711 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x5590e37e7590 0x5590e3928820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:33.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.773+0000 7f0a2198c080 1 -- 172.21.15.156:0/874994711 --> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590e3928d60 con 0x5590e37e7590 2023-11-28T13:09:33.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.773+0000 7f0a1279a700 1 --2- 172.21.15.156:0/874994711 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0a00066c10 0x7f0a000690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.774+0000 7f0a1379c700 1 --2- 172.21.15.156:0/874994711 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x5590e37e7590 0x5590e3928820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:33.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.774+0000 7f0a1379c700 1 --2- 172.21.15.156:0/874994711 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x5590e37e7590 0x5590e3928820 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.774+0000 7f0a1279a700 1 --2- 172.21.15.156:0/874994711 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0a00066c10 0x7f0a000690c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5590e37ec600 tx=0x7f0a08012000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:33.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.775+0000 7f0a1379c700 1 -- 172.21.15.156:0/874994711 <== osd.7 v2:172.21.15.186:6808/132936 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f0a0c017040 con 0x5590e37e7590 2023-11-28T13:09:33.779 INFO:teuthology.orchestra.run.smithi156.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2023-11-28T13:09:33.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.775+0000 7f0a2198c080 1 -- 172.21.15.156:0/874994711 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x5590e37e7590 msgr2=0x5590e3928820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.775+0000 7f0a2198c080 1 --2- 172.21.15.156:0/874994711 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x5590e37e7590 0x5590e3928820 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:33.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.775+0000 7f0a2198c080 1 -- 172.21.15.156:0/874994711 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0a00066c10 msgr2=0x7f0a000690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.775+0000 7f0a2198c080 1 --2- 172.21.15.156:0/874994711 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0a00066c10 0x7f0a000690c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5590e37ec600 tx=0x7f0a08012000 comp rx=0 tx=0).stop 2023-11-28T13:09:33.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.775+0000 7f0a2198c080 1 -- 172.21.15.156:0/874994711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590e3802920 msgr2=0x5590e3895200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:33.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.775+0000 7f0a2198c080 1 --2- 172.21.15.156:0/874994711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5590e3802920 0x5590e3895200 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009fa0 tx=0x7f0a04008790 comp rx=0 tx=0).stop 2023-11-28T13:09:33.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.775+0000 7f0a1379c700 1 -- 172.21.15.156:0/874994711 reap_dead start 2023-11-28T13:09:33.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.775+0000 7f0a2198c080 1 -- 172.21.15.156:0/874994711 shutdown_connections 2023-11-28T13:09:33.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.775+0000 7f0a2198c080 1 -- 172.21.15.156:0/874994711 >> 172.21.15.156:0/874994711 conn(0x5590e38069d0 msgr2=0x5590e37fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:33.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.775+0000 7f0a2198c080 1 -- 172.21.15.156:0/874994711 shutdown_connections 2023-11-28T13:09:33.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:33.775+0000 7f0a2198c080 1 -- 172.21.15.156:0/874994711 wait complete. 2023-11-28T13:09:33.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:09:33.781 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:09:33.781 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:09:33.867 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:09:33.959 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:09:33.960 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:09:33.960 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:09:34.004 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:09:34.004 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:09:34.004 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:09:34.050 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:09:34.050 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:09:34.050 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:09:34.094 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:09:34.094 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:09:34.094 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:09:34.138 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:09:34.138 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:09:34.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.333+0000 7ff2cddff700 1 Processor -- start 2023-11-28T13:09:34.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.334+0000 7ff2cddff700 1 -- start start 2023-11-28T13:09:34.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.335+0000 7ff2cddff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2c80d01b0 0x7ff2c80d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:34.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.335+0000 7ff2cddff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff2c80b20b0 0x7ff2c80d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:34.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.335+0000 7ff2cddff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff2c80b29b0 0x7ff2c80b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:34.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.335+0000 7ff2cddff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c815cec0 con 0x7ff2c80b20b0 2023-11-28T13:09:34.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.335+0000 7ff2cddff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c80b0230 con 0x7ff2c80b29b0 2023-11-28T13:09:34.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.335+0000 7ff2cddff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c80b4570 con 0x7ff2c80d01b0 2023-11-28T13:09:34.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.335+0000 7ff2bffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff2c80b20b0 0x7ff2c80d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:34.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.335+0000 7ff2bffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff2c80b20b0 0x7ff2c80d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44596/0 (socket says 172.21.15.156:44596) 2023-11-28T13:09:34.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.335+0000 7ff2bffff700 1 -- 172.21.15.156:0/2631610287 learned_addr learned my addr 172.21.15.156:0/2631610287 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:34.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.335+0000 7ff2c77fe700 1 --2- 172.21.15.156:0/2631610287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2c80d01b0 0x7ff2c80d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:34.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.335+0000 7ff2c7fff700 1 --2- 172.21.15.156:0/2631610287 >> v2:172.21.15.186:3300/0 conn(0x7ff2c80b29b0 0x7ff2c80b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:34.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.335+0000 7ff2bffff700 1 -- 172.21.15.156:0/2631610287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2c80d01b0 msgr2=0x7ff2c80d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:34.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.335+0000 7ff2bffff700 1 --2- 172.21.15.156:0/2631610287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2c80d01b0 0x7ff2c80d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.336+0000 7ff2bffff700 1 -- 172.21.15.156:0/2631610287 >> v2:172.21.15.186:3300/0 conn(0x7ff2c80b29b0 msgr2=0x7ff2c80b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:34.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.336+0000 7ff2bffff700 1 --2- 172.21.15.156:0/2631610287 >> v2:172.21.15.186:3300/0 conn(0x7ff2c80b29b0 0x7ff2c80b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.336+0000 7ff2bffff700 1 -- 172.21.15.156:0/2631610287 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c80b4850 con 0x7ff2c80b20b0 2023-11-28T13:09:34.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.336+0000 7ff2c77fe700 1 --2- 172.21.15.156:0/2631610287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2c80d01b0 0x7ff2c80d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:34.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.336+0000 7ff2c7fff700 1 --2- 172.21.15.156:0/2631610287 >> v2:172.21.15.186:3300/0 conn(0x7ff2c80b29b0 0x7ff2c80b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:34.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.336+0000 7ff2bffff700 1 --2- 172.21.15.156:0/2631610287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c80b20b0 0x7ff2c80d1160 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8066e80 tx=0x7ff2b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e954b819ccc6f23 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:34.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.336+0000 7ff2c6ffd700 1 -- 172.21.15.156:0/2631610287 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b000b870 con 0x7ff2c80b20b0 2023-11-28T13:09:34.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.336+0000 7ff2c6ffd700 1 -- 172.21.15.156:0/2631610287 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b000b9d0 con 0x7ff2c80b20b0 2023-11-28T13:09:34.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.336+0000 7ff2c6ffd700 1 -- 172.21.15.156:0/2631610287 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b000bd60 con 0x7ff2c80b20b0 2023-11-28T13:09:34.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.337+0000 7ff2cddff700 1 -- 172.21.15.156:0/2631610287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c80b20b0 msgr2=0x7ff2c80d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.337+0000 7ff2cddff700 1 --2- 172.21.15.156:0/2631610287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c80b20b0 0x7ff2c80d1160 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8066e80 tx=0x7ff2b0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.337+0000 7ff2cddff700 1 -- 172.21.15.156:0/2631610287 shutdown_connections 2023-11-28T13:09:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.337+0000 7ff2cddff700 1 --2- 172.21.15.156:0/2631610287 >> v2:172.21.15.186:3300/0 conn(0x7ff2c80b29b0 0x7ff2c80b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.337+0000 7ff2cddff700 1 --2- 172.21.15.156:0/2631610287 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2c80d01b0 0x7ff2c80d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.337+0000 7ff2cddff700 1 --2- 172.21.15.156:0/2631610287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c80b20b0 0x7ff2c80d1160 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.337+0000 7ff2cddff700 1 -- 172.21.15.156:0/2631610287 >> 172.21.15.156:0/2631610287 conn(0x7ff2c809c3f0 msgr2=0x7ff2c80d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.337+0000 7ff2cddff700 1 -- 172.21.15.156:0/2631610287 shutdown_connections 2023-11-28T13:09:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.337+0000 7ff2cddff700 1 -- 172.21.15.156:0/2631610287 wait complete. 2023-11-28T13:09:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.339+0000 7ff2cddff700 1 Processor -- start 2023-11-28T13:09:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.339+0000 7ff2cddff700 1 -- start start 2023-11-28T13:09:34.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2cddff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2c80b20b0 0x7ff2c81551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:34.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2cddff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c80b29b0 0x7ff2c81f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:34.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2cddff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2c80d01b0 0x7ff2c80ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:34.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2cddff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c815e1f0 con 0x7ff2c80b29b0 2023-11-28T13:09:34.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2cddff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c815d5e0 con 0x7ff2c80b20b0 2023-11-28T13:09:34.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2cddff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c815d8c0 con 0x7ff2c80d01b0 2023-11-28T13:09:34.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2bffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c80b29b0 0x7ff2c81f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:34.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2bffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c80b29b0 0x7ff2c81f1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44612/0 (socket says 172.21.15.156:44612) 2023-11-28T13:09:34.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2bffff700 1 -- 172.21.15.156:0/524728273 learned_addr learned my addr 172.21.15.156:0/524728273 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:34.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2c77fe700 1 --2- 172.21.15.156:0/524728273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2c80b20b0 0x7ff2c81551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:34.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2bffff700 1 -- 172.21.15.156:0/524728273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2c80d01b0 msgr2=0x7ff2c80ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:34.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2c7fff700 1 --2- 172.21.15.156:0/524728273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2c80d01b0 0x7ff2c80ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:34.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2bffff700 1 --2- 172.21.15.156:0/524728273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2c80d01b0 0x7ff2c80ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2bffff700 1 -- 172.21.15.156:0/524728273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2c80b20b0 msgr2=0x7ff2c81551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2bffff700 1 --2- 172.21.15.156:0/524728273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2c80b20b0 0x7ff2c81551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2bffff700 1 -- 172.21.15.156:0/524728273 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b0009550 con 0x7ff2c80b29b0 2023-11-28T13:09:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2c77fe700 1 --2- 172.21.15.156:0/524728273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2c80b20b0 0x7ff2c81551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.340+0000 7ff2c7fff700 1 --2- 172.21.15.156:0/524728273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2c80d01b0 0x7ff2c80ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.341+0000 7ff2bffff700 1 --2- 172.21.15.156:0/524728273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c80b29b0 0x7ff2c81f1b90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80b5c40 tx=0x7ff2b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.341+0000 7ff2c4ff9700 1 -- 172.21.15.156:0/524728273 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0004310 con 0x7ff2c80b29b0 2023-11-28T13:09:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.341+0000 7ff2c4ff9700 1 -- 172.21.15.156:0/524728273 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b001f070 con 0x7ff2c80b29b0 2023-11-28T13:09:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.341+0000 7ff2cddff700 1 -- 172.21.15.156:0/524728273 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c80afec0 con 0x7ff2c80b29b0 2023-11-28T13:09:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.341+0000 7ff2c4ff9700 1 -- 172.21.15.156:0/524728273 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b00036a0 con 0x7ff2c80b29b0 2023-11-28T13:09:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.341+0000 7ff2cddff700 1 -- 172.21.15.156:0/524728273 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c80afbe0 con 0x7ff2c80b29b0 2023-11-28T13:09:34.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.342+0000 7ff2cddff700 1 -- 172.21.15.156:0/524728273 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c80b2600 con 0x7ff2c80b29b0 2023-11-28T13:09:34.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.343+0000 7ff2c4ff9700 1 -- 172.21.15.156:0/524728273 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff2b002a020 con 0x7ff2c80b29b0 2023-11-28T13:09:34.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.344+0000 7ff2c4ff9700 1 --2- 172.21.15.156:0/524728273 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2a80682f0 0x7ff2a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:34.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.344+0000 7ff2c4ff9700 1 -- 172.21.15.156:0/524728273 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff2b00873c0 con 0x7ff2c80b29b0 2023-11-28T13:09:34.346 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.344+0000 7ff2c77fe700 1 --2- 172.21.15.156:0/524728273 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2a80682f0 0x7ff2a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:34.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.345+0000 7ff2c77fe700 1 --2- 172.21.15.156:0/524728273 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2a80682f0 0x7ff2a806a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8008910 tx=0x7ff2b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:34.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.348+0000 7ff2c4ff9700 1 -- 172.21.15.156:0/524728273 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff2b0053b30 con 0x7ff2c80b29b0 2023-11-28T13:09:34.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.584+0000 7ff2cddff700 1 -- 172.21.15.156:0/524728273 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2c8068690 con 0x7ff2c80b29b0 2023-11-28T13:09:34.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.585+0000 7ff2c4ff9700 1 -- 172.21.15.156:0/524728273 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7ff2b0057150 con 0x7ff2c80b29b0 2023-11-28T13:09:34.587 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:34.587 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1128367125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1128367125},{"type":"v1","addr":"172.21.15.156:6835","nonce":1128367125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4942,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/4096182008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4096182008},{"type":"v1","addr":"172.21.15.186:6835","nonce":4096182008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4945,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/561812165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":561812165},{"type":"v1","addr":"172.21.15.186:6837","nonce":561812165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:09:08.167940+0000","modified":"2023-11-28T13:09:11.184420+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5013":{"gid":5013,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6834/3471743149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3471743149},{"type":"v1","addr":"172.21.15.156:6834","nonce":3471743149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:09:34.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.587+0000 7ff2cddff700 1 -- 172.21.15.156:0/524728273 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2a80682f0 msgr2=0x7ff2a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:34.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.587+0000 7ff2cddff700 1 --2- 172.21.15.156:0/524728273 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2a80682f0 0x7ff2a806a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8008910 tx=0x7ff2b8008040 comp rx=0 tx=0).stop 2023-11-28T13:09:34.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.587+0000 7ff2cddff700 1 -- 172.21.15.156:0/524728273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c80b29b0 msgr2=0x7ff2c81f1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:34.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.587+0000 7ff2cddff700 1 --2- 172.21.15.156:0/524728273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c80b29b0 0x7ff2c81f1b90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80b5c40 tx=0x7ff2b0004750 comp rx=0 tx=0).stop 2023-11-28T13:09:34.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.587+0000 7ff2cddff700 1 -- 172.21.15.156:0/524728273 shutdown_connections 2023-11-28T13:09:34.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.587+0000 7ff2cddff700 1 --2- 172.21.15.156:0/524728273 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2a80682f0 0x7ff2a806a7a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.587+0000 7ff2cddff700 1 --2- 172.21.15.156:0/524728273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2c80d01b0 0x7ff2c80ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.587+0000 7ff2cddff700 1 --2- 172.21.15.156:0/524728273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2c80b20b0 0x7ff2c81551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.587+0000 7ff2cddff700 1 --2- 172.21.15.156:0/524728273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c80b29b0 0x7ff2c81f1b90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.587+0000 7ff2cddff700 1 -- 172.21.15.156:0/524728273 >> 172.21.15.156:0/524728273 conn(0x7ff2c809c3f0 msgr2=0x7ff2c80b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:34.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.587+0000 7ff2cddff700 1 -- 172.21.15.156:0/524728273 shutdown_connections 2023-11-28T13:09:34.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.587+0000 7ff2cddff700 1 -- 172.21.15.156:0/524728273 wait complete. 2023-11-28T13:09:34.590 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2023-11-28T13:09:34.608 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:09:34.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.803+0000 7f5d4d7ff700 1 Processor -- start 2023-11-28T13:09:34.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.804+0000 7f5d4d7ff700 1 -- start start 2023-11-28T13:09:34.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.805+0000 7f5d4d7ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5d4815c9b0 0x7f5d480b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:34.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.805+0000 7f5d4d7ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5d48161340 0x7f5d480d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:34.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.805+0000 7f5d4d7ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d480d1950 con 0x7f5d480d2320 2023-11-28T13:09:34.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.805+0000 7f5d4d7ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d480b5800 con 0x7f5d48161340 2023-11-28T13:09:34.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.805+0000 7f5d4d7ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d480b5520 con 0x7f5d4815c9b0 2023-11-28T13:09:34.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.805+0000 7f5d467fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5d48161340 0x7f5d480d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:34.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.805+0000 7f5d467fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5d48161340 0x7f5d480d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44618/0 (socket says 172.21.15.156:44618) 2023-11-28T13:09:34.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.805+0000 7f5d467fc700 1 -- 172.21.15.156:0/1376108791 learned_addr learned my addr 172.21.15.156:0/1376108791 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:34.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.805+0000 7f5d46ffd700 1 --2- 172.21.15.156:0/1376108791 >> v2:172.21.15.186:3300/0 conn(0x7f5d4815c9b0 0x7f5d480b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:34.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.805+0000 7f5d467fc700 1 -- 172.21.15.156:0/1376108791 >> v2:172.21.15.186:3300/0 conn(0x7f5d4815c9b0 msgr2=0x7f5d480b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:34.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.805+0000 7f5d467fc700 1 --2- 172.21.15.156:0/1376108791 >> v2:172.21.15.186:3300/0 conn(0x7f5d4815c9b0 0x7f5d480b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.806+0000 7f5d467fc700 1 -- 172.21.15.156:0/1376108791 >> v1:172.21.15.186:6789/0 conn(0x7f5d480d2320 legacy=0x7f5d480cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:34.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.806+0000 7f5d467fc700 1 -- 172.21.15.156:0/1376108791 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d480d1300 con 0x7f5d48161340 2023-11-28T13:09:34.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.806+0000 7f5d46ffd700 1 --2- 172.21.15.156:0/1376108791 >> v2:172.21.15.186:3300/0 conn(0x7f5d4815c9b0 0x7f5d480b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:34.810 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.806+0000 7f5d467fc700 1 --2- 172.21.15.156:0/1376108791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d48161340 0x7f5d480d1de0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5d48067460 tx=0x7f5d3c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cfb4b13f837920 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:34.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.806+0000 7f5d45ffb700 1 -- 172.21.15.156:0/1376108791 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c00b870 con 0x7f5d48161340 2023-11-28T13:09:34.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.806+0000 7f5d45ffb700 1 -- 172.21.15.156:0/1376108791 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c00b9d0 con 0x7f5d48161340 2023-11-28T13:09:34.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.806+0000 7f5d45ffb700 1 -- 172.21.15.156:0/1376108791 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c00bd60 con 0x7f5d48161340 2023-11-28T13:09:34.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.807+0000 7f5d4d7ff700 1 -- 172.21.15.156:0/1376108791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d48161340 msgr2=0x7f5d480d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:34.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.807+0000 7f5d4d7ff700 1 --2- 172.21.15.156:0/1376108791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d48161340 0x7f5d480d1de0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5d48067460 tx=0x7f5d3c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.807+0000 7f5d4d7ff700 1 -- 172.21.15.156:0/1376108791 shutdown_connections 2023-11-28T13:09:34.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.807+0000 7f5d4d7ff700 1 --2- 172.21.15.156:0/1376108791 >> v2:172.21.15.186:3300/0 conn(0x7f5d4815c9b0 0x7f5d480b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.807+0000 7f5d4d7ff700 1 --2- 172.21.15.156:0/1376108791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d48161340 0x7f5d480d1de0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.807+0000 7f5d4d7ff700 1 -- 172.21.15.156:0/1376108791 >> 172.21.15.156:0/1376108791 conn(0x7f5d48072600 msgr2=0x7f5d480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:34.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.807+0000 7f5d4d7ff700 1 -- 172.21.15.156:0/1376108791 shutdown_connections 2023-11-28T13:09:34.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.807+0000 7f5d4d7ff700 1 -- 172.21.15.156:0/1376108791 wait complete. 2023-11-28T13:09:34.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.808+0000 7f5d4d7ff700 1 Processor -- start 2023-11-28T13:09:34.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.809+0000 7f5d4d7ff700 1 -- start start 2023-11-28T13:09:34.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.809+0000 7f5d4d7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d480d2320 0x7f5d480c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:34.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.809+0000 7f5d4d7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d4815c9b0 0x7f5d481f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:34.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.809+0000 7f5d4d7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d48161340 0x7f5d48152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:34.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.809+0000 7f5d4d7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d481f5d50 con 0x7f5d480d2320 2023-11-28T13:09:34.818 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.809+0000 7f5d4d7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480c8fe0 con 0x7f5d48161340 2023-11-28T13:09:34.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.809+0000 7f5d4d7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d48153060 con 0x7f5d4815c9b0 2023-11-28T13:09:34.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.810+0000 7f5d46ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d480d2320 0x7f5d480c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:34.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.810+0000 7f5d46ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d480d2320 0x7f5d480c8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44624/0 (socket says 172.21.15.156:44624) 2023-11-28T13:09:34.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.810+0000 7f5d46ffd700 1 -- 172.21.15.156:0/3957639325 learned_addr learned my addr 172.21.15.156:0/3957639325 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:34.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.810+0000 7f5d467fc700 1 --2- 172.21.15.156:0/3957639325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d4815c9b0 0x7f5d481f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:34.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.810+0000 7f5d477fe700 1 --2- 172.21.15.156:0/3957639325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d48161340 0x7f5d48152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:34.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.810+0000 7f5d46ffd700 1 -- 172.21.15.156:0/3957639325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d4815c9b0 msgr2=0x7f5d481f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:34.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.810+0000 7f5d46ffd700 1 --2- 172.21.15.156:0/3957639325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d4815c9b0 0x7f5d481f5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.810+0000 7f5d46ffd700 1 -- 172.21.15.156:0/3957639325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d48161340 msgr2=0x7f5d48152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:34.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.810+0000 7f5d46ffd700 1 --2- 172.21.15.156:0/3957639325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d48161340 0x7f5d48152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:34.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.810+0000 7f5d46ffd700 1 -- 172.21.15.156:0/3957639325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c009550 con 0x7f5d480d2320 2023-11-28T13:09:34.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.810+0000 7f5d46ffd700 1 --2- 172.21.15.156:0/3957639325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d480d2320 0x7f5d480c8650 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800e810 tx=0x7f5d3800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:34.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.811+0000 7f5d2f7fe700 1 -- 172.21.15.156:0/3957639325 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38013610 con 0x7f5d480d2320 2023-11-28T13:09:34.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.811+0000 7f5d2f7fe700 1 -- 172.21.15.156:0/3957639325 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d38013770 con 0x7f5d480d2320 2023-11-28T13:09:34.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.811+0000 7f5d4d7ff700 1 -- 172.21.15.156:0/3957639325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d480b5130 con 0x7f5d480d2320 2023-11-28T13:09:34.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.811+0000 7f5d2f7fe700 1 -- 172.21.15.156:0/3957639325 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38013b20 con 0x7f5d480d2320 2023-11-28T13:09:34.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.811+0000 7f5d4d7ff700 1 -- 172.21.15.156:0/3957639325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d480d15e0 con 0x7f5d480d2320 2023-11-28T13:09:34.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.812+0000 7f5d4d7ff700 1 -- 172.21.15.156:0/3957639325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d480b52c0 con 0x7f5d480d2320 2023-11-28T13:09:34.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.812+0000 7f5d2f7fe700 1 -- 172.21.15.156:0/3957639325 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5d38023070 con 0x7f5d480d2320 2023-11-28T13:09:34.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.813+0000 7f5d2f7fe700 1 --2- 172.21.15.156:0/3957639325 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5d340682f0 0x7f5d3406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:34.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.813+0000 7f5d2f7fe700 1 -- 172.21.15.156:0/3957639325 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5d38085910 con 0x7f5d480d2320 2023-11-28T13:09:34.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.814+0000 7f5d467fc700 1 --2- 172.21.15.156:0/3957639325 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5d340682f0 0x7f5d3406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:34.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.814+0000 7f5d467fc700 1 --2- 172.21.15.156:0/3957639325 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5d340682f0 0x7f5d3406a7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c000c00 tx=0x7f5d3c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:34.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:34.817+0000 7f5d2f7fe700 1 -- 172.21.15.156:0/3957639325 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5d38052080 con 0x7f5d480d2320 2023-11-28T13:09:35.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.009+0000 7f5d4d7ff700 1 -- 172.21.15.156:0/3957639325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d480745c0 con 0x7f5d480d2320 2023-11-28T13:09:35.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.010+0000 7f5d2f7fe700 1 -- 172.21.15.156:0/3957639325 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+17969 (secure 0 0 0) 0x7f5d380556a0 con 0x7f5d480d2320 2023-11-28T13:09:35.013 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:35.013 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":35,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:09:09.170289+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:09:06.618698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T13:09:07.151554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:09:35.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.013+0000 7f5d4d7ff700 1 -- 172.21.15.156:0/3957639325 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5d340682f0 msgr2=0x7f5d3406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:35.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.013+0000 7f5d4d7ff700 1 --2- 172.21.15.156:0/3957639325 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5d340682f0 0x7f5d3406a7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c000c00 tx=0x7f5d3c008030 comp rx=0 tx=0).stop 2023-11-28T13:09:35.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.013+0000 7f5d4d7ff700 1 -- 172.21.15.156:0/3957639325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d480d2320 msgr2=0x7f5d480c8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:35.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.013+0000 7f5d4d7ff700 1 --2- 172.21.15.156:0/3957639325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d480d2320 0x7f5d480c8650 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800e810 tx=0x7f5d3800eb20 comp rx=0 tx=0).stop 2023-11-28T13:09:35.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.013+0000 7f5d4d7ff700 1 -- 172.21.15.156:0/3957639325 shutdown_connections 2023-11-28T13:09:35.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.013+0000 7f5d4d7ff700 1 --2- 172.21.15.156:0/3957639325 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5d340682f0 0x7f5d3406a7a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.013+0000 7f5d4d7ff700 1 --2- 172.21.15.156:0/3957639325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d48161340 0x7f5d48152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.013+0000 7f5d4d7ff700 1 --2- 172.21.15.156:0/3957639325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d480d2320 0x7f5d480c8650 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.013+0000 7f5d4d7ff700 1 --2- 172.21.15.156:0/3957639325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d4815c9b0 0x7f5d481f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.013+0000 7f5d4d7ff700 1 -- 172.21.15.156:0/3957639325 >> 172.21.15.156:0/3957639325 conn(0x7f5d48072600 msgr2=0x7f5d480b8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:35.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.013+0000 7f5d4d7ff700 1 -- 172.21.15.156:0/3957639325 shutdown_connections 2023-11-28T13:09:35.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.013+0000 7f5d4d7ff700 1 -- 172.21.15.156:0/3957639325 wait complete. 2023-11-28T13:09:35.033 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:09:06.618698+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:09:07.151554+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:09:35.033 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:09:35.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.235+0000 7f6b6cbff700 1 Processor -- start 2023-11-28T13:09:35.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.235+0000 7f6b6cbff700 1 -- start start 2023-11-28T13:09:35.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.237+0000 7f6b6cbff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b6815c9b0 0x7f6b680b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:35.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.237+0000 7f6b6cbff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6b68161340 0x7f6b680d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:35.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.237+0000 7f6b6cbff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6b680d2320 0x7f6b680cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:35.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.237+0000 7f6b6cbff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b680b5800 con 0x7f6b68161340 2023-11-28T13:09:35.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.237+0000 7f6b6cbff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b680d1950 con 0x7f6b680d2320 2023-11-28T13:09:35.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.237+0000 7f6b6cbff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b680b5520 con 0x7f6b6815c9b0 2023-11-28T13:09:35.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.237+0000 7f6b5ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6b68161340 0x7f6b680d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:35.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.237+0000 7f6b5ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6b68161340 0x7f6b680d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44630/0 (socket says 172.21.15.156:44630) 2023-11-28T13:09:35.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.237+0000 7f6b5ffff700 1 -- 172.21.15.156:0/309767958 learned_addr learned my addr 172.21.15.156:0/309767958 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:35.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.237+0000 7f6b5ffff700 1 -- 172.21.15.156:0/309767958 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b6815c9b0 msgr2=0x7f6b680b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:35.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.237+0000 7f6b5ffff700 1 --2- 172.21.15.156:0/309767958 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b6815c9b0 0x7f6b680b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.237+0000 7f6b5ffff700 1 -- 172.21.15.156:0/309767958 >> v2:172.21.15.186:3300/0 conn(0x7f6b680d2320 msgr2=0x7f6b680cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:35.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.237+0000 7f6b5ffff700 1 --2- 172.21.15.156:0/309767958 >> v2:172.21.15.186:3300/0 conn(0x7f6b680d2320 0x7f6b680cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.237+0000 7f6b5ffff700 1 -- 172.21.15.156:0/309767958 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b680d1300 con 0x7f6b68161340 2023-11-28T13:09:35.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.238+0000 7f6b5ffff700 1 --2- 172.21.15.156:0/309767958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b68161340 0x7f6b680d1de0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6b6808c4d0 tx=0x7f6b50009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f78bea1a275cbd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:35.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.238+0000 7f6b65d9b700 1 -- 172.21.15.156:0/309767958 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5000b870 con 0x7f6b68161340 2023-11-28T13:09:35.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.238+0000 7f6b65d9b700 1 -- 172.21.15.156:0/309767958 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5000b9d0 con 0x7f6b68161340 2023-11-28T13:09:35.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.238+0000 7f6b65d9b700 1 -- 172.21.15.156:0/309767958 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5000bd60 con 0x7f6b68161340 2023-11-28T13:09:35.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.238+0000 7f6b6cbff700 1 -- 172.21.15.156:0/309767958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b68161340 msgr2=0x7f6b680d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:35.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.238+0000 7f6b6cbff700 1 --2- 172.21.15.156:0/309767958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b68161340 0x7f6b680d1de0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6b6808c4d0 tx=0x7f6b50009bd0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.238+0000 7f6b6cbff700 1 -- 172.21.15.156:0/309767958 shutdown_connections 2023-11-28T13:09:35.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.238+0000 7f6b6cbff700 1 --2- 172.21.15.156:0/309767958 >> v2:172.21.15.186:3300/0 conn(0x7f6b680d2320 0x7f6b680cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.238+0000 7f6b6cbff700 1 --2- 172.21.15.156:0/309767958 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b6815c9b0 0x7f6b680b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.238+0000 7f6b6cbff700 1 --2- 172.21.15.156:0/309767958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b68161340 0x7f6b680d1de0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.238+0000 7f6b6cbff700 1 -- 172.21.15.156:0/309767958 >> 172.21.15.156:0/309767958 conn(0x7f6b68072600 msgr2=0x7f6b680cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:35.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.238+0000 7f6b6cbff700 1 -- 172.21.15.156:0/309767958 shutdown_connections 2023-11-28T13:09:35.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.239+0000 7f6b6cbff700 1 -- 172.21.15.156:0/309767958 wait complete. 2023-11-28T13:09:35.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.241+0000 7f6b6cbff700 1 Processor -- start 2023-11-28T13:09:35.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.241+0000 7f6b6cbff700 1 -- start start 2023-11-28T13:09:35.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.242+0000 7f6b6cbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b680d2320 0x7f6b6815f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:35.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.242+0000 7f6b6cbff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b6815c9b0 0x7f6b681f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:35.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.242+0000 7f6b6cbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b68161340 0x7f6b680c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:35.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.242+0000 7f6b6cbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68154cb0 con 0x7f6b680d2320 2023-11-28T13:09:35.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.242+0000 7f6b6cbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b681f6bf0 con 0x7f6b68161340 2023-11-28T13:09:35.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.242+0000 7f6b6cbff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b681f6ed0 con 0x7f6b6815c9b0 2023-11-28T13:09:35.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.242+0000 7f6b6659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b680d2320 0x7f6b6815f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:35.246 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.243+0000 7f6b6659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b680d2320 0x7f6b6815f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44640/0 (socket says 172.21.15.156:44640) 2023-11-28T13:09:35.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.243+0000 7f6b6659c700 1 -- 172.21.15.156:0/3787576058 learned_addr learned my addr 172.21.15.156:0/3787576058 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:35.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.243+0000 7f6b66d9d700 1 --2- 172.21.15.156:0/3787576058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b68161340 0x7f6b680c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:35.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.243+0000 7f6b5ffff700 1 --2- 172.21.15.156:0/3787576058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b6815c9b0 0x7f6b681f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:35.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.243+0000 7f6b6659c700 1 -- 172.21.15.156:0/3787576058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b6815c9b0 msgr2=0x7f6b681f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:35.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.243+0000 7f6b6659c700 1 --2- 172.21.15.156:0/3787576058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b6815c9b0 0x7f6b681f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.243+0000 7f6b6659c700 1 -- 172.21.15.156:0/3787576058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b68161340 msgr2=0x7f6b680c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:35.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.243+0000 7f6b6659c700 1 --2- 172.21.15.156:0/3787576058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b68161340 0x7f6b680c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.243+0000 7f6b6659c700 1 -- 172.21.15.156:0/3787576058 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b50009550 con 0x7f6b680d2320 2023-11-28T13:09:35.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.243+0000 7f6b5ffff700 1 --2- 172.21.15.156:0/3787576058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b6815c9b0 0x7f6b681f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:35.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.243+0000 7f6b6659c700 1 --2- 172.21.15.156:0/3787576058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b680d2320 0x7f6b6815f130 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800b4a0 tx=0x7f6b5800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:35.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.243+0000 7f6b5effd700 1 -- 172.21.15.156:0/3787576058 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58011830 con 0x7f6b680d2320 2023-11-28T13:09:35.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.243+0000 7f6b5effd700 1 -- 172.21.15.156:0/3787576058 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b58011990 con 0x7f6b680d2320 2023-11-28T13:09:35.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.243+0000 7f6b6cbff700 1 -- 172.21.15.156:0/3787576058 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b680b5130 con 0x7f6b680d2320 2023-11-28T13:09:35.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.243+0000 7f6b5effd700 1 -- 172.21.15.156:0/3787576058 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58011d40 con 0x7f6b680d2320 2023-11-28T13:09:35.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.244+0000 7f6b6cbff700 1 -- 172.21.15.156:0/3787576058 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b680d15e0 con 0x7f6b680d2320 2023-11-28T13:09:35.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.245+0000 7f6b6cbff700 1 -- 172.21.15.156:0/3787576058 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b680b52c0 con 0x7f6b680d2320 2023-11-28T13:09:35.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.245+0000 7f6b5effd700 1 -- 172.21.15.156:0/3787576058 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6b58021070 con 0x7f6b680d2320 2023-11-28T13:09:35.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.247+0000 7f6b5effd700 1 --2- 172.21.15.156:0/3787576058 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6b480682f0 0x7f6b4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:35.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.247+0000 7f6b5effd700 1 -- 172.21.15.156:0/3787576058 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6b58085ce0 con 0x7f6b680d2320 2023-11-28T13:09:35.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.247+0000 7f6b5ffff700 1 --2- 172.21.15.156:0/3787576058 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6b480682f0 0x7f6b4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:35.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.248+0000 7f6b5ffff700 1 --2- 172.21.15.156:0/3787576058 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6b480682f0 0x7f6b4806a7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6b681581e0 tx=0x7f6b50008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:35.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.251+0000 7f6b5effd700 1 -- 172.21.15.156:0/3787576058 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6b580523d0 con 0x7f6b680d2320 2023-11-28T13:09:35.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.443+0000 7f6b6cbff700 1 -- 172.21.15.156:0/3787576058 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6b68067b70 con 0x7f6b680d2320 2023-11-28T13:09:35.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.444+0000 7f6b5effd700 1 -- 172.21.15.156:0/3787576058 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6b58015020 con 0x7f6b680d2320 2023-11-28T13:09:35.446 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:09:35.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.446+0000 7f6b6cbff700 1 -- 172.21.15.156:0/3787576058 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6b480682f0 msgr2=0x7f6b4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:35.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.446+0000 7f6b6cbff700 1 --2- 172.21.15.156:0/3787576058 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6b480682f0 0x7f6b4806a7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6b681581e0 tx=0x7f6b50008030 comp rx=0 tx=0).stop 2023-11-28T13:09:35.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.446+0000 7f6b6cbff700 1 -- 172.21.15.156:0/3787576058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b680d2320 msgr2=0x7f6b6815f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:35.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.446+0000 7f6b6cbff700 1 --2- 172.21.15.156:0/3787576058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b680d2320 0x7f6b6815f130 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800b4a0 tx=0x7f6b5800b860 comp rx=0 tx=0).stop 2023-11-28T13:09:35.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.446+0000 7f6b6cbff700 1 -- 172.21.15.156:0/3787576058 shutdown_connections 2023-11-28T13:09:35.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.446+0000 7f6b6cbff700 1 --2- 172.21.15.156:0/3787576058 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6b480682f0 0x7f6b4806a7a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.446+0000 7f6b6cbff700 1 --2- 172.21.15.156:0/3787576058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b68161340 0x7f6b680c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.446+0000 7f6b6cbff700 1 --2- 172.21.15.156:0/3787576058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b680d2320 0x7f6b6815f130 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.446+0000 7f6b6cbff700 1 --2- 172.21.15.156:0/3787576058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6b6815c9b0 0x7f6b681f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.446+0000 7f6b6cbff700 1 -- 172.21.15.156:0/3787576058 >> 172.21.15.156:0/3787576058 conn(0x7f6b68072600 msgr2=0x7f6b6815bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:35.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.446+0000 7f6b6cbff700 1 -- 172.21.15.156:0/3787576058 shutdown_connections 2023-11-28T13:09:35.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.446+0000 7f6b6cbff700 1 -- 172.21.15.156:0/3787576058 wait complete. 2023-11-28T13:09:35.466 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:09:35.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.663+0000 7fb7d0fff700 1 Processor -- start 2023-11-28T13:09:35.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.664+0000 7fb7d0fff700 1 -- start start 2023-11-28T13:09:35.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.665+0000 7fb7d0fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb7cc161340 0x7fb7cc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:35.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.665+0000 7fb7d0fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb7cc0d2320 0x7fb7cc0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:35.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.665+0000 7fb7d0fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7cc0d1950 con 0x7fb7cc15c9b0 2023-11-28T13:09:35.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.665+0000 7fb7d0fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7cc0b5800 con 0x7fb7cc161340 2023-11-28T13:09:35.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.665+0000 7fb7d0fff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7cc0b5300 con 0x7fb7cc0d2320 2023-11-28T13:09:35.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.665+0000 7fb7c9d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb7cc161340 0x7fb7cc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:35.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.665+0000 7fb7c9d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb7cc161340 0x7fb7cc0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44656/0 (socket says 172.21.15.156:44656) 2023-11-28T13:09:35.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.665+0000 7fb7ca59c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb7cc15c9b0 0x7fb7cc0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:46654/0 (socket says 172.21.15.156:46654) 2023-11-28T13:09:35.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.665+0000 7fb7c9d9b700 1 -- 172.21.15.156:0/870834191 learned_addr learned my addr 172.21.15.156:0/870834191 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:35.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.665+0000 7fb7cad9d700 1 --2- 172.21.15.156:0/870834191 >> v2:172.21.15.186:3300/0 conn(0x7fb7cc0d2320 0x7fb7cc0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:35.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.665+0000 7fb7c9d9b700 1 -- 172.21.15.156:0/870834191 >> v2:172.21.15.186:3300/0 conn(0x7fb7cc0d2320 msgr2=0x7fb7cc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:35.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.665+0000 7fb7c9d9b700 1 --2- 172.21.15.156:0/870834191 >> v2:172.21.15.186:3300/0 conn(0x7fb7cc0d2320 0x7fb7cc0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.665+0000 7fb7c9d9b700 1 -- 172.21.15.156:0/870834191 >> v1:172.21.15.156:6789/0 conn(0x7fb7cc15c9b0 legacy=0x7fb7cc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:35.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.665+0000 7fb7c9d9b700 1 -- 172.21.15.156:0/870834191 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc0b5520 con 0x7fb7cc161340 2023-11-28T13:09:35.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.666+0000 7fb7c9d9b700 1 --2- 172.21.15.156:0/870834191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc161340 0x7fb7cc0d1de0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc06c170 tx=0x7fb7c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6911b2646d68b292 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:35.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.666+0000 7fb7c959a700 1 -- 172.21.15.156:0/870834191 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c000b860 con 0x7fb7cc161340 2023-11-28T13:09:35.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.666+0000 7fb7c959a700 1 -- 172.21.15.156:0/870834191 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c000b9c0 con 0x7fb7cc161340 2023-11-28T13:09:35.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.666+0000 7fb7c959a700 1 -- 172.21.15.156:0/870834191 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c000bd50 con 0x7fb7cc161340 2023-11-28T13:09:35.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.666+0000 7fb7d0fff700 1 -- 172.21.15.156:0/870834191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc161340 msgr2=0x7fb7cc0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:35.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.666+0000 7fb7d0fff700 1 --2- 172.21.15.156:0/870834191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc161340 0x7fb7cc0d1de0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc06c170 tx=0x7fb7c0009c00 comp rx=0 tx=0).stop 2023-11-28T13:09:35.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.667+0000 7fb7d0fff700 1 -- 172.21.15.156:0/870834191 shutdown_connections 2023-11-28T13:09:35.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.667+0000 7fb7d0fff700 1 --2- 172.21.15.156:0/870834191 >> v2:172.21.15.186:3300/0 conn(0x7fb7cc0d2320 0x7fb7cc0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.667+0000 7fb7d0fff700 1 --2- 172.21.15.156:0/870834191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc161340 0x7fb7cc0d1de0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.667+0000 7fb7d0fff700 1 -- 172.21.15.156:0/870834191 >> 172.21.15.156:0/870834191 conn(0x7fb7cc072600 msgr2=0x7fb7cc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:35.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.667+0000 7fb7d0fff700 1 -- 172.21.15.156:0/870834191 shutdown_connections 2023-11-28T13:09:35.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.667+0000 7fb7d0fff700 1 -- 172.21.15.156:0/870834191 wait complete. 2023-11-28T13:09:35.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.669+0000 7fb7d0fff700 1 Processor -- start 2023-11-28T13:09:35.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.669+0000 7fb7d0fff700 1 -- start start 2023-11-28T13:09:35.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.670+0000 7fb7d0fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc0d2320 0x7fb7cc1474c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:35.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.670+0000 7fb7d0fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7cc15c9b0 0x7fb7cc156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:35.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.671+0000 7fb7d0fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7cc161340 0x7fb7cc157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:35.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.671+0000 7fb7d0fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc147a00 con 0x7fb7cc0d2320 2023-11-28T13:09:35.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.671+0000 7fb7d0fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc15d0c0 con 0x7fb7cc15c9b0 2023-11-28T13:09:35.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.671+0000 7fb7d0fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc15d310 con 0x7fb7cc161340 2023-11-28T13:09:35.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.671+0000 7fb7ca59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc0d2320 0x7fb7cc1474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:35.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.671+0000 7fb7ca59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc0d2320 0x7fb7cc1474c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44672/0 (socket says 172.21.15.156:44672) 2023-11-28T13:09:35.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.671+0000 7fb7ca59c700 1 -- 172.21.15.156:0/3583201471 learned_addr learned my addr 172.21.15.156:0/3583201471 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:35.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.671+0000 7fb7c9d9b700 1 --2- 172.21.15.156:0/3583201471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7cc15c9b0 0x7fb7cc156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:35.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.671+0000 7fb7cad9d700 1 --2- 172.21.15.156:0/3583201471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7cc161340 0x7fb7cc157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:35.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.671+0000 7fb7ca59c700 1 -- 172.21.15.156:0/3583201471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7cc161340 msgr2=0x7fb7cc157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:35.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.671+0000 7fb7ca59c700 1 --2- 172.21.15.156:0/3583201471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7cc161340 0x7fb7cc157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.671+0000 7fb7ca59c700 1 -- 172.21.15.156:0/3583201471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7cc15c9b0 msgr2=0x7fb7cc156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:35.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.671+0000 7fb7ca59c700 1 --2- 172.21.15.156:0/3583201471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7cc15c9b0 0x7fb7cc156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.671+0000 7fb7ca59c700 1 -- 172.21.15.156:0/3583201471 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c0009550 con 0x7fb7cc0d2320 2023-11-28T13:09:35.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.671+0000 7fb7ca59c700 1 --2- 172.21.15.156:0/3583201471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc0d2320 0x7fb7cc1474c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc00c500 tx=0x7fb7bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:35.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.671+0000 7fb7baffd700 1 -- 172.21.15.156:0/3583201471 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc005cb0 con 0x7fb7cc0d2320 2023-11-28T13:09:35.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.672+0000 7fb7baffd700 1 -- 172.21.15.156:0/3583201471 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7bc005e10 con 0x7fb7cc0d2320 2023-11-28T13:09:35.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.672+0000 7fb7d0fff700 1 -- 172.21.15.156:0/3583201471 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7cc0d15e0 con 0x7fb7cc0d2320 2023-11-28T13:09:35.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.672+0000 7fb7baffd700 1 -- 172.21.15.156:0/3583201471 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc0072d0 con 0x7fb7cc0d2320 2023-11-28T13:09:35.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.672+0000 7fb7d0fff700 1 -- 172.21.15.156:0/3583201471 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7cc0d1300 con 0x7fb7cc0d2320 2023-11-28T13:09:35.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.673+0000 7fb7d0fff700 1 -- 172.21.15.156:0/3583201471 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7cc15cd80 con 0x7fb7cc0d2320 2023-11-28T13:09:35.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.673+0000 7fb7baffd700 1 -- 172.21.15.156:0/3583201471 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb7bc019030 con 0x7fb7cc0d2320 2023-11-28T13:09:35.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.674+0000 7fb7baffd700 1 --2- 172.21.15.156:0/3583201471 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb7ac0682f0 0x7fb7ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:35.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.674+0000 7fb7baffd700 1 -- 172.21.15.156:0/3583201471 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb7bc084f60 con 0x7fb7cc0d2320 2023-11-28T13:09:35.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.675+0000 7fb7c9d9b700 1 --2- 172.21.15.156:0/3583201471 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb7ac0682f0 0x7fb7ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:35.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.675+0000 7fb7c9d9b700 1 --2- 172.21.15.156:0/3583201471 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb7ac0682f0 0x7fb7ac06a7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0000c00 tx=0x7fb7c0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:35.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.678+0000 7fb7baffd700 1 -- 172.21.15.156:0/3583201471 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb7bc04b5e0 con 0x7fb7cc0d2320 2023-11-28T13:09:35.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.868+0000 7fb7d0fff700 1 -- 172.21.15.156:0/3583201471 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7cc0745c0 con 0x7fb7cc0d2320 2023-11-28T13:09:35.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.868+0000 7fb7baffd700 1 -- 172.21.15.156:0/3583201471 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7bc017020 con 0x7fb7cc0d2320 2023-11-28T13:09:35.870 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:09:35.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.870+0000 7fb7d0fff700 1 -- 172.21.15.156:0/3583201471 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb7ac0682f0 msgr2=0x7fb7ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:35.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.870+0000 7fb7d0fff700 1 --2- 172.21.15.156:0/3583201471 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb7ac0682f0 0x7fb7ac06a7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0000c00 tx=0x7fb7c0008030 comp rx=0 tx=0).stop 2023-11-28T13:09:35.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.870+0000 7fb7d0fff700 1 -- 172.21.15.156:0/3583201471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc0d2320 msgr2=0x7fb7cc1474c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:35.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.870+0000 7fb7d0fff700 1 --2- 172.21.15.156:0/3583201471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc0d2320 0x7fb7cc1474c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc00c500 tx=0x7fb7bc00c8c0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.870+0000 7fb7d0fff700 1 -- 172.21.15.156:0/3583201471 shutdown_connections 2023-11-28T13:09:35.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.870+0000 7fb7d0fff700 1 --2- 172.21.15.156:0/3583201471 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb7ac0682f0 0x7fb7ac06a7a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.870+0000 7fb7d0fff700 1 --2- 172.21.15.156:0/3583201471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7cc161340 0x7fb7cc157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.870+0000 7fb7d0fff700 1 --2- 172.21.15.156:0/3583201471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc0d2320 0x7fb7cc1474c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.870+0000 7fb7d0fff700 1 --2- 172.21.15.156:0/3583201471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7cc15c9b0 0x7fb7cc156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:35.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.870+0000 7fb7d0fff700 1 -- 172.21.15.156:0/3583201471 >> 172.21.15.156:0/3583201471 conn(0x7fb7cc072600 msgr2=0x7fb7cc0b82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:35.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.870+0000 7fb7d0fff700 1 -- 172.21.15.156:0/3583201471 shutdown_connections 2023-11-28T13:09:35.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:35.870+0000 7fb7d0fff700 1 -- 172.21.15.156:0/3583201471 wait complete. 2023-11-28T13:09:35.890 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:09:36.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.089+0000 7fcfd4dff700 1 Processor -- start 2023-11-28T13:09:36.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.089+0000 7fcfd4dff700 1 -- start start 2023-11-28T13:09:36.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.090+0000 7fcfd4dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfd015c9b0 0x7fcfd00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:36.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.090+0000 7fcfd4dff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd00d1950 con 0x7fcfd0161340 2023-11-28T13:09:36.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.090+0000 7fcfd4dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd00d24f0 con 0x7fcfd00cd7c0 2023-11-28T13:09:36.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.090+0000 7fcfd4dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd00b5800 con 0x7fcfd015c9b0 2023-11-28T13:09:36.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.090+0000 7fcfc7fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fcfd0161340 0x7fcfd00d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53250/0 (socket says 172.21.15.156:53250) 2023-11-28T13:09:36.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.090+0000 7fcfc7fff700 1 -- 172.21.15.156:0/2906508417 learned_addr learned my addr 172.21.15.156:0/2906508417 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:36.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.090+0000 7fcfce59c700 1 --2- 172.21.15.156:0/2906508417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfd015c9b0 0x7fcfd00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:36.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.091+0000 7fcfcdd9b700 1 -- 172.21.15.156:0/2906508417 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 928212799 0 0) 0x7fcfd00d1950 con 0x7fcfd0161340 2023-11-28T13:09:36.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.091+0000 7fcfcdd9b700 1 -- 172.21.15.156:0/2906508417 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfbc0035b0 con 0x7fcfd0161340 2023-11-28T13:09:36.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.091+0000 7fcfcdd9b700 1 -- 172.21.15.156:0/2906508417 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1304966908 0 0) 0x7fcfd00d24f0 con 0x7fcfd00cd7c0 2023-11-28T13:09:36.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.091+0000 7fcfcdd9b700 1 -- 172.21.15.156:0/2906508417 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfd00d1950 con 0x7fcfd00cd7c0 2023-11-28T13:09:36.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.091+0000 7fcfce59c700 1 -- 172.21.15.156:0/2906508417 >> v1:172.21.15.186:6789/0 conn(0x7fcfd00cd7c0 legacy=0x7fcfd00b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.091+0000 7fcfce59c700 1 -- 172.21.15.156:0/2906508417 >> v1:172.21.15.156:6789/0 conn(0x7fcfd0161340 legacy=0x7fcfd00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.091+0000 7fcfce59c700 1 -- 172.21.15.156:0/2906508417 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd00b5520 con 0x7fcfd015c9b0 2023-11-28T13:09:36.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.091+0000 7fcfce59c700 1 --2- 172.21.15.156:0/2906508417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfd015c9b0 0x7fcfd00b4bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009870 tx=0x7fcfc0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24531b1706e34235 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:36.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.092+0000 7fcfcdd9b700 1 -- 172.21.15.156:0/2906508417 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0011830 con 0x7fcfd015c9b0 2023-11-28T13:09:36.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.092+0000 7fcfcdd9b700 1 -- 172.21.15.156:0/2906508417 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc0011990 con 0x7fcfd015c9b0 2023-11-28T13:09:36.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.092+0000 7fcfcdd9b700 1 -- 172.21.15.156:0/2906508417 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0011d20 con 0x7fcfd015c9b0 2023-11-28T13:09:36.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.092+0000 7fcfd4dff700 1 -- 172.21.15.156:0/2906508417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfd015c9b0 msgr2=0x7fcfd00b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.092+0000 7fcfd4dff700 1 --2- 172.21.15.156:0/2906508417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfd015c9b0 0x7fcfd00b4bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009870 tx=0x7fcfc0009b80 comp rx=0 tx=0).stop 2023-11-28T13:09:36.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.092+0000 7fcfd4dff700 1 -- 172.21.15.156:0/2906508417 shutdown_connections 2023-11-28T13:09:36.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.092+0000 7fcfd4dff700 1 --2- 172.21.15.156:0/2906508417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfd015c9b0 0x7fcfd00b4bf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.092+0000 7fcfd4dff700 1 -- 172.21.15.156:0/2906508417 >> 172.21.15.156:0/2906508417 conn(0x7fcfd0072600 msgr2=0x7fcfd00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:36.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.092+0000 7fcfd4dff700 1 -- 172.21.15.156:0/2906508417 shutdown_connections 2023-11-28T13:09:36.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.092+0000 7fcfd4dff700 1 -- 172.21.15.156:0/2906508417 wait complete. 2023-11-28T13:09:36.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.094+0000 7fcfd4dff700 1 Processor -- start 2023-11-28T13:09:36.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.094+0000 7fcfd4dff700 1 -- start start 2023-11-28T13:09:36.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.095+0000 7fcfd4dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd00cd7c0 0x7fcfd00c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:36.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.095+0000 7fcfd4dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfd015c9b0 0x7fcfd00c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:36.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.095+0000 7fcfd4dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfd0161340 0x7fcfd015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:36.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.095+0000 7fcfd4dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd01e7490 con 0x7fcfd00cd7c0 2023-11-28T13:09:36.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.095+0000 7fcfd4dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd01e6480 con 0x7fcfd0161340 2023-11-28T13:09:36.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.095+0000 7fcfd4dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd01e66d0 con 0x7fcfd015c9b0 2023-11-28T13:09:36.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.095+0000 7fcfce59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd00cd7c0 0x7fcfd00c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:36.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.095+0000 7fcfce59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd00cd7c0 0x7fcfd00c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36334/0 (socket says 172.21.15.156:36334) 2023-11-28T13:09:36.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.095+0000 7fcfce59c700 1 -- 172.21.15.156:0/668697383 learned_addr learned my addr 172.21.15.156:0/668697383 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:36.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.095+0000 7fcfced9d700 1 --2- 172.21.15.156:0/668697383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfd0161340 0x7fcfd015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:36.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.095+0000 7fcfc7fff700 1 --2- 172.21.15.156:0/668697383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfd015c9b0 0x7fcfd00c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:36.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.096+0000 7fcfce59c700 1 -- 172.21.15.156:0/668697383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfd015c9b0 msgr2=0x7fcfd00c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.096+0000 7fcfce59c700 1 --2- 172.21.15.156:0/668697383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfd015c9b0 0x7fcfd00c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.096+0000 7fcfce59c700 1 -- 172.21.15.156:0/668697383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfd0161340 msgr2=0x7fcfd015a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.096+0000 7fcfce59c700 1 --2- 172.21.15.156:0/668697383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfd0161340 0x7fcfd015a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.096+0000 7fcfce59c700 1 -- 172.21.15.156:0/668697383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc00094a0 con 0x7fcfd00cd7c0 2023-11-28T13:09:36.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.096+0000 7fcfce59c700 1 --2- 172.21.15.156:0/668697383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd00cd7c0 0x7fcfd00c99f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0014040 tx=0x7fcfc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:36.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.096+0000 7fcfc6ffd700 1 -- 172.21.15.156:0/668697383 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc001a860 con 0x7fcfd00cd7c0 2023-11-28T13:09:36.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.096+0000 7fcfc6ffd700 1 -- 172.21.15.156:0/668697383 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc001a9c0 con 0x7fcfd00cd7c0 2023-11-28T13:09:36.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.096+0000 7fcfd4dff700 1 -- 172.21.15.156:0/668697383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd00d15e0 con 0x7fcfd00cd7c0 2023-11-28T13:09:36.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.096+0000 7fcfc6ffd700 1 -- 172.21.15.156:0/668697383 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc001ad30 con 0x7fcfd00cd7c0 2023-11-28T13:09:36.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.096+0000 7fcfd4dff700 1 -- 172.21.15.156:0/668697383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd00d1300 con 0x7fcfd00cd7c0 2023-11-28T13:09:36.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.097+0000 7fcfd4dff700 1 -- 172.21.15.156:0/668697383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd00b5130 con 0x7fcfd00cd7c0 2023-11-28T13:09:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.098+0000 7fcfc6ffd700 1 -- 172.21.15.156:0/668697383 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcfc002b030 con 0x7fcfd00cd7c0 2023-11-28T13:09:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.099+0000 7fcfc6ffd700 1 --2- 172.21.15.156:0/668697383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcfb00682f0 0x7fcfb006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.099+0000 7fcfc6ffd700 1 -- 172.21.15.156:0/668697383 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcfc0086fc0 con 0x7fcfd00cd7c0 2023-11-28T13:09:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.099+0000 7fcfc7fff700 1 --2- 172.21.15.156:0/668697383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcfb00682f0 0x7fcfb006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:36.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.100+0000 7fcfc7fff700 1 --2- 172.21.15.156:0/668697383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcfb00682f0 0x7fcfb006a7a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcfd015a3b0 tx=0x7fcfb8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:36.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.103+0000 7fcfc6ffd700 1 -- 172.21.15.156:0/668697383 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcfc0053730 con 0x7fcfd00cd7c0 2023-11-28T13:09:36.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.294+0000 7fcfd4dff700 1 -- 172.21.15.156:0/668697383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcfd00745c0 con 0x7fcfd00cd7c0 2023-11-28T13:09:36.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.294+0000 7fcfc6ffd700 1 -- 172.21.15.156:0/668697383 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcfc00290a0 con 0x7fcfd00cd7c0 2023-11-28T13:09:36.296 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T13:09:36.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.297+0000 7fcfd4dff700 1 -- 172.21.15.156:0/668697383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcfb00682f0 msgr2=0x7fcfb006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.297+0000 7fcfd4dff700 1 --2- 172.21.15.156:0/668697383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcfb00682f0 0x7fcfb006a7a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcfd015a3b0 tx=0x7fcfb8009040 comp rx=0 tx=0).stop 2023-11-28T13:09:36.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.297+0000 7fcfd4dff700 1 -- 172.21.15.156:0/668697383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd00cd7c0 msgr2=0x7fcfd00c99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.297+0000 7fcfd4dff700 1 --2- 172.21.15.156:0/668697383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd00cd7c0 0x7fcfd00c99f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0014040 tx=0x7fcfc0004750 comp rx=0 tx=0).stop 2023-11-28T13:09:36.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.297+0000 7fcfd4dff700 1 -- 172.21.15.156:0/668697383 shutdown_connections 2023-11-28T13:09:36.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.297+0000 7fcfd4dff700 1 --2- 172.21.15.156:0/668697383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcfb00682f0 0x7fcfb006a7a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.297+0000 7fcfd4dff700 1 --2- 172.21.15.156:0/668697383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfd0161340 0x7fcfd015a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.297+0000 7fcfd4dff700 1 --2- 172.21.15.156:0/668697383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd00cd7c0 0x7fcfd00c99f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.297+0000 7fcfd4dff700 1 --2- 172.21.15.156:0/668697383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfd015c9b0 0x7fcfd00c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.297+0000 7fcfd4dff700 1 -- 172.21.15.156:0/668697383 >> 172.21.15.156:0/668697383 conn(0x7fcfd0072600 msgr2=0x7fcfd00b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:36.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.297+0000 7fcfd4dff700 1 -- 172.21.15.156:0/668697383 shutdown_connections 2023-11-28T13:09:36.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.297+0000 7fcfd4dff700 1 -- 172.21.15.156:0/668697383 wait complete. 2023-11-28T13:09:36.316 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:09:36.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.515+0000 7efe8bbff700 1 Processor -- start 2023-11-28T13:09:36.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.515+0000 7efe8bbff700 1 -- start start 2023-11-28T13:09:36.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.516+0000 7efe8bbff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe8415c9b0 0x7efe840b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:36.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.516+0000 7efe8bbff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe840d1950 con 0x7efe840cd7c0 2023-11-28T13:09:36.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.516+0000 7efe8bbff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe840d24f0 con 0x7efe84161340 2023-11-28T13:09:36.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.516+0000 7efe8bbff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe840b5800 con 0x7efe8415c9b0 2023-11-28T13:09:36.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.516+0000 7efe8a19c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7efe840cd7c0 0x7efe840b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53260/0 (socket says 172.21.15.156:53260) 2023-11-28T13:09:36.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.516+0000 7efe8a19c700 1 -- 172.21.15.156:0/1559729743 learned_addr learned my addr 172.21.15.156:0/1559729743 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:36.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.517+0000 7efe8999b700 1 --2- 172.21.15.156:0/1559729743 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe8415c9b0 0x7efe840b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:36.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.517+0000 7efe88999700 1 -- 172.21.15.156:0/1559729743 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589078679 0 0) 0x7efe840d1950 con 0x7efe840cd7c0 2023-11-28T13:09:36.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.517+0000 7efe88999700 1 -- 172.21.15.156:0/1559729743 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe700035b0 con 0x7efe840cd7c0 2023-11-28T13:09:36.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.517+0000 7efe88999700 1 -- 172.21.15.156:0/1559729743 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2805082634 0 0) 0x7efe840d24f0 con 0x7efe84161340 2023-11-28T13:09:36.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.517+0000 7efe88999700 1 -- 172.21.15.156:0/1559729743 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe840d1950 con 0x7efe84161340 2023-11-28T13:09:36.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.517+0000 7efe8999b700 1 -- 172.21.15.156:0/1559729743 >> v1:172.21.15.186:6789/0 conn(0x7efe84161340 legacy=0x7efe840d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.517+0000 7efe8999b700 1 -- 172.21.15.156:0/1559729743 >> v1:172.21.15.156:6789/0 conn(0x7efe840cd7c0 legacy=0x7efe840b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.517+0000 7efe8999b700 1 -- 172.21.15.156:0/1559729743 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe840b5520 con 0x7efe8415c9b0 2023-11-28T13:09:36.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.517+0000 7efe8999b700 1 --2- 172.21.15.156:0/1559729743 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe8415c9b0 0x7efe840b4bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efe80009870 tx=0x7efe80009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21a53b3fc71544aa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:36.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.518+0000 7efe88999700 1 -- 172.21.15.156:0/1559729743 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe80011830 con 0x7efe8415c9b0 2023-11-28T13:09:36.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.518+0000 7efe88999700 1 -- 172.21.15.156:0/1559729743 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe80011990 con 0x7efe8415c9b0 2023-11-28T13:09:36.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.518+0000 7efe88999700 1 -- 172.21.15.156:0/1559729743 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe80011d20 con 0x7efe8415c9b0 2023-11-28T13:09:36.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.518+0000 7efe8bbff700 1 -- 172.21.15.156:0/1559729743 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe8415c9b0 msgr2=0x7efe840b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.518+0000 7efe8bbff700 1 --2- 172.21.15.156:0/1559729743 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe8415c9b0 0x7efe840b4bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efe80009870 tx=0x7efe80009b80 comp rx=0 tx=0).stop 2023-11-28T13:09:36.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.518+0000 7efe8bbff700 1 -- 172.21.15.156:0/1559729743 shutdown_connections 2023-11-28T13:09:36.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.518+0000 7efe8bbff700 1 --2- 172.21.15.156:0/1559729743 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe8415c9b0 0x7efe840b4bf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.518+0000 7efe8bbff700 1 -- 172.21.15.156:0/1559729743 >> 172.21.15.156:0/1559729743 conn(0x7efe84072600 msgr2=0x7efe840cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:36.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.518+0000 7efe8bbff700 1 -- 172.21.15.156:0/1559729743 shutdown_connections 2023-11-28T13:09:36.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.518+0000 7efe8bbff700 1 -- 172.21.15.156:0/1559729743 wait complete. 2023-11-28T13:09:36.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.520+0000 7efe8bbff700 1 Processor -- start 2023-11-28T13:09:36.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.520+0000 7efe8bbff700 1 -- start start 2023-11-28T13:09:36.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.521+0000 7efe8bbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe840cd7c0 0x7efe840c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:36.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.521+0000 7efe8bbff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe8415c9b0 0x7efe841576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:36.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.521+0000 7efe8bbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe84161340 0x7efe841e7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:36.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.521+0000 7efe8bbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe840c9af0 con 0x7efe84161340 2023-11-28T13:09:36.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.521+0000 7efe8bbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe84157c00 con 0x7efe840cd7c0 2023-11-28T13:09:36.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.521+0000 7efe8bbff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe840c8f70 con 0x7efe8415c9b0 2023-11-28T13:09:36.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.521+0000 7efe8a19c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe84161340 0x7efe841e7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:36.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.521+0000 7efe8a19c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe84161340 0x7efe841e7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36342/0 (socket says 172.21.15.156:36342) 2023-11-28T13:09:36.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.521+0000 7efe8a19c700 1 -- 172.21.15.156:0/3666729804 learned_addr learned my addr 172.21.15.156:0/3666729804 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.521+0000 7efe8999b700 1 --2- 172.21.15.156:0/3666729804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe840cd7c0 0x7efe840c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.521+0000 7efe8919a700 1 --2- 172.21.15.156:0/3666729804 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe8415c9b0 0x7efe841576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.522+0000 7efe8a19c700 1 -- 172.21.15.156:0/3666729804 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe8415c9b0 msgr2=0x7efe841576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.522+0000 7efe8a19c700 1 --2- 172.21.15.156:0/3666729804 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe8415c9b0 0x7efe841576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.522+0000 7efe8a19c700 1 -- 172.21.15.156:0/3666729804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe840cd7c0 msgr2=0x7efe840c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.522+0000 7efe8a19c700 1 --2- 172.21.15.156:0/3666729804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe840cd7c0 0x7efe840c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.522+0000 7efe8a19c700 1 -- 172.21.15.156:0/3666729804 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe800094a0 con 0x7efe84161340 2023-11-28T13:09:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.522+0000 7efe8a19c700 1 --2- 172.21.15.156:0/3666729804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe84161340 0x7efe841e7620 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00cad0 tx=0x7efe7c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.522+0000 7efe7a7fc700 1 -- 172.21.15.156:0/3666729804 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00a9a0 con 0x7efe84161340 2023-11-28T13:09:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.522+0000 7efe7a7fc700 1 -- 172.21.15.156:0/3666729804 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c00ab00 con 0x7efe84161340 2023-11-28T13:09:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.522+0000 7efe8bbff700 1 -- 172.21.15.156:0/3666729804 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe840d15e0 con 0x7efe84161340 2023-11-28T13:09:36.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.522+0000 7efe7a7fc700 1 -- 172.21.15.156:0/3666729804 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00ae70 con 0x7efe84161340 2023-11-28T13:09:36.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.522+0000 7efe8bbff700 1 -- 172.21.15.156:0/3666729804 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe840d1300 con 0x7efe84161340 2023-11-28T13:09:36.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.523+0000 7efe8bbff700 1 -- 172.21.15.156:0/3666729804 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe840b5130 con 0x7efe84161340 2023-11-28T13:09:36.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.524+0000 7efe7a7fc700 1 -- 172.21.15.156:0/3666729804 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efe7c019030 con 0x7efe84161340 2023-11-28T13:09:36.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.525+0000 7efe7a7fc700 1 --2- 172.21.15.156:0/3666729804 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efe680682f0 0x7efe6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:36.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.525+0000 7efe7a7fc700 1 -- 172.21.15.156:0/3666729804 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efe7c0856e0 con 0x7efe84161340 2023-11-28T13:09:36.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.526+0000 7efe8999b700 1 --2- 172.21.15.156:0/3666729804 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efe680682f0 0x7efe6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:36.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.526+0000 7efe8999b700 1 --2- 172.21.15.156:0/3666729804 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efe680682f0 0x7efe6806a7a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efe800097c0 tx=0x7efe8001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:36.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.529+0000 7efe7a7fc700 1 -- 172.21.15.156:0/3666729804 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efe7c04b5e0 con 0x7efe84161340 2023-11-28T13:09:36.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.722+0000 7efe8bbff700 1 -- 172.21.15.156:0/3666729804 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe840ca2d0 con 0x7efe84161340 2023-11-28T13:09:36.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.722+0000 7efe7a7fc700 1 -- 172.21.15.156:0/3666729804 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7efe840ca2d0 con 0x7efe84161340 2023-11-28T13:09:36.724 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T13:09:36.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.725+0000 7efe8bbff700 1 -- 172.21.15.156:0/3666729804 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efe680682f0 msgr2=0x7efe6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.725+0000 7efe8bbff700 1 --2- 172.21.15.156:0/3666729804 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efe680682f0 0x7efe6806a7a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efe800097c0 tx=0x7efe8001c040 comp rx=0 tx=0).stop 2023-11-28T13:09:36.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.725+0000 7efe8bbff700 1 -- 172.21.15.156:0/3666729804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe84161340 msgr2=0x7efe841e7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.725+0000 7efe8bbff700 1 --2- 172.21.15.156:0/3666729804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe84161340 0x7efe841e7620 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00cad0 tx=0x7efe7c00cde0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.725+0000 7efe8bbff700 1 -- 172.21.15.156:0/3666729804 shutdown_connections 2023-11-28T13:09:36.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.725+0000 7efe8bbff700 1 --2- 172.21.15.156:0/3666729804 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efe680682f0 0x7efe6806a7a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.725+0000 7efe8bbff700 1 --2- 172.21.15.156:0/3666729804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe84161340 0x7efe841e7620 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.725+0000 7efe8bbff700 1 --2- 172.21.15.156:0/3666729804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe840cd7c0 0x7efe840c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.725+0000 7efe8bbff700 1 --2- 172.21.15.156:0/3666729804 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe8415c9b0 0x7efe841576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.725+0000 7efe8bbff700 1 -- 172.21.15.156:0/3666729804 >> 172.21.15.156:0/3666729804 conn(0x7efe84072600 msgr2=0x7efe840b7160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:36.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.725+0000 7efe8bbff700 1 -- 172.21.15.156:0/3666729804 shutdown_connections 2023-11-28T13:09:36.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.725+0000 7efe8bbff700 1 -- 172.21.15.156:0/3666729804 wait complete. 2023-11-28T13:09:36.745 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:09:36.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.939+0000 7fbda25ff700 1 Processor -- start 2023-11-28T13:09:36.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.940+0000 7fbda25ff700 1 -- start start 2023-11-28T13:09:36.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.941+0000 7fbda25ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbd9c0d1330 0x7fbd9c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:36.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.941+0000 7fbda25ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbd9c0b20b0 0x7fbd9c0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:36.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.941+0000 7fbda25ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9c0b0230 con 0x7fbd9c0d01b0 2023-11-28T13:09:36.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.941+0000 7fbda25ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9c15cec0 con 0x7fbd9c0d1330 2023-11-28T13:09:36.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.941+0000 7fbda25ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9c0b4570 con 0x7fbd9c0b20b0 2023-11-28T13:09:36.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.941+0000 7fbd9b7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbd9c0d1330 0x7fbd9c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:36.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.941+0000 7fbd9b7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbd9c0d1330 0x7fbd9c0b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36358/0 (socket says 172.21.15.156:36358) 2023-11-28T13:09:36.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.941+0000 7fbd9b7fe700 1 -- 172.21.15.156:0/3425863435 learned_addr learned my addr 172.21.15.156:0/3425863435 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:36.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.941+0000 7fbda0b9c700 1 --2- 172.21.15.156:0/3425863435 >> v2:172.21.15.186:3300/0 conn(0x7fbd9c0b20b0 0x7fbd9c0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:36.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.942+0000 7fbd9b7fe700 1 -- 172.21.15.156:0/3425863435 >> v2:172.21.15.186:3300/0 conn(0x7fbd9c0b20b0 msgr2=0x7fbd9c0b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:36.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.942+0000 7fbd9b7fe700 1 --2- 172.21.15.156:0/3425863435 >> v2:172.21.15.186:3300/0 conn(0x7fbd9c0b20b0 0x7fbd9c0b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.942+0000 7fbd9b7fe700 1 -- 172.21.15.156:0/3425863435 >> v1:172.21.15.186:6789/0 conn(0x7fbd9c0d01b0 legacy=0x7fbd9c0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.942+0000 7fbd9b7fe700 1 -- 172.21.15.156:0/3425863435 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c0b4850 con 0x7fbd9c0d1330 2023-11-28T13:09:36.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.942+0000 7fbda0b9c700 1 --2- 172.21.15.156:0/3425863435 >> v2:172.21.15.186:3300/0 conn(0x7fbd9c0b20b0 0x7fbd9c0b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:36.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.942+0000 7fbd9b7fe700 1 --2- 172.21.15.156:0/3425863435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c0d1330 0x7fbd9c0b06c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c066e80 tx=0x7fbd90009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8155570b8e385c08 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:36.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.942+0000 7fbd9affd700 1 -- 172.21.15.156:0/3425863435 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000b870 con 0x7fbd9c0d1330 2023-11-28T13:09:36.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.942+0000 7fbd9affd700 1 -- 172.21.15.156:0/3425863435 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9000b9d0 con 0x7fbd9c0d1330 2023-11-28T13:09:36.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.942+0000 7fbd9affd700 1 -- 172.21.15.156:0/3425863435 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000bd60 con 0x7fbd9c0d1330 2023-11-28T13:09:36.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.943+0000 7fbda25ff700 1 -- 172.21.15.156:0/3425863435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c0d1330 msgr2=0x7fbd9c0b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.943+0000 7fbda25ff700 1 --2- 172.21.15.156:0/3425863435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c0d1330 0x7fbd9c0b06c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c066e80 tx=0x7fbd90009bd0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.943+0000 7fbda25ff700 1 -- 172.21.15.156:0/3425863435 shutdown_connections 2023-11-28T13:09:36.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.943+0000 7fbda25ff700 1 --2- 172.21.15.156:0/3425863435 >> v2:172.21.15.186:3300/0 conn(0x7fbd9c0b20b0 0x7fbd9c0b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.943+0000 7fbda25ff700 1 --2- 172.21.15.156:0/3425863435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c0d1330 0x7fbd9c0b06c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.943+0000 7fbda25ff700 1 -- 172.21.15.156:0/3425863435 >> 172.21.15.156:0/3425863435 conn(0x7fbd9c09c3f0 msgr2=0x7fbd9c0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:36.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.943+0000 7fbda25ff700 1 -- 172.21.15.156:0/3425863435 shutdown_connections 2023-11-28T13:09:36.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.943+0000 7fbda25ff700 1 -- 172.21.15.156:0/3425863435 wait complete. 2023-11-28T13:09:36.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.945+0000 7fbda25ff700 1 Processor -- start 2023-11-28T13:09:36.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.945+0000 7fbda25ff700 1 -- start start 2023-11-28T13:09:36.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.946+0000 7fbda25ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd9c0b20b0 0x7fbd9c15b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:36.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.946+0000 7fbda25ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd9c0d01b0 0x7fbd9c1f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:36.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.946+0000 7fbda25ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c0d1330 0x7fbd9c06b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:36.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.946+0000 7fbda25ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c1f3250 con 0x7fbd9c0d1330 2023-11-28T13:09:36.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.946+0000 7fbda25ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c09e5f0 con 0x7fbd9c0b20b0 2023-11-28T13:09:36.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.946+0000 7fbda25ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c1f2f70 con 0x7fbd9c0d01b0 2023-11-28T13:09:36.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.946+0000 7fbda0b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c0d1330 0x7fbd9c06b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:36.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.946+0000 7fbda0b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c0d1330 0x7fbd9c06b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36360/0 (socket says 172.21.15.156:36360) 2023-11-28T13:09:36.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.946+0000 7fbda0b9c700 1 -- 172.21.15.156:0/2717164760 learned_addr learned my addr 172.21.15.156:0/2717164760 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:36.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.946+0000 7fbd9bfff700 1 --2- 172.21.15.156:0/2717164760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd9c0b20b0 0x7fbd9c15b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:36.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.946+0000 7fbd9b7fe700 1 --2- 172.21.15.156:0/2717164760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd9c0d01b0 0x7fbd9c1f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:36.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.946+0000 7fbda0b9c700 1 -- 172.21.15.156:0/2717164760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd9c0d01b0 msgr2=0x7fbd9c1f33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.946+0000 7fbda0b9c700 1 --2- 172.21.15.156:0/2717164760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd9c0d01b0 0x7fbd9c1f33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.946+0000 7fbda0b9c700 1 -- 172.21.15.156:0/2717164760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd9c0b20b0 msgr2=0x7fbd9c15b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:36.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.946+0000 7fbda0b9c700 1 --2- 172.21.15.156:0/2717164760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd9c0b20b0 0x7fbd9c15b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:36.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.946+0000 7fbda0b9c700 1 -- 172.21.15.156:0/2717164760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd90009550 con 0x7fbd9c0d1330 2023-11-28T13:09:36.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.947+0000 7fbda0b9c700 1 --2- 172.21.15.156:0/2717164760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c0d1330 0x7fbd9c06b3f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbd94007e70 tx=0x7fbd9400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:36.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.947+0000 7fbd98ff9700 1 -- 172.21.15.156:0/2717164760 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9400dd20 con 0x7fbd9c0d1330 2023-11-28T13:09:36.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.947+0000 7fbd98ff9700 1 -- 172.21.15.156:0/2717164760 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9400f040 con 0x7fbd9c0d1330 2023-11-28T13:09:36.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.947+0000 7fbda25ff700 1 -- 172.21.15.156:0/2717164760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9c0afec0 con 0x7fbd9c0d1330 2023-11-28T13:09:36.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.947+0000 7fbd98ff9700 1 -- 172.21.15.156:0/2717164760 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94015640 con 0x7fbd9c0d1330 2023-11-28T13:09:36.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.947+0000 7fbda25ff700 1 -- 172.21.15.156:0/2717164760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9c0afbe0 con 0x7fbd9c0d1330 2023-11-28T13:09:36.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.948+0000 7fbda25ff700 1 -- 172.21.15.156:0/2717164760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9c0b2600 con 0x7fbd9c0d1330 2023-11-28T13:09:36.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.949+0000 7fbd98ff9700 1 -- 172.21.15.156:0/2717164760 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbd9401c030 con 0x7fbd9c0d1330 2023-11-28T13:09:36.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.950+0000 7fbd98ff9700 1 --2- 172.21.15.156:0/2717164760 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbd880682f0 0x7fbd8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:36.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.950+0000 7fbd98ff9700 1 -- 172.21.15.156:0/2717164760 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbd94085cc0 con 0x7fbd9c0d1330 2023-11-28T13:09:36.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.951+0000 7fbd9bfff700 1 --2- 172.21.15.156:0/2717164760 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbd880682f0 0x7fbd8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:36.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.951+0000 7fbd9bfff700 1 --2- 172.21.15.156:0/2717164760 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbd880682f0 0x7fbd8806a7a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c007fd0 tx=0x7fbd8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:36.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:36.954+0000 7fbd98ff9700 1 -- 172.21.15.156:0/2717164760 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbd9408c080 con 0x7fbd9c0d1330 2023-11-28T13:09:37.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.188+0000 7fbda25ff700 1 -- 172.21.15.156:0/2717164760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd9c09e780 con 0x7fbd9c0d1330 2023-11-28T13:09:37.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.189+0000 7fbd98ff9700 1 -- 172.21.15.156:0/2717164760 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fbd9401a090 con 0x7fbd9c0d1330 2023-11-28T13:09:37.191 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:37.191 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1128367125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1128367125},{"type":"v1","addr":"172.21.15.156:6835","nonce":1128367125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4942,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/4096182008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4096182008},{"type":"v1","addr":"172.21.15.186:6835","nonce":4096182008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4945,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/561812165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":561812165},{"type":"v1","addr":"172.21.15.186:6837","nonce":561812165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:09:08.167940+0000","modified":"2023-11-28T13:09:11.184420+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5013":{"gid":5013,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6834/3471743149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3471743149},{"type":"v1","addr":"172.21.15.156:6834","nonce":3471743149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:09:37.193 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.191+0000 7fbda25ff700 1 -- 172.21.15.156:0/2717164760 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbd880682f0 msgr2=0x7fbd8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:37.193 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.191+0000 7fbda25ff700 1 --2- 172.21.15.156:0/2717164760 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbd880682f0 0x7fbd8806a7a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c007fd0 tx=0x7fbd8c009040 comp rx=0 tx=0).stop 2023-11-28T13:09:37.193 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.191+0000 7fbda25ff700 1 -- 172.21.15.156:0/2717164760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c0d1330 msgr2=0x7fbd9c06b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:37.193 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.191+0000 7fbda25ff700 1 --2- 172.21.15.156:0/2717164760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c0d1330 0x7fbd9c06b3f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbd94007e70 tx=0x7fbd9400d3b0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.191+0000 7fbda25ff700 1 -- 172.21.15.156:0/2717164760 shutdown_connections 2023-11-28T13:09:37.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.191+0000 7fbda25ff700 1 --2- 172.21.15.156:0/2717164760 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbd880682f0 0x7fbd8806a7a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.191+0000 7fbda25ff700 1 --2- 172.21.15.156:0/2717164760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c0d1330 0x7fbd9c06b3f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.191+0000 7fbda25ff700 1 --2- 172.21.15.156:0/2717164760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd9c0b20b0 0x7fbd9c15b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.191+0000 7fbda25ff700 1 --2- 172.21.15.156:0/2717164760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd9c0d01b0 0x7fbd9c1f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.191+0000 7fbda25ff700 1 -- 172.21.15.156:0/2717164760 >> 172.21.15.156:0/2717164760 conn(0x7fbd9c09c3f0 msgr2=0x7fbd9c15c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:37.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.192+0000 7fbda25ff700 1 -- 172.21.15.156:0/2717164760 shutdown_connections 2023-11-28T13:09:37.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.192+0000 7fbda25ff700 1 -- 172.21.15.156:0/2717164760 wait complete. 2023-11-28T13:09:37.197 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2023-11-28T13:09:37.212 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:09:37.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.407+0000 7f79725ff700 1 Processor -- start 2023-11-28T13:09:37.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.408+0000 7f79725ff700 1 -- start start 2023-11-28T13:09:37.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.409+0000 7f79725ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f796c15c9b0 0x7f796c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:37.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.409+0000 7f79725ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f796c0cd7c0 0x7f796c0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:37.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.409+0000 7f79725ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796c0ce160 con 0x7f796c161340 2023-11-28T13:09:37.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.409+0000 7f79725ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f796c0b5800 con 0x7f796c0cd7c0 2023-11-28T13:09:37.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.409+0000 7f79725ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796c0d1950 con 0x7f796c15c9b0 2023-11-28T13:09:37.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.409+0000 7f7970b9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f796c0cd7c0 0x7f796c0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:37.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.409+0000 7f7970b9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f796c0cd7c0 0x7f796c0cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36362/0 (socket says 172.21.15.156:36362) 2023-11-28T13:09:37.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.409+0000 7f7970b9c700 1 -- 172.21.15.156:0/1350460546 learned_addr learned my addr 172.21.15.156:0/1350460546 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:37.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.409+0000 7f796bfff700 1 --2- 172.21.15.156:0/1350460546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f796c15c9b0 0x7f796c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:37.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.410+0000 7f7970b9c700 1 -- 172.21.15.156:0/1350460546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f796c15c9b0 msgr2=0x7f796c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:37.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.410+0000 7f7970b9c700 1 --2- 172.21.15.156:0/1350460546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f796c15c9b0 0x7f796c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.410+0000 7f7970b9c700 1 -- 172.21.15.156:0/1350460546 >> v1:172.21.15.186:6789/0 conn(0x7f796c161340 legacy=0x7f796c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:37.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.410+0000 7f7970b9c700 1 -- 172.21.15.156:0/1350460546 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c0d24f0 con 0x7f796c0cd7c0 2023-11-28T13:09:37.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.410+0000 7f796bfff700 1 --2- 172.21.15.156:0/1350460546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f796c15c9b0 0x7f796c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:37.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.410+0000 7f7970b9c700 1 --2- 172.21.15.156:0/1350460546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f796c0cd7c0 0x7f796c0cdc20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7964009e50 tx=0x7f796400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f48cdb5ec8d54c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:37.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.411+0000 7f796affd700 1 -- 172.21.15.156:0/1350460546 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796400e070 con 0x7f796c0cd7c0 2023-11-28T13:09:37.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.411+0000 7f796affd700 1 -- 172.21.15.156:0/1350460546 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796400b9c0 con 0x7f796c0cd7c0 2023-11-28T13:09:37.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.411+0000 7f796affd700 1 -- 172.21.15.156:0/1350460546 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796400bd70 con 0x7f796c0cd7c0 2023-11-28T13:09:37.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.411+0000 7f79725ff700 1 -- 172.21.15.156:0/1350460546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f796c0cd7c0 msgr2=0x7f796c0cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:37.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.411+0000 7f79725ff700 1 --2- 172.21.15.156:0/1350460546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f796c0cd7c0 0x7f796c0cdc20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7964009e50 tx=0x7f796400b330 comp rx=0 tx=0).stop 2023-11-28T13:09:37.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.411+0000 7f79725ff700 1 -- 172.21.15.156:0/1350460546 shutdown_connections 2023-11-28T13:09:37.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.411+0000 7f79725ff700 1 --2- 172.21.15.156:0/1350460546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f796c0cd7c0 0x7f796c0cdc20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.411+0000 7f79725ff700 1 --2- 172.21.15.156:0/1350460546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f796c15c9b0 0x7f796c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.411+0000 7f79725ff700 1 -- 172.21.15.156:0/1350460546 >> 172.21.15.156:0/1350460546 conn(0x7f796c072600 msgr2=0x7f796c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:37.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.411+0000 7f79725ff700 1 -- 172.21.15.156:0/1350460546 shutdown_connections 2023-11-28T13:09:37.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.412+0000 7f79725ff700 1 -- 172.21.15.156:0/1350460546 wait complete. 2023-11-28T13:09:37.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.413+0000 7f79725ff700 1 Processor -- start 2023-11-28T13:09:37.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.413+0000 7f79725ff700 1 -- start start 2023-11-28T13:09:37.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.414+0000 7f79725ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f796c0cd7c0 0x7f796c0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:37.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.414+0000 7f79725ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f796c15c9b0 0x7f796c1e7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:37.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.415+0000 7f79725ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f796c161340 0x7f796c0ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:37.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.415+0000 7f79725ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c1e7d90 con 0x7f796c0cd7c0 2023-11-28T13:09:37.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.415+0000 7f79725ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c0cc6e0 con 0x7f796c15c9b0 2023-11-28T13:09:37.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.415+0000 7f79725ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796c0cd0b0 con 0x7f796c161340 2023-11-28T13:09:37.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.415+0000 7f796bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f796c0cd7c0 0x7f796c0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:37.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.415+0000 7f796bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f796c0cd7c0 0x7f796c0cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36372/0 (socket says 172.21.15.156:36372) 2023-11-28T13:09:37.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.415+0000 7f796bfff700 1 -- 172.21.15.156:0/3517429310 learned_addr learned my addr 172.21.15.156:0/3517429310 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:37.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.415+0000 7f7970b9c700 1 --2- 172.21.15.156:0/3517429310 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f796c161340 0x7f796c0ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:37.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.415+0000 7f796b7fe700 1 --2- 172.21.15.156:0/3517429310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f796c15c9b0 0x7f796c1e7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:37.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.415+0000 7f796bfff700 1 -- 172.21.15.156:0/3517429310 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f796c161340 msgr2=0x7f796c0ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:37.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.415+0000 7f796bfff700 1 --2- 172.21.15.156:0/3517429310 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f796c161340 0x7f796c0ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.415+0000 7f796bfff700 1 -- 172.21.15.156:0/3517429310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f796c15c9b0 msgr2=0x7f796c1e7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:37.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.415+0000 7f796bfff700 1 --2- 172.21.15.156:0/3517429310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f796c15c9b0 0x7f796c1e7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.415+0000 7f796bfff700 1 -- 172.21.15.156:0/3517429310 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7964009b30 con 0x7f796c0cd7c0 2023-11-28T13:09:37.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.415+0000 7f796b7fe700 1 --2- 172.21.15.156:0/3517429310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f796c15c9b0 0x7f796c1e7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:37.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.415+0000 7f796bfff700 1 --2- 172.21.15.156:0/3517429310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f796c0cd7c0 0x7f796c0cb4f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f795c00e760 tx=0x7f795c00ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:37.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.416+0000 7f7968ff9700 1 -- 172.21.15.156:0/3517429310 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c013610 con 0x7f796c0cd7c0 2023-11-28T13:09:37.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.416+0000 7f7968ff9700 1 -- 172.21.15.156:0/3517429310 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795c013770 con 0x7f796c0cd7c0 2023-11-28T13:09:37.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.416+0000 7f7968ff9700 1 -- 172.21.15.156:0/3517429310 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c013b20 con 0x7f796c0cd7c0 2023-11-28T13:09:37.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.416+0000 7f79725ff700 1 -- 172.21.15.156:0/3517429310 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796c0d1300 con 0x7f796c0cd7c0 2023-11-28T13:09:37.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.416+0000 7f79725ff700 1 -- 172.21.15.156:0/3517429310 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796c0b5520 con 0x7f796c0cd7c0 2023-11-28T13:09:37.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.417+0000 7f79725ff700 1 -- 172.21.15.156:0/3517429310 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796c0d15e0 con 0x7f796c0cd7c0 2023-11-28T13:09:37.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.418+0000 7f7968ff9700 1 -- 172.21.15.156:0/3517429310 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f795c023070 con 0x7f796c0cd7c0 2023-11-28T13:09:37.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.419+0000 7f7968ff9700 1 --2- 172.21.15.156:0/3517429310 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f79580682f0 0x7f795806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:37.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.419+0000 7f7968ff9700 1 -- 172.21.15.156:0/3517429310 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f795c085820 con 0x7f796c0cd7c0 2023-11-28T13:09:37.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.419+0000 7f796b7fe700 1 --2- 172.21.15.156:0/3517429310 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f79580682f0 0x7f795806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:37.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.420+0000 7f796b7fe700 1 --2- 172.21.15.156:0/3517429310 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f79580682f0 0x7f795806a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f796c15f500 tx=0x7f7960009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:37.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.423+0000 7f7968ff9700 1 -- 172.21.15.156:0/3517429310 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f795c052080 con 0x7f796c0cd7c0 2023-11-28T13:09:37.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.616+0000 7f79725ff700 1 -- 172.21.15.156:0/3517429310 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f796c067b10 con 0x7f796c0cd7c0 2023-11-28T13:09:37.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.618+0000 7f7968ff9700 1 -- 172.21.15.156:0/3517429310 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+17969 (secure 0 0 0) 0x7f795c0556a0 con 0x7f796c0cd7c0 2023-11-28T13:09:37.619 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:37.620 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":35,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:09:09.170289+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:09:06.618698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T13:09:07.151554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:09:37.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.620+0000 7f79725ff700 1 -- 172.21.15.156:0/3517429310 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f79580682f0 msgr2=0x7f795806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:37.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.620+0000 7f79725ff700 1 --2- 172.21.15.156:0/3517429310 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f79580682f0 0x7f795806a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f796c15f500 tx=0x7f7960009040 comp rx=0 tx=0).stop 2023-11-28T13:09:37.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.620+0000 7f79725ff700 1 -- 172.21.15.156:0/3517429310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f796c0cd7c0 msgr2=0x7f796c0cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:37.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.620+0000 7f79725ff700 1 --2- 172.21.15.156:0/3517429310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f796c0cd7c0 0x7f796c0cb4f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f795c00e760 tx=0x7f795c00ea70 comp rx=0 tx=0).stop 2023-11-28T13:09:37.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.620+0000 7f79725ff700 1 -- 172.21.15.156:0/3517429310 shutdown_connections 2023-11-28T13:09:37.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.620+0000 7f79725ff700 1 --2- 172.21.15.156:0/3517429310 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f79580682f0 0x7f795806a7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.620+0000 7f79725ff700 1 --2- 172.21.15.156:0/3517429310 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f796c161340 0x7f796c0ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.620+0000 7f79725ff700 1 --2- 172.21.15.156:0/3517429310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f796c0cd7c0 0x7f796c0cb4f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.620+0000 7f79725ff700 1 --2- 172.21.15.156:0/3517429310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f796c15c9b0 0x7f796c1e7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.620+0000 7f79725ff700 1 -- 172.21.15.156:0/3517429310 >> 172.21.15.156:0/3517429310 conn(0x7f796c072600 msgr2=0x7f796c0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:37.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.620+0000 7f79725ff700 1 -- 172.21.15.156:0/3517429310 shutdown_connections 2023-11-28T13:09:37.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.620+0000 7f79725ff700 1 -- 172.21.15.156:0/3517429310 wait complete. 2023-11-28T13:09:37.640 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T13:09:37.641 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:09:37.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.837+0000 7f18c7dff700 1 Processor -- start 2023-11-28T13:09:37.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.838+0000 7f18c7dff700 1 -- start start 2023-11-28T13:09:37.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.839+0000 7f18c7dff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f18c0159860 0x7f18c00c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:37.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.839+0000 7f18c7dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18c00b0d80 0x7f18c00c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:37.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.839+0000 7f18c7dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c00ccc00 con 0x7f18c0159490 2023-11-28T13:09:37.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.839+0000 7f18c7dff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c00ccec0 con 0x7f18c0159860 2023-11-28T13:09:37.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.839+0000 7f18c7dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c00cd1a0 con 0x7f18c00b0d80 2023-11-28T13:09:37.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.839+0000 7f18c539a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f18c0159860 0x7f18c00c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:37.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.839+0000 7f18c539a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f18c0159860 0x7f18c00c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36376/0 (socket says 172.21.15.156:36376) 2023-11-28T13:09:37.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.839+0000 7f18c539a700 1 -- 172.21.15.156:0/2072114219 learned_addr learned my addr 172.21.15.156:0/2072114219 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:37.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.839+0000 7f18c639c700 1 --2- 172.21.15.156:0/2072114219 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18c00b0d80 0x7f18c00c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:37.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.839+0000 7f18c539a700 1 -- 172.21.15.156:0/2072114219 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18c00b0d80 msgr2=0x7f18c00c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:37.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.839+0000 7f18c539a700 1 --2- 172.21.15.156:0/2072114219 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18c00b0d80 0x7f18c00c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.839+0000 7f18c539a700 1 -- 172.21.15.156:0/2072114219 >> v1:172.21.15.186:6789/0 conn(0x7f18c0159490 legacy=0x7f18c00b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:37.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.839+0000 7f18c539a700 1 -- 172.21.15.156:0/2072114219 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c00cd480 con 0x7f18c0159860 2023-11-28T13:09:37.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.840+0000 7f18c539a700 1 --2- 172.21.15.156:0/2072114219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18c0159860 0x7f18c00c94c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f18b000bb90 tx=0x7f18b000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a140c69cc25e3155 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:37.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.840+0000 7f18c4b99700 1 -- 172.21.15.156:0/2072114219 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b000dd50 con 0x7f18c0159860 2023-11-28T13:09:37.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.840+0000 7f18c4b99700 1 -- 172.21.15.156:0/2072114219 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b0010470 con 0x7f18c0159860 2023-11-28T13:09:37.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.840+0000 7f18c4b99700 1 -- 172.21.15.156:0/2072114219 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b0010800 con 0x7f18c0159860 2023-11-28T13:09:37.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.840+0000 7f18c7dff700 1 -- 172.21.15.156:0/2072114219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18c0159860 msgr2=0x7f18c00c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:37.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.840+0000 7f18c7dff700 1 --2- 172.21.15.156:0/2072114219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18c0159860 0x7f18c00c94c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f18b000bb90 tx=0x7f18b000bea0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.841+0000 7f18c7dff700 1 -- 172.21.15.156:0/2072114219 shutdown_connections 2023-11-28T13:09:37.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.841+0000 7f18c7dff700 1 --2- 172.21.15.156:0/2072114219 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18c00b0d80 0x7f18c00c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.841+0000 7f18c7dff700 1 --2- 172.21.15.156:0/2072114219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18c0159860 0x7f18c00c94c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.841+0000 7f18c7dff700 1 -- 172.21.15.156:0/2072114219 >> 172.21.15.156:0/2072114219 conn(0x7f18c00cb670 msgr2=0x7f18c00cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:37.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.841+0000 7f18c7dff700 1 -- 172.21.15.156:0/2072114219 shutdown_connections 2023-11-28T13:09:37.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.841+0000 7f18c7dff700 1 -- 172.21.15.156:0/2072114219 wait complete. 2023-11-28T13:09:37.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.842+0000 7f18c7dff700 1 Processor -- start 2023-11-28T13:09:37.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.843+0000 7f18c7dff700 1 -- start start 2023-11-28T13:09:37.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.843+0000 7f18c7dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18c00b0d80 0x7f18c015d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:37.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.843+0000 7f18c7dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18c0159490 0x7f18c00c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:37.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.844+0000 7f18c7dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18c00b0200 0x7f18c01f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:37.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.844+0000 7f18c7dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c00b2b20 con 0x7f18c00b0200 2023-11-28T13:09:37.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.844+0000 7f18c7dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c01f2b40 con 0x7f18c0159490 2023-11-28T13:09:37.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.844+0000 7f18c7dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c01f2550 con 0x7f18c00b0d80 2023-11-28T13:09:37.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.844+0000 7f18c5b9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18c00b0d80 0x7f18c015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:37.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.844+0000 7f18c639c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18c00b0200 0x7f18c01f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:37.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.844+0000 7f18c5b9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18c00b0d80 0x7f18c015d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:43424/0 (socket says 172.21.15.156:43424) 2023-11-28T13:09:37.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.844+0000 7f18c5b9b700 1 -- 172.21.15.156:0/1585540462 learned_addr learned my addr 172.21.15.156:0/1585540462 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:37.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.844+0000 7f18c539a700 1 --2- 172.21.15.156:0/1585540462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18c0159490 0x7f18c00c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:37.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.844+0000 7f18c639c700 1 -- 172.21.15.156:0/1585540462 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18c00b0d80 msgr2=0x7f18c015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:37.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.844+0000 7f18c639c700 1 --2- 172.21.15.156:0/1585540462 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18c00b0d80 0x7f18c015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.844+0000 7f18c639c700 1 -- 172.21.15.156:0/1585540462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18c0159490 msgr2=0x7f18c00c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:37.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.844+0000 7f18c639c700 1 --2- 172.21.15.156:0/1585540462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18c0159490 0x7f18c00c3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:37.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.844+0000 7f18c639c700 1 -- 172.21.15.156:0/1585540462 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b000b7f0 con 0x7f18c00b0200 2023-11-28T13:09:37.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.844+0000 7f18c639c700 1 --2- 172.21.15.156:0/1585540462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18c00b0200 0x7f18c01f2f60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f18b8007d90 tx=0x7f18b800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:37.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.845+0000 7f18b67fc700 1 -- 172.21.15.156:0/1585540462 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8018070 con 0x7f18c00b0200 2023-11-28T13:09:37.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.845+0000 7f18b67fc700 1 -- 172.21.15.156:0/1585540462 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b800ead0 con 0x7f18c00b0200 2023-11-28T13:09:37.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.845+0000 7f18b67fc700 1 -- 172.21.15.156:0/1585540462 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b800ee60 con 0x7f18c00b0200 2023-11-28T13:09:37.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.845+0000 7f18c7dff700 1 -- 172.21.15.156:0/1585540462 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c00c9f40 con 0x7f18c00b0200 2023-11-28T13:09:37.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.845+0000 7f18c7dff700 1 -- 172.21.15.156:0/1585540462 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c00cd760 con 0x7f18c00b0200 2023-11-28T13:09:37.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.846+0000 7f18c7dff700 1 -- 172.21.15.156:0/1585540462 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c00afe30 con 0x7f18c00b0200 2023-11-28T13:09:37.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.847+0000 7f18b67fc700 1 -- 172.21.15.156:0/1585540462 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f18b8012050 con 0x7f18c00b0200 2023-11-28T13:09:37.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.848+0000 7f18b67fc700 1 --2- 172.21.15.156:0/1585540462 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f18a40682f0 0x7f18a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:37.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.848+0000 7f18b67fc700 1 -- 172.21.15.156:0/1585540462 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f18b80858f0 con 0x7f18c00b0200 2023-11-28T13:09:37.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.848+0000 7f18c5b9b700 1 --2- 172.21.15.156:0/1585540462 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f18a40682f0 0x7f18a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:37.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.849+0000 7f18c5b9b700 1 --2- 172.21.15.156:0/1585540462 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f18a40682f0 0x7f18a406a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f18bc001920 tx=0x7f18bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:37.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:37.852+0000 7f18b67fc700 1 -- 172.21.15.156:0/1585540462 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f18b8052060 con 0x7f18c00b0200 2023-11-28T13:09:38.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.085+0000 7f18c7dff700 1 -- 172.21.15.156:0/1585540462 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18c00cdc80 con 0x7f18c00b0200 2023-11-28T13:09:38.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.086+0000 7f18b67fc700 1 -- 172.21.15.156:0/1585540462 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f18b8055680 con 0x7f18c00b0200 2023-11-28T13:09:38.087 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:38.088 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1128367125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1128367125},{"type":"v1","addr":"172.21.15.156:6835","nonce":1128367125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4942,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/4096182008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4096182008},{"type":"v1","addr":"172.21.15.186:6835","nonce":4096182008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4945,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/561812165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":561812165},{"type":"v1","addr":"172.21.15.186:6837","nonce":561812165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:09:08.167940+0000","modified":"2023-11-28T13:09:11.184420+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5013":{"gid":5013,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6834/3471743149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3471743149},{"type":"v1","addr":"172.21.15.156:6834","nonce":3471743149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:09:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.088+0000 7f18c7dff700 1 -- 172.21.15.156:0/1585540462 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f18a40682f0 msgr2=0x7f18a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.088+0000 7f18c7dff700 1 --2- 172.21.15.156:0/1585540462 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f18a40682f0 0x7f18a406a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f18bc001920 tx=0x7f18bc009040 comp rx=0 tx=0).stop 2023-11-28T13:09:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.088+0000 7f18c7dff700 1 -- 172.21.15.156:0/1585540462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18c00b0200 msgr2=0x7f18c01f2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.088+0000 7f18c7dff700 1 --2- 172.21.15.156:0/1585540462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18c00b0200 0x7f18c01f2f60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f18b8007d90 tx=0x7f18b800e3c0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.088+0000 7f18c7dff700 1 -- 172.21.15.156:0/1585540462 shutdown_connections 2023-11-28T13:09:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.088+0000 7f18c7dff700 1 --2- 172.21.15.156:0/1585540462 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f18a40682f0 0x7f18a406a7a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.088+0000 7f18c7dff700 1 --2- 172.21.15.156:0/1585540462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18c00b0200 0x7f18c01f2f60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.088+0000 7f18c7dff700 1 --2- 172.21.15.156:0/1585540462 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18c00b0d80 0x7f18c015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.088+0000 7f18c7dff700 1 --2- 172.21.15.156:0/1585540462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18c0159490 0x7f18c00c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.088+0000 7f18c7dff700 1 -- 172.21.15.156:0/1585540462 >> 172.21.15.156:0/1585540462 conn(0x7f18c00cb670 msgr2=0x7f18c0153af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.088+0000 7f18c7dff700 1 -- 172.21.15.156:0/1585540462 shutdown_connections 2023-11-28T13:09:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.088+0000 7f18c7dff700 1 -- 172.21.15.156:0/1585540462 wait complete. 2023-11-28T13:09:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2023-11-28T13:09:38.108 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:09:38.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.302+0000 7f17969ff700 1 Processor -- start 2023-11-28T13:09:38.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.302+0000 7f17969ff700 1 -- start start 2023-11-28T13:09:38.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.303+0000 7f17969ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f179015cd30 0x7f1790161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:38.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.303+0000 7f17969ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f17900cfda0 0x7f17900d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:38.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.303+0000 7f17969ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f179006a270 0x7f179006a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:38.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.303+0000 7f17969ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f179006ac50 con 0x7f17900cfda0 2023-11-28T13:09:38.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.303+0000 7f17969ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17900d0840 con 0x7f179015cd30 2023-11-28T13:09:38.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.303+0000 7f17969ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17900d0b20 con 0x7f179006a270 2023-11-28T13:09:38.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.304+0000 7f178f7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f17900cfda0 0x7f17900d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:38.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.304+0000 7f178f7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f17900cfda0 0x7f17900d0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36404/0 (socket says 172.21.15.156:36404) 2023-11-28T13:09:38.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.304+0000 7f178f7fe700 1 -- 172.21.15.156:0/3435172606 learned_addr learned my addr 172.21.15.156:0/3435172606 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:38.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.304+0000 7f178ffff700 1 --2- 172.21.15.156:0/3435172606 >> v2:172.21.15.186:3300/0 conn(0x7f179015cd30 0x7f1790161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:38.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.304+0000 7f1794f9c700 1 --2- 172.21.15.156:0/3435172606 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f179006a270 0x7f179006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:38.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.304+0000 7f178f7fe700 1 -- 172.21.15.156:0/3435172606 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f179006a270 msgr2=0x7f179006a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:38.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.304+0000 7f178f7fe700 1 --2- 172.21.15.156:0/3435172606 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f179006a270 0x7f179006a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.304+0000 7f178f7fe700 1 -- 172.21.15.156:0/3435172606 >> v2:172.21.15.186:3300/0 conn(0x7f179015cd30 msgr2=0x7f1790161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:38.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.304+0000 7f178f7fe700 1 --2- 172.21.15.156:0/3435172606 >> v2:172.21.15.186:3300/0 conn(0x7f179015cd30 0x7f1790161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.304+0000 7f178f7fe700 1 -- 172.21.15.156:0/3435172606 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1790160670 con 0x7f17900cfda0 2023-11-28T13:09:38.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.304+0000 7f178ffff700 1 --2- 172.21.15.156:0/3435172606 >> v2:172.21.15.186:3300/0 conn(0x7f179015cd30 0x7f1790161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:38.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.304+0000 7f178f7fe700 1 --2- 172.21.15.156:0/3435172606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17900cfda0 0x7f17900d0240 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f17840098f0 tx=0x7f1784009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973a55a610b31aa6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:38.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.305+0000 7f178effd700 1 -- 172.21.15.156:0/3435172606 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178400b860 con 0x7f17900cfda0 2023-11-28T13:09:38.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.305+0000 7f178effd700 1 -- 172.21.15.156:0/3435172606 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178400b9c0 con 0x7f17900cfda0 2023-11-28T13:09:38.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.305+0000 7f178effd700 1 -- 172.21.15.156:0/3435172606 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178400bd50 con 0x7f17900cfda0 2023-11-28T13:09:38.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.305+0000 7f17969ff700 1 -- 172.21.15.156:0/3435172606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17900cfda0 msgr2=0x7f17900d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:38.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.305+0000 7f17969ff700 1 --2- 172.21.15.156:0/3435172606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17900cfda0 0x7f17900d0240 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f17840098f0 tx=0x7f1784009c00 comp rx=0 tx=0).stop 2023-11-28T13:09:38.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.305+0000 7f17969ff700 1 -- 172.21.15.156:0/3435172606 shutdown_connections 2023-11-28T13:09:38.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.305+0000 7f17969ff700 1 --2- 172.21.15.156:0/3435172606 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f179006a270 0x7f179006a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.305+0000 7f17969ff700 1 --2- 172.21.15.156:0/3435172606 >> v2:172.21.15.186:3300/0 conn(0x7f179015cd30 0x7f1790161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.305+0000 7f17969ff700 1 --2- 172.21.15.156:0/3435172606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17900cfda0 0x7f17900d0240 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.305+0000 7f17969ff700 1 -- 172.21.15.156:0/3435172606 >> 172.21.15.156:0/3435172606 conn(0x7f179006b380 msgr2=0x7f179006b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:38.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.305+0000 7f17969ff700 1 -- 172.21.15.156:0/3435172606 shutdown_connections 2023-11-28T13:09:38.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.305+0000 7f17969ff700 1 -- 172.21.15.156:0/3435172606 wait complete. 2023-11-28T13:09:38.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.307+0000 7f17969ff700 1 Processor -- start 2023-11-28T13:09:38.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.307+0000 7f17969ff700 1 -- start start 2023-11-28T13:09:38.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.308+0000 7f17969ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f179006a270 0x7f17900c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:38.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.308+0000 7f17969ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17900cfda0 0x7f179014f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:38.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.308+0000 7f17969ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f179015cd30 0x7f1790157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:38.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.308+0000 7f17969ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17900c54e0 con 0x7f179015cd30 2023-11-28T13:09:38.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.308+0000 7f17969ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17900c48d0 con 0x7f179006a270 2023-11-28T13:09:38.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.308+0000 7f17969ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17900c4bb0 con 0x7f17900cfda0 2023-11-28T13:09:38.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.308+0000 7f1794f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f179015cd30 0x7f1790157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:38.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.308+0000 7f1794f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f179015cd30 0x7f1790157950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36418/0 (socket says 172.21.15.156:36418) 2023-11-28T13:09:38.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.309+0000 7f1794f9c700 1 -- 172.21.15.156:0/3045351069 learned_addr learned my addr 172.21.15.156:0/3045351069 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:38.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.309+0000 7f178ffff700 1 --2- 172.21.15.156:0/3045351069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f179006a270 0x7f17900c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:38.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.309+0000 7f178f7fe700 1 --2- 172.21.15.156:0/3045351069 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17900cfda0 0x7f179014f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:38.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.309+0000 7f1794f9c700 1 -- 172.21.15.156:0/3045351069 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17900cfda0 msgr2=0x7f179014f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:38.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.309+0000 7f1794f9c700 1 --2- 172.21.15.156:0/3045351069 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17900cfda0 0x7f179014f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.309+0000 7f1794f9c700 1 -- 172.21.15.156:0/3045351069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f179006a270 msgr2=0x7f17900c42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:38.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.309+0000 7f1794f9c700 1 --2- 172.21.15.156:0/3045351069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f179006a270 0x7f17900c42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.309+0000 7f1794f9c700 1 -- 172.21.15.156:0/3045351069 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1784009550 con 0x7f179015cd30 2023-11-28T13:09:38.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.309+0000 7f1794f9c700 1 --2- 172.21.15.156:0/3045351069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f179015cd30 0x7f1790157950 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1788007e60 tx=0x7f178800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:38.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.309+0000 7f178cff9700 1 -- 172.21.15.156:0/3045351069 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178800ed30 con 0x7f179015cd30 2023-11-28T13:09:38.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.309+0000 7f178cff9700 1 -- 172.21.15.156:0/3045351069 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1788014420 con 0x7f179015cd30 2023-11-28T13:09:38.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.309+0000 7f17969ff700 1 -- 172.21.15.156:0/3045351069 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1790160c30 con 0x7f179015cd30 2023-11-28T13:09:38.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.309+0000 7f178cff9700 1 -- 172.21.15.156:0/3045351069 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17880147d0 con 0x7f179015cd30 2023-11-28T13:09:38.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.309+0000 7f17969ff700 1 -- 172.21.15.156:0/3045351069 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1790160950 con 0x7f179015cd30 2023-11-28T13:09:38.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.310+0000 7f17969ff700 1 -- 172.21.15.156:0/3045351069 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17900cc280 con 0x7f179015cd30 2023-11-28T13:09:38.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.311+0000 7f178cff9700 1 -- 172.21.15.156:0/3045351069 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1788012050 con 0x7f179015cd30 2023-11-28T13:09:38.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.312+0000 7f178cff9700 1 --2- 172.21.15.156:0/3045351069 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f177c0682f0 0x7f177c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:38.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.312+0000 7f178cff9700 1 -- 172.21.15.156:0/3045351069 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1788086280 con 0x7f179015cd30 2023-11-28T13:09:38.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.313+0000 7f178ffff700 1 --2- 172.21.15.156:0/3045351069 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f177c0682f0 0x7f177c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:38.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.313+0000 7f178ffff700 1 --2- 172.21.15.156:0/3045351069 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f177c0682f0 0x7f177c06a7a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f178000c940 tx=0x7f1780008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:38.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.316+0000 7f178cff9700 1 -- 172.21.15.156:0/3045351069 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f17880529f0 con 0x7f179015cd30 2023-11-28T13:09:38.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.547+0000 7f17969ff700 1 -- 172.21.15.156:0/3045351069 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17900745c0 con 0x7f179015cd30 2023-11-28T13:09:38.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.548+0000 7f178cff9700 1 -- 172.21.15.156:0/3045351069 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f1788056010 con 0x7f179015cd30 2023-11-28T13:09:38.550 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:38.550 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1128367125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1128367125},{"type":"v1","addr":"172.21.15.156:6835","nonce":1128367125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4942,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/4096182008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4096182008},{"type":"v1","addr":"172.21.15.186:6835","nonce":4096182008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4945,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/561812165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":561812165},{"type":"v1","addr":"172.21.15.186:6837","nonce":561812165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:09:08.167940+0000","modified":"2023-11-28T13:09:11.184420+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5013":{"gid":5013,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6834/3471743149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3471743149},{"type":"v1","addr":"172.21.15.156:6834","nonce":3471743149}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:09:38.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.550+0000 7f17969ff700 1 -- 172.21.15.156:0/3045351069 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f177c0682f0 msgr2=0x7f177c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:38.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.550+0000 7f17969ff700 1 --2- 172.21.15.156:0/3045351069 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f177c0682f0 0x7f177c06a7a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f178000c940 tx=0x7f1780008040 comp rx=0 tx=0).stop 2023-11-28T13:09:38.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.550+0000 7f17969ff700 1 -- 172.21.15.156:0/3045351069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f179015cd30 msgr2=0x7f1790157950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:38.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.550+0000 7f17969ff700 1 --2- 172.21.15.156:0/3045351069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f179015cd30 0x7f1790157950 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1788007e60 tx=0x7f178800e3c0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.550+0000 7f17969ff700 1 -- 172.21.15.156:0/3045351069 shutdown_connections 2023-11-28T13:09:38.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.550+0000 7f17969ff700 1 --2- 172.21.15.156:0/3045351069 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f177c0682f0 0x7f177c06a7a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.550+0000 7f17969ff700 1 --2- 172.21.15.156:0/3045351069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f179015cd30 0x7f1790157950 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.550+0000 7f17969ff700 1 --2- 172.21.15.156:0/3045351069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f179006a270 0x7f17900c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.550+0000 7f17969ff700 1 --2- 172.21.15.156:0/3045351069 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f17900cfda0 0x7f179014f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.550+0000 7f17969ff700 1 -- 172.21.15.156:0/3045351069 >> 172.21.15.156:0/3045351069 conn(0x7f179006b380 msgr2=0x7f17900714e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:38.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.550+0000 7f17969ff700 1 -- 172.21.15.156:0/3045351069 shutdown_connections 2023-11-28T13:09:38.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.550+0000 7f17969ff700 1 -- 172.21.15.156:0/3045351069 wait complete. 2023-11-28T13:09:38.557 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2023-11-28T13:09:38.571 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:09:38.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.769+0000 7f009afff700 1 Processor -- start 2023-11-28T13:09:38.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.769+0000 7f009afff700 1 -- start start 2023-11-28T13:09:38.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.770+0000 7f009afff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f009415c9b0 0x7f00940b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:38.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.770+0000 7f009afff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0094161340 0x7f00940d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:38.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.770+0000 7f009afff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00940d1950 con 0x7f00940d2320 2023-11-28T13:09:38.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.770+0000 7f009afff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00940b5800 con 0x7f0094161340 2023-11-28T13:09:38.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.770+0000 7f009afff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00940b5520 con 0x7f009415c9b0 2023-11-28T13:09:38.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.770+0000 7f0093fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0094161340 0x7f00940d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:38.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.770+0000 7f0093fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0094161340 0x7f00940d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36424/0 (socket says 172.21.15.156:36424) 2023-11-28T13:09:38.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.770+0000 7f0093fff700 1 -- 172.21.15.156:0/3229637413 learned_addr learned my addr 172.21.15.156:0/3229637413 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:38.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.771+0000 7f0098d9b700 1 --2- 172.21.15.156:0/3229637413 >> v2:172.21.15.186:3300/0 conn(0x7f009415c9b0 0x7f00940b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:38.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.771+0000 7f0093fff700 1 -- 172.21.15.156:0/3229637413 >> v2:172.21.15.186:3300/0 conn(0x7f009415c9b0 msgr2=0x7f00940b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:38.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.771+0000 7f0093fff700 1 --2- 172.21.15.156:0/3229637413 >> v2:172.21.15.186:3300/0 conn(0x7f009415c9b0 0x7f00940b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.771+0000 7f0093fff700 1 -- 172.21.15.156:0/3229637413 >> v1:172.21.15.186:6789/0 conn(0x7f00940d2320 legacy=0x7f00940cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:38.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.771+0000 7f0093fff700 1 -- 172.21.15.156:0/3229637413 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00940d1300 con 0x7f0094161340 2023-11-28T13:09:38.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.771+0000 7f0093fff700 1 --2- 172.21.15.156:0/3229637413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0094161340 0x7f00940d1de0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0094067460 tx=0x7f0080009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0db360a4c08c7bb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:38.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.771+0000 7f00937fe700 1 -- 172.21.15.156:0/3229637413 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008000b870 con 0x7f0094161340 2023-11-28T13:09:38.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.771+0000 7f00937fe700 1 -- 172.21.15.156:0/3229637413 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008000b9d0 con 0x7f0094161340 2023-11-28T13:09:38.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.771+0000 7f00937fe700 1 -- 172.21.15.156:0/3229637413 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008000bd60 con 0x7f0094161340 2023-11-28T13:09:38.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.772+0000 7f009afff700 1 -- 172.21.15.156:0/3229637413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0094161340 msgr2=0x7f00940d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:38.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.772+0000 7f009afff700 1 --2- 172.21.15.156:0/3229637413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0094161340 0x7f00940d1de0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0094067460 tx=0x7f0080009bd0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.772+0000 7f009afff700 1 -- 172.21.15.156:0/3229637413 shutdown_connections 2023-11-28T13:09:38.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.772+0000 7f009afff700 1 --2- 172.21.15.156:0/3229637413 >> v2:172.21.15.186:3300/0 conn(0x7f009415c9b0 0x7f00940b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.772+0000 7f009afff700 1 --2- 172.21.15.156:0/3229637413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0094161340 0x7f00940d1de0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.772+0000 7f009afff700 1 -- 172.21.15.156:0/3229637413 >> 172.21.15.156:0/3229637413 conn(0x7f0094072600 msgr2=0x7f00940cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:38.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.772+0000 7f009afff700 1 -- 172.21.15.156:0/3229637413 shutdown_connections 2023-11-28T13:09:38.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.772+0000 7f009afff700 1 -- 172.21.15.156:0/3229637413 wait complete. 2023-11-28T13:09:38.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.774+0000 7f009afff700 1 Processor -- start 2023-11-28T13:09:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.774+0000 7f009afff700 1 -- start start 2023-11-28T13:09:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.775+0000 7f009afff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00940d2320 0x7f00940c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.775+0000 7f009afff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f009415c9b0 0x7f00941f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.775+0000 7f009afff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0094161340 0x7f009406a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.775+0000 7f009afff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00940c65a0 con 0x7f00940d2320 2023-11-28T13:09:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.775+0000 7f009afff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0094154970 con 0x7f0094161340 2023-11-28T13:09:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.775+0000 7f009afff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00940c8830 con 0x7f009415c9b0 2023-11-28T13:09:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.775+0000 7f0098d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00940d2320 0x7f00940c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.775+0000 7f0098d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00940d2320 0x7f00940c6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36432/0 (socket says 172.21.15.156:36432) 2023-11-28T13:09:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.775+0000 7f0098d9b700 1 -- 172.21.15.156:0/1479441778 learned_addr learned my addr 172.21.15.156:0/1479441778 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.775+0000 7f0093fff700 1 --2- 172.21.15.156:0/1479441778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f009415c9b0 0x7f00941f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.775+0000 7f009959c700 1 --2- 172.21.15.156:0/1479441778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0094161340 0x7f009406a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.775+0000 7f0098d9b700 1 -- 172.21.15.156:0/1479441778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f009415c9b0 msgr2=0x7f00941f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.775+0000 7f0098d9b700 1 --2- 172.21.15.156:0/1479441778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f009415c9b0 0x7f00941f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.775+0000 7f0098d9b700 1 -- 172.21.15.156:0/1479441778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0094161340 msgr2=0x7f009406a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.775+0000 7f0098d9b700 1 --2- 172.21.15.156:0/1479441778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0094161340 0x7f009406a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.775+0000 7f0098d9b700 1 -- 172.21.15.156:0/1479441778 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0080009550 con 0x7f00940d2320 2023-11-28T13:09:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.775+0000 7f0093fff700 1 --2- 172.21.15.156:0/1479441778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f009415c9b0 0x7f00941f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.776+0000 7f0098d9b700 1 --2- 172.21.15.156:0/1479441778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00940d2320 0x7f00940c6060 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f008800e810 tx=0x7f008800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.776+0000 7f00917fa700 1 -- 172.21.15.156:0/1479441778 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088013610 con 0x7f00940d2320 2023-11-28T13:09:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.776+0000 7f00917fa700 1 -- 172.21.15.156:0/1479441778 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0088013770 con 0x7f00940d2320 2023-11-28T13:09:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.776+0000 7f009afff700 1 -- 172.21.15.156:0/1479441778 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00940b5130 con 0x7f00940d2320 2023-11-28T13:09:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.776+0000 7f00917fa700 1 -- 172.21.15.156:0/1479441778 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088013b20 con 0x7f00940d2320 2023-11-28T13:09:38.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.776+0000 7f009afff700 1 -- 172.21.15.156:0/1479441778 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00940d15e0 con 0x7f00940d2320 2023-11-28T13:09:38.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.777+0000 7f009afff700 1 -- 172.21.15.156:0/1479441778 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00940b52c0 con 0x7f00940d2320 2023-11-28T13:09:38.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.778+0000 7f00917fa700 1 -- 172.21.15.156:0/1479441778 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0088023070 con 0x7f00940d2320 2023-11-28T13:09:38.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.779+0000 7f00917fa700 1 --2- 172.21.15.156:0/1479441778 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f00780682f0 0x7f007806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:38.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.779+0000 7f00917fa700 1 -- 172.21.15.156:0/1479441778 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0088085a90 con 0x7f00940d2320 2023-11-28T13:09:38.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.779+0000 7f0093fff700 1 --2- 172.21.15.156:0/1479441778 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f00780682f0 0x7f007806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:38.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.780+0000 7f0093fff700 1 --2- 172.21.15.156:0/1479441778 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f00780682f0 0x7f007806a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0080000c00 tx=0x7f0080008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:38.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.782+0000 7f00917fa700 1 -- 172.21.15.156:0/1479441778 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0088052180 con 0x7f00940d2320 2023-11-28T13:09:38.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.981+0000 7f009afff700 1 -- 172.21.15.156:0/1479441778 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00940745c0 con 0x7f00940d2320 2023-11-28T13:09:38.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.982+0000 7f00917fa700 1 -- 172.21.15.156:0/1479441778 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+17969 (secure 0 0 0) 0x7f00880557a0 con 0x7f00940d2320 2023-11-28T13:09:38.984 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:38.984 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":35,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:09:09.170289+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:09:06.618698+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T13:09:07.151554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:09:38.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.984+0000 7f009afff700 1 -- 172.21.15.156:0/1479441778 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f00780682f0 msgr2=0x7f007806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:38.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.984+0000 7f009afff700 1 --2- 172.21.15.156:0/1479441778 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f00780682f0 0x7f007806a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0080000c00 tx=0x7f0080008030 comp rx=0 tx=0).stop 2023-11-28T13:09:38.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.984+0000 7f009afff700 1 -- 172.21.15.156:0/1479441778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00940d2320 msgr2=0x7f00940c6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:38.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.984+0000 7f009afff700 1 --2- 172.21.15.156:0/1479441778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00940d2320 0x7f00940c6060 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f008800e810 tx=0x7f008800ebd0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.984+0000 7f009afff700 1 -- 172.21.15.156:0/1479441778 shutdown_connections 2023-11-28T13:09:38.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.984+0000 7f009afff700 1 --2- 172.21.15.156:0/1479441778 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f00780682f0 0x7f007806a7a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.984+0000 7f009afff700 1 --2- 172.21.15.156:0/1479441778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0094161340 0x7f009406a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.984+0000 7f009afff700 1 --2- 172.21.15.156:0/1479441778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00940d2320 0x7f00940c6060 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.984+0000 7f009afff700 1 --2- 172.21.15.156:0/1479441778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f009415c9b0 0x7f00941f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:38.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.985+0000 7f009afff700 1 -- 172.21.15.156:0/1479441778 >> 172.21.15.156:0/1479441778 conn(0x7f0094072600 msgr2=0x7f00940b8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:38.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.985+0000 7f009afff700 1 -- 172.21.15.156:0/1479441778 shutdown_connections 2023-11-28T13:09:38.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:38.985+0000 7f009afff700 1 -- 172.21.15.156:0/1479441778 wait complete. 2023-11-28T13:09:39.005 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T13:09:39.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.199+0000 7f35271ff700 1 Processor -- start 2023-11-28T13:09:39.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.200+0000 7f35271ff700 1 -- start start 2023-11-28T13:09:39.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.201+0000 7f35271ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f352015c9b0 0x7f35200b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:39.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.201+0000 7f35271ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3520161340 0x7f35200d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:39.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.201+0000 7f35271ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35200d1950 con 0x7f35200d2320 2023-11-28T13:09:39.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.201+0000 7f35271ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35200b5800 con 0x7f352015c9b0 2023-11-28T13:09:39.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.201+0000 7f35271ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35200b5520 con 0x7f3520161340 2023-11-28T13:09:39.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.201+0000 7f3524f9b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f352015c9b0 0x7f35200b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:39.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.201+0000 7f351ffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3520161340 0x7f35200d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:39.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.201+0000 7f3524f9b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f352015c9b0 0x7f35200b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:49912/0 (socket says 172.21.15.156:49912) 2023-11-28T13:09:39.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.201+0000 7f3524f9b700 1 -- 172.21.15.156:0/1248401839 learned_addr learned my addr 172.21.15.156:0/1248401839 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:39.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.201+0000 7f352579c700 1 --1- 172.21.15.156:0/1248401839 >> v1:172.21.15.186:6789/0 conn(0x7f35200d2320 0x7f35200cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:42014/0 (socket says 172.21.15.156:42014) 2023-11-28T13:09:39.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.202+0000 7f351f7fe700 1 -- 172.21.15.156:0/1248401839 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2700965080 0 0) 0x7f35200d1950 con 0x7f35200d2320 2023-11-28T13:09:39.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.202+0000 7f351f7fe700 1 -- 172.21.15.156:0/1248401839 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35100035b0 con 0x7f35200d2320 2023-11-28T13:09:39.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.202+0000 7f351ffff700 1 -- 172.21.15.156:0/1248401839 >> v2:172.21.15.186:3300/0 conn(0x7f352015c9b0 msgr2=0x7f35200b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:39.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.202+0000 7f351ffff700 1 --2- 172.21.15.156:0/1248401839 >> v2:172.21.15.186:3300/0 conn(0x7f352015c9b0 0x7f35200b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:39.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.202+0000 7f351ffff700 1 -- 172.21.15.156:0/1248401839 >> v1:172.21.15.186:6789/0 conn(0x7f35200d2320 legacy=0x7f35200cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:39.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.202+0000 7f351ffff700 1 -- 172.21.15.156:0/1248401839 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35200d1300 con 0x7f3520161340 2023-11-28T13:09:39.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.202+0000 7f3524f9b700 1 --2- 172.21.15.156:0/1248401839 >> v2:172.21.15.186:3300/0 conn(0x7f352015c9b0 0x7f35200b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:09:39.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.202+0000 7f351ffff700 1 --2- 172.21.15.156:0/1248401839 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3520161340 0x7f35200d1de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f350c009840 tx=0x7f350c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca94c9feb262ff5a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:39.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.202+0000 7f351f7fe700 1 -- 172.21.15.156:0/1248401839 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c011830 con 0x7f3520161340 2023-11-28T13:09:39.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.203+0000 7f351f7fe700 1 -- 172.21.15.156:0/1248401839 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350c011990 con 0x7f3520161340 2023-11-28T13:09:39.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.203+0000 7f351f7fe700 1 -- 172.21.15.156:0/1248401839 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c011d00 con 0x7f3520161340 2023-11-28T13:09:39.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.203+0000 7f35271ff700 1 -- 172.21.15.156:0/1248401839 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3520161340 msgr2=0x7f35200d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:39.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.203+0000 7f35271ff700 1 --2- 172.21.15.156:0/1248401839 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3520161340 0x7f35200d1de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f350c009840 tx=0x7f350c009b50 comp rx=0 tx=0).stop 2023-11-28T13:09:39.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.203+0000 7f35271ff700 1 -- 172.21.15.156:0/1248401839 shutdown_connections 2023-11-28T13:09:39.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.203+0000 7f35271ff700 1 --2- 172.21.15.156:0/1248401839 >> v2:172.21.15.186:3300/0 conn(0x7f352015c9b0 0x7f35200b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:39.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.203+0000 7f35271ff700 1 --2- 172.21.15.156:0/1248401839 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3520161340 0x7f35200d1de0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:39.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.203+0000 7f35271ff700 1 -- 172.21.15.156:0/1248401839 >> 172.21.15.156:0/1248401839 conn(0x7f3520072600 msgr2=0x7f35200cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:39.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.203+0000 7f35271ff700 1 -- 172.21.15.156:0/1248401839 shutdown_connections 2023-11-28T13:09:39.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.203+0000 7f35271ff700 1 -- 172.21.15.156:0/1248401839 wait complete. 2023-11-28T13:09:39.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.205+0000 7f35271ff700 1 Processor -- start 2023-11-28T13:09:39.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.205+0000 7f35271ff700 1 -- start start 2023-11-28T13:09:39.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.206+0000 7f35271ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35200d2320 0x7f35201f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:39.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.206+0000 7f35271ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f352015c9b0 0x7f35200b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:39.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.206+0000 7f35271ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3520161340 0x7f35200b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:39.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.206+0000 7f35271ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35200b6700 con 0x7f3520161340 2023-11-28T13:09:39.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.206+0000 7f35271ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3520150f20 con 0x7f35200d2320 2023-11-28T13:09:39.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.206+0000 7f35271ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35201f2fd0 con 0x7f352015c9b0 2023-11-28T13:09:39.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.206+0000 7f352579c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3520161340 0x7f35200b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:39.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.206+0000 7f352579c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3520161340 0x7f35200b74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36444/0 (socket says 172.21.15.156:36444) 2023-11-28T13:09:39.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.206+0000 7f352579c700 1 -- 172.21.15.156:0/2738592167 learned_addr learned my addr 172.21.15.156:0/2738592167 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:39.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.206+0000 7f3524f9b700 1 --2- 172.21.15.156:0/2738592167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35200d2320 0x7f35201f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:39.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.206+0000 7f351ffff700 1 --2- 172.21.15.156:0/2738592167 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f352015c9b0 0x7f35200b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:39.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.207+0000 7f352579c700 1 -- 172.21.15.156:0/2738592167 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f352015c9b0 msgr2=0x7f35200b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:39.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.207+0000 7f352579c700 1 --2- 172.21.15.156:0/2738592167 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f352015c9b0 0x7f35200b61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:39.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.207+0000 7f352579c700 1 -- 172.21.15.156:0/2738592167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35200d2320 msgr2=0x7f35201f29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:39.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.207+0000 7f352579c700 1 --2- 172.21.15.156:0/2738592167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35200d2320 0x7f35201f29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:39.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.207+0000 7f352579c700 1 -- 172.21.15.156:0/2738592167 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3514009550 con 0x7f3520161340 2023-11-28T13:09:39.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.207+0000 7f352579c700 1 --2- 172.21.15.156:0/2738592167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3520161340 0x7f35200b74c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f351800cbc0 tx=0x7f351800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:39.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.207+0000 7f351d7fa700 1 -- 172.21.15.156:0/2738592167 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351800aa40 con 0x7f3520161340 2023-11-28T13:09:39.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.207+0000 7f351d7fa700 1 -- 172.21.15.156:0/2738592167 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351800aba0 con 0x7f3520161340 2023-11-28T13:09:39.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.207+0000 7f35271ff700 1 -- 172.21.15.156:0/2738592167 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350c0094a0 con 0x7f3520161340 2023-11-28T13:09:39.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.207+0000 7f351d7fa700 1 -- 172.21.15.156:0/2738592167 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518007070 con 0x7f3520161340 2023-11-28T13:09:39.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.207+0000 7f35271ff700 1 -- 172.21.15.156:0/2738592167 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35200b5130 con 0x7f3520161340 2023-11-28T13:09:39.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.208+0000 7f35271ff700 1 -- 172.21.15.156:0/2738592167 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35200d15e0 con 0x7f3520161340 2023-11-28T13:09:39.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.209+0000 7f351d7fa700 1 -- 172.21.15.156:0/2738592167 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3518019030 con 0x7f3520161340 2023-11-28T13:09:39.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.210+0000 7f351d7fa700 1 --2- 172.21.15.156:0/2738592167 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35040682f0 0x7f350406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:39.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.211+0000 7f351d7fa700 1 -- 172.21.15.156:0/2738592167 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3518085a10 con 0x7f3520161340 2023-11-28T13:09:39.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.211+0000 7f3524f9b700 1 --2- 172.21.15.156:0/2738592167 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35040682f0 0x7f350406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:39.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.211+0000 7f3524f9b700 1 --2- 172.21.15.156:0/2738592167 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35040682f0 0x7f350406a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3514006d30 tx=0x7f35140092c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:39.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.214+0000 7f351d7fa700 1 -- 172.21.15.156:0/2738592167 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3518052100 con 0x7f3520161340 2023-11-28T13:09:39.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.450+0000 7f35271ff700 1 -- 172.21.15.156:0/2738592167 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f35200b52c0 con 0x7f3520161340 2023-11-28T13:09:39.617 INFO:tasks.ceph.mon.a.smithi156.stderr:2023-11-28T13:09:39.616+0000 7f61ad1f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T13:09:39.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.619+0000 7f351d7fa700 1 -- 172.21.15.156:0/2738592167 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3518017020 con 0x7f3520161340 2023-11-28T13:09:39.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.621+0000 7f35271ff700 1 -- 172.21.15.156:0/2738592167 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35040682f0 msgr2=0x7f350406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:39.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.621+0000 7f35271ff700 1 --2- 172.21.15.156:0/2738592167 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35040682f0 0x7f350406a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3514006d30 tx=0x7f35140092c0 comp rx=0 tx=0).stop 2023-11-28T13:09:39.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.621+0000 7f35271ff700 1 -- 172.21.15.156:0/2738592167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3520161340 msgr2=0x7f35200b74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:39.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.621+0000 7f35271ff700 1 --2- 172.21.15.156:0/2738592167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3520161340 0x7f35200b74c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f351800cbc0 tx=0x7f351800ced0 comp rx=0 tx=0).stop 2023-11-28T13:09:39.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.622+0000 7f35271ff700 1 -- 172.21.15.156:0/2738592167 shutdown_connections 2023-11-28T13:09:39.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.622+0000 7f35271ff700 1 --2- 172.21.15.156:0/2738592167 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f35040682f0 0x7f350406a7a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:39.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.622+0000 7f35271ff700 1 --2- 172.21.15.156:0/2738592167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3520161340 0x7f35200b74c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:39.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.622+0000 7f35271ff700 1 --2- 172.21.15.156:0/2738592167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35200d2320 0x7f35201f29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:39.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.622+0000 7f35271ff700 1 --2- 172.21.15.156:0/2738592167 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f352015c9b0 0x7f35200b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:39.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.622+0000 7f35271ff700 1 -- 172.21.15.156:0/2738592167 >> 172.21.15.156:0/2738592167 conn(0x7f3520072600 msgr2=0x7f3520160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:39.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.622+0000 7f35271ff700 1 -- 172.21.15.156:0/2738592167 shutdown_connections 2023-11-28T13:09:39.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.622+0000 7f35271ff700 1 -- 172.21.15.156:0/2738592167 wait complete. 2023-11-28T13:09:39.625 INFO:teuthology.orchestra.run.smithi156.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T13:09:39.633 INFO:tasks.ceph.mds.c.smithi156.stderr: -7281> 2023-11-28T13:09:10.590+0000 7f01f9ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:09:39.644 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T13:09:39.688 INFO:tasks.ceph.mds.c.smithi156.stdout:starting mds.c at 2023-11-28T13:09:39.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.846+0000 7fc9ebbff700 1 Processor -- start 2023-11-28T13:09:39.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.846+0000 7fc9ebbff700 1 -- start start 2023-11-28T13:09:39.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.847+0000 7fc9ebbff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc9e415c9b0 0x7fc9e40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:39.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.847+0000 7fc9ebbff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9e4161340 0x7fc9e40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:39.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.847+0000 7fc9ebbff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e40d1950 con 0x7fc9e40d2320 2023-11-28T13:09:39.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.847+0000 7fc9ebbff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e40b5800 con 0x7fc9e415c9b0 2023-11-28T13:09:39.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.847+0000 7fc9ebbff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40b5520 con 0x7fc9e4161340 2023-11-28T13:09:39.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.847+0000 7fc9ea19c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fc9e40d2320 0x7fc9e40cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53274/0 (socket says 172.21.15.156:53274) 2023-11-28T13:09:39.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.847+0000 7fc9ea19c700 1 -- 172.21.15.156:0/2434586542 learned_addr learned my addr 172.21.15.156:0/2434586542 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:39.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.847+0000 7fc9e999b700 1 --2- 172.21.15.156:0/2434586542 >> v2:172.21.15.186:3300/0 conn(0x7fc9e415c9b0 0x7fc9e40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:39.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.847+0000 7fc9e919a700 1 --2- 172.21.15.156:0/2434586542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9e4161340 0x7fc9e40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:39.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.848+0000 7fc9e8999700 1 -- 172.21.15.156:0/2434586542 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940232051 0 0) 0x7fc9e40d1950 con 0x7fc9e40d2320 2023-11-28T13:09:39.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.848+0000 7fc9e8999700 1 -- 172.21.15.156:0/2434586542 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9d00035b0 con 0x7fc9e40d2320 2023-11-28T13:09:39.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.848+0000 7fc9e919a700 1 -- 172.21.15.156:0/2434586542 >> v2:172.21.15.186:3300/0 conn(0x7fc9e415c9b0 msgr2=0x7fc9e40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:39.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.848+0000 7fc9e8999700 1 -- 172.21.15.156:0/2434586542 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2346629891 0 0) 0x7fc9d00035b0 con 0x7fc9e40d2320 2023-11-28T13:09:39.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.848+0000 7fc9e919a700 1 --2- 172.21.15.156:0/2434586542 >> v2:172.21.15.186:3300/0 conn(0x7fc9e415c9b0 0x7fc9e40b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:39.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.848+0000 7fc9e919a700 1 -- 172.21.15.156:0/2434586542 >> v1:172.21.15.156:6789/0 conn(0x7fc9e40d2320 legacy=0x7fc9e40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:39.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.848+0000 7fc9e919a700 1 -- 172.21.15.156:0/2434586542 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e40d1300 con 0x7fc9e4161340 2023-11-28T13:09:39.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.848+0000 7fc9e999b700 1 --2- 172.21.15.156:0/2434586542 >> v2:172.21.15.186:3300/0 conn(0x7fc9e415c9b0 0x7fc9e40b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:09:39.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.848+0000 7fc9e919a700 1 --2- 172.21.15.156:0/2434586542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9e4161340 0x7fc9e40d1de0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009840 tx=0x7fc9d4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91c27bd127961e78 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:39.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.849+0000 7fc9e8999700 1 -- 172.21.15.156:0/2434586542 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0019f0 con 0x7fc9e4161340 2023-11-28T13:09:39.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.849+0000 7fc9e8999700 1 -- 172.21.15.156:0/2434586542 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d4004030 con 0x7fc9e4161340 2023-11-28T13:09:39.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.849+0000 7fc9e8999700 1 -- 172.21.15.156:0/2434586542 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40043c0 con 0x7fc9e4161340 2023-11-28T13:09:39.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.849+0000 7fc9ebbff700 1 -- 172.21.15.156:0/2434586542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9e4161340 msgr2=0x7fc9e40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:39.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.849+0000 7fc9ebbff700 1 --2- 172.21.15.156:0/2434586542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9e4161340 0x7fc9e40d1de0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009840 tx=0x7fc9d4009b50 comp rx=0 tx=0).stop 2023-11-28T13:09:39.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.849+0000 7fc9ebbff700 1 -- 172.21.15.156:0/2434586542 shutdown_connections 2023-11-28T13:09:39.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.849+0000 7fc9ebbff700 1 --2- 172.21.15.156:0/2434586542 >> v2:172.21.15.186:3300/0 conn(0x7fc9e415c9b0 0x7fc9e40b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:39.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.849+0000 7fc9ebbff700 1 --2- 172.21.15.156:0/2434586542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9e4161340 0x7fc9e40d1de0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:39.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.849+0000 7fc9ebbff700 1 -- 172.21.15.156:0/2434586542 >> 172.21.15.156:0/2434586542 conn(0x7fc9e4072600 msgr2=0x7fc9e40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:39.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.849+0000 7fc9ebbff700 1 -- 172.21.15.156:0/2434586542 shutdown_connections 2023-11-28T13:09:39.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.849+0000 7fc9ebbff700 1 -- 172.21.15.156:0/2434586542 wait complete. 2023-11-28T13:09:39.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.851+0000 7fc9ebbff700 1 Processor -- start 2023-11-28T13:09:39.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.851+0000 7fc9ebbff700 1 -- start start 2023-11-28T13:09:39.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.852+0000 7fc9ebbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9e40d2320 0x7fc9e415fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:39.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.852+0000 7fc9ebbff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9e415c9b0 0x7fc9e415dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:39.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.852+0000 7fc9ebbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e4161340 0x7fc9e414e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:39.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.852+0000 7fc9ebbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e41f6b20 con 0x7fc9e4161340 2023-11-28T13:09:39.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.852+0000 7fc9ebbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e41600b0 con 0x7fc9e40d2320 2023-11-28T13:09:39.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.852+0000 7fc9ebbff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e415e150 con 0x7fc9e415c9b0 2023-11-28T13:09:39.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.852+0000 7fc9ea19c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e4161340 0x7fc9e414e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:39.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.852+0000 7fc9e999b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9e40d2320 0x7fc9e415fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:39.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.852+0000 7fc9e919a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9e415c9b0 0x7fc9e415dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:39.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.852+0000 7fc9ea19c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e4161340 0x7fc9e414e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36468/0 (socket says 172.21.15.156:36468) 2023-11-28T13:09:39.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.852+0000 7fc9ea19c700 1 -- 172.21.15.156:0/312506045 learned_addr learned my addr 172.21.15.156:0/312506045 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:39.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.853+0000 7fc9ea19c700 1 -- 172.21.15.156:0/312506045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9e415c9b0 msgr2=0x7fc9e415dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:39.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.853+0000 7fc9ea19c700 1 --2- 172.21.15.156:0/312506045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9e415c9b0 0x7fc9e415dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:39.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.853+0000 7fc9ea19c700 1 -- 172.21.15.156:0/312506045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9e40d2320 msgr2=0x7fc9e415fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:39.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.853+0000 7fc9ea19c700 1 --2- 172.21.15.156:0/312506045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9e40d2320 0x7fc9e415fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:39.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.853+0000 7fc9ea19c700 1 -- 172.21.15.156:0/312506045 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e00094a0 con 0x7fc9e4161340 2023-11-28T13:09:39.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.853+0000 7fc9ea19c700 1 --2- 172.21.15.156:0/312506045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e4161340 0x7fc9e414e750 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00cc00 tx=0x7fc9dc00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:39.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.853+0000 7fc9da7fc700 1 -- 172.21.15.156:0/312506045 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc00aa80 con 0x7fc9e4161340 2023-11-28T13:09:39.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.853+0000 7fc9da7fc700 1 -- 172.21.15.156:0/312506045 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9dc00ac00 con 0x7fc9e4161340 2023-11-28T13:09:39.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.853+0000 7fc9ebbff700 1 -- 172.21.15.156:0/312506045 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d40094a0 con 0x7fc9e4161340 2023-11-28T13:09:39.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.853+0000 7fc9da7fc700 1 -- 172.21.15.156:0/312506045 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0070d0 con 0x7fc9e4161340 2023-11-28T13:09:39.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.853+0000 7fc9ebbff700 1 -- 172.21.15.156:0/312506045 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e40b5130 con 0x7fc9e4161340 2023-11-28T13:09:39.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.855+0000 7fc9ebbff700 1 -- 172.21.15.156:0/312506045 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e40d15e0 con 0x7fc9e4161340 2023-11-28T13:09:39.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.856+0000 7fc9da7fc700 1 -- 172.21.15.156:0/312506045 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc9dc010070 con 0x7fc9e4161340 2023-11-28T13:09:39.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.856+0000 7fc9da7fc700 1 --2- 172.21.15.156:0/312506045 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc9c8068320 0x7fc9c806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:39.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.856+0000 7fc9da7fc700 1 -- 172.21.15.156:0/312506045 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fc9dc0855c0 con 0x7fc9e4161340 2023-11-28T13:09:39.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.857+0000 7fc9e999b700 1 --2- 172.21.15.156:0/312506045 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc9c8068320 0x7fc9c806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:39.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.858+0000 7fc9e999b700 1 --2- 172.21.15.156:0/312506045 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc9c8068320 0x7fc9c806a7d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc9e415d5b0 tx=0x7fc9e0009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:39.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:39.860+0000 7fc9da7fc700 1 -- 172.21.15.156:0/312506045 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc9dc051dc0 con 0x7fc9e4161340 2023-11-28T13:09:40.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.096+0000 7fc9ebbff700 1 -- 172.21.15.156:0/312506045 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc9e40b52c0 con 0x7fc9e4161340 2023-11-28T13:09:40.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.630+0000 7fc9da7fc700 1 -- 172.21.15.156:0/312506045 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc9dc0553e0 con 0x7fc9e4161340 2023-11-28T13:09:40.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.633+0000 7fc9ebbff700 1 -- 172.21.15.156:0/312506045 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc9c8068320 msgr2=0x7fc9c806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:40.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.633+0000 7fc9ebbff700 1 --2- 172.21.15.156:0/312506045 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc9c8068320 0x7fc9c806a7d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc9e415d5b0 tx=0x7fc9e0009210 comp rx=0 tx=0).stop 2023-11-28T13:09:40.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.633+0000 7fc9ebbff700 1 -- 172.21.15.156:0/312506045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e4161340 msgr2=0x7fc9e414e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:40.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.633+0000 7fc9ebbff700 1 --2- 172.21.15.156:0/312506045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e4161340 0x7fc9e414e750 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00cc00 tx=0x7fc9dc00cf10 comp rx=0 tx=0).stop 2023-11-28T13:09:40.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.633+0000 7fc9ebbff700 1 -- 172.21.15.156:0/312506045 shutdown_connections 2023-11-28T13:09:40.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.633+0000 7fc9ebbff700 1 --2- 172.21.15.156:0/312506045 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc9c8068320 0x7fc9c806a7d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:40.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.633+0000 7fc9ebbff700 1 --2- 172.21.15.156:0/312506045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e4161340 0x7fc9e414e750 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:40.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.633+0000 7fc9ebbff700 1 --2- 172.21.15.156:0/312506045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9e40d2320 0x7fc9e415fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:40.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.633+0000 7fc9ebbff700 1 --2- 172.21.15.156:0/312506045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc9e415c9b0 0x7fc9e415dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:40.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.633+0000 7fc9ebbff700 1 -- 172.21.15.156:0/312506045 >> 172.21.15.156:0/312506045 conn(0x7fc9e4072600 msgr2=0x7fc9e415cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:40.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.633+0000 7fc9ebbff700 1 -- 172.21.15.156:0/312506045 shutdown_connections 2023-11-28T13:09:40.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.633+0000 7fc9ebbff700 1 -- 172.21.15.156:0/312506045 wait complete. 2023-11-28T13:09:40.655 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T13:09:40.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.897+0000 7f20b41ff700 1 Processor -- start 2023-11-28T13:09:40.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.897+0000 7f20b41ff700 1 -- start start 2023-11-28T13:09:40.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.898+0000 7f20b41ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f20ac15a760 0x7f20ac161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:40.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.898+0000 7f20b41ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ac15dc30 0x7f20ac0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:40.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.898+0000 7f20b41ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac15e0a0 con 0x7f20ac0cd7c0 2023-11-28T13:09:40.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.898+0000 7f20b41ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ac15ab30 con 0x7f20ac15a760 2023-11-28T13:09:40.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.898+0000 7f20b41ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac160d70 con 0x7f20ac15dc30 2023-11-28T13:09:40.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.898+0000 7f20b279c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f20ac0cd7c0 0x7f20ac0d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53286/0 (socket says 172.21.15.156:53286) 2023-11-28T13:09:40.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.898+0000 7f20b279c700 1 -- 172.21.15.156:0/3135897906 learned_addr learned my addr 172.21.15.156:0/3135897906 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:40.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.898+0000 7f20b1f9b700 1 --2- 172.21.15.156:0/3135897906 >> v2:172.21.15.156:3300/0 conn(0x7f20ac15a760 0x7f20ac161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:40.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.899+0000 7f20b179a700 1 --2- 172.21.15.156:0/3135897906 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ac15dc30 0x7f20ac0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:40.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.899+0000 7f20b0f99700 1 -- 172.21.15.156:0/3135897906 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3462726051 0 0) 0x7f20ac15e0a0 con 0x7f20ac0cd7c0 2023-11-28T13:09:40.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.899+0000 7f20b0f99700 1 -- 172.21.15.156:0/3135897906 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20980035b0 con 0x7f20ac0cd7c0 2023-11-28T13:09:40.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.899+0000 7f20b1f9b700 1 -- 172.21.15.156:0/3135897906 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ac15dc30 msgr2=0x7f20ac0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:40.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.899+0000 7f20b1f9b700 1 --2- 172.21.15.156:0/3135897906 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ac15dc30 0x7f20ac0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:40.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.899+0000 7f20b1f9b700 1 -- 172.21.15.156:0/3135897906 >> v1:172.21.15.156:6789/0 conn(0x7f20ac0cd7c0 legacy=0x7f20ac0d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:40.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.899+0000 7f20b1f9b700 1 -- 172.21.15.156:0/3135897906 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac0b5130 con 0x7f20ac15a760 2023-11-28T13:09:40.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.899+0000 7f20b1f9b700 1 --2- 172.21.15.156:0/3135897906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20ac15a760 0x7f20ac161200 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009d90 tx=0x7f20a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9577a2ebecf64114 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:40.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.900+0000 7f20b0f99700 1 -- 172.21.15.156:0/3135897906 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a800e070 con 0x7f20ac15a760 2023-11-28T13:09:40.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.900+0000 7f20b0f99700 1 -- 172.21.15.156:0/3135897906 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a800bb20 con 0x7f20ac15a760 2023-11-28T13:09:40.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.900+0000 7f20b0f99700 1 -- 172.21.15.156:0/3135897906 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8012430 con 0x7f20ac15a760 2023-11-28T13:09:40.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.900+0000 7f20b41ff700 1 -- 172.21.15.156:0/3135897906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20ac15a760 msgr2=0x7f20ac161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:40.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.900+0000 7f20b41ff700 1 --2- 172.21.15.156:0/3135897906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20ac15a760 0x7f20ac161200 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009d90 tx=0x7f20a800b330 comp rx=0 tx=0).stop 2023-11-28T13:09:40.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.900+0000 7f20b41ff700 1 -- 172.21.15.156:0/3135897906 shutdown_connections 2023-11-28T13:09:40.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.900+0000 7f20b41ff700 1 --2- 172.21.15.156:0/3135897906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20ac15a760 0x7f20ac161200 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:40.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.900+0000 7f20b41ff700 1 --2- 172.21.15.156:0/3135897906 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ac15dc30 0x7f20ac0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:40.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.900+0000 7f20b41ff700 1 -- 172.21.15.156:0/3135897906 >> 172.21.15.156:0/3135897906 conn(0x7f20ac072620 msgr2=0x7f20ac0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:40.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.900+0000 7f20b41ff700 1 -- 172.21.15.156:0/3135897906 shutdown_connections 2023-11-28T13:09:40.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.900+0000 7f20b41ff700 1 -- 172.21.15.156:0/3135897906 wait complete. 2023-11-28T13:09:40.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.903+0000 7f20b41ff700 1 Processor -- start 2023-11-28T13:09:40.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.903+0000 7f20b41ff700 1 -- start start 2023-11-28T13:09:40.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.904+0000 7f20b41ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20ac0cd7c0 0x7f20ac15fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:40.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.904+0000 7f20b41ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ac15a760 0x7f20ac0cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:40.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.904+0000 7f20b41ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ac15dc30 0x7f20ac1575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:40.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.904+0000 7f20b41ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0b61b0 con 0x7f20ac0cd7c0 2023-11-28T13:09:40.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.904+0000 7f20b41ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0cb7d0 con 0x7f20ac15dc30 2023-11-28T13:09:40.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.904+0000 7f20b41ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0cba50 con 0x7f20ac15a760 2023-11-28T13:09:40.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.904+0000 7f20b1f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20ac0cd7c0 0x7f20ac15fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:40.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.904+0000 7f20b1f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20ac0cd7c0 0x7f20ac15fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36478/0 (socket says 172.21.15.156:36478) 2023-11-28T13:09:40.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.904+0000 7f20b1f9b700 1 -- 172.21.15.156:0/740071768 learned_addr learned my addr 172.21.15.156:0/740071768 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:40.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.904+0000 7f20b179a700 1 --2- 172.21.15.156:0/740071768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ac15a760 0x7f20ac0cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:40.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.905+0000 7f20b279c700 1 --2- 172.21.15.156:0/740071768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ac15dc30 0x7f20ac1575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:40.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.905+0000 7f20b1f9b700 1 -- 172.21.15.156:0/740071768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ac15a760 msgr2=0x7f20ac0cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:40.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.905+0000 7f20b1f9b700 1 --2- 172.21.15.156:0/740071768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ac15a760 0x7f20ac0cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:40.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.905+0000 7f20b1f9b700 1 -- 172.21.15.156:0/740071768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ac15dc30 msgr2=0x7f20ac1575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:40.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.905+0000 7f20b1f9b700 1 --2- 172.21.15.156:0/740071768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ac15dc30 0x7f20ac1575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:40.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.905+0000 7f20b1f9b700 1 -- 172.21.15.156:0/740071768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a8009a70 con 0x7f20ac0cd7c0 2023-11-28T13:09:40.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.905+0000 7f20b279c700 1 --2- 172.21.15.156:0/740071768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ac15dc30 0x7f20ac1575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:40.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.905+0000 7f20b1f9b700 1 --2- 172.21.15.156:0/740071768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20ac0cd7c0 0x7f20ac15fc20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0675d0 tx=0x7f20a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:40.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.905+0000 7f20a27fc700 1 -- 172.21.15.156:0/740071768 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8012430 con 0x7f20ac0cd7c0 2023-11-28T13:09:40.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.905+0000 7f20a27fc700 1 -- 172.21.15.156:0/740071768 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a8020070 con 0x7f20ac0cd7c0 2023-11-28T13:09:40.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.905+0000 7f20b41ff700 1 -- 172.21.15.156:0/740071768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ac0b5670 con 0x7f20ac0cd7c0 2023-11-28T13:09:40.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.905+0000 7f20a27fc700 1 -- 172.21.15.156:0/740071768 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8004280 con 0x7f20ac0cd7c0 2023-11-28T13:09:40.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.906+0000 7f20b41ff700 1 -- 172.21.15.156:0/740071768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ac0b5390 con 0x7f20ac0cd7c0 2023-11-28T13:09:40.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.907+0000 7f20b41ff700 1 -- 172.21.15.156:0/740071768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ac160720 con 0x7f20ac0cd7c0 2023-11-28T13:09:40.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.907+0000 7f20a27fc700 1 -- 172.21.15.156:0/740071768 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f20a802e070 con 0x7f20ac0cd7c0 2023-11-28T13:09:40.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.908+0000 7f20a27fc700 1 --2- 172.21.15.156:0/740071768 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f20900682f0 0x7f209006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:40.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.908+0000 7f20a27fc700 1 -- 172.21.15.156:0/740071768 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f20a8022070 con 0x7f20ac0cd7c0 2023-11-28T13:09:40.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.909+0000 7f20b179a700 1 --2- 172.21.15.156:0/740071768 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f20900682f0 0x7f209006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:40.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.909+0000 7f20b179a700 1 --2- 172.21.15.156:0/740071768 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f20900682f0 0x7f209006a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0cf190 tx=0x7f209c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:40.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:40.912+0000 7f20a27fc700 1 -- 172.21.15.156:0/740071768 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f20a8054100 con 0x7f20ac0cd7c0 2023-11-28T13:09:41.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.102+0000 7f20b41ff700 1 -- 172.21.15.156:0/740071768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f20ac04e620 con 0x7f20ac0cd7c0 2023-11-28T13:09:41.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.701+0000 7f20a27fc700 1 -- 172.21.15.156:0/740071768 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) v1 ==== 181+0+0 (secure 0 0 0) 0x7f20a8057720 con 0x7f20ac0cd7c0 2023-11-28T13:09:41.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.703+0000 7f20b41ff700 1 -- 172.21.15.156:0/740071768 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f20900682f0 msgr2=0x7f209006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:41.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.703+0000 7f20b41ff700 1 --2- 172.21.15.156:0/740071768 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f20900682f0 0x7f209006a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0cf190 tx=0x7f209c009210 comp rx=0 tx=0).stop 2023-11-28T13:09:41.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.703+0000 7f20b41ff700 1 -- 172.21.15.156:0/740071768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20ac0cd7c0 msgr2=0x7f20ac15fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:41.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.703+0000 7f20b41ff700 1 --2- 172.21.15.156:0/740071768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20ac0cd7c0 0x7f20ac15fc20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0675d0 tx=0x7f20a8004ad0 comp rx=0 tx=0).stop 2023-11-28T13:09:41.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.704+0000 7f20b41ff700 1 -- 172.21.15.156:0/740071768 shutdown_connections 2023-11-28T13:09:41.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.704+0000 7f20b41ff700 1 --2- 172.21.15.156:0/740071768 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f20900682f0 0x7f209006a7a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:41.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.704+0000 7f20b41ff700 1 --2- 172.21.15.156:0/740071768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ac15dc30 0x7f20ac1575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:41.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.704+0000 7f20b41ff700 1 --2- 172.21.15.156:0/740071768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20ac0cd7c0 0x7f20ac15fc20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:41.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.704+0000 7f20b41ff700 1 --2- 172.21.15.156:0/740071768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20ac15a760 0x7f20ac0cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:41.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.704+0000 7f20b41ff700 1 -- 172.21.15.156:0/740071768 >> 172.21.15.156:0/740071768 conn(0x7f20ac072620 msgr2=0x7f20ac158b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:41.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.704+0000 7f20b41ff700 1 -- 172.21.15.156:0/740071768 shutdown_connections 2023-11-28T13:09:41.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.704+0000 7f20b41ff700 1 -- 172.21.15.156:0/740071768 wait complete. 2023-11-28T13:09:41.707 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_metadata' removed 2023-11-28T13:09:41.725 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T13:09:41.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.934+0000 7f1f72dff700 1 Processor -- start 2023-11-28T13:09:41.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.934+0000 7f1f72dff700 1 -- start start 2023-11-28T13:09:41.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.936+0000 7f1f72dff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1f6c0cded0 0x7f1f6c161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:41.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.936+0000 7f1f72dff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6c160d70 con 0x7f1f6c15dc30 2023-11-28T13:09:41.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.936+0000 7f1f72dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6c0b5390 con 0x7f1f6c15a760 2023-11-28T13:09:41.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.936+0000 7f1f72dff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f6c15e0d0 con 0x7f1f6c0cded0 2023-11-28T13:09:41.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.936+0000 7f1f7139c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f1f6c15dc30 0x7f1f6c0d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53296/0 (socket says 172.21.15.156:53296) 2023-11-28T13:09:41.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.936+0000 7f1f7139c700 1 -- 172.21.15.156:0/2725827934 learned_addr learned my addr 172.21.15.156:0/2725827934 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:41.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.936+0000 7f1f6bfff700 1 --2- 172.21.15.156:0/2725827934 >> v2:172.21.15.156:3300/0 conn(0x7f1f6c0cded0 0x7f1f6c161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:41.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.936+0000 7f1f6bfff700 1 -- 172.21.15.156:0/2725827934 >> v1:172.21.15.186:6789/0 conn(0x7f1f6c15a760 legacy=0x7f1f6c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:41.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.937+0000 7f1f6bfff700 1 -- 172.21.15.156:0/2725827934 >> v1:172.21.15.156:6789/0 conn(0x7f1f6c15dc30 legacy=0x7f1f6c0d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:41.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.937+0000 7f1f6bfff700 1 -- 172.21.15.156:0/2725827934 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c0b5670 con 0x7f1f6c0cded0 2023-11-28T13:09:41.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.937+0000 7f1f6b7fe700 1 -- 172.21.15.156:0/2725827934 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131365951 0 0) 0x7f1f6c160d70 con 0x7f1f6c15dc30 2023-11-28T13:09:41.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.937+0000 7f1f6bfff700 1 --2- 172.21.15.156:0/2725827934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6c0cded0 0x7f1f6c161200 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009840 tx=0x7f1f58009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e758c4974d849a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:41.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.937+0000 7f1f6b7fe700 1 -- 172.21.15.156:0/2725827934 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58011830 con 0x7f1f6c0cded0 2023-11-28T13:09:41.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.937+0000 7f1f6b7fe700 1 -- 172.21.15.156:0/2725827934 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f58011990 con 0x7f1f6c0cded0 2023-11-28T13:09:41.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.937+0000 7f1f6b7fe700 1 -- 172.21.15.156:0/2725827934 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58011d20 con 0x7f1f6c0cded0 2023-11-28T13:09:41.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.937+0000 7f1f72dff700 1 -- 172.21.15.156:0/2725827934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6c0cded0 msgr2=0x7f1f6c161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:41.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.937+0000 7f1f72dff700 1 --2- 172.21.15.156:0/2725827934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6c0cded0 0x7f1f6c161200 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009840 tx=0x7f1f58009b50 comp rx=0 tx=0).stop 2023-11-28T13:09:41.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.938+0000 7f1f72dff700 1 -- 172.21.15.156:0/2725827934 shutdown_connections 2023-11-28T13:09:41.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.938+0000 7f1f72dff700 1 --2- 172.21.15.156:0/2725827934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6c0cded0 0x7f1f6c161200 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:41.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.938+0000 7f1f72dff700 1 -- 172.21.15.156:0/2725827934 >> 172.21.15.156:0/2725827934 conn(0x7f1f6c072620 msgr2=0x7f1f6c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:41.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.938+0000 7f1f72dff700 1 -- 172.21.15.156:0/2725827934 shutdown_connections 2023-11-28T13:09:41.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.938+0000 7f1f72dff700 1 -- 172.21.15.156:0/2725827934 wait complete. 2023-11-28T13:09:41.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.940+0000 7f1f72dff700 1 Processor -- start 2023-11-28T13:09:41.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.940+0000 7f1f72dff700 1 -- start start 2023-11-28T13:09:41.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.941+0000 7f1f72dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6c0cded0 0x7f1f6c1536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:41.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.942+0000 7f1f72dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f6c15a760 0x7f1f6c0b69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:41.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.942+0000 7f1f72dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f6c15dc30 0x7f1f6c1f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:41.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.942+0000 7f1f72dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c153be0 con 0x7f1f6c0cded0 2023-11-28T13:09:41.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.942+0000 7f1f72dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c0b7400 con 0x7f1f6c15a760 2023-11-28T13:09:41.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.942+0000 7f1f72dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c0b76e0 con 0x7f1f6c15dc30 2023-11-28T13:09:41.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.942+0000 7f1f70b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6c0cded0 0x7f1f6c1536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:41.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.942+0000 7f1f70b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6c0cded0 0x7f1f6c1536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36494/0 (socket says 172.21.15.156:36494) 2023-11-28T13:09:41.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.942+0000 7f1f70b9b700 1 -- 172.21.15.156:0/143005605 learned_addr learned my addr 172.21.15.156:0/143005605 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:41.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.942+0000 7f1f6bfff700 1 --2- 172.21.15.156:0/143005605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f6c15a760 0x7f1f6c0b69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:41.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.942+0000 7f1f7139c700 1 --2- 172.21.15.156:0/143005605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f6c15dc30 0x7f1f6c1f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:41.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.942+0000 7f1f70b9b700 1 -- 172.21.15.156:0/143005605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f6c15dc30 msgr2=0x7f1f6c1f29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:41.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.942+0000 7f1f70b9b700 1 --2- 172.21.15.156:0/143005605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f6c15dc30 0x7f1f6c1f29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:41.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.942+0000 7f1f70b9b700 1 -- 172.21.15.156:0/143005605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f6c15a760 msgr2=0x7f1f6c0b69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:41.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.942+0000 7f1f70b9b700 1 --2- 172.21.15.156:0/143005605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f6c15a760 0x7f1f6c0b69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:41.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.942+0000 7f1f70b9b700 1 -- 172.21.15.156:0/143005605 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f580094a0 con 0x7f1f6c0cded0 2023-11-28T13:09:41.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.942+0000 7f1f7139c700 1 --2- 172.21.15.156:0/143005605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f6c15dc30 0x7f1f6c1f29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:41.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.943+0000 7f1f70b9b700 1 --2- 172.21.15.156:0/143005605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6c0cded0 0x7f1f6c1536a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000c4e0 tx=0x7f1f6000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:41.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.943+0000 7f1f697fa700 1 -- 172.21.15.156:0/143005605 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f600052d0 con 0x7f1f6c0cded0 2023-11-28T13:09:41.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.943+0000 7f1f697fa700 1 -- 172.21.15.156:0/143005605 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f60005430 con 0x7f1f6c0cded0 2023-11-28T13:09:41.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.943+0000 7f1f72dff700 1 -- 172.21.15.156:0/143005605 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6c160a00 con 0x7f1f6c0cded0 2023-11-28T13:09:41.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.943+0000 7f1f697fa700 1 -- 172.21.15.156:0/143005605 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f600038c0 con 0x7f1f6c0cded0 2023-11-28T13:09:41.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.943+0000 7f1f72dff700 1 -- 172.21.15.156:0/143005605 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c160720 con 0x7f1f6c0cded0 2023-11-28T13:09:41.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.944+0000 7f1f72dff700 1 -- 172.21.15.156:0/143005605 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6c0b4bf0 con 0x7f1f6c0cded0 2023-11-28T13:09:41.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.945+0000 7f1f697fa700 1 -- 172.21.15.156:0/143005605 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1f60019030 con 0x7f1f6c0cded0 2023-11-28T13:09:41.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.946+0000 7f1f697fa700 1 --2- 172.21.15.156:0/143005605 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1f500682f0 0x7f1f5006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:41.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.946+0000 7f1f697fa700 1 -- 172.21.15.156:0/143005605 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f1f60084e60 con 0x7f1f6c0cded0 2023-11-28T13:09:41.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.947+0000 7f1f6bfff700 1 --2- 172.21.15.156:0/143005605 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1f500682f0 0x7f1f5006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:41.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.947+0000 7f1f6bfff700 1 --2- 172.21.15.156:0/143005605 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1f500682f0 0x7f1f5006a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0ce2c0 tx=0x7f1f5801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:41.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:41.950+0000 7f1f697fa700 1 -- 172.21.15.156:0/143005605 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1f60051820 con 0x7f1f6c0cded0 2023-11-28T13:09:42.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.140+0000 7f1f72dff700 1 -- 172.21.15.156:0/143005605 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1f6c04e620 con 0x7f1f6c0cded0 2023-11-28T13:09:42.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.707+0000 7f1f697fa700 1 -- 172.21.15.156:0/143005605 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1f60054e40 con 0x7f1f6c0cded0 2023-11-28T13:09:42.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.709+0000 7f1f72dff700 1 -- 172.21.15.156:0/143005605 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1f500682f0 msgr2=0x7f1f5006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:42.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.709+0000 7f1f72dff700 1 --2- 172.21.15.156:0/143005605 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1f500682f0 0x7f1f5006a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0ce2c0 tx=0x7f1f5801c040 comp rx=0 tx=0).stop 2023-11-28T13:09:42.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.709+0000 7f1f72dff700 1 -- 172.21.15.156:0/143005605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6c0cded0 msgr2=0x7f1f6c1536a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:42.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.709+0000 7f1f72dff700 1 --2- 172.21.15.156:0/143005605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6c0cded0 0x7f1f6c1536a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000c4e0 tx=0x7f1f6000c7f0 comp rx=0 tx=0).stop 2023-11-28T13:09:42.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.710+0000 7f1f72dff700 1 -- 172.21.15.156:0/143005605 shutdown_connections 2023-11-28T13:09:42.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.710+0000 7f1f72dff700 1 --2- 172.21.15.156:0/143005605 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1f500682f0 0x7f1f5006a7a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:42.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.710+0000 7f1f72dff700 1 --2- 172.21.15.156:0/143005605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f6c15dc30 0x7f1f6c1f29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:42.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.710+0000 7f1f72dff700 1 --2- 172.21.15.156:0/143005605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f6c0cded0 0x7f1f6c1536a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:42.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.710+0000 7f1f72dff700 1 --2- 172.21.15.156:0/143005605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f6c15a760 0x7f1f6c0b69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:42.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.710+0000 7f1f72dff700 1 -- 172.21.15.156:0/143005605 >> 172.21.15.156:0/143005605 conn(0x7f1f6c072620 msgr2=0x7f1f6c0d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:42.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.710+0000 7f1f72dff700 1 -- 172.21.15.156:0/143005605 shutdown_connections 2023-11-28T13:09:42.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.710+0000 7f1f72dff700 1 -- 172.21.15.156:0/143005605 wait complete. 2023-11-28T13:09:42.713 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_data' removed 2023-11-28T13:09:42.742 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T13:09:42.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.943+0000 7f96cc1ff700 1 Processor -- start 2023-11-28T13:09:42.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.943+0000 7f96cc1ff700 1 -- start start 2023-11-28T13:09:42.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.944+0000 7f96cc1ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f96c40cd7c0 0x7f96c40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:42.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.944+0000 7f96cc1ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c40d1950 con 0x7f96c415c9b0 2023-11-28T13:09:42.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.945+0000 7f96cc1ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c40ce140 con 0x7f96c4161340 2023-11-28T13:09:42.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.945+0000 7f96cc1ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c40b5800 con 0x7f96c40cd7c0 2023-11-28T13:09:42.946 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.945+0000 7f96c9f9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f96c415c9b0 0x7f96c40b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53298/0 (socket says 172.21.15.156:53298) 2023-11-28T13:09:42.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.945+0000 7f96c9f9b700 1 -- 172.21.15.156:0/1924897505 learned_addr learned my addr 172.21.15.156:0/1924897505 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:42.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.945+0000 7f96ca79c700 1 --2- 172.21.15.156:0/1924897505 >> v2:172.21.15.186:3300/0 conn(0x7f96c40cd7c0 0x7f96c40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:42.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.945+0000 7f96c8f99700 1 -- 172.21.15.156:0/1924897505 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498080082 0 0) 0x7f96c40d1950 con 0x7f96c415c9b0 2023-11-28T13:09:42.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.945+0000 7f96c8f99700 1 -- 172.21.15.156:0/1924897505 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96b00035b0 con 0x7f96c415c9b0 2023-11-28T13:09:42.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.945+0000 7f96c8f99700 1 -- 172.21.15.156:0/1924897505 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 660969974 0 0) 0x7f96b00035b0 con 0x7f96c415c9b0 2023-11-28T13:09:42.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.945+0000 7f96c8f99700 1 -- 172.21.15.156:0/1924897505 >> v2:172.21.15.186:3300/0 conn(0x7f96c40cd7c0 msgr2=0x7f96c40cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:42.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.945+0000 7f96c8f99700 1 --2- 172.21.15.156:0/1924897505 >> v2:172.21.15.186:3300/0 conn(0x7f96c40cd7c0 0x7f96c40cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:42.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.945+0000 7f96c8f99700 1 -- 172.21.15.156:0/1924897505 >> v1:172.21.15.186:6789/0 conn(0x7f96c4161340 legacy=0x7f96c40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:42.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.945+0000 7f96c8f99700 1 -- 172.21.15.156:0/1924897505 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c40d24f0 con 0x7f96c415c9b0 2023-11-28T13:09:42.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.946+0000 7f96c8f99700 1 -- 172.21.15.156:0/1924897505 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f96c0001670 con 0x7f96c415c9b0 2023-11-28T13:09:42.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.946+0000 7f96c8f99700 1 -- 172.21.15.156:0/1924897505 >> v1:172.21.15.156:6789/0 conn(0x7f96c415c9b0 legacy=0x7f96c40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:42.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.946+0000 7f96c8f99700 1 --2- 172.21.15.156:0/1924897505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96b0003a00 0x7f96b0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:42.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.946+0000 7f96c8f99700 1 --2- 172.21.15.156:0/1924897505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96b0004850 0x7f96b0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:42.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.946+0000 7f96c8f99700 1 --2- 172.21.15.156:0/1924897505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96b0005200 0x7f96b000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:42.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.946+0000 7f96c8f99700 1 -- 172.21.15.156:0/1924897505 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c40b5800 con 0x7f96b0004850 2023-11-28T13:09:42.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.946+0000 7f96c8f99700 1 -- 172.21.15.156:0/1924897505 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b000c110 con 0x7f96b0005200 2023-11-28T13:09:42.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.946+0000 7f96c8f99700 1 -- 172.21.15.156:0/1924897505 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b000c2d0 con 0x7f96b0003a00 2023-11-28T13:09:42.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.946+0000 7f96c979a700 1 --2- 172.21.15.156:0/1924897505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96b0004850 0x7f96b0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:42.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.946+0000 7f96c9f9b700 1 --2- 172.21.15.156:0/1924897505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96b0003a00 0x7f96b0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:42.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.946+0000 7f96ca79c700 1 --2- 172.21.15.156:0/1924897505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96b0005200 0x7f96b000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:42.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.946+0000 7f96c979a700 1 -- 172.21.15.156:0/1924897505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96b0003a00 msgr2=0x7f96b0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:42.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.946+0000 7f96c979a700 1 --2- 172.21.15.156:0/1924897505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96b0003a00 0x7f96b0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:42.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.946+0000 7f96c979a700 1 -- 172.21.15.156:0/1924897505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96b0005200 msgr2=0x7f96b000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:42.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.946+0000 7f96c979a700 1 --2- 172.21.15.156:0/1924897505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96b0005200 0x7f96b000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:42.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.946+0000 7f96c979a700 1 -- 172.21.15.156:0/1924897505 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00035b0 con 0x7f96b0004850 2023-11-28T13:09:42.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.946+0000 7f96ca79c700 1 -- 172.21.15.156:0/1924897505 reap_dead start 2023-11-28T13:09:42.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.947+0000 7f96c979a700 1 --2- 172.21.15.156:0/1924897505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96b0004850 0x7f96b0004cc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f96b40017c0 tx=0x7f96b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748906884df57cbd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:42.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.947+0000 7f96c8f99700 1 -- 172.21.15.156:0/1924897505 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b40052d0 con 0x7f96b0004850 2023-11-28T13:09:42.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.947+0000 7f96c8f99700 1 -- 172.21.15.156:0/1924897505 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b4005450 con 0x7f96b0004850 2023-11-28T13:09:42.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.947+0000 7f96c8f99700 1 -- 172.21.15.156:0/1924897505 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b40038c0 con 0x7f96b0004850 2023-11-28T13:09:42.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.947+0000 7f96cc1ff700 1 -- 172.21.15.156:0/1924897505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96b0004850 msgr2=0x7f96b0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:42.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.947+0000 7f96cc1ff700 1 --2- 172.21.15.156:0/1924897505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96b0004850 0x7f96b0004cc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f96b40017c0 tx=0x7f96b0005880 comp rx=0 tx=0).stop 2023-11-28T13:09:42.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.947+0000 7f96cc1ff700 1 -- 172.21.15.156:0/1924897505 shutdown_connections 2023-11-28T13:09:42.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.947+0000 7f96cc1ff700 1 --2- 172.21.15.156:0/1924897505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96b0004850 0x7f96b0004cc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:42.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.947+0000 7f96cc1ff700 1 -- 172.21.15.156:0/1924897505 >> 172.21.15.156:0/1924897505 conn(0x7f96c4072600 msgr2=0x7f96c40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:42.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.948+0000 7f96cc1ff700 1 -- 172.21.15.156:0/1924897505 shutdown_connections 2023-11-28T13:09:42.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.948+0000 7f96cc1ff700 1 -- 172.21.15.156:0/1924897505 wait complete. 2023-11-28T13:09:42.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.949+0000 7f96cc1ff700 1 Processor -- start 2023-11-28T13:09:42.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.949+0000 7f96cc1ff700 1 -- start start 2023-11-28T13:09:42.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.950+0000 7f96cc1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96b0004850 0x7f96c41f1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:42.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.950+0000 7f96cc1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96c40c8c70 0x7f96c41e6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:42.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.950+0000 7f96cc1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96c41e6ff0 0x7f96c40c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:42.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.950+0000 7f96cc1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c4147240 con 0x7f96c40c8c70 2023-11-28T13:09:42.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.950+0000 7f96cc1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c40c9150 con 0x7f96b0004850 2023-11-28T13:09:42.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.950+0000 7f96cc1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c40c9430 con 0x7f96c41e6ff0 2023-11-28T13:09:42.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.951+0000 7f96c979a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96c40c8c70 0x7f96c41e6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:42.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.951+0000 7f96c979a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96c40c8c70 0x7f96c41e6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36512/0 (socket says 172.21.15.156:36512) 2023-11-28T13:09:42.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.951+0000 7f96c979a700 1 -- 172.21.15.156:0/1936788360 learned_addr learned my addr 172.21.15.156:0/1936788360 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:42.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.951+0000 7f96ca79c700 1 --2- 172.21.15.156:0/1936788360 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96c41e6ff0 0x7f96c40c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:42.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.951+0000 7f96c9f9b700 1 --2- 172.21.15.156:0/1936788360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96b0004850 0x7f96c41f1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:42.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.951+0000 7f96c979a700 1 -- 172.21.15.156:0/1936788360 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96c41e6ff0 msgr2=0x7f96c40c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:42.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.951+0000 7f96c979a700 1 --2- 172.21.15.156:0/1936788360 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96c41e6ff0 0x7f96c40c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:42.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.951+0000 7f96c979a700 1 -- 172.21.15.156:0/1936788360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96b0004850 msgr2=0x7f96c41f1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:42.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.951+0000 7f96c979a700 1 --2- 172.21.15.156:0/1936788360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96b0004850 0x7f96c41f1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:42.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.951+0000 7f96c979a700 1 -- 172.21.15.156:0/1936788360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c40d1300 con 0x7f96c40c8c70 2023-11-28T13:09:42.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.951+0000 7f96c979a700 1 --2- 172.21.15.156:0/1936788360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96c40c8c70 0x7f96c41e6ab0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f96b000c0e0 tx=0x7f96b4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:42.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.952+0000 7f96ba7fc700 1 -- 172.21.15.156:0/1936788360 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4001290 con 0x7f96c40c8c70 2023-11-28T13:09:42.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.952+0000 7f96ba7fc700 1 -- 172.21.15.156:0/1936788360 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b4007040 con 0x7f96c40c8c70 2023-11-28T13:09:42.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.952+0000 7f96cc1ff700 1 -- 172.21.15.156:0/1936788360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c40b5520 con 0x7f96c40c8c70 2023-11-28T13:09:42.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.952+0000 7f96ba7fc700 1 -- 172.21.15.156:0/1936788360 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b40073d0 con 0x7f96c40c8c70 2023-11-28T13:09:42.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.952+0000 7f96cc1ff700 1 -- 172.21.15.156:0/1936788360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c40d15e0 con 0x7f96c40c8c70 2023-11-28T13:09:42.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.953+0000 7f96cc1ff700 1 -- 172.21.15.156:0/1936788360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c41473f0 con 0x7f96c40c8c70 2023-11-28T13:09:42.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.954+0000 7f96ba7fc700 1 -- 172.21.15.156:0/1936788360 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f96b402d070 con 0x7f96c40c8c70 2023-11-28T13:09:42.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.954+0000 7f96ba7fc700 1 --2- 172.21.15.156:0/1936788360 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f96a80682f0 0x7f96a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:42.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.955+0000 7f96ba7fc700 1 -- 172.21.15.156:0/1936788360 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f96b4086110 con 0x7f96c40c8c70 2023-11-28T13:09:42.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.955+0000 7f96c9f9b700 1 --2- 172.21.15.156:0/1936788360 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f96a80682f0 0x7f96a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:42.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.956+0000 7f96c9f9b700 1 --2- 172.21.15.156:0/1936788360 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f96a80682f0 0x7f96a806a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f96c0000c00 tx=0x7f96c0001b20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:42.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:42.959+0000 7f96ba7fc700 1 -- 172.21.15.156:0/1936788360 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f96c41473f0 con 0x7f96c40c8c70 2023-11-28T13:09:43.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.152+0000 7f96cc1ff700 1 -- 172.21.15.156:0/1936788360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f96c40745c0 con 0x7f96c40c8c70 2023-11-28T13:09:43.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.156+0000 7f96ba7fc700 1 -- 172.21.15.156:0/1936788360 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f96b4052ba0 con 0x7f96c40c8c70 2023-11-28T13:09:43.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.158+0000 7f96cc1ff700 1 -- 172.21.15.156:0/1936788360 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f96a80682f0 msgr2=0x7f96a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:43.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.158+0000 7f96cc1ff700 1 --2- 172.21.15.156:0/1936788360 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f96a80682f0 0x7f96a806a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f96c0000c00 tx=0x7f96c0001b20 comp rx=0 tx=0).stop 2023-11-28T13:09:43.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.158+0000 7f96cc1ff700 1 -- 172.21.15.156:0/1936788360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96c40c8c70 msgr2=0x7f96c41e6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:43.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.158+0000 7f96cc1ff700 1 --2- 172.21.15.156:0/1936788360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96c40c8c70 0x7f96c41e6ab0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f96b000c0e0 tx=0x7f96b4001120 comp rx=0 tx=0).stop 2023-11-28T13:09:43.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.158+0000 7f96cc1ff700 1 -- 172.21.15.156:0/1936788360 shutdown_connections 2023-11-28T13:09:43.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.158+0000 7f96cc1ff700 1 --2- 172.21.15.156:0/1936788360 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f96a80682f0 0x7f96a806a7a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.158+0000 7f96cc1ff700 1 --2- 172.21.15.156:0/1936788360 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96c41e6ff0 0x7f96c40c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.158+0000 7f96cc1ff700 1 --2- 172.21.15.156:0/1936788360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96b0004850 0x7f96c41f1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.158+0000 7f96cc1ff700 1 --2- 172.21.15.156:0/1936788360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96c40c8c70 0x7f96c41e6ab0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.158+0000 7f96cc1ff700 1 -- 172.21.15.156:0/1936788360 >> 172.21.15.156:0/1936788360 conn(0x7f96c4072600 msgr2=0x7f96c40b7250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:43.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.158+0000 7f96cc1ff700 1 -- 172.21.15.156:0/1936788360 shutdown_connections 2023-11-28T13:09:43.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.158+0000 7f96cc1ff700 1 -- 172.21.15.156:0/1936788360 wait complete. 2023-11-28T13:09:43.178 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T13:09:43.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.375+0000 7f15c57ff700 1 Processor -- start 2023-11-28T13:09:43.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.375+0000 7f15c57ff700 1 -- start start 2023-11-28T13:09:43.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.376+0000 7f15c57ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15c015c9b0 0x7f15c00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:43.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.376+0000 7f15c57ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f15c0161340 0x7f15c00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:43.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.376+0000 7f15c57ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c00d1950 con 0x7f15c00d2320 2023-11-28T13:09:43.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.376+0000 7f15c57ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c00b5800 con 0x7f15c0161340 2023-11-28T13:09:43.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.376+0000 7f15c57ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c00b5520 con 0x7f15c015c9b0 2023-11-28T13:09:43.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.376+0000 7f15be7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f15c0161340 0x7f15c00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:43.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.376+0000 7f15bf7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f15c00d2320 0x7f15c00cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53304/0 (socket says 172.21.15.156:53304) 2023-11-28T13:09:43.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.376+0000 7f15bf7fe700 1 -- 172.21.15.156:0/134395467 learned_addr learned my addr 172.21.15.156:0/134395467 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:43.380 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.377+0000 7f15beffd700 1 --2- 172.21.15.156:0/134395467 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15c015c9b0 0x7f15c00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:43.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.377+0000 7f15be7fc700 1 -- 172.21.15.156:0/134395467 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15c015c9b0 msgr2=0x7f15c00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:43.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.377+0000 7f15be7fc700 1 --2- 172.21.15.156:0/134395467 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15c015c9b0 0x7f15c00b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.377+0000 7f15be7fc700 1 -- 172.21.15.156:0/134395467 >> v1:172.21.15.156:6789/0 conn(0x7f15c00d2320 legacy=0x7f15c00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:43.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.377+0000 7f15be7fc700 1 -- 172.21.15.156:0/134395467 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c00d1300 con 0x7f15c0161340 2023-11-28T13:09:43.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.377+0000 7f15bdffb700 1 -- 172.21.15.156:0/134395467 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918498942 0 0) 0x7f15c00d1950 con 0x7f15c00d2320 2023-11-28T13:09:43.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.377+0000 7f15be7fc700 1 --2- 172.21.15.156:0/134395467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15c0161340 0x7f15c00d1de0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009840 tx=0x7f15b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b418adf5b029f109 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:43.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.377+0000 7f15bdffb700 1 -- 172.21.15.156:0/134395467 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4011830 con 0x7f15c0161340 2023-11-28T13:09:43.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.377+0000 7f15bdffb700 1 -- 172.21.15.156:0/134395467 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b4011990 con 0x7f15c0161340 2023-11-28T13:09:43.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.377+0000 7f15bdffb700 1 -- 172.21.15.156:0/134395467 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4011d20 con 0x7f15c0161340 2023-11-28T13:09:43.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.378+0000 7f15c57ff700 1 -- 172.21.15.156:0/134395467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15c0161340 msgr2=0x7f15c00d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:43.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.378+0000 7f15c57ff700 1 --2- 172.21.15.156:0/134395467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15c0161340 0x7f15c00d1de0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009840 tx=0x7f15b4009b50 comp rx=0 tx=0).stop 2023-11-28T13:09:43.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.378+0000 7f15c57ff700 1 -- 172.21.15.156:0/134395467 shutdown_connections 2023-11-28T13:09:43.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.378+0000 7f15c57ff700 1 --2- 172.21.15.156:0/134395467 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15c015c9b0 0x7f15c00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.378+0000 7f15c57ff700 1 --2- 172.21.15.156:0/134395467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15c0161340 0x7f15c00d1de0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.382 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.378+0000 7f15c57ff700 1 -- 172.21.15.156:0/134395467 >> 172.21.15.156:0/134395467 conn(0x7f15c0072600 msgr2=0x7f15c00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:43.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.378+0000 7f15c57ff700 1 -- 172.21.15.156:0/134395467 shutdown_connections 2023-11-28T13:09:43.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.378+0000 7f15c57ff700 1 -- 172.21.15.156:0/134395467 wait complete. 2023-11-28T13:09:43.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.380+0000 7f15c57ff700 1 Processor -- start 2023-11-28T13:09:43.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.380+0000 7f15c57ff700 1 -- start start 2023-11-28T13:09:43.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.381+0000 7f15c57ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15c00d2320 0x7f15c01e7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:43.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.381+0000 7f15c57ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15c015c9b0 0x7f15c00b7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:43.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.381+0000 7f15c57ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15c0161340 0x7f15c00b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:43.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.381+0000 7f15c57ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c0140e10 con 0x7f15c015c9b0 2023-11-28T13:09:43.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.381+0000 7f15c57ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c01e8430 con 0x7f15c0161340 2023-11-28T13:09:43.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.381+0000 7f15c57ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c01e6460 con 0x7f15c00d2320 2023-11-28T13:09:43.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.381+0000 7f15be7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15c015c9b0 0x7f15c00b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:43.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.381+0000 7f15be7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15c015c9b0 0x7f15c00b7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36522/0 (socket says 172.21.15.156:36522) 2023-11-28T13:09:43.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.381+0000 7f15be7fc700 1 -- 172.21.15.156:0/462926681 learned_addr learned my addr 172.21.15.156:0/462926681 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:43.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.382+0000 7f15be7fc700 1 -- 172.21.15.156:0/462926681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15c00d2320 msgr2=0x7f15c01e7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:43.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.382+0000 7f15be7fc700 1 --2- 172.21.15.156:0/462926681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15c00d2320 0x7f15c01e7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.382+0000 7f15be7fc700 1 -- 172.21.15.156:0/462926681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15c0161340 msgr2=0x7f15c00b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:43.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.382+0000 7f15be7fc700 1 --2- 172.21.15.156:0/462926681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15c0161340 0x7f15c00b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.382+0000 7f15be7fc700 1 -- 172.21.15.156:0/462926681 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b40094a0 con 0x7f15c015c9b0 2023-11-28T13:09:43.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.382+0000 7f15be7fc700 1 --2- 172.21.15.156:0/462926681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15c015c9b0 0x7f15c00b7270 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f15c00675d0 tx=0x7f15b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:43.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.382+0000 7f15a77fe700 1 -- 172.21.15.156:0/462926681 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b40038a0 con 0x7f15c015c9b0 2023-11-28T13:09:43.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.382+0000 7f15a77fe700 1 -- 172.21.15.156:0/462926681 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b401f070 con 0x7f15c015c9b0 2023-11-28T13:09:43.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.382+0000 7f15c57ff700 1 -- 172.21.15.156:0/462926681 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c00b5130 con 0x7f15c015c9b0 2023-11-28T13:09:43.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.382+0000 7f15a77fe700 1 -- 172.21.15.156:0/462926681 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4003c20 con 0x7f15c015c9b0 2023-11-28T13:09:43.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.382+0000 7f15c57ff700 1 -- 172.21.15.156:0/462926681 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c00d15e0 con 0x7f15c015c9b0 2023-11-28T13:09:43.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.383+0000 7f15c57ff700 1 -- 172.21.15.156:0/462926681 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c00b52c0 con 0x7f15c015c9b0 2023-11-28T13:09:43.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.384+0000 7f15a77fe700 1 -- 172.21.15.156:0/462926681 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f15b400f050 con 0x7f15c015c9b0 2023-11-28T13:09:43.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.385+0000 7f15a77fe700 1 --2- 172.21.15.156:0/462926681 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f15ac0682f0 0x7f15ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:43.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.385+0000 7f15a77fe700 1 -- 172.21.15.156:0/462926681 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f15b4021070 con 0x7f15c015c9b0 2023-11-28T13:09:43.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.385+0000 7f15beffd700 1 --2- 172.21.15.156:0/462926681 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f15ac0682f0 0x7f15ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:43.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.386+0000 7f15beffd700 1 --2- 172.21.15.156:0/462926681 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f15ac0682f0 0x7f15ac06a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f15b00097f0 tx=0x7f15b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:43.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.389+0000 7f15a77fe700 1 -- 172.21.15.156:0/462926681 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f15b4053eb0 con 0x7f15c015c9b0 2023-11-28T13:09:43.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.658+0000 7f15c57ff700 1 -- 172.21.15.156:0/462926681 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f15c00b9150 con 0x7f15c015c9b0 2023-11-28T13:09:43.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.708+0000 7f15a77fe700 1 -- 172.21.15.156:0/462926681 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v107) v1 ==== 142+0+0 (secure 0 0 0) 0x7f15b40574d0 con 0x7f15c015c9b0 2023-11-28T13:09:43.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.711+0000 7f15c57ff700 1 -- 172.21.15.156:0/462926681 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f15ac0682f0 msgr2=0x7f15ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:43.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.711+0000 7f15c57ff700 1 --2- 172.21.15.156:0/462926681 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f15ac0682f0 0x7f15ac06a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f15b00097f0 tx=0x7f15b0008040 comp rx=0 tx=0).stop 2023-11-28T13:09:43.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.711+0000 7f15c57ff700 1 -- 172.21.15.156:0/462926681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15c015c9b0 msgr2=0x7f15c00b7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:43.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.711+0000 7f15c57ff700 1 --2- 172.21.15.156:0/462926681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15c015c9b0 0x7f15c00b7270 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f15c00675d0 tx=0x7f15b4004750 comp rx=0 tx=0).stop 2023-11-28T13:09:43.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.711+0000 7f15c57ff700 1 -- 172.21.15.156:0/462926681 shutdown_connections 2023-11-28T13:09:43.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.711+0000 7f15c57ff700 1 --2- 172.21.15.156:0/462926681 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f15ac0682f0 0x7f15ac06a7a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.711+0000 7f15c57ff700 1 --2- 172.21.15.156:0/462926681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15c0161340 0x7f15c00b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.711+0000 7f15c57ff700 1 --2- 172.21.15.156:0/462926681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15c00d2320 0x7f15c01e7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.711+0000 7f15c57ff700 1 --2- 172.21.15.156:0/462926681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15c015c9b0 0x7f15c00b7270 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.711+0000 7f15c57ff700 1 -- 172.21.15.156:0/462926681 >> 172.21.15.156:0/462926681 conn(0x7f15c0072600 msgr2=0x7f15c0160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:43.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.711+0000 7f15c57ff700 1 -- 172.21.15.156:0/462926681 shutdown_connections 2023-11-28T13:09:43.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.711+0000 7f15c57ff700 1 -- 172.21.15.156:0/462926681 wait complete. 2023-11-28T13:09:43.731 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2023-11-28T13:09:43.732 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2023-11-28T13:09:43.732 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:09:43.732 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T13:09:43.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.926+0000 7feb173ff700 1 Processor -- start 2023-11-28T13:09:43.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.927+0000 7feb173ff700 1 -- start start 2023-11-28T13:09:43.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.928+0000 7feb173ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb10161340 0x7feb100d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:43.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.928+0000 7feb173ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7feb100d2320 0x7feb100cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:43.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.928+0000 7feb173ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb100d1950 con 0x7feb1015c9b0 2023-11-28T13:09:43.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.928+0000 7feb173ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb100b5800 con 0x7feb100d2320 2023-11-28T13:09:43.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.928+0000 7feb173ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb100b5300 con 0x7feb10161340 2023-11-28T13:09:43.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.928+0000 7feb1599c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7feb100d2320 0x7feb100cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:43.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.928+0000 7feb1519b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7feb1015c9b0 0x7feb100b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53308/0 (socket says 172.21.15.156:53308) 2023-11-28T13:09:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.928+0000 7feb1519b700 1 -- 172.21.15.156:0/2835932411 learned_addr learned my addr 172.21.15.156:0/2835932411 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.928+0000 7feb1499a700 1 --2- 172.21.15.156:0/2835932411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb10161340 0x7feb100d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.928+0000 7feb1599c700 1 -- 172.21.15.156:0/2835932411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb10161340 msgr2=0x7feb100d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.928+0000 7feb1599c700 1 --2- 172.21.15.156:0/2835932411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb10161340 0x7feb100d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.928+0000 7feb07fff700 1 -- 172.21.15.156:0/2835932411 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448246036 0 0) 0x7feb100d1950 con 0x7feb1015c9b0 2023-11-28T13:09:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.928+0000 7feb1599c700 1 -- 172.21.15.156:0/2835932411 >> v1:172.21.15.156:6789/0 conn(0x7feb1015c9b0 legacy=0x7feb100b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.928+0000 7feb1599c700 1 -- 172.21.15.156:0/2835932411 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb100b5520 con 0x7feb100d2320 2023-11-28T13:09:43.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.929+0000 7feb1599c700 1 --2- 172.21.15.156:0/2835932411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb100d2320 0x7feb100cd7c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7feb08009dc0 tx=0x7feb0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93c451f82ce7883 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:43.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.929+0000 7feb07fff700 1 -- 172.21.15.156:0/2835932411 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0800e070 con 0x7feb100d2320 2023-11-28T13:09:43.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.929+0000 7feb07fff700 1 -- 172.21.15.156:0/2835932411 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0800bb90 con 0x7feb100d2320 2023-11-28T13:09:43.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.929+0000 7feb07fff700 1 -- 172.21.15.156:0/2835932411 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb08012430 con 0x7feb100d2320 2023-11-28T13:09:43.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.929+0000 7feb173ff700 1 -- 172.21.15.156:0/2835932411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb100d2320 msgr2=0x7feb100cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:43.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.929+0000 7feb173ff700 1 --2- 172.21.15.156:0/2835932411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb100d2320 0x7feb100cd7c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7feb08009dc0 tx=0x7feb0800b330 comp rx=0 tx=0).stop 2023-11-28T13:09:43.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.930+0000 7feb173ff700 1 -- 172.21.15.156:0/2835932411 shutdown_connections 2023-11-28T13:09:43.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.930+0000 7feb173ff700 1 --2- 172.21.15.156:0/2835932411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb100d2320 0x7feb100cd7c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.930+0000 7feb173ff700 1 --2- 172.21.15.156:0/2835932411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb10161340 0x7feb100d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.930+0000 7feb173ff700 1 -- 172.21.15.156:0/2835932411 >> 172.21.15.156:0/2835932411 conn(0x7feb10072600 msgr2=0x7feb100cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:43.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.930+0000 7feb173ff700 1 -- 172.21.15.156:0/2835932411 shutdown_connections 2023-11-28T13:09:43.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.930+0000 7feb173ff700 1 -- 172.21.15.156:0/2835932411 wait complete. 2023-11-28T13:09:43.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.931+0000 7feb173ff700 1 Processor -- start 2023-11-28T13:09:43.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.931+0000 7feb173ff700 1 -- start start 2023-11-28T13:09:43.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.932+0000 7feb173ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb100d2320 0x7feb101469b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:43.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.932+0000 7feb173ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb1015c9b0 0x7feb101594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:43.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.932+0000 7feb173ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb10161340 0x7feb10159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:43.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.932+0000 7feb173ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb10154350 con 0x7feb1015c9b0 2023-11-28T13:09:43.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.932+0000 7feb173ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb10146ef0 con 0x7feb100d2320 2023-11-28T13:09:43.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.932+0000 7feb173ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb10152130 con 0x7feb10161340 2023-11-28T13:09:43.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.933+0000 7feb1499a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb1015c9b0 0x7feb101594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:43.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.933+0000 7feb1499a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb1015c9b0 0x7feb101594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36550/0 (socket says 172.21.15.156:36550) 2023-11-28T13:09:43.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.933+0000 7feb1499a700 1 -- 172.21.15.156:0/149740875 learned_addr learned my addr 172.21.15.156:0/149740875 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:43.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.933+0000 7feb1599c700 1 --2- 172.21.15.156:0/149740875 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb10161340 0x7feb10159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:43.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.933+0000 7feb1519b700 1 --2- 172.21.15.156:0/149740875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb100d2320 0x7feb101469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:43.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.933+0000 7feb1499a700 1 -- 172.21.15.156:0/149740875 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb10161340 msgr2=0x7feb10159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:43.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.933+0000 7feb1499a700 1 --2- 172.21.15.156:0/149740875 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb10161340 0x7feb10159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.933+0000 7feb1499a700 1 -- 172.21.15.156:0/149740875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb100d2320 msgr2=0x7feb101469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:43.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.933+0000 7feb1499a700 1 --2- 172.21.15.156:0/149740875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb100d2320 0x7feb101469b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:43.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.933+0000 7feb1499a700 1 -- 172.21.15.156:0/149740875 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb08009a70 con 0x7feb1015c9b0 2023-11-28T13:09:43.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.933+0000 7feb1499a700 1 --2- 172.21.15.156:0/149740875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb1015c9b0 0x7feb101594d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7feb0000d800 tx=0x7feb0000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:43.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.934+0000 7feb05ffb700 1 -- 172.21.15.156:0/149740875 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb000125f0 con 0x7feb1015c9b0 2023-11-28T13:09:43.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.934+0000 7feb05ffb700 1 -- 172.21.15.156:0/149740875 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb00012750 con 0x7feb1015c9b0 2023-11-28T13:09:43.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.934+0000 7feb05ffb700 1 -- 172.21.15.156:0/149740875 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00012b00 con 0x7feb1015c9b0 2023-11-28T13:09:43.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.934+0000 7feb173ff700 1 -- 172.21.15.156:0/149740875 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb100d15e0 con 0x7feb1015c9b0 2023-11-28T13:09:43.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.934+0000 7feb173ff700 1 -- 172.21.15.156:0/149740875 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb100d1300 con 0x7feb1015c9b0 2023-11-28T13:09:43.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.935+0000 7feb173ff700 1 -- 172.21.15.156:0/149740875 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb1015cd80 con 0x7feb1015c9b0 2023-11-28T13:09:43.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.936+0000 7feb05ffb700 1 -- 172.21.15.156:0/149740875 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feb00012c60 con 0x7feb1015c9b0 2023-11-28T13:09:43.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.937+0000 7feb05ffb700 1 --2- 172.21.15.156:0/149740875 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feaf40682f0 0x7feaf406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:43.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.937+0000 7feb05ffb700 1 -- 172.21.15.156:0/149740875 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7feb00085b70 con 0x7feb1015c9b0 2023-11-28T13:09:43.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.937+0000 7feb1519b700 1 --2- 172.21.15.156:0/149740875 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feaf40682f0 0x7feaf406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:43.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.938+0000 7feb1519b700 1 --2- 172.21.15.156:0/149740875 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feaf40682f0 0x7feaf406a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7feb0c000f60 tx=0x7feb0c007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:43.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:43.941+0000 7feb05ffb700 1 -- 172.21.15.156:0/149740875 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feb00052600 con 0x7feb1015c9b0 2023-11-28T13:09:44.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.205+0000 7feb173ff700 1 -- 172.21.15.156:0/149740875 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7feb100745c0 con 0x7feb1015c9b0 2023-11-28T13:09:44.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.714+0000 7feb05ffb700 1 -- 172.21.15.156:0/149740875 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v108) v1 ==== 152+0+0 (secure 0 0 0) 0x7feb00055c20 con 0x7feb1015c9b0 2023-11-28T13:09:44.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.716+0000 7feb173ff700 1 -- 172.21.15.156:0/149740875 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feaf40682f0 msgr2=0x7feaf406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:44.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.716+0000 7feb173ff700 1 --2- 172.21.15.156:0/149740875 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feaf40682f0 0x7feaf406a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7feb0c000f60 tx=0x7feb0c007e70 comp rx=0 tx=0).stop 2023-11-28T13:09:44.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.716+0000 7feb173ff700 1 -- 172.21.15.156:0/149740875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb1015c9b0 msgr2=0x7feb101594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:44.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.716+0000 7feb173ff700 1 --2- 172.21.15.156:0/149740875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb1015c9b0 0x7feb101594d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7feb0000d800 tx=0x7feb0000db10 comp rx=0 tx=0).stop 2023-11-28T13:09:44.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.716+0000 7feb173ff700 1 -- 172.21.15.156:0/149740875 shutdown_connections 2023-11-28T13:09:44.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.716+0000 7feb173ff700 1 --2- 172.21.15.156:0/149740875 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feaf40682f0 0x7feaf406a7a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:44.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.716+0000 7feb173ff700 1 --2- 172.21.15.156:0/149740875 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb10161340 0x7feb10159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:44.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.716+0000 7feb173ff700 1 --2- 172.21.15.156:0/149740875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb100d2320 0x7feb101469b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:44.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.716+0000 7feb173ff700 1 --2- 172.21.15.156:0/149740875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb1015c9b0 0x7feb101594d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:44.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.716+0000 7feb173ff700 1 -- 172.21.15.156:0/149740875 >> 172.21.15.156:0/149740875 conn(0x7feb10072600 msgr2=0x7feb1006b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:44.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.717+0000 7feb173ff700 1 -- 172.21.15.156:0/149740875 shutdown_connections 2023-11-28T13:09:44.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.717+0000 7feb173ff700 1 -- 172.21.15.156:0/149740875 wait complete. 2023-11-28T13:09:44.737 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T13:09:44.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.937+0000 7fa2457ff700 1 Processor -- start 2023-11-28T13:09:44.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.937+0000 7fa2457ff700 1 -- start start 2023-11-28T13:09:44.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.938+0000 7fa2457ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa240161340 0x7fa2400d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:44.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.938+0000 7fa2457ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa2400d2320 0x7fa2400cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:44.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.938+0000 7fa2457ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2400d1950 con 0x7fa24015c9b0 2023-11-28T13:09:44.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.938+0000 7fa2457ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2400b5800 con 0x7fa240161340 2023-11-28T13:09:44.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.938+0000 7fa2457ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2400b5300 con 0x7fa2400d2320 2023-11-28T13:09:44.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.938+0000 7fa237fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa240161340 0x7fa2400d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:44.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.938+0000 7fa237fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa240161340 0x7fa2400d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36562/0 (socket says 172.21.15.156:36562) 2023-11-28T13:09:44.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.938+0000 7fa237fff700 1 -- 172.21.15.156:0/639916196 learned_addr learned my addr 172.21.15.156:0/639916196 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:44.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.938+0000 7fa23f7fe700 1 --2- 172.21.15.156:0/639916196 >> v2:172.21.15.186:3300/0 conn(0x7fa2400d2320 0x7fa2400cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:44.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.939+0000 7fa237fff700 1 -- 172.21.15.156:0/639916196 >> v2:172.21.15.186:3300/0 conn(0x7fa2400d2320 msgr2=0x7fa2400cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:44.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.939+0000 7fa237fff700 1 --2- 172.21.15.156:0/639916196 >> v2:172.21.15.186:3300/0 conn(0x7fa2400d2320 0x7fa2400cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:44.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.939+0000 7fa237fff700 1 -- 172.21.15.156:0/639916196 >> v1:172.21.15.186:6789/0 conn(0x7fa24015c9b0 legacy=0x7fa2400b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:44.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.939+0000 7fa237fff700 1 -- 172.21.15.156:0/639916196 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2400b5520 con 0x7fa240161340 2023-11-28T13:09:44.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.939+0000 7fa237fff700 1 --2- 172.21.15.156:0/639916196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa240161340 0x7fa2400d1de0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa24006c170 tx=0x7fa228009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f377be8ca746acd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:44.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.939+0000 7fa23e7fc700 1 -- 172.21.15.156:0/639916196 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228004230 con 0x7fa240161340 2023-11-28T13:09:44.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.939+0000 7fa23e7fc700 1 -- 172.21.15.156:0/639916196 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa228004390 con 0x7fa240161340 2023-11-28T13:09:44.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.939+0000 7fa23e7fc700 1 -- 172.21.15.156:0/639916196 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2280036a0 con 0x7fa240161340 2023-11-28T13:09:44.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.940+0000 7fa2457ff700 1 -- 172.21.15.156:0/639916196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa240161340 msgr2=0x7fa2400d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:44.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.940+0000 7fa2457ff700 1 --2- 172.21.15.156:0/639916196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa240161340 0x7fa2400d1de0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa24006c170 tx=0x7fa228009c00 comp rx=0 tx=0).stop 2023-11-28T13:09:44.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.940+0000 7fa2457ff700 1 -- 172.21.15.156:0/639916196 shutdown_connections 2023-11-28T13:09:44.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.940+0000 7fa2457ff700 1 --2- 172.21.15.156:0/639916196 >> v2:172.21.15.186:3300/0 conn(0x7fa2400d2320 0x7fa2400cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:44.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.940+0000 7fa2457ff700 1 --2- 172.21.15.156:0/639916196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa240161340 0x7fa2400d1de0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:44.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.940+0000 7fa2457ff700 1 -- 172.21.15.156:0/639916196 >> 172.21.15.156:0/639916196 conn(0x7fa240072600 msgr2=0x7fa2400cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:44.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.940+0000 7fa2457ff700 1 -- 172.21.15.156:0/639916196 shutdown_connections 2023-11-28T13:09:44.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.940+0000 7fa2457ff700 1 -- 172.21.15.156:0/639916196 wait complete. 2023-11-28T13:09:44.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.942+0000 7fa2457ff700 1 Processor -- start 2023-11-28T13:09:44.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.942+0000 7fa2457ff700 1 -- start start 2023-11-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.944+0000 7fa2457ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2400d2320 0x7fa240150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.944+0000 7fa2457ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa24015c9b0 0x7fa2401577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.944+0000 7fa2457ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa240161340 0x7fa24015f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.944+0000 7fa2457ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2401e6fa0 con 0x7fa2400d2320 2023-11-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.944+0000 7fa2457ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24015fa90 con 0x7fa240161340 2023-11-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.944+0000 7fa2457ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2401e6600 con 0x7fa24015c9b0 2023-11-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.944+0000 7fa23effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2400d2320 0x7fa240150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.944+0000 7fa23effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2400d2320 0x7fa240150550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36576/0 (socket says 172.21.15.156:36576) 2023-11-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.944+0000 7fa23effd700 1 -- 172.21.15.156:0/1933480059 learned_addr learned my addr 172.21.15.156:0/1933480059 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.944+0000 7fa237fff700 1 --2- 172.21.15.156:0/1933480059 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa24015c9b0 0x7fa2401577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.944+0000 7fa23f7fe700 1 --2- 172.21.15.156:0/1933480059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa240161340 0x7fa24015f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:44.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.944+0000 7fa23effd700 1 -- 172.21.15.156:0/1933480059 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa24015c9b0 msgr2=0x7fa2401577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:44.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.944+0000 7fa23effd700 1 --2- 172.21.15.156:0/1933480059 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa24015c9b0 0x7fa2401577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:44.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.944+0000 7fa23effd700 1 -- 172.21.15.156:0/1933480059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa240161340 msgr2=0x7fa24015f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:44.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.944+0000 7fa23effd700 1 --2- 172.21.15.156:0/1933480059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa240161340 0x7fa24015f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:44.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.944+0000 7fa23effd700 1 -- 172.21.15.156:0/1933480059 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa228009550 con 0x7fa2400d2320 2023-11-28T13:09:44.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.945+0000 7fa23effd700 1 --2- 172.21.15.156:0/1933480059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2400d2320 0x7fa240150550 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa23000b4d0 tx=0x7fa23000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:44.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.945+0000 7fa2377fe700 1 -- 172.21.15.156:0/1933480059 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230011830 con 0x7fa2400d2320 2023-11-28T13:09:44.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.945+0000 7fa2377fe700 1 -- 172.21.15.156:0/1933480059 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa230011990 con 0x7fa2400d2320 2023-11-28T13:09:44.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.945+0000 7fa2457ff700 1 -- 172.21.15.156:0/1933480059 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2400d15e0 con 0x7fa2400d2320 2023-11-28T13:09:44.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.945+0000 7fa2377fe700 1 -- 172.21.15.156:0/1933480059 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230011d40 con 0x7fa2400d2320 2023-11-28T13:09:44.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.945+0000 7fa2457ff700 1 -- 172.21.15.156:0/1933480059 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2400d1300 con 0x7fa2400d2320 2023-11-28T13:09:44.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.946+0000 7fa2457ff700 1 -- 172.21.15.156:0/1933480059 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa24015cd80 con 0x7fa2400d2320 2023-11-28T13:09:44.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.947+0000 7fa2377fe700 1 -- 172.21.15.156:0/1933480059 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa23000f050 con 0x7fa2400d2320 2023-11-28T13:09:44.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.948+0000 7fa2377fe700 1 --2- 172.21.15.156:0/1933480059 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa2200682f0 0x7fa22006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:44.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.948+0000 7fa2377fe700 1 -- 172.21.15.156:0/1933480059 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa2300855b0 con 0x7fa2400d2320 2023-11-28T13:09:44.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.948+0000 7fa237fff700 1 --2- 172.21.15.156:0/1933480059 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa2200682f0 0x7fa22006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:44.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.949+0000 7fa237fff700 1 --2- 172.21.15.156:0/1933480059 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa2200682f0 0x7fa22006a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa228009870 tx=0x7fa228008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:44.953 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:44.952+0000 7fa2377fe700 1 -- 172.21.15.156:0/1933480059 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa230052020 con 0x7fa2400d2320 2023-11-28T13:09:45.148 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.146+0000 7fa2457ff700 1 -- 172.21.15.156:0/1933480059 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa2400745c0 con 0x7fa2400d2320 2023-11-28T13:09:45.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.150+0000 7fa2377fe700 1 -- 172.21.15.156:0/1933480059 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa230015020 con 0x7fa2400d2320 2023-11-28T13:09:45.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.153+0000 7fa2457ff700 1 -- 172.21.15.156:0/1933480059 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa2200682f0 msgr2=0x7fa22006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:45.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.153+0000 7fa2457ff700 1 --2- 172.21.15.156:0/1933480059 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa2200682f0 0x7fa22006a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa228009870 tx=0x7fa228008030 comp rx=0 tx=0).stop 2023-11-28T13:09:45.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.153+0000 7fa2457ff700 1 -- 172.21.15.156:0/1933480059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2400d2320 msgr2=0x7fa240150550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:45.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.153+0000 7fa2457ff700 1 --2- 172.21.15.156:0/1933480059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2400d2320 0x7fa240150550 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa23000b4d0 tx=0x7fa23000b890 comp rx=0 tx=0).stop 2023-11-28T13:09:45.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.153+0000 7fa2457ff700 1 -- 172.21.15.156:0/1933480059 shutdown_connections 2023-11-28T13:09:45.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.153+0000 7fa2457ff700 1 --2- 172.21.15.156:0/1933480059 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa2200682f0 0x7fa22006a7a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:45.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.153+0000 7fa2457ff700 1 --2- 172.21.15.156:0/1933480059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa240161340 0x7fa24015f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:45.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.153+0000 7fa2457ff700 1 --2- 172.21.15.156:0/1933480059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2400d2320 0x7fa240150550 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:45.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.153+0000 7fa2457ff700 1 --2- 172.21.15.156:0/1933480059 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa24015c9b0 0x7fa2401577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:45.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.153+0000 7fa2457ff700 1 -- 172.21.15.156:0/1933480059 >> 172.21.15.156:0/1933480059 conn(0x7fa240072600 msgr2=0x7fa2400b6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:45.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.153+0000 7fa2457ff700 1 -- 172.21.15.156:0/1933480059 shutdown_connections 2023-11-28T13:09:45.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.153+0000 7fa2457ff700 1 -- 172.21.15.156:0/1933480059 wait complete. 2023-11-28T13:09:45.173 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:09:45.173 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:09:45.274 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:09:45.370 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:09:45.370 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:09:45.370 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:09:45.415 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:09:45.415 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:09:45.415 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:09:45.461 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:09:45.461 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:09:45.461 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:09:45.505 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:09:45.506 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:09:45.506 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:09:45.549 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:09:45.549 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T13:09:45.549 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:09:45.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.747+0000 7fc1f6fff700 1 Processor -- start 2023-11-28T13:09:45.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.747+0000 7fc1f6fff700 1 -- start start 2023-11-28T13:09:45.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.748+0000 7fc1f6fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc1f015ca50 0x7fc1f006b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:45.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.748+0000 7fc1f6fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1f006b8c0 0x7fc1f00cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:45.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.748+0000 7fc1f6fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f00cf5e0 con 0x7fc1f00ccc40 2023-11-28T13:09:45.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.748+0000 7fc1f6fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f00cd470 con 0x7fc1f015ca50 2023-11-28T13:09:45.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.748+0000 7fc1f6fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f006bd30 con 0x7fc1f006b8c0 2023-11-28T13:09:45.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.749+0000 7fc1f559c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fc1f00ccc40 0x7fc1f00cd7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53316/0 (socket says 172.21.15.156:53316) 2023-11-28T13:09:45.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.749+0000 7fc1f4d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc1f015ca50 0x7fc1f006b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:45.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.749+0000 7fc1f559c700 1 -- 172.21.15.156:0/817022555 learned_addr learned my addr 172.21.15.156:0/817022555 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:45.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.749+0000 7fc1effff700 1 --2- 172.21.15.156:0/817022555 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1f006b8c0 0x7fc1f00cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:45.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.749+0000 7fc1ef7fe700 1 -- 172.21.15.156:0/817022555 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260875685 0 0) 0x7fc1f00cf5e0 con 0x7fc1f00ccc40 2023-11-28T13:09:45.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.749+0000 7fc1ef7fe700 1 -- 172.21.15.156:0/817022555 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1e00035b0 con 0x7fc1f00ccc40 2023-11-28T13:09:45.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.749+0000 7fc1f4d9b700 1 -- 172.21.15.156:0/817022555 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1f006b8c0 msgr2=0x7fc1f00cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:45.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.749+0000 7fc1f4d9b700 1 --2- 172.21.15.156:0/817022555 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1f006b8c0 0x7fc1f00cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:45.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.749+0000 7fc1f4d9b700 1 -- 172.21.15.156:0/817022555 >> v1:172.21.15.156:6789/0 conn(0x7fc1f00ccc40 legacy=0x7fc1f00cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:45.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.749+0000 7fc1f4d9b700 1 -- 172.21.15.156:0/817022555 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f00cecb0 con 0x7fc1f015ca50 2023-11-28T13:09:45.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.749+0000 7fc1f4d9b700 1 --2- 172.21.15.156:0/817022555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1f015ca50 0x7fc1f006b380 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009870 tx=0x7fc1e4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263309ce28c5ea07 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:45.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.750+0000 7fc1ef7fe700 1 -- 172.21.15.156:0/817022555 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4011830 con 0x7fc1f015ca50 2023-11-28T13:09:45.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.750+0000 7fc1ef7fe700 1 -- 172.21.15.156:0/817022555 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e4011990 con 0x7fc1f015ca50 2023-11-28T13:09:45.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.750+0000 7fc1ef7fe700 1 -- 172.21.15.156:0/817022555 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4011d20 con 0x7fc1f015ca50 2023-11-28T13:09:45.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.750+0000 7fc1f6fff700 1 -- 172.21.15.156:0/817022555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1f015ca50 msgr2=0x7fc1f006b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:45.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.750+0000 7fc1f6fff700 1 --2- 172.21.15.156:0/817022555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1f015ca50 0x7fc1f006b380 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009870 tx=0x7fc1e4009b80 comp rx=0 tx=0).stop 2023-11-28T13:09:45.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.750+0000 7fc1f6fff700 1 -- 172.21.15.156:0/817022555 shutdown_connections 2023-11-28T13:09:45.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.750+0000 7fc1f6fff700 1 --2- 172.21.15.156:0/817022555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1f015ca50 0x7fc1f006b380 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:45.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.750+0000 7fc1f6fff700 1 --2- 172.21.15.156:0/817022555 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1f006b8c0 0x7fc1f00cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:45.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.750+0000 7fc1f6fff700 1 -- 172.21.15.156:0/817022555 >> 172.21.15.156:0/817022555 conn(0x7fc1f009a230 msgr2=0x7fc1f008afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:45.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.750+0000 7fc1f6fff700 1 -- 172.21.15.156:0/817022555 shutdown_connections 2023-11-28T13:09:45.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.750+0000 7fc1f6fff700 1 -- 172.21.15.156:0/817022555 wait complete. 2023-11-28T13:09:45.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.752+0000 7fc1f6fff700 1 Processor -- start 2023-11-28T13:09:45.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.752+0000 7fc1f6fff700 1 -- start start 2023-11-28T13:09:45.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.753+0000 7fc1f6fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1f006b8c0 0x7fc1f015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:45.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.753+0000 7fc1f6fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1f00ccc40 0x7fc1f0149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:45.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.753+0000 7fc1f6fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1f015ca50 0x7fc1f00ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:45.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.753+0000 7fc1f6fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f014aee0 con 0x7fc1f00ccc40 2023-11-28T13:09:45.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.753+0000 7fc1f6fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f014a920 con 0x7fc1f015ca50 2023-11-28T13:09:45.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.753+0000 7fc1f6fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f014ac00 con 0x7fc1f006b8c0 2023-11-28T13:09:45.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.753+0000 7fc1effff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1f00ccc40 0x7fc1f0149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:45.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.753+0000 7fc1effff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1f00ccc40 0x7fc1f0149f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36596/0 (socket says 172.21.15.156:36596) 2023-11-28T13:09:45.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.753+0000 7fc1f4d9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1f006b8c0 0x7fc1f015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:45.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.753+0000 7fc1effff700 1 -- 172.21.15.156:0/1384199874 learned_addr learned my addr 172.21.15.156:0/1384199874 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:45.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.753+0000 7fc1f559c700 1 --2- 172.21.15.156:0/1384199874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1f015ca50 0x7fc1f00ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:45.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.754+0000 7fc1effff700 1 -- 172.21.15.156:0/1384199874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1f006b8c0 msgr2=0x7fc1f015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:45.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.754+0000 7fc1effff700 1 --2- 172.21.15.156:0/1384199874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1f006b8c0 0x7fc1f015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:45.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.754+0000 7fc1effff700 1 -- 172.21.15.156:0/1384199874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1f015ca50 msgr2=0x7fc1f00ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:45.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.754+0000 7fc1effff700 1 --2- 172.21.15.156:0/1384199874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1f015ca50 0x7fc1f00ae530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:45.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.754+0000 7fc1effff700 1 -- 172.21.15.156:0/1384199874 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e40094a0 con 0x7fc1f00ccc40 2023-11-28T13:09:45.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.754+0000 7fc1effff700 1 --2- 172.21.15.156:0/1384199874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1f00ccc40 0x7fc1f0149f40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00e9b0 tx=0x7fc1dc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:45.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.754+0000 7fc1ed7fa700 1 -- 172.21.15.156:0/1384199874 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc009710 con 0x7fc1f00ccc40 2023-11-28T13:09:45.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.754+0000 7fc1ed7fa700 1 -- 172.21.15.156:0/1384199874 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1dc00ee50 con 0x7fc1f00ccc40 2023-11-28T13:09:45.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.754+0000 7fc1ed7fa700 1 -- 172.21.15.156:0/1384199874 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc00ca50 con 0x7fc1f00ccc40 2023-11-28T13:09:45.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.754+0000 7fc1f6fff700 1 -- 172.21.15.156:0/1384199874 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f00cf270 con 0x7fc1f00ccc40 2023-11-28T13:09:45.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.754+0000 7fc1f6fff700 1 -- 172.21.15.156:0/1384199874 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f00cef90 con 0x7fc1f00ccc40 2023-11-28T13:09:45.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.756+0000 7fc1f6fff700 1 -- 172.21.15.156:0/1384199874 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f00cd0c0 con 0x7fc1f00ccc40 2023-11-28T13:09:45.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.756+0000 7fc1ed7fa700 1 -- 172.21.15.156:0/1384199874 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc1dc019030 con 0x7fc1f00ccc40 2023-11-28T13:09:45.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.757+0000 7fc1ed7fa700 1 --2- 172.21.15.156:0/1384199874 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc1d40682f0 0x7fc1d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:45.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.758+0000 7fc1ed7fa700 1 -- 172.21.15.156:0/1384199874 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc1dc085960 con 0x7fc1f00ccc40 2023-11-28T13:09:45.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.758+0000 7fc1f4d9b700 1 --2- 172.21.15.156:0/1384199874 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc1d40682f0 0x7fc1d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:45.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.758+0000 7fc1f4d9b700 1 --2- 172.21.15.156:0/1384199874 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc1d40682f0 0x7fc1d406a7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40097c0 tx=0x7fc1e401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:45.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.761+0000 7fc1ed7fa700 1 -- 172.21.15.156:0/1384199874 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc1dc052390 con 0x7fc1f00ccc40 2023-11-28T13:09:45.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.994+0000 7fc1f6fff700 1 -- 172.21.15.156:0/1384199874 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1f00b39f0 con 0x7fc1f00ccc40 2023-11-28T13:09:45.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.995+0000 7fc1ed7fa700 1 -- 172.21.15.156:0/1384199874 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3538 (secure 0 0 0) 0x7fc1dc021df0 con 0x7fc1f00ccc40 2023-11-28T13:09:45.997 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:45.997 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1128367125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":1128367125},{"type":"v1","addr":"172.21.15.156:6835","nonce":1128367125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4942,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/4096182008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":4096182008},{"type":"v1","addr":"172.21.15.186:6835","nonce":4096182008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4945,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/561812165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":561812165},{"type":"v1","addr":"172.21.15.186:6837","nonce":561812165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4241943773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4241943773},{"type":"v1","addr":"172.21.15.156:6834","nonce":4241943773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2023-11-28T13:09:45.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.997+0000 7fc1f6fff700 1 -- 172.21.15.156:0/1384199874 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc1d40682f0 msgr2=0x7fc1d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:45.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.997+0000 7fc1f6fff700 1 --2- 172.21.15.156:0/1384199874 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc1d40682f0 0x7fc1d406a7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40097c0 tx=0x7fc1e401c040 comp rx=0 tx=0).stop 2023-11-28T13:09:45.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.997+0000 7fc1f6fff700 1 -- 172.21.15.156:0/1384199874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1f00ccc40 msgr2=0x7fc1f0149f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:45.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.997+0000 7fc1f6fff700 1 --2- 172.21.15.156:0/1384199874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1f00ccc40 0x7fc1f0149f40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00e9b0 tx=0x7fc1dc00ecc0 comp rx=0 tx=0).stop 2023-11-28T13:09:45.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.997+0000 7fc1f6fff700 1 -- 172.21.15.156:0/1384199874 shutdown_connections 2023-11-28T13:09:45.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.997+0000 7fc1f6fff700 1 --2- 172.21.15.156:0/1384199874 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc1d40682f0 0x7fc1d406a7a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:45.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.997+0000 7fc1f6fff700 1 --2- 172.21.15.156:0/1384199874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1f015ca50 0x7fc1f00ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:46.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.997+0000 7fc1f6fff700 1 --2- 172.21.15.156:0/1384199874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc1f006b8c0 0x7fc1f015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:46.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.997+0000 7fc1f6fff700 1 --2- 172.21.15.156:0/1384199874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1f00ccc40 0x7fc1f0149f40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:46.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.997+0000 7fc1f6fff700 1 -- 172.21.15.156:0/1384199874 >> 172.21.15.156:0/1384199874 conn(0x7fc1f009a230 msgr2=0x7fc1f00d0600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:46.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.997+0000 7fc1f6fff700 1 -- 172.21.15.156:0/1384199874 shutdown_connections 2023-11-28T13:09:46.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:45.998+0000 7fc1f6fff700 1 -- 172.21.15.156:0/1384199874 wait complete. 2023-11-28T13:09:46.000 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 18 2023-11-28T13:09:46.018 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T13:09:46.022 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T13:09:46.022 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T13:09:46.022 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:09:46.022 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T13:09:46.022 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T13:09:46.023 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T13:09:46.023 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:09:46.023 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T13:09:46.023 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T13:09:46.023 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T13:09:46.023 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:09:46.023 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T13:09:46.024 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T13:09:46.024 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T13:09:46.024 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:09:46.038 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T13:09:46.038 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T13:09:46.042 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T13:09:46.042 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T13:09:46.043 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T13:09:46.043 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T13:09:46.044 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T13:09:46.045 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T13:09:46.084 INFO:tasks.ceph.mds.a:Started 2023-11-28T13:09:46.084 DEBUG:teuthology.parallel:result is None 2023-11-28T13:09:46.088 INFO:tasks.ceph.mds.c:Started 2023-11-28T13:09:46.088 INFO:tasks.ceph.mds.b:Started 2023-11-28T13:09:46.089 DEBUG:teuthology.parallel:result is None 2023-11-28T13:09:46.089 DEBUG:teuthology.parallel:result is None 2023-11-28T13:09:46.092 INFO:tasks.ceph.mds.d:Started 2023-11-28T13:09:46.092 DEBUG:teuthology.parallel:result is None 2023-11-28T13:09:46.092 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T13:09:46.233 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2023-11-28T13:09:46.234 INFO:tasks.ceph.mds.c.smithi156.stdout:starting mds.c at 2023-11-28T13:09:46.236 INFO:tasks.ceph.mds.a.smithi156.stdout:starting mds.a at 2023-11-28T13:09:46.236 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2023-11-28T13:09:46.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.308+0000 7f03705ff700 1 Processor -- start 2023-11-28T13:09:46.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.308+0000 7f03705ff700 1 -- start start 2023-11-28T13:09:46.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.309+0000 7f03705ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f036815c9b0 0x7f03680b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:46.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.309+0000 7f03705ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f03680cd7c0 0x7f03680cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:46.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.309+0000 7f03705ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03680ce160 con 0x7f0368161340 2023-11-28T13:09:46.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.309+0000 7f03705ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03680b5800 con 0x7f03680cd7c0 2023-11-28T13:09:46.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.309+0000 7f03705ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03680d1950 con 0x7f036815c9b0 2023-11-28T13:09:46.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.310+0000 7f036eb9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f03680cd7c0 0x7f03680cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:46.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.310+0000 7f036eb9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f03680cd7c0 0x7f03680cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38788/0 (socket says 172.21.15.156:38788) 2023-11-28T13:09:46.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.310+0000 7f036eb9c700 1 -- 172.21.15.156:0/879473822 learned_addr learned my addr 172.21.15.156:0/879473822 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:46.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.310+0000 7f036e39b700 1 --2- 172.21.15.156:0/879473822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f036815c9b0 0x7f03680b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:46.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.310+0000 7f036eb9c700 1 -- 172.21.15.156:0/879473822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f036815c9b0 msgr2=0x7f03680b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:46.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.310+0000 7f036eb9c700 1 --2- 172.21.15.156:0/879473822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f036815c9b0 0x7f03680b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:46.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.310+0000 7f036eb9c700 1 -- 172.21.15.156:0/879473822 >> v1:172.21.15.186:6789/0 conn(0x7f0368161340 legacy=0x7f03680d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:46.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.310+0000 7f036eb9c700 1 -- 172.21.15.156:0/879473822 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03680d24f0 con 0x7f03680cd7c0 2023-11-28T13:09:46.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.310+0000 7f036eb9c700 1 --2- 172.21.15.156:0/879473822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03680cd7c0 0x7f03680cdc20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0360009e50 tx=0x7f036000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7519282cb8d6645e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:46.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.311+0000 7f036d399700 1 -- 172.21.15.156:0/879473822 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036000e070 con 0x7f03680cd7c0 2023-11-28T13:09:46.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.311+0000 7f036d399700 1 -- 172.21.15.156:0/879473822 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036000b910 con 0x7f03680cd7c0 2023-11-28T13:09:46.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.311+0000 7f036d399700 1 -- 172.21.15.156:0/879473822 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036000bcc0 con 0x7f03680cd7c0 2023-11-28T13:09:46.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.311+0000 7f03705ff700 1 -- 172.21.15.156:0/879473822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03680cd7c0 msgr2=0x7f03680cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:46.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.311+0000 7f03705ff700 1 --2- 172.21.15.156:0/879473822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03680cd7c0 0x7f03680cdc20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0360009e50 tx=0x7f036000b330 comp rx=0 tx=0).stop 2023-11-28T13:09:46.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.311+0000 7f03705ff700 1 -- 172.21.15.156:0/879473822 shutdown_connections 2023-11-28T13:09:46.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.311+0000 7f03705ff700 1 --2- 172.21.15.156:0/879473822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03680cd7c0 0x7f03680cdc20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:46.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.311+0000 7f03705ff700 1 --2- 172.21.15.156:0/879473822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f036815c9b0 0x7f03680b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:46.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.311+0000 7f03705ff700 1 -- 172.21.15.156:0/879473822 >> 172.21.15.156:0/879473822 conn(0x7f0368072600 msgr2=0x7f03680cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:46.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.311+0000 7f03705ff700 1 -- 172.21.15.156:0/879473822 shutdown_connections 2023-11-28T13:09:46.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.311+0000 7f03705ff700 1 -- 172.21.15.156:0/879473822 wait complete. 2023-11-28T13:09:46.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.313+0000 7f03705ff700 1 Processor -- start 2023-11-28T13:09:46.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.313+0000 7f03705ff700 1 -- start start 2023-11-28T13:09:46.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.314+0000 7f03705ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03680cd7c0 0x7f03681544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:46.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.314+0000 7f03705ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f036815c9b0 0x7f03681593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:46.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.314+0000 7f03705ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0368161340 0x7f03681598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:46.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.314+0000 7f03705ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03681e6a90 con 0x7f03680cd7c0 2023-11-28T13:09:46.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.314+0000 7f03705ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03680cc0f0 con 0x7f036815c9b0 2023-11-28T13:09:46.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.314+0000 7f03705ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03680cc3d0 con 0x7f0368161340 2023-11-28T13:09:46.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.315+0000 7f036e39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03680cd7c0 0x7f03681544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:46.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.315+0000 7f036e39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03680cd7c0 0x7f03681544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38804/0 (socket says 172.21.15.156:38804) 2023-11-28T13:09:46.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.315+0000 7f036e39b700 1 -- 172.21.15.156:0/3993086223 learned_addr learned my addr 172.21.15.156:0/3993086223 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:46.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.315+0000 7f036db9a700 1 --2- 172.21.15.156:0/3993086223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f036815c9b0 0x7f03681593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:46.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.315+0000 7f036eb9c700 1 --2- 172.21.15.156:0/3993086223 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0368161340 0x7f03681598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:46.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.315+0000 7f036e39b700 1 -- 172.21.15.156:0/3993086223 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0368161340 msgr2=0x7f03681598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:46.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.315+0000 7f036e39b700 1 --2- 172.21.15.156:0/3993086223 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0368161340 0x7f03681598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:46.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.315+0000 7f036e39b700 1 -- 172.21.15.156:0/3993086223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f036815c9b0 msgr2=0x7f03681593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:46.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.315+0000 7f036e39b700 1 --2- 172.21.15.156:0/3993086223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f036815c9b0 0x7f03681593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:46.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.315+0000 7f036e39b700 1 -- 172.21.15.156:0/3993086223 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0360009b30 con 0x7f03680cd7c0 2023-11-28T13:09:46.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.315+0000 7f036e39b700 1 --2- 172.21.15.156:0/3993086223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03680cd7c0 0x7f03681544f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f036400e870 tx=0x7f036400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:46.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.315+0000 7f035effd700 1 -- 172.21.15.156:0/3993086223 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364009590 con 0x7f03680cd7c0 2023-11-28T13:09:46.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.316+0000 7f035effd700 1 -- 172.21.15.156:0/3993086223 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0364013680 con 0x7f03680cd7c0 2023-11-28T13:09:46.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.316+0000 7f035effd700 1 -- 172.21.15.156:0/3993086223 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03640139f0 con 0x7f03680cd7c0 2023-11-28T13:09:46.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.316+0000 7f03705ff700 1 -- 172.21.15.156:0/3993086223 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03680d1300 con 0x7f03680cd7c0 2023-11-28T13:09:46.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.316+0000 7f03705ff700 1 -- 172.21.15.156:0/3993086223 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03680b5520 con 0x7f03680cd7c0 2023-11-28T13:09:46.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.317+0000 7f03705ff700 1 -- 172.21.15.156:0/3993086223 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03680d15e0 con 0x7f03680cd7c0 2023-11-28T13:09:46.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.317+0000 7f035effd700 1 -- 172.21.15.156:0/3993086223 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0364019030 con 0x7f03680cd7c0 2023-11-28T13:09:46.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.318+0000 7f035effd700 1 --2- 172.21.15.156:0/3993086223 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f034c0682f0 0x7f034c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:46.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.318+0000 7f035effd700 1 -- 172.21.15.156:0/3993086223 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0364085650 con 0x7f03680cd7c0 2023-11-28T13:09:46.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.319+0000 7f036db9a700 1 --2- 172.21.15.156:0/3993086223 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f034c0682f0 0x7f034c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:46.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.319+0000 7f036db9a700 1 --2- 172.21.15.156:0/3993086223 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f034c0682f0 0x7f034c06a7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0358007fd0 tx=0x7f0358009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:46.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.322+0000 7f035effd700 1 -- 172.21.15.156:0/3993086223 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f03640521b0 con 0x7f03680cd7c0 2023-11-28T13:09:46.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:46.529+0000 7f03705ff700 1 -- 172.21.15.156:0/3993086223 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f03680cc510 con 0x7f03680cd7c0 2023-11-28T13:09:47.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.273+0000 7f035effd700 1 -- 172.21.15.156:0/3993086223 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v43) v1 ==== 97+0+0 (secure 0 0 0) 0x7f03680cc510 con 0x7f03680cd7c0 2023-11-28T13:09:47.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.275+0000 7f03705ff700 1 -- 172.21.15.156:0/3993086223 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f034c0682f0 msgr2=0x7f034c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:47.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.275+0000 7f03705ff700 1 --2- 172.21.15.156:0/3993086223 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f034c0682f0 0x7f034c06a7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0358007fd0 tx=0x7f0358009040 comp rx=0 tx=0).stop 2023-11-28T13:09:47.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.276+0000 7f03705ff700 1 -- 172.21.15.156:0/3993086223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03680cd7c0 msgr2=0x7f03681544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:47.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.276+0000 7f03705ff700 1 --2- 172.21.15.156:0/3993086223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03680cd7c0 0x7f03681544f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f036400e870 tx=0x7f036400eb80 comp rx=0 tx=0).stop 2023-11-28T13:09:47.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.276+0000 7f03705ff700 1 -- 172.21.15.156:0/3993086223 shutdown_connections 2023-11-28T13:09:47.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.276+0000 7f03705ff700 1 --2- 172.21.15.156:0/3993086223 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f034c0682f0 0x7f034c06a7a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:47.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.276+0000 7f03705ff700 1 --2- 172.21.15.156:0/3993086223 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0368161340 0x7f03681598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:47.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.276+0000 7f03705ff700 1 --2- 172.21.15.156:0/3993086223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03680cd7c0 0x7f03681544f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:47.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.276+0000 7f03705ff700 1 --2- 172.21.15.156:0/3993086223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f036815c9b0 0x7f03681593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:47.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.276+0000 7f03705ff700 1 -- 172.21.15.156:0/3993086223 >> 172.21.15.156:0/3993086223 conn(0x7f0368072600 msgr2=0x7f03680b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:47.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.276+0000 7f03705ff700 1 -- 172.21.15.156:0/3993086223 shutdown_connections 2023-11-28T13:09:47.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.276+0000 7f03705ff700 1 -- 172.21.15.156:0/3993086223 wait complete. 2023-11-28T13:09:47.279 INFO:teuthology.orchestra.run.smithi156.stderr: removed all blocklist entries 2023-11-28T13:09:47.297 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T13:09:47.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.497+0000 7f9bcc7ff700 1 Processor -- start 2023-11-28T13:09:47.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.498+0000 7f9bcc7ff700 1 -- start start 2023-11-28T13:09:47.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.499+0000 7f9bcc7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bc415c9b0 0x7f9bc40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:47.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.499+0000 7f9bcc7ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9bc40cd7c0 0x7f9bc40cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:47.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.499+0000 7f9bcc7ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc40ce160 con 0x7f9bc4161340 2023-11-28T13:09:47.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.499+0000 7f9bcc7ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bc40b5800 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.499+0000 7f9bcc7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc40d1950 con 0x7f9bc415c9b0 2023-11-28T13:09:47.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.499+0000 7f9bcad9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9bc40cd7c0 0x7f9bc40cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:47.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.499+0000 7f9bcad9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9bc40cd7c0 0x7f9bc40cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38816/0 (socket says 172.21.15.156:38816) 2023-11-28T13:09:47.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.499+0000 7f9bcad9c700 1 -- 172.21.15.156:0/968544871 learned_addr learned my addr 172.21.15.156:0/968544871 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:47.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.499+0000 7f9bca59b700 1 --2- 172.21.15.156:0/968544871 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bc415c9b0 0x7f9bc40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:47.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.499+0000 7f9bcad9c700 1 -- 172.21.15.156:0/968544871 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bc415c9b0 msgr2=0x7f9bc40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:47.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.499+0000 7f9bcad9c700 1 --2- 172.21.15.156:0/968544871 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bc415c9b0 0x7f9bc40b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:47.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.499+0000 7f9bcad9c700 1 -- 172.21.15.156:0/968544871 >> v1:172.21.15.186:6789/0 conn(0x7f9bc4161340 legacy=0x7f9bc40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:47.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.500+0000 7f9bcad9c700 1 -- 172.21.15.156:0/968544871 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc40d24f0 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.500+0000 7f9bcad9c700 1 --2- 172.21.15.156:0/968544871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9bc40cd7c0 0x7f9bc40cdc20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009e50 tx=0x7f9bbc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50450d2889ee1cf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:47.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.500+0000 7f9bc9599700 1 -- 172.21.15.156:0/968544871 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc00e070 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.500+0000 7f9bc9599700 1 -- 172.21.15.156:0/968544871 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bbc00b9c0 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.500+0000 7f9bc9599700 1 -- 172.21.15.156:0/968544871 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc00bd70 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.500+0000 7f9bcc7ff700 1 -- 172.21.15.156:0/968544871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9bc40cd7c0 msgr2=0x7f9bc40cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:47.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.500+0000 7f9bcc7ff700 1 --2- 172.21.15.156:0/968544871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9bc40cd7c0 0x7f9bc40cdc20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009e50 tx=0x7f9bbc00b330 comp rx=0 tx=0).stop 2023-11-28T13:09:47.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.501+0000 7f9bcc7ff700 1 -- 172.21.15.156:0/968544871 shutdown_connections 2023-11-28T13:09:47.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.501+0000 7f9bcc7ff700 1 --2- 172.21.15.156:0/968544871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9bc40cd7c0 0x7f9bc40cdc20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:47.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.501+0000 7f9bcc7ff700 1 --2- 172.21.15.156:0/968544871 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bc415c9b0 0x7f9bc40b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:47.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.501+0000 7f9bcc7ff700 1 -- 172.21.15.156:0/968544871 >> 172.21.15.156:0/968544871 conn(0x7f9bc4072600 msgr2=0x7f9bc40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:47.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.501+0000 7f9bcc7ff700 1 -- 172.21.15.156:0/968544871 shutdown_connections 2023-11-28T13:09:47.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.501+0000 7f9bcc7ff700 1 -- 172.21.15.156:0/968544871 wait complete. 2023-11-28T13:09:47.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.502+0000 7f9bcc7ff700 1 Processor -- start 2023-11-28T13:09:47.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.503+0000 7f9bcc7ff700 1 -- start start 2023-11-28T13:09:47.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bcc7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9bc40cd7c0 0x7f9bc4157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:47.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bcc7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bc415c9b0 0x7f9bc40c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:47.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bcc7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bc4161340 0x7f9bc414a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:47.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bcc7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc414a900 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bcc7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc40c8620 con 0x7f9bc4161340 2023-11-28T13:09:47.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bcc7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc40c8900 con 0x7f9bc415c9b0 2023-11-28T13:09:47.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bca59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9bc40cd7c0 0x7f9bc4157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:47.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bca59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9bc40cd7c0 0x7f9bc4157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38828/0 (socket says 172.21.15.156:38828) 2023-11-28T13:09:47.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bca59b700 1 -- 172.21.15.156:0/1672181009 learned_addr learned my addr 172.21.15.156:0/1672181009 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:47.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bc9d9a700 1 --2- 172.21.15.156:0/1672181009 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bc415c9b0 0x7f9bc40c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:47.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bcad9c700 1 --2- 172.21.15.156:0/1672181009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bc4161340 0x7f9bc414a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:47.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bca59b700 1 -- 172.21.15.156:0/1672181009 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bc415c9b0 msgr2=0x7f9bc40c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:47.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bca59b700 1 --2- 172.21.15.156:0/1672181009 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bc415c9b0 0x7f9bc40c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:47.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bca59b700 1 -- 172.21.15.156:0/1672181009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bc4161340 msgr2=0x7f9bc414a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:47.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bca59b700 1 --2- 172.21.15.156:0/1672181009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bc4161340 0x7f9bc414a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:47.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bca59b700 1 -- 172.21.15.156:0/1672181009 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc009b30 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bcad9c700 1 --2- 172.21.15.156:0/1672181009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bc4161340 0x7f9bc414a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:47.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.504+0000 7f9bca59b700 1 --2- 172.21.15.156:0/1672181009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9bc40cd7c0 0x7f9bc4157910 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000d800 tx=0x7f9bc000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:47.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.505+0000 7f9bbaffd700 1 -- 172.21.15.156:0/1672181009 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00125f0 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.505+0000 7f9bbaffd700 1 -- 172.21.15.156:0/1672181009 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc0012750 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.505+0000 7f9bbaffd700 1 -- 172.21.15.156:0/1672181009 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0012b00 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.505+0000 7f9bcc7ff700 1 -- 172.21.15.156:0/1672181009 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc40d1300 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.505+0000 7f9bcc7ff700 1 -- 172.21.15.156:0/1672181009 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc40b5520 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.506+0000 7f9bcc7ff700 1 -- 172.21.15.156:0/1672181009 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc40d15e0 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.507+0000 7f9bbaffd700 1 -- 172.21.15.156:0/1672181009 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9bc0012c60 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.508+0000 7f9bbaffd700 1 --2- 172.21.15.156:0/1672181009 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9ba80682f0 0x7f9ba806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:47.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.508+0000 7f9bbaffd700 1 -- 172.21.15.156:0/1672181009 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9bc0085c10 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.508+0000 7f9bc9d9a700 1 --2- 172.21.15.156:0/1672181009 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9ba80682f0 0x7f9ba806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:47.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.509+0000 7f9bc9d9a700 1 --2- 172.21.15.156:0/1672181009 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9ba80682f0 0x7f9ba806a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9bc40cf230 tx=0x7f9bb4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:47.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.512+0000 7f9bbaffd700 1 -- 172.21.15.156:0/1672181009 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9bc00527c0 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.778+0000 7f9bcc7ff700 1 -- 172.21.15.156:0/1672181009 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9bc40d26d0 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.779+0000 7f9bbaffd700 1 -- 172.21.15.156:0/1672181009 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f9bc0055de0 con 0x7f9bc40cd7c0 2023-11-28T13:09:47.781 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:47.781 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:09:47.781 INFO:teuthology.orchestra.run.smithi156.stdout: "auth_dump": [ 2023-11-28T13:09:47.781 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.781 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.a", 2023-11-28T13:09:47.781 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVl34w0ERAA9H2i6sMS60YUKvNCit0idg==", 2023-11-28T13:09:47.781 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.781 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:09:47.782 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T13:09:47.782 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T13:09:47.782 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:47.782 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.782 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.782 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.782 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.b", 2023-11-28T13:09:47.782 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVlzVSSKRAA3APxxYZXNOOOnG1t7DIIDg==", 2023-11-28T13:09:47.782 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.782 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:09:47.783 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T13:09:47.783 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T13:09:47.783 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:47.783 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.783 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.783 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.783 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.c", 2023-11-28T13:09:47.783 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVl++RtHRAAsenmAb/Y3eRvWw22940lkw==", 2023-11-28T13:09:47.783 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.783 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:09:47.783 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T13:09:47.783 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T13:09:47.784 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:47.784 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.784 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.784 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.784 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.d", 2023-11-28T13:09:47.784 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVlFDP1NRAASlTZM2P0RL+lcFskrr08uQ==", 2023-11-28T13:09:47.784 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.784 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:09:47.784 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T13:09:47.784 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T13:09:47.784 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:47.784 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.785 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.785 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.785 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.0", 2023-11-28T13:09:47.785 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD95WVlF1g2BBAAM1AkiYFhmzcy6f2xOUHuSA==", 2023-11-28T13:09:47.785 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.785 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:09:47.785 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:09:47.785 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:47.785 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.785 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.785 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.785 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.1", 2023-11-28T13:09:47.785 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAA5mVlMP+VChAAvBK9dslAIznG9cJCnmarfg==", 2023-11-28T13:09:47.786 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.786 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:09:47.786 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:09:47.786 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:47.786 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.787 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.787 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.787 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.2", 2023-11-28T13:09:47.787 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAD5mVlY5TMERAAH2W59d+/mWQ7vVOFV0uH5g==", 2023-11-28T13:09:47.787 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.787 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:09:47.787 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:09:47.787 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:47.787 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.787 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.787 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.788 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.3", 2023-11-28T13:09:47.788 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAG5mVlKreoGBAAHwbSd1vt4sF/Ahmz97lv5A==", 2023-11-28T13:09:47.788 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.788 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:09:47.788 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:09:47.788 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:47.788 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.788 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.788 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.788 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.4", 2023-11-28T13:09:47.788 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAM5mVlVb7BORAA2/l64tOgJl6WRDwI1Ee+YA==", 2023-11-28T13:09:47.788 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.789 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:09:47.789 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:09:47.789 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:47.789 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.789 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.789 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.789 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.5", 2023-11-28T13:09:47.789 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAQ5mVlIl0jBRAAYrEb4PGL8amZxfsGUhE0BQ==", 2023-11-28T13:09:47.789 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.789 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:09:47.789 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:09:47.789 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:47.789 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.790 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.790 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.790 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.6", 2023-11-28T13:09:47.790 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAT5mVlicggDRAANowLXao2cG7JHDi+ytdREg==", 2023-11-28T13:09:47.790 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.790 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:09:47.790 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:09:47.790 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:47.790 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.790 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.791 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.791 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.7", 2023-11-28T13:09:47.791 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAW5mVlPTz9EhAAyyq2SF/TYYStii1bExbDew==", 2023-11-28T13:09:47.791 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.791 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:09:47.791 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:09:47.791 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:47.791 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.791 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.791 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.791 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.0", 2023-11-28T13:09:47.792 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD75WVlhwXEBBAAu/NSgck5Joi7bKQoBne7MA==", 2023-11-28T13:09:47.792 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.792 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:09:47.792 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow r", 2023-11-28T13:09:47.792 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T13:09:47.792 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.792 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.792 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.792 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.1", 2023-11-28T13:09:47.792 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD75WVlSLGyFBAATUac8v+U0A55jCLv6SCsAA==", 2023-11-28T13:09:47.792 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.792 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:09:47.793 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T13:09:47.793 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T13:09:47.793 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T13:09:47.793 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.793 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.793 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.793 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.2", 2023-11-28T13:09:47.793 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD75WVl49+oDBAA/rsj8joMrNDo2FOlvvLvWQ==", 2023-11-28T13:09:47.793 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.793 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:09:47.794 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T13:09:47.794 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T13:09:47.794 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T13:09:47.794 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.794 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.794 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.794 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.3", 2023-11-28T13:09:47.794 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD75WVlmfuWHBAAbf1HvrNxzm3zJwTF6De2Kg==", 2023-11-28T13:09:47.794 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.794 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:09:47.794 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T13:09:47.794 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T13:09:47.795 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T13:09:47.795 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.795 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.795 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.795 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.admin", 2023-11-28T13:09:47.795 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD55WVld9ddFBAA6s9ybzLqB7e7Uj9gOd65/Q==", 2023-11-28T13:09:47.795 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.795 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T13:09:47.795 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T13:09:47.795 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T13:09:47.795 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:47.795 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.796 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.796 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.796 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mgr.x", 2023-11-28T13:09:47.796 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVl3oYaCRAALjx/GgP3kX/gyD4ML7Af7Q==", 2023-11-28T13:09:47.796 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.796 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T13:09:47.796 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile mgr", 2023-11-28T13:09:47.796 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:47.796 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.796 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:47.796 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:47.797 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mgr.y", 2023-11-28T13:09:47.797 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVlhmWPABAA5F0d66op3eUWwWPKJ6Mc6w==", 2023-11-28T13:09:47.797 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:09:47.797 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T13:09:47.797 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile mgr", 2023-11-28T13:09:47.797 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:09:47.797 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.797 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:47.797 INFO:teuthology.orchestra.run.smithi156.stdout: ] 2023-11-28T13:09:47.797 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:09:47.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.781+0000 7f9bcc7ff700 1 -- 172.21.15.156:0/1672181009 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9ba80682f0 msgr2=0x7f9ba806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:47.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.781+0000 7f9bcc7ff700 1 --2- 172.21.15.156:0/1672181009 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9ba80682f0 0x7f9ba806a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9bc40cf230 tx=0x7f9bb4009040 comp rx=0 tx=0).stop 2023-11-28T13:09:47.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.781+0000 7f9bcc7ff700 1 -- 172.21.15.156:0/1672181009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9bc40cd7c0 msgr2=0x7f9bc4157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:47.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.781+0000 7f9bcc7ff700 1 --2- 172.21.15.156:0/1672181009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9bc40cd7c0 0x7f9bc4157910 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000d800 tx=0x7f9bc000db10 comp rx=0 tx=0).stop 2023-11-28T13:09:47.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.781+0000 7f9bcc7ff700 1 -- 172.21.15.156:0/1672181009 shutdown_connections 2023-11-28T13:09:47.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.781+0000 7f9bcc7ff700 1 --2- 172.21.15.156:0/1672181009 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9ba80682f0 0x7f9ba806a7a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:47.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.781+0000 7f9bcc7ff700 1 --2- 172.21.15.156:0/1672181009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bc4161340 0x7f9bc414a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:47.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.781+0000 7f9bcc7ff700 1 --2- 172.21.15.156:0/1672181009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9bc40cd7c0 0x7f9bc4157910 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:47.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.781+0000 7f9bcc7ff700 1 --2- 172.21.15.156:0/1672181009 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bc415c9b0 0x7f9bc40c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:47.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.781+0000 7f9bcc7ff700 1 -- 172.21.15.156:0/1672181009 >> 172.21.15.156:0/1672181009 conn(0x7f9bc4072600 msgr2=0x7f9bc40b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:47.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.781+0000 7f9bcc7ff700 1 -- 172.21.15.156:0/1672181009 shutdown_connections 2023-11-28T13:09:47.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.782+0000 7f9bcc7ff700 1 -- 172.21.15.156:0/1672181009 wait complete. 2023-11-28T13:09:47.801 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:09:47.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.997+0000 7f02de3ff700 1 Processor -- start 2023-11-28T13:09:47.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.997+0000 7f02de3ff700 1 -- start start 2023-11-28T13:09:47.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.998+0000 7f02de3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d815c9b0 0x7f02d80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.998+0000 7f02de3ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f02d80cd7c0 0x7f02d80cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.998+0000 7f02de3ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d80ce160 con 0x7f02d8161340 2023-11-28T13:09:48.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.998+0000 7f02de3ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d80b5800 con 0x7f02d80cd7c0 2023-11-28T13:09:48.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.998+0000 7f02de3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d80d1950 con 0x7f02d815c9b0 2023-11-28T13:09:48.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.998+0000 7f02d77fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f02d8161340 0x7f02d80d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:59924/0 (socket says 172.21.15.156:59924) 2023-11-28T13:09:48.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.998+0000 7f02d77fe700 1 -- 172.21.15.156:0/1023359508 learned_addr learned my addr 172.21.15.156:0/1023359508 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:48.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.998+0000 7f02d7fff700 1 --2- 172.21.15.156:0/1023359508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d815c9b0 0x7f02d80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.998+0000 7f02dc99c700 1 --2- 172.21.15.156:0/1023359508 >> v2:172.21.15.186:3300/0 conn(0x7f02d80cd7c0 0x7f02d80cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.999+0000 7f02d6ffd700 1 -- 172.21.15.156:0/1023359508 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052031466 0 0) 0x7f02d80ce160 con 0x7f02d8161340 2023-11-28T13:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.999+0000 7f02d6ffd700 1 -- 172.21.15.156:0/1023359508 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02c00035b0 con 0x7f02d8161340 2023-11-28T13:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.999+0000 7f02d7fff700 1 -- 172.21.15.156:0/1023359508 >> v2:172.21.15.186:3300/0 conn(0x7f02d80cd7c0 msgr2=0x7f02d80cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.999+0000 7f02d7fff700 1 --2- 172.21.15.156:0/1023359508 >> v2:172.21.15.186:3300/0 conn(0x7f02d80cd7c0 0x7f02d80cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.999+0000 7f02d7fff700 1 -- 172.21.15.156:0/1023359508 >> v1:172.21.15.156:6789/0 conn(0x7f02d8161340 legacy=0x7f02d80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.999+0000 7f02d7fff700 1 -- 172.21.15.156:0/1023359508 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d80d24f0 con 0x7f02d815c9b0 2023-11-28T13:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.999+0000 7f02dc99c700 1 --2- 172.21.15.156:0/1023359508 >> v2:172.21.15.186:3300/0 conn(0x7f02d80cd7c0 0x7f02d80cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.999+0000 7f02d7fff700 1 --2- 172.21.15.156:0/1023359508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d815c9b0 0x7f02d80b4bf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009840 tx=0x7f02c8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18b1eb592edd2f5b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:48.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.999+0000 7f02d6ffd700 1 -- 172.21.15.156:0/1023359508 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c8011830 con 0x7f02d815c9b0 2023-11-28T13:09:48.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:47.999+0000 7f02d6ffd700 1 -- 172.21.15.156:0/1023359508 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c8011990 con 0x7f02d815c9b0 2023-11-28T13:09:48.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.000+0000 7f02d6ffd700 1 -- 172.21.15.156:0/1023359508 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c8011d00 con 0x7f02d815c9b0 2023-11-28T13:09:48.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.000+0000 7f02de3ff700 1 -- 172.21.15.156:0/1023359508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d815c9b0 msgr2=0x7f02d80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:48.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.000+0000 7f02de3ff700 1 --2- 172.21.15.156:0/1023359508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d815c9b0 0x7f02d80b4bf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009840 tx=0x7f02c8009b50 comp rx=0 tx=0).stop 2023-11-28T13:09:48.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.000+0000 7f02de3ff700 1 -- 172.21.15.156:0/1023359508 shutdown_connections 2023-11-28T13:09:48.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.000+0000 7f02de3ff700 1 --2- 172.21.15.156:0/1023359508 >> v2:172.21.15.186:3300/0 conn(0x7f02d80cd7c0 0x7f02d80cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.000+0000 7f02de3ff700 1 --2- 172.21.15.156:0/1023359508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d815c9b0 0x7f02d80b4bf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.000+0000 7f02de3ff700 1 -- 172.21.15.156:0/1023359508 >> 172.21.15.156:0/1023359508 conn(0x7f02d8072600 msgr2=0x7f02d80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:48.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.000+0000 7f02de3ff700 1 -- 172.21.15.156:0/1023359508 shutdown_connections 2023-11-28T13:09:48.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.000+0000 7f02de3ff700 1 -- 172.21.15.156:0/1023359508 wait complete. 2023-11-28T13:09:48.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.002+0000 7f02de3ff700 1 Processor -- start 2023-11-28T13:09:48.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.002+0000 7f02de3ff700 1 -- start start 2023-11-28T13:09:48.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.003+0000 7f02de3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d80cd7c0 0x7f02d80b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.003+0000 7f02de3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d815c9b0 0x7f02d814d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.003+0000 7f02de3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02d8161340 0x7f02d814d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.003+0000 7f02de3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d814db00 con 0x7f02d80cd7c0 2023-11-28T13:09:48.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.003+0000 7f02de3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d81f5ba0 con 0x7f02d8161340 2023-11-28T13:09:48.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.003+0000 7f02de3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d81f5e80 con 0x7f02d815c9b0 2023-11-28T13:09:48.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.004+0000 7f02d7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d80cd7c0 0x7f02d80b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.004+0000 7f02d7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d80cd7c0 0x7f02d80b7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38830/0 (socket says 172.21.15.156:38830) 2023-11-28T13:09:48.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.004+0000 7f02d7fff700 1 -- 172.21.15.156:0/45178016 learned_addr learned my addr 172.21.15.156:0/45178016 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:48.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.004+0000 7f02d77fe700 1 --2- 172.21.15.156:0/45178016 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d815c9b0 0x7f02d814d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.004+0000 7f02dc99c700 1 --2- 172.21.15.156:0/45178016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02d8161340 0x7f02d814d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.004+0000 7f02d7fff700 1 -- 172.21.15.156:0/45178016 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d815c9b0 msgr2=0x7f02d814d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:48.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.004+0000 7f02d7fff700 1 --2- 172.21.15.156:0/45178016 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d815c9b0 0x7f02d814d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.004+0000 7f02d7fff700 1 -- 172.21.15.156:0/45178016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02d8161340 msgr2=0x7f02d814d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:48.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.004+0000 7f02d7fff700 1 --2- 172.21.15.156:0/45178016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02d8161340 0x7f02d814d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.004+0000 7f02d7fff700 1 -- 172.21.15.156:0/45178016 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d0009a70 con 0x7f02d80cd7c0 2023-11-28T13:09:48.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.004+0000 7f02d7fff700 1 --2- 172.21.15.156:0/45178016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d80cd7c0 0x7f02d80b7b70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009fd0 tx=0x7f02c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:48.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.005+0000 7f02d4ff9700 1 -- 172.21.15.156:0/45178016 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c8004550 con 0x7f02d80cd7c0 2023-11-28T13:09:48.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.005+0000 7f02d4ff9700 1 -- 172.21.15.156:0/45178016 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c801e070 con 0x7f02d80cd7c0 2023-11-28T13:09:48.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.005+0000 7f02d4ff9700 1 -- 172.21.15.156:0/45178016 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c80038c0 con 0x7f02d80cd7c0 2023-11-28T13:09:48.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.005+0000 7f02de3ff700 1 -- 172.21.15.156:0/45178016 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c80094a0 con 0x7f02d80cd7c0 2023-11-28T13:09:48.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.005+0000 7f02de3ff700 1 -- 172.21.15.156:0/45178016 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d80d1300 con 0x7f02d80cd7c0 2023-11-28T13:09:48.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.006+0000 7f02de3ff700 1 -- 172.21.15.156:0/45178016 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d80b5520 con 0x7f02d80cd7c0 2023-11-28T13:09:48.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.007+0000 7f02d4ff9700 1 -- 172.21.15.156:0/45178016 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f02c800f050 con 0x7f02d80cd7c0 2023-11-28T13:09:48.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.008+0000 7f02d4ff9700 1 --2- 172.21.15.156:0/45178016 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02c40682f0 0x7f02c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.008+0000 7f02d4ff9700 1 -- 172.21.15.156:0/45178016 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f02c8086750 con 0x7f02d80cd7c0 2023-11-28T13:09:48.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.008+0000 7f02d77fe700 1 --2- 172.21.15.156:0/45178016 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02c40682f0 0x7f02c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.009+0000 7f02d77fe700 1 --2- 172.21.15.156:0/45178016 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02c40682f0 0x7f02c406a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f02d815fbd0 tx=0x7f02cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:48.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.012+0000 7f02d4ff9700 1 -- 172.21.15.156:0/45178016 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f02c8053300 con 0x7f02d80cd7c0 2023-11-28T13:09:48.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.204+0000 7f02de3ff700 1 -- 172.21.15.156:0/45178016 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02d80d15e0 con 0x7f02d80cd7c0 2023-11-28T13:09:48.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.205+0000 7f02d4ff9700 1 -- 172.21.15.156:0/45178016 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+14257 (secure 0 0 0) 0x7f02c8020090 con 0x7f02d80cd7c0 2023-11-28T13:09:48.206 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:48.206 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":43,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:09:47.270429+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:09:48.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.207+0000 7f02de3ff700 1 -- 172.21.15.156:0/45178016 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02c40682f0 msgr2=0x7f02c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:48.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.207+0000 7f02de3ff700 1 --2- 172.21.15.156:0/45178016 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02c40682f0 0x7f02c406a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f02d815fbd0 tx=0x7f02cc009040 comp rx=0 tx=0).stop 2023-11-28T13:09:48.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.207+0000 7f02de3ff700 1 -- 172.21.15.156:0/45178016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d80cd7c0 msgr2=0x7f02d80b7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:48.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.207+0000 7f02de3ff700 1 --2- 172.21.15.156:0/45178016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d80cd7c0 0x7f02d80b7b70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009fd0 tx=0x7f02c8004750 comp rx=0 tx=0).stop 2023-11-28T13:09:48.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.207+0000 7f02de3ff700 1 -- 172.21.15.156:0/45178016 shutdown_connections 2023-11-28T13:09:48.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.207+0000 7f02de3ff700 1 --2- 172.21.15.156:0/45178016 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02c40682f0 0x7f02c406a7a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.207+0000 7f02de3ff700 1 --2- 172.21.15.156:0/45178016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02d8161340 0x7f02d814d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.207+0000 7f02de3ff700 1 --2- 172.21.15.156:0/45178016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d80cd7c0 0x7f02d80b7b70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.207+0000 7f02de3ff700 1 --2- 172.21.15.156:0/45178016 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02d815c9b0 0x7f02d814d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.207+0000 7f02de3ff700 1 -- 172.21.15.156:0/45178016 >> 172.21.15.156:0/45178016 conn(0x7f02d8072600 msgr2=0x7f02d806a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:48.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.207+0000 7f02de3ff700 1 -- 172.21.15.156:0/45178016 shutdown_connections 2023-11-28T13:09:48.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.207+0000 7f02de3ff700 1 -- 172.21.15.156:0/45178016 wait complete. 2023-11-28T13:09:48.228 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:09:48.228 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:09:48.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.427+0000 7f4b425ff700 1 Processor -- start 2023-11-28T13:09:48.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.428+0000 7f4b425ff700 1 -- start start 2023-11-28T13:09:48.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.429+0000 7f4b425ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4b3c15c9b0 0x7f4b3c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.429+0000 7f4b425ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4b3c161340 0x7f4b3c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.429+0000 7f4b425ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b3c0d2320 0x7f4b3c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.429+0000 7f4b425ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b3c0b5800 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.429+0000 7f4b425ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b3c0d1950 con 0x7f4b3c161340 2023-11-28T13:09:48.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.429+0000 7f4b425ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c0b5520 con 0x7f4b3c0d2320 2023-11-28T13:09:48.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.429+0000 7f4b3bfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4b3c15c9b0 0x7f4b3c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.429+0000 7f4b3bfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4b3c15c9b0 0x7f4b3c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38844/0 (socket says 172.21.15.156:38844) 2023-11-28T13:09:48.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.429+0000 7f4b3bfff700 1 -- 172.21.15.156:0/795114444 learned_addr learned my addr 172.21.15.156:0/795114444 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:48.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.429+0000 7f4b3b7fe700 1 --2- 172.21.15.156:0/795114444 >> v2:172.21.15.186:3300/0 conn(0x7f4b3c161340 0x7f4b3c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.429+0000 7f4b40b9c700 1 --2- 172.21.15.156:0/795114444 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b3c0d2320 0x7f4b3c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.429+0000 7f4b3bfff700 1 -- 172.21.15.156:0/795114444 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b3c0d2320 msgr2=0x7f4b3c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:48.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.429+0000 7f4b3bfff700 1 --2- 172.21.15.156:0/795114444 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b3c0d2320 0x7f4b3c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.429+0000 7f4b3bfff700 1 -- 172.21.15.156:0/795114444 >> v2:172.21.15.186:3300/0 conn(0x7f4b3c161340 msgr2=0x7f4b3c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:48.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.429+0000 7f4b3bfff700 1 --2- 172.21.15.156:0/795114444 >> v2:172.21.15.186:3300/0 conn(0x7f4b3c161340 0x7f4b3c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.430+0000 7f4b3bfff700 1 -- 172.21.15.156:0/795114444 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c0d1300 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.430+0000 7f4b3bfff700 1 --2- 172.21.15.156:0/795114444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3c15c9b0 0x7f4b3c0b4bf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0685c0 tx=0x7f4b2c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3180e9c54a7505 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:48.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.430+0000 7f4b3affd700 1 -- 172.21.15.156:0/795114444 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c00b870 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.430+0000 7f4b3affd700 1 -- 172.21.15.156:0/795114444 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2c00b9d0 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.430+0000 7f4b3affd700 1 -- 172.21.15.156:0/795114444 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c00bd60 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.431+0000 7f4b425ff700 1 -- 172.21.15.156:0/795114444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3c15c9b0 msgr2=0x7f4b3c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:48.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.431+0000 7f4b425ff700 1 --2- 172.21.15.156:0/795114444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3c15c9b0 0x7f4b3c0b4bf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0685c0 tx=0x7f4b2c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.431+0000 7f4b425ff700 1 -- 172.21.15.156:0/795114444 shutdown_connections 2023-11-28T13:09:48.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.431+0000 7f4b425ff700 1 --2- 172.21.15.156:0/795114444 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b3c0d2320 0x7f4b3c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.431+0000 7f4b425ff700 1 --2- 172.21.15.156:0/795114444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3c15c9b0 0x7f4b3c0b4bf0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.431+0000 7f4b425ff700 1 --2- 172.21.15.156:0/795114444 >> v2:172.21.15.186:3300/0 conn(0x7f4b3c161340 0x7f4b3c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.431+0000 7f4b425ff700 1 -- 172.21.15.156:0/795114444 >> 172.21.15.156:0/795114444 conn(0x7f4b3c072600 msgr2=0x7f4b3c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:48.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.431+0000 7f4b425ff700 1 -- 172.21.15.156:0/795114444 shutdown_connections 2023-11-28T13:09:48.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.431+0000 7f4b425ff700 1 -- 172.21.15.156:0/795114444 wait complete. 2023-11-28T13:09:48.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.433+0000 7f4b425ff700 1 Processor -- start 2023-11-28T13:09:48.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.433+0000 7f4b425ff700 1 -- start start 2023-11-28T13:09:48.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b425ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b3c0d2320 0x7f4b3c1f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b425ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3c15c9b0 0x7f4b3c155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b425ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b3c161340 0x7f4b3c1506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b425ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c152fe0 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b425ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c1559c0 con 0x7f4b3c161340 2023-11-28T13:09:48.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b425ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c150c20 con 0x7f4b3c0d2320 2023-11-28T13:09:48.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b3b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3c15c9b0 0x7f4b3c155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b3b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3c15c9b0 0x7f4b3c155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38850/0 (socket says 172.21.15.156:38850) 2023-11-28T13:09:48.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b3b7fe700 1 -- 172.21.15.156:0/1852243510 learned_addr learned my addr 172.21.15.156:0/1852243510 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:48.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b3bfff700 1 --2- 172.21.15.156:0/1852243510 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b3c0d2320 0x7f4b3c1f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b3b7fe700 1 -- 172.21.15.156:0/1852243510 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b3c0d2320 msgr2=0x7f4b3c1f2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:48.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b40b9c700 1 --2- 172.21.15.156:0/1852243510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b3c161340 0x7f4b3c1506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b3b7fe700 1 --2- 172.21.15.156:0/1852243510 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b3c0d2320 0x7f4b3c1f2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b3b7fe700 1 -- 172.21.15.156:0/1852243510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b3c161340 msgr2=0x7f4b3c1506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:48.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b3b7fe700 1 --2- 172.21.15.156:0/1852243510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b3c161340 0x7f4b3c1506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b3b7fe700 1 -- 172.21.15.156:0/1852243510 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c009550 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b3bfff700 1 --2- 172.21.15.156:0/1852243510 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b3c0d2320 0x7f4b3c1f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:48.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.434+0000 7f4b40b9c700 1 --2- 172.21.15.156:0/1852243510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b3c161340 0x7f4b3c1506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:48.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.435+0000 7f4b3b7fe700 1 --2- 172.21.15.156:0/1852243510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3c15c9b0 0x7f4b3c155480 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000e840 tx=0x7f4b3000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:48.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.435+0000 7f4b38ff9700 1 -- 172.21.15.156:0/1852243510 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b30013610 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.435+0000 7f4b38ff9700 1 -- 172.21.15.156:0/1852243510 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b30013770 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.435+0000 7f4b425ff700 1 -- 172.21.15.156:0/1852243510 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3c0b5130 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.435+0000 7f4b38ff9700 1 -- 172.21.15.156:0/1852243510 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b30013b20 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.435+0000 7f4b425ff700 1 -- 172.21.15.156:0/1852243510 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3c0d15e0 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.436+0000 7f4b425ff700 1 -- 172.21.15.156:0/1852243510 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b3c0b52c0 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.437+0000 7f4b38ff9700 1 -- 172.21.15.156:0/1852243510 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4b30023070 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.438+0000 7f4b38ff9700 1 --2- 172.21.15.156:0/1852243510 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4b280682f0 0x7f4b2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.438+0000 7f4b38ff9700 1 -- 172.21.15.156:0/1852243510 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4b300855f0 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.438+0000 7f4b3bfff700 1 --2- 172.21.15.156:0/1852243510 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4b280682f0 0x7f4b2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.439+0000 7f4b3bfff700 1 --2- 172.21.15.156:0/1852243510 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4b280682f0 0x7f4b2806a7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0d0c00 tx=0x7f4b2c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:48.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.441+0000 7f4b38ff9700 1 -- 172.21.15.156:0/1852243510 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4b300520b0 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.634+0000 7f4b425ff700 1 -- 172.21.15.156:0/1852243510 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4b3c153120 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.635+0000 7f4b38ff9700 1 -- 172.21.15.156:0/1852243510 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v43) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4b300556d0 con 0x7f4b3c15c9b0 2023-11-28T13:09:48.636 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:09:48.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.637+0000 7f4b425ff700 1 -- 172.21.15.156:0/1852243510 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4b280682f0 msgr2=0x7f4b2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:48.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.637+0000 7f4b425ff700 1 --2- 172.21.15.156:0/1852243510 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4b280682f0 0x7f4b2806a7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0d0c00 tx=0x7f4b2c008030 comp rx=0 tx=0).stop 2023-11-28T13:09:48.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.637+0000 7f4b425ff700 1 -- 172.21.15.156:0/1852243510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3c15c9b0 msgr2=0x7f4b3c155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:48.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.637+0000 7f4b425ff700 1 --2- 172.21.15.156:0/1852243510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3c15c9b0 0x7f4b3c155480 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000e840 tx=0x7f4b3000ec00 comp rx=0 tx=0).stop 2023-11-28T13:09:48.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.637+0000 7f4b425ff700 1 -- 172.21.15.156:0/1852243510 shutdown_connections 2023-11-28T13:09:48.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.637+0000 7f4b425ff700 1 --2- 172.21.15.156:0/1852243510 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4b280682f0 0x7f4b2806a7a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.637+0000 7f4b425ff700 1 --2- 172.21.15.156:0/1852243510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b3c161340 0x7f4b3c1506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.637+0000 7f4b425ff700 1 --2- 172.21.15.156:0/1852243510 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b3c0d2320 0x7f4b3c1f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.637+0000 7f4b425ff700 1 --2- 172.21.15.156:0/1852243510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3c15c9b0 0x7f4b3c155480 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.637+0000 7f4b425ff700 1 -- 172.21.15.156:0/1852243510 >> 172.21.15.156:0/1852243510 conn(0x7f4b3c072600 msgr2=0x7f4b3c15cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:48.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.637+0000 7f4b425ff700 1 -- 172.21.15.156:0/1852243510 shutdown_connections 2023-11-28T13:09:48.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.637+0000 7f4b425ff700 1 -- 172.21.15.156:0/1852243510 wait complete. 2023-11-28T13:09:48.657 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:09:48.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.856+0000 7ff2b47ff700 1 Processor -- start 2023-11-28T13:09:48.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.856+0000 7ff2b47ff700 1 -- start start 2023-11-28T13:09:48.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.857+0000 7ff2b47ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff2ac0b0d80 0x7ff2ac0c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.857+0000 7ff2b47ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2ac0ccc00 con 0x7ff2ac159860 2023-11-28T13:09:48.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.857+0000 7ff2b47ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2ac0ccec0 con 0x7ff2ac159490 2023-11-28T13:09:48.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.857+0000 7ff2b47ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2ac0cd1a0 con 0x7ff2ac0b0d80 2023-11-28T13:09:48.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.857+0000 7ff2b2d9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff2ac0b0d80 0x7ff2ac0c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.857+0000 7ff2b1d9a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff2ac159860 0x7ff2ac0c94c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:59928/0 (socket says 172.21.15.156:59928) 2023-11-28T13:09:48.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.857+0000 7ff2b1d9a700 1 -- 172.21.15.156:0/1385190197 learned_addr learned my addr 172.21.15.156:0/1385190197 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:48.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.858+0000 7ff2b1599700 1 -- 172.21.15.156:0/1385190197 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890815996 0 0) 0x7ff2ac0ccc00 con 0x7ff2ac159860 2023-11-28T13:09:48.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.858+0000 7ff2b2d9c700 1 -- 172.21.15.156:0/1385190197 >> v1:172.21.15.186:6789/0 conn(0x7ff2ac159490 legacy=0x7ff2ac0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:48.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.858+0000 7ff2b2d9c700 1 -- 172.21.15.156:0/1385190197 >> v1:172.21.15.156:6789/0 conn(0x7ff2ac159860 legacy=0x7ff2ac0c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:48.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.858+0000 7ff2b2d9c700 1 -- 172.21.15.156:0/1385190197 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac0cd480 con 0x7ff2ac0b0d80 2023-11-28T13:09:48.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.858+0000 7ff2b2d9c700 1 --2- 172.21.15.156:0/1385190197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2ac0b0d80 0x7ff2ac0c9bd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009d90 tx=0x7ff2a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2309f8e43e1d33 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:48.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.858+0000 7ff2b1599700 1 -- 172.21.15.156:0/1385190197 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a400bdd0 con 0x7ff2ac0b0d80 2023-11-28T13:09:48.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.858+0000 7ff2b1599700 1 -- 172.21.15.156:0/1385190197 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a4011460 con 0x7ff2ac0b0d80 2023-11-28T13:09:48.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.858+0000 7ff2b1599700 1 -- 172.21.15.156:0/1385190197 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4011810 con 0x7ff2ac0b0d80 2023-11-28T13:09:48.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.859+0000 7ff2b47ff700 1 -- 172.21.15.156:0/1385190197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2ac0b0d80 msgr2=0x7ff2ac0c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:48.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.859+0000 7ff2b47ff700 1 --2- 172.21.15.156:0/1385190197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2ac0b0d80 0x7ff2ac0c9bd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009d90 tx=0x7ff2a400b330 comp rx=0 tx=0).stop 2023-11-28T13:09:48.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.859+0000 7ff2b47ff700 1 -- 172.21.15.156:0/1385190197 shutdown_connections 2023-11-28T13:09:48.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.859+0000 7ff2b47ff700 1 --2- 172.21.15.156:0/1385190197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2ac0b0d80 0x7ff2ac0c9bd0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.859+0000 7ff2b47ff700 1 -- 172.21.15.156:0/1385190197 >> 172.21.15.156:0/1385190197 conn(0x7ff2ac0cb670 msgr2=0x7ff2ac0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:48.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.859+0000 7ff2b47ff700 1 -- 172.21.15.156:0/1385190197 shutdown_connections 2023-11-28T13:09:48.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.859+0000 7ff2b47ff700 1 -- 172.21.15.156:0/1385190197 wait complete. 2023-11-28T13:09:48.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.861+0000 7ff2b47ff700 1 Processor -- start 2023-11-28T13:09:48.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.861+0000 7ff2b47ff700 1 -- start start 2023-11-28T13:09:48.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.862+0000 7ff2b47ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2ac0b0d80 0x7ff2ac0b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.862+0000 7ff2b47ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2ac159490 0x7ff2ac149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.862+0000 7ff2b47ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2ac0b30a0 0x7ff2ac1f2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.862+0000 7ff2b47ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0b2000 con 0x7ff2ac159490 2023-11-28T13:09:48.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.862+0000 7ff2b47ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0b22e0 con 0x7ff2ac0b30a0 2023-11-28T13:09:48.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.862+0000 7ff2b47ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0b25c0 con 0x7ff2ac0b0d80 2023-11-28T13:09:48.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.862+0000 7ff2b1d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2ac159490 0x7ff2ac149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.862+0000 7ff2b1d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2ac159490 0x7ff2ac149cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38882/0 (socket says 172.21.15.156:38882) 2023-11-28T13:09:48.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.862+0000 7ff2b1d9a700 1 -- 172.21.15.156:0/750556336 learned_addr learned my addr 172.21.15.156:0/750556336 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:48.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.862+0000 7ff2b2d9c700 1 --2- 172.21.15.156:0/750556336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2ac0b30a0 0x7ff2ac1f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.862+0000 7ff2b259b700 1 --2- 172.21.15.156:0/750556336 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2ac0b0d80 0x7ff2ac0b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.862+0000 7ff2b1d9a700 1 -- 172.21.15.156:0/750556336 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2ac0b0d80 msgr2=0x7ff2ac0b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:48.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.862+0000 7ff2b1d9a700 1 --2- 172.21.15.156:0/750556336 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2ac0b0d80 0x7ff2ac0b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.862+0000 7ff2b1d9a700 1 -- 172.21.15.156:0/750556336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2ac0b30a0 msgr2=0x7ff2ac1f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:48.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.862+0000 7ff2b1d9a700 1 --2- 172.21.15.156:0/750556336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2ac0b30a0 0x7ff2ac1f2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:48.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.862+0000 7ff2b1d9a700 1 -- 172.21.15.156:0/750556336 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a4009a70 con 0x7ff2ac159490 2023-11-28T13:09:48.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.863+0000 7ff2b1d9a700 1 --2- 172.21.15.156:0/750556336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2ac159490 0x7ff2ac149cc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00c520 tx=0x7ff29c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:48.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.863+0000 7ff2a2ffd700 1 -- 172.21.15.156:0/750556336 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c004c80 con 0x7ff2ac159490 2023-11-28T13:09:48.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.863+0000 7ff2a2ffd700 1 -- 172.21.15.156:0/750556336 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c004e20 con 0x7ff2ac159490 2023-11-28T13:09:48.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.863+0000 7ff2b47ff700 1 -- 172.21.15.156:0/750556336 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ac0cdc80 con 0x7ff2ac159490 2023-11-28T13:09:48.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.863+0000 7ff2a2ffd700 1 -- 172.21.15.156:0/750556336 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c007310 con 0x7ff2ac159490 2023-11-28T13:09:48.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.863+0000 7ff2b47ff700 1 -- 172.21.15.156:0/750556336 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ac0cd760 con 0x7ff2ac159490 2023-11-28T13:09:48.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.864+0000 7ff2b47ff700 1 -- 172.21.15.156:0/750556336 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ac0cde10 con 0x7ff2ac159490 2023-11-28T13:09:48.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.865+0000 7ff2a2ffd700 1 -- 172.21.15.156:0/750556336 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff29c01a020 con 0x7ff2ac159490 2023-11-28T13:09:48.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.866+0000 7ff2a2ffd700 1 --2- 172.21.15.156:0/750556336 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2900682f0 0x7ff29006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:48.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.866+0000 7ff2a2ffd700 1 -- 172.21.15.156:0/750556336 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff29c085560 con 0x7ff2ac159490 2023-11-28T13:09:48.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.866+0000 7ff2b259b700 1 --2- 172.21.15.156:0/750556336 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2900682f0 0x7ff29006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:48.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.867+0000 7ff2b259b700 1 --2- 172.21.15.156:0/750556336 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2900682f0 0x7ff29006a7a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8000f60 tx=0x7ff2a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:48.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:48.870+0000 7ff2a2ffd700 1 -- 172.21.15.156:0/750556336 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff29c052020 con 0x7ff2ac159490 2023-11-28T13:09:49.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.058+0000 7ff2b47ff700 1 -- 172.21.15.156:0/750556336 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2ac1538d0 con 0x7ff2ac159490 2023-11-28T13:09:49.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.059+0000 7ff2a2ffd700 1 -- 172.21.15.156:0/750556336 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff29c055640 con 0x7ff2ac159490 2023-11-28T13:09:49.061 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:09:49.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.062+0000 7ff2b47ff700 1 -- 172.21.15.156:0/750556336 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2900682f0 msgr2=0x7ff29006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:49.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.062+0000 7ff2b47ff700 1 --2- 172.21.15.156:0/750556336 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2900682f0 0x7ff29006a7a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8000f60 tx=0x7ff2a8009040 comp rx=0 tx=0).stop 2023-11-28T13:09:49.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.062+0000 7ff2b47ff700 1 -- 172.21.15.156:0/750556336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2ac159490 msgr2=0x7ff2ac149cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:49.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.062+0000 7ff2b47ff700 1 --2- 172.21.15.156:0/750556336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2ac159490 0x7ff2ac149cc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00c520 tx=0x7ff29c00c8e0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.062+0000 7ff2b47ff700 1 -- 172.21.15.156:0/750556336 shutdown_connections 2023-11-28T13:09:49.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.062+0000 7ff2b47ff700 1 --2- 172.21.15.156:0/750556336 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2900682f0 0x7ff29006a7a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.062+0000 7ff2b47ff700 1 --2- 172.21.15.156:0/750556336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2ac0b30a0 0x7ff2ac1f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.062+0000 7ff2b47ff700 1 --2- 172.21.15.156:0/750556336 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2ac0b0d80 0x7ff2ac0b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.062+0000 7ff2b47ff700 1 --2- 172.21.15.156:0/750556336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2ac159490 0x7ff2ac149cc0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.062+0000 7ff2b47ff700 1 -- 172.21.15.156:0/750556336 >> 172.21.15.156:0/750556336 conn(0x7ff2ac0cb670 msgr2=0x7ff2ac06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:49.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.062+0000 7ff2b47ff700 1 -- 172.21.15.156:0/750556336 shutdown_connections 2023-11-28T13:09:49.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.062+0000 7ff2b47ff700 1 -- 172.21.15.156:0/750556336 wait complete. 2023-11-28T13:09:49.082 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T13:09:49.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.277+0000 7f4b35fff700 1 Processor -- start 2023-11-28T13:09:49.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.277+0000 7f4b35fff700 1 -- start start 2023-11-28T13:09:49.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.278+0000 7f4b35fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4b3015c9b0 0x7f4b300b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:49.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.279+0000 7f4b35fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4b300cd7c0 0x7f4b300cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:49.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.279+0000 7f4b35fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b300d1950 con 0x7f4b30161340 2023-11-28T13:09:49.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.279+0000 7f4b35fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b300b5800 con 0x7f4b3015c9b0 2023-11-28T13:09:49.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.279+0000 7f4b35fff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b300ce140 con 0x7f4b300cd7c0 2023-11-28T13:09:49.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.279+0000 7f4b2f7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4b3015c9b0 0x7f4b300b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:49.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.279+0000 7f4b2f7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4b3015c9b0 0x7f4b300b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38894/0 (socket says 172.21.15.156:38894) 2023-11-28T13:09:49.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.279+0000 7f4b2f7fe700 1 -- 172.21.15.156:0/1855274374 learned_addr learned my addr 172.21.15.156:0/1855274374 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:49.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.279+0000 7f4b2ffff700 1 --2- 172.21.15.156:0/1855274374 >> v2:172.21.15.186:3300/0 conn(0x7f4b300cd7c0 0x7f4b300cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:49.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.279+0000 7f4b2f7fe700 1 -- 172.21.15.156:0/1855274374 >> v2:172.21.15.186:3300/0 conn(0x7f4b300cd7c0 msgr2=0x7f4b300cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:49.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.279+0000 7f4b2f7fe700 1 --2- 172.21.15.156:0/1855274374 >> v2:172.21.15.186:3300/0 conn(0x7f4b300cd7c0 0x7f4b300cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.279+0000 7f4b2f7fe700 1 -- 172.21.15.156:0/1855274374 >> v1:172.21.15.186:6789/0 conn(0x7f4b30161340 legacy=0x7f4b300d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:49.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.279+0000 7f4b2f7fe700 1 -- 172.21.15.156:0/1855274374 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b300d24f0 con 0x7f4b3015c9b0 2023-11-28T13:09:49.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.280+0000 7f4b2f7fe700 1 --2- 172.21.15.156:0/1855274374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3015c9b0 0x7f4b300b4bf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4b3006c170 tx=0x7f4b20009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b738bb5301c56f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:49.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.280+0000 7f4b2effd700 1 -- 172.21.15.156:0/1855274374 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2000b870 con 0x7f4b3015c9b0 2023-11-28T13:09:49.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.280+0000 7f4b2effd700 1 -- 172.21.15.156:0/1855274374 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2000b9d0 con 0x7f4b3015c9b0 2023-11-28T13:09:49.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.280+0000 7f4b2effd700 1 -- 172.21.15.156:0/1855274374 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2000bd60 con 0x7f4b3015c9b0 2023-11-28T13:09:49.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.280+0000 7f4b35fff700 1 -- 172.21.15.156:0/1855274374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3015c9b0 msgr2=0x7f4b300b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:49.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.280+0000 7f4b35fff700 1 --2- 172.21.15.156:0/1855274374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3015c9b0 0x7f4b300b4bf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4b3006c170 tx=0x7f4b20009bd0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.280+0000 7f4b35fff700 1 -- 172.21.15.156:0/1855274374 shutdown_connections 2023-11-28T13:09:49.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.281+0000 7f4b35fff700 1 --2- 172.21.15.156:0/1855274374 >> v2:172.21.15.186:3300/0 conn(0x7f4b300cd7c0 0x7f4b300cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.281+0000 7f4b35fff700 1 --2- 172.21.15.156:0/1855274374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3015c9b0 0x7f4b300b4bf0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.281+0000 7f4b35fff700 1 -- 172.21.15.156:0/1855274374 >> 172.21.15.156:0/1855274374 conn(0x7f4b30072600 msgr2=0x7f4b300cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:49.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.281+0000 7f4b35fff700 1 -- 172.21.15.156:0/1855274374 shutdown_connections 2023-11-28T13:09:49.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.281+0000 7f4b35fff700 1 -- 172.21.15.156:0/1855274374 wait complete. 2023-11-28T13:09:49.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.282+0000 7f4b35fff700 1 Processor -- start 2023-11-28T13:09:49.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.283+0000 7f4b35fff700 1 -- start start 2023-11-28T13:09:49.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.283+0000 7f4b35fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b300cd7c0 0x7f4b300cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.283+0000 7f4b35fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3015c9b0 0x7f4b3015e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.283+0000 7f4b35fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b30161340 0x7f4b30156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.283+0000 7f4b35fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b300cbe50 con 0x7f4b3015c9b0 2023-11-28T13:09:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.283+0000 7f4b35fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b300cc130 con 0x7f4b300cd7c0 2023-11-28T13:09:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.283+0000 7f4b35fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b300cc410 con 0x7f4b30161340 2023-11-28T13:09:49.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.284+0000 7f4b27fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3015c9b0 0x7f4b3015e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:49.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.284+0000 7f4b27fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3015c9b0 0x7f4b3015e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38902/0 (socket says 172.21.15.156:38902) 2023-11-28T13:09:49.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.284+0000 7f4b27fff700 1 -- 172.21.15.156:0/3420569176 learned_addr learned my addr 172.21.15.156:0/3420569176 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:49.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.284+0000 7f4b2f7fe700 1 --2- 172.21.15.156:0/3420569176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b300cd7c0 0x7f4b300cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:49.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.284+0000 7f4b2ffff700 1 --2- 172.21.15.156:0/3420569176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b30161340 0x7f4b30156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:49.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.284+0000 7f4b27fff700 1 -- 172.21.15.156:0/3420569176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b30161340 msgr2=0x7f4b30156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:49.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.284+0000 7f4b27fff700 1 --2- 172.21.15.156:0/3420569176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b30161340 0x7f4b30156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.284+0000 7f4b27fff700 1 -- 172.21.15.156:0/3420569176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b300cd7c0 msgr2=0x7f4b300cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:49.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.284+0000 7f4b27fff700 1 --2- 172.21.15.156:0/3420569176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b300cd7c0 0x7f4b300cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.284+0000 7f4b27fff700 1 -- 172.21.15.156:0/3420569176 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b20009550 con 0x7f4b3015c9b0 2023-11-28T13:09:49.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.284+0000 7f4b27fff700 1 --2- 172.21.15.156:0/3420569176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3015c9b0 0x7f4b3015e7d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c450 tx=0x7f4b1800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:49.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.285+0000 7f4b2cff9700 1 -- 172.21.15.156:0/3420569176 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b180052d0 con 0x7f4b3015c9b0 2023-11-28T13:09:49.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.285+0000 7f4b2cff9700 1 -- 172.21.15.156:0/3420569176 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b18005430 con 0x7f4b3015c9b0 2023-11-28T13:09:49.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.285+0000 7f4b2cff9700 1 -- 172.21.15.156:0/3420569176 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b180038c0 con 0x7f4b3015c9b0 2023-11-28T13:09:49.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.285+0000 7f4b35fff700 1 -- 172.21.15.156:0/3420569176 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b300d1300 con 0x7f4b3015c9b0 2023-11-28T13:09:49.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.285+0000 7f4b35fff700 1 -- 172.21.15.156:0/3420569176 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b300b5520 con 0x7f4b3015c9b0 2023-11-28T13:09:49.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.286+0000 7f4b35fff700 1 -- 172.21.15.156:0/3420569176 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b300d15e0 con 0x7f4b3015c9b0 2023-11-28T13:09:49.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.287+0000 7f4b2cff9700 1 -- 172.21.15.156:0/3420569176 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4b18019030 con 0x7f4b3015c9b0 2023-11-28T13:09:49.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.287+0000 7f4b2cff9700 1 --2- 172.21.15.156:0/3420569176 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4b100682f0 0x7f4b1006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:49.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.288+0000 7f4b2cff9700 1 -- 172.21.15.156:0/3420569176 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4b18084bb0 con 0x7f4b3015c9b0 2023-11-28T13:09:49.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.288+0000 7f4b2f7fe700 1 --2- 172.21.15.156:0/3420569176 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4b100682f0 0x7f4b1006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:49.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.288+0000 7f4b2f7fe700 1 --2- 172.21.15.156:0/3420569176 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4b100682f0 0x7f4b1006a7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4b20014040 tx=0x7f4b20008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:49.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.291+0000 7f4b2cff9700 1 -- 172.21.15.156:0/3420569176 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4b18051670 con 0x7f4b3015c9b0 2023-11-28T13:09:49.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.487+0000 7f4b35fff700 1 -- 172.21.15.156:0/3420569176 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4b300745c0 con 0x7f4b3015c9b0 2023-11-28T13:09:49.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.487+0000 7f4b2cff9700 1 -- 172.21.15.156:0/3420569176 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) v1 ==== 84+0+126 (secure 0 0 0) 0x7f4b18017090 con 0x7f4b3015c9b0 2023-11-28T13:09:49.489 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:49.489 INFO:teuthology.orchestra.run.smithi156.stdout:[ 2023-11-28T13:09:49.489 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:49.489 INFO:teuthology.orchestra.run.smithi156.stdout: "poolnum": 1, 2023-11-28T13:09:49.489 INFO:teuthology.orchestra.run.smithi156.stdout: "poolname": "rbd" 2023-11-28T13:09:49.489 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:09:49.489 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:09:49.489 INFO:teuthology.orchestra.run.smithi156.stdout: "poolnum": 2, 2023-11-28T13:09:49.490 INFO:teuthology.orchestra.run.smithi156.stdout: "poolname": ".mgr" 2023-11-28T13:09:49.490 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:09:49.490 INFO:teuthology.orchestra.run.smithi156.stdout:] 2023-11-28T13:09:49.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.489+0000 7f4b35fff700 1 -- 172.21.15.156:0/3420569176 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4b100682f0 msgr2=0x7f4b1006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:49.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.489+0000 7f4b35fff700 1 --2- 172.21.15.156:0/3420569176 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4b100682f0 0x7f4b1006a7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4b20014040 tx=0x7f4b20008030 comp rx=0 tx=0).stop 2023-11-28T13:09:49.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.489+0000 7f4b35fff700 1 -- 172.21.15.156:0/3420569176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3015c9b0 msgr2=0x7f4b3015e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:49.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.489+0000 7f4b35fff700 1 --2- 172.21.15.156:0/3420569176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3015c9b0 0x7f4b3015e7d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c450 tx=0x7f4b1800c760 comp rx=0 tx=0).stop 2023-11-28T13:09:49.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.490+0000 7f4b35fff700 1 -- 172.21.15.156:0/3420569176 shutdown_connections 2023-11-28T13:09:49.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.490+0000 7f4b35fff700 1 --2- 172.21.15.156:0/3420569176 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4b100682f0 0x7f4b1006a7a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.490+0000 7f4b35fff700 1 --2- 172.21.15.156:0/3420569176 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4b30161340 0x7f4b30156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.490+0000 7f4b35fff700 1 --2- 172.21.15.156:0/3420569176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b300cd7c0 0x7f4b300cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.490+0000 7f4b35fff700 1 --2- 172.21.15.156:0/3420569176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b3015c9b0 0x7f4b3015e7d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.490+0000 7f4b35fff700 1 -- 172.21.15.156:0/3420569176 >> 172.21.15.156:0/3420569176 conn(0x7f4b30072600 msgr2=0x7f4b300b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:49.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.490+0000 7f4b35fff700 1 -- 172.21.15.156:0/3420569176 shutdown_connections 2023-11-28T13:09:49.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.490+0000 7f4b35fff700 1 -- 172.21.15.156:0/3420569176 wait complete. 2023-11-28T13:09:49.510 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T13:09:49.510 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T13:09:49.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.709+0000 7efebe7ff700 1 Processor -- start 2023-11-28T13:09:49.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.709+0000 7efebe7ff700 1 -- start start 2023-11-28T13:09:49.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.711+0000 7efebe7ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7efeb8161200 0x7efeb815dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:49.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.711+0000 7efebe7ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb8160d70 con 0x7efeb80cded0 2023-11-28T13:09:49.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.711+0000 7efebe7ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb80b5390 con 0x7efeb815a760 2023-11-28T13:09:49.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.711+0000 7efebe7ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb815ab30 con 0x7efeb8161200 2023-11-28T13:09:49.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.711+0000 7efeaffff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7efeb80cded0 0x7efeb80d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:59932/0 (socket says 172.21.15.156:59932) 2023-11-28T13:09:49.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.711+0000 7efeaffff700 1 -- 172.21.15.156:0/4077225495 learned_addr learned my addr 172.21.15.156:0/4077225495 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:49.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.711+0000 7efebcd9c700 1 --2- 172.21.15.156:0/4077225495 >> v2:172.21.15.186:3300/0 conn(0x7efeb8161200 0x7efeb815dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:49.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.711+0000 7efeb77fe700 1 -- 172.21.15.156:0/4077225495 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4188343133 0 0) 0x7efeb8160d70 con 0x7efeb80cded0 2023-11-28T13:09:49.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.711+0000 7efeb77fe700 1 -- 172.21.15.156:0/4077225495 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efea40035b0 con 0x7efeb80cded0 2023-11-28T13:09:49.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.711+0000 7efeb77fe700 1 -- 172.21.15.156:0/4077225495 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248090164 0 0) 0x7efeb80b5390 con 0x7efeb815a760 2023-11-28T13:09:49.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.711+0000 7efeb77fe700 1 -- 172.21.15.156:0/4077225495 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeb8160d70 con 0x7efeb815a760 2023-11-28T13:09:49.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.711+0000 7efeb77fe700 1 -- 172.21.15.156:0/4077225495 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1320922748 0 0) 0x7efea40035b0 con 0x7efeb80cded0 2023-11-28T13:09:49.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efeb77fe700 1 -- 172.21.15.156:0/4077225495 >> v2:172.21.15.186:3300/0 conn(0x7efeb8161200 msgr2=0x7efeb815dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:49.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efeb77fe700 1 --2- 172.21.15.156:0/4077225495 >> v2:172.21.15.186:3300/0 conn(0x7efeb8161200 0x7efeb815dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efeb77fe700 1 -- 172.21.15.156:0/4077225495 >> v1:172.21.15.186:6789/0 conn(0x7efeb815a760 legacy=0x7efeb80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:49.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efeb77fe700 1 -- 172.21.15.156:0/4077225495 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb80b5670 con 0x7efeb80cded0 2023-11-28T13:09:49.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efeb77fe700 1 -- 172.21.15.156:0/4077225495 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7efea0001670 con 0x7efeb80cded0 2023-11-28T13:09:49.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efeb77fe700 1 -- 172.21.15.156:0/4077225495 >> v1:172.21.15.156:6789/0 conn(0x7efeb80cded0 legacy=0x7efeb80d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:49.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efeb77fe700 1 --2- 172.21.15.156:0/4077225495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efea4003b90 0x7efea4003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:49.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efeb77fe700 1 --2- 172.21.15.156:0/4077225495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efea4004850 0x7efea4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:49.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efeb77fe700 1 --2- 172.21.15.156:0/4077225495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efea4005200 0x7efea400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:49.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efeb77fe700 1 -- 172.21.15.156:0/4077225495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb815ab30 con 0x7efea4005200 2023-11-28T13:09:49.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efeb77fe700 1 -- 172.21.15.156:0/4077225495 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea400c060 con 0x7efea4004850 2023-11-28T13:09:49.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efeb77fe700 1 -- 172.21.15.156:0/4077225495 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea400c220 con 0x7efea4003b90 2023-11-28T13:09:49.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efebcd9c700 1 --2- 172.21.15.156:0/4077225495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efea4005200 0x7efea400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:49.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efeaffff700 1 --2- 172.21.15.156:0/4077225495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efea4004850 0x7efea4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:49.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efeb7fff700 1 --2- 172.21.15.156:0/4077225495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efea4003b90 0x7efea4003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:49.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efebcd9c700 1 -- 172.21.15.156:0/4077225495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efea4003b90 msgr2=0x7efea4003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:49.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efebcd9c700 1 --2- 172.21.15.156:0/4077225495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efea4003b90 0x7efea4003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efebcd9c700 1 -- 172.21.15.156:0/4077225495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efea4004850 msgr2=0x7efea4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:49.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efebcd9c700 1 --2- 172.21.15.156:0/4077225495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efea4004850 0x7efea4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.712+0000 7efebcd9c700 1 -- 172.21.15.156:0/4077225495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea40035b0 con 0x7efea4005200 2023-11-28T13:09:49.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.713+0000 7efebcd9c700 1 -- 172.21.15.156:0/4077225495 reap_dead start 2023-11-28T13:09:49.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.713+0000 7efebcd9c700 1 --2- 172.21.15.156:0/4077225495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efea4005200 0x7efea400ba10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009fc0 tx=0x7efea4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a603cfd6f077d0a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:49.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.713+0000 7efeb77fe700 1 -- 172.21.15.156:0/4077225495 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0003a20 con 0x7efea4005200 2023-11-28T13:09:49.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.713+0000 7efeb77fe700 1 -- 172.21.15.156:0/4077225495 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb000f040 con 0x7efea4005200 2023-11-28T13:09:49.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.713+0000 7efeb77fe700 1 -- 172.21.15.156:0/4077225495 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0003da0 con 0x7efea4005200 2023-11-28T13:09:49.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.713+0000 7efebe7ff700 1 -- 172.21.15.156:0/4077225495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efea4005200 msgr2=0x7efea400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:49.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.713+0000 7efebe7ff700 1 --2- 172.21.15.156:0/4077225495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efea4005200 0x7efea400ba10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009fc0 tx=0x7efea4005880 comp rx=0 tx=0).stop 2023-11-28T13:09:49.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.714+0000 7efebe7ff700 1 -- 172.21.15.156:0/4077225495 shutdown_connections 2023-11-28T13:09:49.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.714+0000 7efebe7ff700 1 --2- 172.21.15.156:0/4077225495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efea4005200 0x7efea400ba10 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.714+0000 7efebe7ff700 1 -- 172.21.15.156:0/4077225495 >> 172.21.15.156:0/4077225495 conn(0x7efeb8072620 msgr2=0x7efeb80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:49.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.714+0000 7efebe7ff700 1 -- 172.21.15.156:0/4077225495 shutdown_connections 2023-11-28T13:09:49.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.714+0000 7efebe7ff700 1 -- 172.21.15.156:0/4077225495 wait complete. 2023-11-28T13:09:49.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.715+0000 7efebe7ff700 1 Processor -- start 2023-11-28T13:09:49.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.716+0000 7efebe7ff700 1 -- start start 2023-11-28T13:09:49.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.716+0000 7efebe7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efea4005200 0x7efeb81572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:49.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.716+0000 7efebe7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efeb80c60f0 0x7efeb8145240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:49.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.717+0000 7efebe7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efeb80c6560 0x7efeb8145780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:49.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.717+0000 7efebe7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb8156eb0 con 0x7efeb80c6560 2023-11-28T13:09:49.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.717+0000 7efebe7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb8145cc0 con 0x7efea4005200 2023-11-28T13:09:49.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.717+0000 7efebe7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb8156230 con 0x7efeb80c60f0 2023-11-28T13:09:49.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.717+0000 7efebcd9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efeb80c6560 0x7efeb8145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:49.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.717+0000 7efebcd9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efeb80c6560 0x7efeb8145780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38908/0 (socket says 172.21.15.156:38908) 2023-11-28T13:09:49.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.717+0000 7efebcd9c700 1 -- 172.21.15.156:0/3697094292 learned_addr learned my addr 172.21.15.156:0/3697094292 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:49.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.717+0000 7efeb7fff700 1 --2- 172.21.15.156:0/3697094292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efea4005200 0x7efeb81572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:49.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.717+0000 7efeaffff700 1 --2- 172.21.15.156:0/3697094292 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efeb80c60f0 0x7efeb8145240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:49.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.717+0000 7efebcd9c700 1 -- 172.21.15.156:0/3697094292 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efeb80c60f0 msgr2=0x7efeb8145240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:49.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.717+0000 7efebcd9c700 1 --2- 172.21.15.156:0/3697094292 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efeb80c60f0 0x7efeb8145240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.717+0000 7efebcd9c700 1 -- 172.21.15.156:0/3697094292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efea4005200 msgr2=0x7efeb81572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:49.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.717+0000 7efebcd9c700 1 --2- 172.21.15.156:0/3697094292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efea4005200 0x7efeb81572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:49.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.717+0000 7efebcd9c700 1 -- 172.21.15.156:0/3697094292 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb8160a00 con 0x7efeb80c6560 2023-11-28T13:09:49.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.717+0000 7efebcd9c700 1 --2- 172.21.15.156:0/3697094292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efeb80c6560 0x7efeb8145780 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efeb808c3b0 tx=0x7efeb0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:49.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.718+0000 7efeb57fa700 1 -- 172.21.15.156:0/3697094292 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0003e90 con 0x7efeb80c6560 2023-11-28T13:09:49.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.718+0000 7efeb57fa700 1 -- 172.21.15.156:0/3697094292 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb0004250 con 0x7efeb80c6560 2023-11-28T13:09:49.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.718+0000 7efeb57fa700 1 -- 172.21.15.156:0/3697094292 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0004620 con 0x7efeb80c6560 2023-11-28T13:09:49.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.718+0000 7efebe7ff700 1 -- 172.21.15.156:0/3697094292 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb8160720 con 0x7efeb80c6560 2023-11-28T13:09:49.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.718+0000 7efebe7ff700 1 -- 172.21.15.156:0/3697094292 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb815dc30 con 0x7efeb80c6560 2023-11-28T13:09:49.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.719+0000 7efebe7ff700 1 -- 172.21.15.156:0/3697094292 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb804e620 con 0x7efeb80c6560 2023-11-28T13:09:49.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.720+0000 7efeb57fa700 1 -- 172.21.15.156:0/3697094292 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efeb0004780 con 0x7efeb80c6560 2023-11-28T13:09:49.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.720+0000 7efeb57fa700 1 --2- 172.21.15.156:0/3697094292 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efe980682f0 0x7efe9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:49.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.721+0000 7efeb57fa700 1 -- 172.21.15.156:0/3697094292 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5210+0+0 (secure 0 0 0) 0x7efeb0086d20 con 0x7efeb80c6560 2023-11-28T13:09:49.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.721+0000 7efeb7fff700 1 --2- 172.21.15.156:0/3697094292 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efe980682f0 0x7efe9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:49.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.722+0000 7efeb7fff700 1 --2- 172.21.15.156:0/3697094292 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efe980682f0 0x7efe9806a7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efea8007fd0 tx=0x7efea8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:49.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.725+0000 7efeb57fa700 1 -- 172.21.15.156:0/3697094292 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efeb0053870 con 0x7efeb80c6560 2023-11-28T13:09:49.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:49.917+0000 7efebe7ff700 1 -- 172.21.15.156:0/3697094292 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7efeb80d08a0 con 0x7efeb80c6560 2023-11-28T13:09:50.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.738+0000 7efeb57fa700 1 -- 172.21.15.156:0/3697094292 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v44) v1 ==== 138+0+0 (secure 0 0 0) 0x7efeb00180b0 con 0x7efeb80c6560 2023-11-28T13:09:50.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.741+0000 7efebe7ff700 1 -- 172.21.15.156:0/3697094292 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efe980682f0 msgr2=0x7efe9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:50.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.741+0000 7efebe7ff700 1 --2- 172.21.15.156:0/3697094292 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efe980682f0 0x7efe9806a7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efea8007fd0 tx=0x7efea8009040 comp rx=0 tx=0).stop 2023-11-28T13:09:50.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.741+0000 7efebe7ff700 1 -- 172.21.15.156:0/3697094292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efeb80c6560 msgr2=0x7efeb8145780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:50.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.741+0000 7efebe7ff700 1 --2- 172.21.15.156:0/3697094292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efeb80c6560 0x7efeb8145780 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efeb808c3b0 tx=0x7efeb0009750 comp rx=0 tx=0).stop 2023-11-28T13:09:50.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.742+0000 7efebe7ff700 1 -- 172.21.15.156:0/3697094292 shutdown_connections 2023-11-28T13:09:50.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.742+0000 7efebe7ff700 1 --2- 172.21.15.156:0/3697094292 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7efe980682f0 0x7efe9806a7a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:50.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.742+0000 7efebe7ff700 1 --2- 172.21.15.156:0/3697094292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efeb80c6560 0x7efeb8145780 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:50.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.742+0000 7efebe7ff700 1 --2- 172.21.15.156:0/3697094292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efea4005200 0x7efeb81572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:50.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.742+0000 7efebe7ff700 1 --2- 172.21.15.156:0/3697094292 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efeb80c60f0 0x7efeb8145240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:50.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.742+0000 7efebe7ff700 1 -- 172.21.15.156:0/3697094292 >> 172.21.15.156:0/3697094292 conn(0x7efeb8072620 msgr2=0x7efeb80b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:50.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.742+0000 7efebe7ff700 1 -- 172.21.15.156:0/3697094292 shutdown_connections 2023-11-28T13:09:50.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.742+0000 7efebe7ff700 1 -- 172.21.15.156:0/3697094292 wait complete. 2023-11-28T13:09:50.745 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_metadata' created 2023-11-28T13:09:50.766 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T13:09:50.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.969+0000 7f2b81bff700 1 Processor -- start 2023-11-28T13:09:50.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.969+0000 7f2b81bff700 1 -- start start 2023-11-28T13:09:50.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.970+0000 7f2b81bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2b7c15c9b0 0x7f2b7c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:50.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.970+0000 7f2b81bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b7c161340 0x7f2b7c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:50.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.970+0000 7f2b81bff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b7c0d1950 con 0x7f2b7c0d2320 2023-11-28T13:09:50.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.970+0000 7f2b81bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b7c0b5800 con 0x7f2b7c15c9b0 2023-11-28T13:09:50.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.970+0000 7f2b81bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c0b5520 con 0x7f2b7c161340 2023-11-28T13:09:50.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.970+0000 7f2b7b7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2b7c15c9b0 0x7f2b7c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:50.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.970+0000 7f2b7bfff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f2b7c0d2320 0x7f2b7c0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:59944/0 (socket says 172.21.15.156:59944) 2023-11-28T13:09:50.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.970+0000 7f2b7bfff700 1 -- 172.21.15.156:0/330696045 learned_addr learned my addr 172.21.15.156:0/330696045 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:50.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.971+0000 7f2b73fff700 1 --2- 172.21.15.156:0/330696045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b7c161340 0x7f2b7c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:50.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.971+0000 7f2b7b7fe700 1 -- 172.21.15.156:0/330696045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b7c161340 msgr2=0x7f2b7c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:50.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.971+0000 7f2b7b7fe700 1 --2- 172.21.15.156:0/330696045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b7c161340 0x7f2b7c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:50.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.971+0000 7f2b7b7fe700 1 -- 172.21.15.156:0/330696045 >> v1:172.21.15.156:6789/0 conn(0x7f2b7c0d2320 legacy=0x7f2b7c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:50.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.971+0000 7f2b7b7fe700 1 -- 172.21.15.156:0/330696045 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c0d1300 con 0x7f2b7c15c9b0 2023-11-28T13:09:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.971+0000 7f2b7b7fe700 1 --2- 172.21.15.156:0/330696045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b7c15c9b0 0x7f2b7c0b4bf0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009840 tx=0x7f2b6c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5260d1865b35bf7a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.971+0000 7f2b7affd700 1 -- 172.21.15.156:0/330696045 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c011830 con 0x7f2b7c15c9b0 2023-11-28T13:09:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.971+0000 7f2b7affd700 1 -- 172.21.15.156:0/330696045 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6c011990 con 0x7f2b7c15c9b0 2023-11-28T13:09:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.971+0000 7f2b7affd700 1 -- 172.21.15.156:0/330696045 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c011d20 con 0x7f2b7c15c9b0 2023-11-28T13:09:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.972+0000 7f2b81bff700 1 -- 172.21.15.156:0/330696045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b7c15c9b0 msgr2=0x7f2b7c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.972+0000 7f2b81bff700 1 --2- 172.21.15.156:0/330696045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b7c15c9b0 0x7f2b7c0b4bf0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009840 tx=0x7f2b6c009b50 comp rx=0 tx=0).stop 2023-11-28T13:09:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.972+0000 7f2b81bff700 1 -- 172.21.15.156:0/330696045 shutdown_connections 2023-11-28T13:09:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.972+0000 7f2b81bff700 1 --2- 172.21.15.156:0/330696045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b7c15c9b0 0x7f2b7c0b4bf0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.972+0000 7f2b81bff700 1 --2- 172.21.15.156:0/330696045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b7c161340 0x7f2b7c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.972+0000 7f2b81bff700 1 -- 172.21.15.156:0/330696045 >> 172.21.15.156:0/330696045 conn(0x7f2b7c072600 msgr2=0x7f2b7c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.972+0000 7f2b81bff700 1 -- 172.21.15.156:0/330696045 shutdown_connections 2023-11-28T13:09:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.972+0000 7f2b81bff700 1 -- 172.21.15.156:0/330696045 wait complete. 2023-11-28T13:09:50.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.974+0000 7f2b81bff700 1 Processor -- start 2023-11-28T13:09:50.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.974+0000 7f2b81bff700 1 -- start start 2023-11-28T13:09:50.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.975+0000 7f2b81bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b7c0d2320 0x7f2b7c151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:50.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.975+0000 7f2b81bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b7c15c9b0 0x7f2b7c150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:50.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.975+0000 7f2b81bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b7c161340 0x7f2b7c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:50.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.975+0000 7f2b81bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c1546d0 con 0x7f2b7c15c9b0 2023-11-28T13:09:50.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.975+0000 7f2b81bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c156700 con 0x7f2b7c161340 2023-11-28T13:09:50.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.975+0000 7f2b81bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c155b80 con 0x7f2b7c0d2320 2023-11-28T13:09:50.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.975+0000 7f2b73fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b7c15c9b0 0x7f2b7c150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.975+0000 7f2b73fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b7c15c9b0 0x7f2b7c150190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38922/0 (socket says 172.21.15.156:38922) 2023-11-28T13:09:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.975+0000 7f2b73fff700 1 -- 172.21.15.156:0/4027053437 learned_addr learned my addr 172.21.15.156:0/4027053437 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.975+0000 7f2b7b7fe700 1 --2- 172.21.15.156:0/4027053437 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b7c0d2320 0x7f2b7c151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.975+0000 7f2b7bfff700 1 --2- 172.21.15.156:0/4027053437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b7c161340 0x7f2b7c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.976+0000 7f2b73fff700 1 -- 172.21.15.156:0/4027053437 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b7c0d2320 msgr2=0x7f2b7c151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.976+0000 7f2b73fff700 1 --2- 172.21.15.156:0/4027053437 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b7c0d2320 0x7f2b7c151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.976+0000 7f2b73fff700 1 -- 172.21.15.156:0/4027053437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b7c161340 msgr2=0x7f2b7c15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.976+0000 7f2b73fff700 1 --2- 172.21.15.156:0/4027053437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b7c161340 0x7f2b7c15f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.976+0000 7f2b73fff700 1 -- 172.21.15.156:0/4027053437 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c0094a0 con 0x7f2b7c15c9b0 2023-11-28T13:09:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.976+0000 7f2b7bfff700 1 --2- 172.21.15.156:0/4027053437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b7c161340 0x7f2b7c15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.976+0000 7f2b73fff700 1 --2- 172.21.15.156:0/4027053437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b7c15c9b0 0x7f2b7c150190 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400e9b0 tx=0x7f2b6400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.976+0000 7f2b78ff9700 1 -- 172.21.15.156:0/4027053437 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64009710 con 0x7f2b7c15c9b0 2023-11-28T13:09:50.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.976+0000 7f2b78ff9700 1 -- 172.21.15.156:0/4027053437 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6400c850 con 0x7f2b7c15c9b0 2023-11-28T13:09:50.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.976+0000 7f2b81bff700 1 -- 172.21.15.156:0/4027053437 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7c0b5130 con 0x7f2b7c15c9b0 2023-11-28T13:09:50.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.976+0000 7f2b78ff9700 1 -- 172.21.15.156:0/4027053437 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6400cbc0 con 0x7f2b7c15c9b0 2023-11-28T13:09:50.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.976+0000 7f2b81bff700 1 -- 172.21.15.156:0/4027053437 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7c0d15e0 con 0x7f2b7c15c9b0 2023-11-28T13:09:50.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.977+0000 7f2b81bff700 1 -- 172.21.15.156:0/4027053437 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b7c0b52c0 con 0x7f2b7c15c9b0 2023-11-28T13:09:50.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.978+0000 7f2b78ff9700 1 -- 172.21.15.156:0/4027053437 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2b64019030 con 0x7f2b7c15c9b0 2023-11-28T13:09:50.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.979+0000 7f2b78ff9700 1 --2- 172.21.15.156:0/4027053437 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2b5c0682f0 0x7f2b5c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:50.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.979+0000 7f2b78ff9700 1 -- 172.21.15.156:0/4027053437 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f2b64085bb0 con 0x7f2b7c15c9b0 2023-11-28T13:09:50.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.980+0000 7f2b7b7fe700 1 --2- 172.21.15.156:0/4027053437 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2b5c0682f0 0x7f2b5c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:50.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.980+0000 7f2b7b7fe700 1 --2- 172.21.15.156:0/4027053437 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2b5c0682f0 0x7f2b5c06a7a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c000c00 tx=0x7f2b6c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:50.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:50.984+0000 7f2b78ff9700 1 -- 172.21.15.156:0/4027053437 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2b64052500 con 0x7f2b7c15c9b0 2023-11-28T13:09:51.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.178+0000 7f2b81bff700 1 -- 172.21.15.156:0/4027053437 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2b7c1544a0 con 0x7f2b7c15c9b0 2023-11-28T13:09:51.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.747+0000 7f2b78ff9700 1 -- 172.21.15.156:0/4027053437 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v45) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2b7c1544a0 con 0x7f2b7c15c9b0 2023-11-28T13:09:51.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.752+0000 7f2b81bff700 1 -- 172.21.15.156:0/4027053437 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2b5c0682f0 msgr2=0x7f2b5c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:51.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.752+0000 7f2b81bff700 1 --2- 172.21.15.156:0/4027053437 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2b5c0682f0 0x7f2b5c06a7a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c000c00 tx=0x7f2b6c008030 comp rx=0 tx=0).stop 2023-11-28T13:09:51.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.752+0000 7f2b81bff700 1 -- 172.21.15.156:0/4027053437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b7c15c9b0 msgr2=0x7f2b7c150190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:51.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.752+0000 7f2b81bff700 1 --2- 172.21.15.156:0/4027053437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b7c15c9b0 0x7f2b7c150190 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400e9b0 tx=0x7f2b6400ed70 comp rx=0 tx=0).stop 2023-11-28T13:09:51.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.754+0000 7f2b81bff700 1 -- 172.21.15.156:0/4027053437 shutdown_connections 2023-11-28T13:09:51.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.754+0000 7f2b81bff700 1 --2- 172.21.15.156:0/4027053437 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2b5c0682f0 0x7f2b5c06a7a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:51.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.754+0000 7f2b81bff700 1 --2- 172.21.15.156:0/4027053437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b7c161340 0x7f2b7c15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:51.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.754+0000 7f2b81bff700 1 --2- 172.21.15.156:0/4027053437 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b7c0d2320 0x7f2b7c151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:51.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.754+0000 7f2b81bff700 1 --2- 172.21.15.156:0/4027053437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b7c15c9b0 0x7f2b7c150190 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:51.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.754+0000 7f2b81bff700 1 -- 172.21.15.156:0/4027053437 >> 172.21.15.156:0/4027053437 conn(0x7f2b7c072600 msgr2=0x7f2b7c15bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:51.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.755+0000 7f2b81bff700 1 -- 172.21.15.156:0/4027053437 shutdown_connections 2023-11-28T13:09:51.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.756+0000 7f2b81bff700 1 -- 172.21.15.156:0/4027053437 wait complete. 2023-11-28T13:09:51.769 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_data' created 2023-11-28T13:09:51.790 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T13:09:51.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.989+0000 7f4f525ff700 1 Processor -- start 2023-11-28T13:09:51.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.989+0000 7f4f525ff700 1 -- start start 2023-11-28T13:09:51.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.990+0000 7f4f525ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4f4c161340 0x7f4f4c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:51.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.990+0000 7f4f525ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f4c0d1950 con 0x7f4f4c15c9b0 2023-11-28T13:09:51.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.990+0000 7f4f525ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f4c0b5520 con 0x7f4f4c0d2320 2023-11-28T13:09:51.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.990+0000 7f4f525ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f4c0b5800 con 0x7f4f4c161340 2023-11-28T13:09:51.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.991+0000 7f4f4bfff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4f4c15c9b0 0x7f4f4c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:59960/0 (socket says 172.21.15.156:59960) 2023-11-28T13:09:51.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.991+0000 7f4f4bfff700 1 -- 172.21.15.156:0/1121184388 learned_addr learned my addr 172.21.15.156:0/1121184388 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:51.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.991+0000 7f4f4b7fe700 1 --2- 172.21.15.156:0/1121184388 >> v2:172.21.15.156:3300/0 conn(0x7f4f4c161340 0x7f4f4c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:51.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.991+0000 7f4f4affd700 1 -- 172.21.15.156:0/1121184388 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716963729 0 0) 0x7f4f4c0d1950 con 0x7f4f4c15c9b0 2023-11-28T13:09:51.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.991+0000 7f4f4affd700 1 -- 172.21.15.156:0/1121184388 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f340035b0 con 0x7f4f4c15c9b0 2023-11-28T13:09:51.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.991+0000 7f4f4b7fe700 1 -- 172.21.15.156:0/1121184388 >> v1:172.21.15.186:6789/0 conn(0x7f4f4c0d2320 legacy=0x7f4f4c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:51.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.991+0000 7f4f4b7fe700 1 -- 172.21.15.156:0/1121184388 >> v1:172.21.15.156:6789/0 conn(0x7f4f4c15c9b0 legacy=0x7f4f4c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:51.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.991+0000 7f4f4b7fe700 1 -- 172.21.15.156:0/1121184388 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c0d1300 con 0x7f4f4c161340 2023-11-28T13:09:51.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.992+0000 7f4f4b7fe700 1 --2- 172.21.15.156:0/1121184388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f4c161340 0x7f4f4c0d1de0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009840 tx=0x7f4f40009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbf9e449d635a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:51.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.992+0000 7f4f4affd700 1 -- 172.21.15.156:0/1121184388 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f40011830 con 0x7f4f4c161340 2023-11-28T13:09:51.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.992+0000 7f4f4affd700 1 -- 172.21.15.156:0/1121184388 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f40011990 con 0x7f4f4c161340 2023-11-28T13:09:51.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.992+0000 7f4f4affd700 1 -- 172.21.15.156:0/1121184388 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f40011d20 con 0x7f4f4c161340 2023-11-28T13:09:51.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.992+0000 7f4f525ff700 1 -- 172.21.15.156:0/1121184388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f4c161340 msgr2=0x7f4f4c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:51.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.992+0000 7f4f525ff700 1 --2- 172.21.15.156:0/1121184388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f4c161340 0x7f4f4c0d1de0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009840 tx=0x7f4f40009b50 comp rx=0 tx=0).stop 2023-11-28T13:09:51.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.992+0000 7f4f525ff700 1 -- 172.21.15.156:0/1121184388 shutdown_connections 2023-11-28T13:09:52.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.992+0000 7f4f525ff700 1 --2- 172.21.15.156:0/1121184388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f4c161340 0x7f4f4c0d1de0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.992+0000 7f4f525ff700 1 -- 172.21.15.156:0/1121184388 >> 172.21.15.156:0/1121184388 conn(0x7f4f4c072600 msgr2=0x7f4f4c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:52.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.993+0000 7f4f525ff700 1 -- 172.21.15.156:0/1121184388 shutdown_connections 2023-11-28T13:09:52.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.993+0000 7f4f525ff700 1 -- 172.21.15.156:0/1121184388 wait complete. 2023-11-28T13:09:52.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.994+0000 7f4f525ff700 1 Processor -- start 2023-11-28T13:09:52.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.994+0000 7f4f525ff700 1 -- start start 2023-11-28T13:09:52.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.995+0000 7f4f525ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4c0d2320 0x7f4f4c1f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.995+0000 7f4f525ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f4c15c9b0 0x7f4f4c149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.995+0000 7f4f525ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f4c161340 0x7f4f4c1e6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.995+0000 7f4f525ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c0c8a00 con 0x7f4f4c161340 2023-11-28T13:09:52.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.995+0000 7f4f525ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c1f3360 con 0x7f4f4c0d2320 2023-11-28T13:09:52.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.995+0000 7f4f525ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c149980 con 0x7f4f4c15c9b0 2023-11-28T13:09:52.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.996+0000 7f4f50b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f4c161340 0x7f4f4c1e6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:52.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.996+0000 7f4f50b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f4c161340 0x7f4f4c1e6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38950/0 (socket says 172.21.15.156:38950) 2023-11-28T13:09:52.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.996+0000 7f4f50b9c700 1 -- 172.21.15.156:0/2579448293 learned_addr learned my addr 172.21.15.156:0/2579448293 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:52.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.996+0000 7f4f50b9c700 1 -- 172.21.15.156:0/2579448293 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f4c15c9b0 msgr2=0x7f4f4c149440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:52.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.996+0000 7f4f50b9c700 1 --2- 172.21.15.156:0/2579448293 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f4c15c9b0 0x7f4f4c149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.996+0000 7f4f50b9c700 1 -- 172.21.15.156:0/2579448293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4c0d2320 msgr2=0x7f4f4c1f2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:52.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.996+0000 7f4f50b9c700 1 --2- 172.21.15.156:0/2579448293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4c0d2320 0x7f4f4c1f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.996+0000 7f4f50b9c700 1 -- 172.21.15.156:0/2579448293 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f400094a0 con 0x7f4f4c161340 2023-11-28T13:09:52.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.996+0000 7f4f50b9c700 1 --2- 172.21.15.156:0/2579448293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f4c161340 0x7f4f4c1e6e50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400cbc0 tx=0x7f4f4400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:52.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.996+0000 7f4f48ff9700 1 -- 172.21.15.156:0/2579448293 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44003c00 con 0x7f4f4c161340 2023-11-28T13:09:52.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.996+0000 7f4f48ff9700 1 -- 172.21.15.156:0/2579448293 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f44003d60 con 0x7f4f4c161340 2023-11-28T13:09:52.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.996+0000 7f4f48ff9700 1 -- 172.21.15.156:0/2579448293 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4400abf0 con 0x7f4f4c161340 2023-11-28T13:09:52.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.996+0000 7f4f525ff700 1 -- 172.21.15.156:0/2579448293 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4c15cd80 con 0x7f4f4c161340 2023-11-28T13:09:52.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.997+0000 7f4f525ff700 1 -- 172.21.15.156:0/2579448293 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4c0d15e0 con 0x7f4f4c161340 2023-11-28T13:09:52.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.998+0000 7f4f525ff700 1 -- 172.21.15.156:0/2579448293 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f4c15cf10 con 0x7f4f4c161340 2023-11-28T13:09:52.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:51.999+0000 7f4f48ff9700 1 -- 172.21.15.156:0/2579448293 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4f44010050 con 0x7f4f4c161340 2023-11-28T13:09:52.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.000+0000 7f4f48ff9700 1 --2- 172.21.15.156:0/2579448293 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4f380682f0 0x7f4f3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.000+0000 7f4f48ff9700 1 -- 172.21.15.156:0/2579448293 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f4f44085ca0 con 0x7f4f4c161340 2023-11-28T13:09:52.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.000+0000 7f4f4bfff700 1 --2- 172.21.15.156:0/2579448293 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4f380682f0 0x7f4f3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:52.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.001+0000 7f4f4bfff700 1 --2- 172.21.15.156:0/2579448293 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4f380682f0 0x7f4f3806a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0016b0 tx=0x7f4f3c0079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:52.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.005+0000 7f4f48ff9700 1 -- 172.21.15.156:0/2579448293 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4f44052470 con 0x7f4f4c161340 2023-11-28T13:09:52.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.242+0000 7f4f525ff700 1 -- 172.21.15.156:0/2579448293 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4f4c0745c0 con 0x7f4f4c161340 2023-11-28T13:09:52.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.250+0000 7f4f48ff9700 1 -- 172.21.15.156:0/2579448293 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4f44055a90 con 0x7f4f4c161340 2023-11-28T13:09:52.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.260+0000 7f4f525ff700 1 -- 172.21.15.156:0/2579448293 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4f380682f0 msgr2=0x7f4f3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:52.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.260+0000 7f4f525ff700 1 --2- 172.21.15.156:0/2579448293 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4f380682f0 0x7f4f3806a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0016b0 tx=0x7f4f3c0079c0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.260+0000 7f4f525ff700 1 -- 172.21.15.156:0/2579448293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f4c161340 msgr2=0x7f4f4c1e6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:52.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.260+0000 7f4f525ff700 1 --2- 172.21.15.156:0/2579448293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f4c161340 0x7f4f4c1e6e50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400cbc0 tx=0x7f4f4400cf80 comp rx=0 tx=0).stop 2023-11-28T13:09:52.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.261+0000 7f4f525ff700 1 -- 172.21.15.156:0/2579448293 shutdown_connections 2023-11-28T13:09:52.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.261+0000 7f4f525ff700 1 --2- 172.21.15.156:0/2579448293 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4f380682f0 0x7f4f3806a7a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.261+0000 7f4f525ff700 1 --2- 172.21.15.156:0/2579448293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f4c161340 0x7f4f4c1e6e50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.261+0000 7f4f525ff700 1 --2- 172.21.15.156:0/2579448293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f4c0d2320 0x7f4f4c1f2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.261+0000 7f4f525ff700 1 --2- 172.21.15.156:0/2579448293 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f4c15c9b0 0x7f4f4c149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.261+0000 7f4f525ff700 1 -- 172.21.15.156:0/2579448293 >> 172.21.15.156:0/2579448293 conn(0x7f4f4c072600 msgr2=0x7f4f4c15f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:52.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.263+0000 7f4f525ff700 1 -- 172.21.15.156:0/2579448293 shutdown_connections 2023-11-28T13:09:52.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.263+0000 7f4f525ff700 1 -- 172.21.15.156:0/2579448293 wait complete. 2023-11-28T13:09:52.270 INFO:teuthology.orchestra.run.smithi156.stderr:new fs with metadata pool 7 and data pool 8 2023-11-28T13:09:52.299 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:09:52.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.497+0000 7f3e325ff700 1 Processor -- start 2023-11-28T13:09:52.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.497+0000 7f3e325ff700 1 -- start start 2023-11-28T13:09:52.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.498+0000 7f3e325ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e2c15ca50 0x7f3e2c06b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.498+0000 7f3e325ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3e2c0cce10 0x7f3e2c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.498+0000 7f3e325ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e2c0cf5e0 con 0x7f3e2c06b8c0 2023-11-28T13:09:52.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.498+0000 7f3e325ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e2c0cd290 con 0x7f3e2c0cce10 2023-11-28T13:09:52.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.498+0000 7f3e325ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c0cd470 con 0x7f3e2c15ca50 2023-11-28T13:09:52.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.499+0000 7f3e23fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f3e2c06b8c0 0x7f3e2c0cc700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:59974/0 (socket says 172.21.15.156:59974) 2023-11-28T13:09:52.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.499+0000 7f3e23fff700 1 -- 172.21.15.156:0/2547849981 learned_addr learned my addr 172.21.15.156:0/2547849981 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:52.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.499+0000 7f3e2bfff700 1 --2- 172.21.15.156:0/2547849981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e2c15ca50 0x7f3e2c06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:52.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.499+0000 7f3e30b9c700 1 --2- 172.21.15.156:0/2547849981 >> v2:172.21.15.186:3300/0 conn(0x7f3e2c0cce10 0x7f3e2c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:52.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.499+0000 7f3e2b7fe700 1 -- 172.21.15.156:0/2547849981 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739807161 0 0) 0x7f3e2c0cf5e0 con 0x7f3e2c06b8c0 2023-11-28T13:09:52.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.499+0000 7f3e2b7fe700 1 -- 172.21.15.156:0/2547849981 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e180035b0 con 0x7f3e2c06b8c0 2023-11-28T13:09:52.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.499+0000 7f3e2b7fe700 1 -- 172.21.15.156:0/2547849981 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 517712373 0 0) 0x7f3e180035b0 con 0x7f3e2c06b8c0 2023-11-28T13:09:52.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.499+0000 7f3e2b7fe700 1 -- 172.21.15.156:0/2547849981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e2c15ca50 msgr2=0x7f3e2c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:52.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.499+0000 7f3e2b7fe700 1 --2- 172.21.15.156:0/2547849981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e2c15ca50 0x7f3e2c06b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.499+0000 7f3e2b7fe700 1 -- 172.21.15.156:0/2547849981 >> v2:172.21.15.186:3300/0 conn(0x7f3e2c0cce10 msgr2=0x7f3e2c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:52.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.499+0000 7f3e2b7fe700 1 --2- 172.21.15.156:0/2547849981 >> v2:172.21.15.186:3300/0 conn(0x7f3e2c0cce10 0x7f3e2c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.499+0000 7f3e2b7fe700 1 -- 172.21.15.156:0/2547849981 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2c0cecb0 con 0x7f3e2c06b8c0 2023-11-28T13:09:52.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.499+0000 7f3e2b7fe700 1 -- 172.21.15.156:0/2547849981 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f3e14001670 con 0x7f3e2c06b8c0 2023-11-28T13:09:52.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.499+0000 7f3e2bfff700 1 --2- 172.21.15.156:0/2547849981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e2c15ca50 0x7f3e2c06b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:09:52.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.499+0000 7f3e2b7fe700 1 -- 172.21.15.156:0/2547849981 >> v1:172.21.15.156:6789/0 conn(0x7f3e2c06b8c0 legacy=0x7f3e2c0cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:52.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.500+0000 7f3e2b7fe700 1 --2- 172.21.15.156:0/2547849981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e2c15ca50 0x7f3e18003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.500+0000 7f3e2b7fe700 1 --2- 172.21.15.156:0/2547849981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e18003f70 0x7f3e18004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.500+0000 7f3e2b7fe700 1 --2- 172.21.15.156:0/2547849981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e18004d90 0x7f3e18005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.500+0000 7f3e2b7fe700 1 -- 172.21.15.156:0/2547849981 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c0cd290 con 0x7f3e18004d90 2023-11-28T13:09:52.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.500+0000 7f3e2b7fe700 1 -- 172.21.15.156:0/2547849981 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c0cd470 con 0x7f3e18003f70 2023-11-28T13:09:52.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.500+0000 7f3e2b7fe700 1 -- 172.21.15.156:0/2547849981 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1800ba10 con 0x7f3e2c15ca50 2023-11-28T13:09:52.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.500+0000 7f3e30b9c700 1 --2- 172.21.15.156:0/2547849981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e18004d90 0x7f3e18005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:52.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.500+0000 7f3e23fff700 1 --2- 172.21.15.156:0/2547849981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e18003f70 0x7f3e18004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:52.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.500+0000 7f3e2bfff700 1 --2- 172.21.15.156:0/2547849981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e2c15ca50 0x7f3e18003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:52.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.500+0000 7f3e30b9c700 1 -- 172.21.15.156:0/2547849981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e2c15ca50 msgr2=0x7f3e18003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:52.509 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.500+0000 7f3e30b9c700 1 --2- 172.21.15.156:0/2547849981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e2c15ca50 0x7f3e18003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.509 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.500+0000 7f3e30b9c700 1 -- 172.21.15.156:0/2547849981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e18003f70 msgr2=0x7f3e18004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:52.509 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.500+0000 7f3e30b9c700 1 --2- 172.21.15.156:0/2547849981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e18003f70 0x7f3e18004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.509 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.500+0000 7f3e30b9c700 1 -- 172.21.15.156:0/2547849981 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1c0094a0 con 0x7f3e18004d90 2023-11-28T13:09:52.509 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.501+0000 7f3e30b9c700 1 --2- 172.21.15.156:0/2547849981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e18004d90 0x7f3e18005200 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009fc0 tx=0x7f3e18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd3c2ebb1f788a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:52.509 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.501+0000 7f3e2b7fe700 1 -- 172.21.15.156:0/2547849981 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2400ec30 con 0x7f3e18004d90 2023-11-28T13:09:52.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.501+0000 7f3e2b7fe700 1 -- 172.21.15.156:0/2547849981 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2400f040 con 0x7f3e18004d90 2023-11-28T13:09:52.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.501+0000 7f3e2b7fe700 1 -- 172.21.15.156:0/2547849981 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e24014480 con 0x7f3e18004d90 2023-11-28T13:09:52.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.501+0000 7f3e325ff700 1 -- 172.21.15.156:0/2547849981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e18004d90 msgr2=0x7f3e18005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:52.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.501+0000 7f3e325ff700 1 --2- 172.21.15.156:0/2547849981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e18004d90 0x7f3e18005200 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009fc0 tx=0x7f3e18005880 comp rx=0 tx=0).stop 2023-11-28T13:09:52.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.501+0000 7f3e30b9c700 1 -- 172.21.15.156:0/2547849981 reap_dead start 2023-11-28T13:09:52.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.501+0000 7f3e325ff700 1 -- 172.21.15.156:0/2547849981 shutdown_connections 2023-11-28T13:09:52.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.501+0000 7f3e325ff700 1 -- 172.21.15.156:0/2547849981 >> 172.21.15.156:0/2547849981 conn(0x7f3e2c09a230 msgr2=0x7f3e2c08afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:52.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.501+0000 7f3e325ff700 1 -- 172.21.15.156:0/2547849981 shutdown_connections 2023-11-28T13:09:52.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.502+0000 7f3e325ff700 1 -- 172.21.15.156:0/2547849981 wait complete. 2023-11-28T13:09:52.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.503+0000 7f3e325ff700 1 Processor -- start 2023-11-28T13:09:52.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.503+0000 7f3e325ff700 1 -- start start 2023-11-28T13:09:52.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.504+0000 7f3e325ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e18004d90 0x7f3e2c1f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.504+0000 7f3e325ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e2c157180 0x7f3e2c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.504+0000 7f3e325ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e2c06f6f0 0x7f3e2c15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.504+0000 7f3e325ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c1f5510 con 0x7f3e18004d90 2023-11-28T13:09:52.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.504+0000 7f3e325ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c1f57f0 con 0x7f3e2c157180 2023-11-28T13:09:52.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.504+0000 7f3e325ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c1f5ad0 con 0x7f3e2c06f6f0 2023-11-28T13:09:52.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.505+0000 7f3e2bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e18004d90 0x7f3e2c1f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:52.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.505+0000 7f3e2bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e18004d90 0x7f3e2c1f1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38958/0 (socket says 172.21.15.156:38958) 2023-11-28T13:09:52.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.505+0000 7f3e2bfff700 1 -- 172.21.15.156:0/4270587297 learned_addr learned my addr 172.21.15.156:0/4270587297 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:52.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.505+0000 7f3e23fff700 1 --2- 172.21.15.156:0/4270587297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e2c157180 0x7f3e2c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:52.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.505+0000 7f3e30b9c700 1 --2- 172.21.15.156:0/4270587297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e2c06f6f0 0x7f3e2c15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:52.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.505+0000 7f3e2bfff700 1 -- 172.21.15.156:0/4270587297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e2c06f6f0 msgr2=0x7f3e2c15f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:52.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.505+0000 7f3e2bfff700 1 --2- 172.21.15.156:0/4270587297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e2c06f6f0 0x7f3e2c15f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.505+0000 7f3e2bfff700 1 -- 172.21.15.156:0/4270587297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e2c157180 msgr2=0x7f3e2c06f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:52.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.505+0000 7f3e2bfff700 1 --2- 172.21.15.156:0/4270587297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e2c157180 0x7f3e2c06f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.505+0000 7f3e2bfff700 1 -- 172.21.15.156:0/4270587297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2c0cf270 con 0x7f3e18004d90 2023-11-28T13:09:52.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.505+0000 7f3e2bfff700 1 --2- 172.21.15.156:0/4270587297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e18004d90 0x7f3e2c1f1db0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c011cd0 tx=0x7f3e1c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:52.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.505+0000 7f3e297fa700 1 -- 172.21.15.156:0/4270587297 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c00fa00 con 0x7f3e18004d90 2023-11-28T13:09:52.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.506+0000 7f3e297fa700 1 -- 172.21.15.156:0/4270587297 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1c00fb60 con 0x7f3e18004d90 2023-11-28T13:09:52.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.506+0000 7f3e325ff700 1 -- 172.21.15.156:0/4270587297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2c0cef90 con 0x7f3e18004d90 2023-11-28T13:09:52.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.506+0000 7f3e297fa700 1 -- 172.21.15.156:0/4270587297 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c004020 con 0x7f3e18004d90 2023-11-28T13:09:52.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.506+0000 7f3e325ff700 1 -- 172.21.15.156:0/4270587297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2c15ce20 con 0x7f3e18004d90 2023-11-28T13:09:52.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.507+0000 7f3e325ff700 1 -- 172.21.15.156:0/4270587297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e2c0745c0 con 0x7f3e18004d90 2023-11-28T13:09:52.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.508+0000 7f3e297fa700 1 -- 172.21.15.156:0/4270587297 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3e1c024070 con 0x7f3e18004d90 2023-11-28T13:09:52.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.508+0000 7f3e297fa700 1 --2- 172.21.15.156:0/4270587297 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3e0c0682f0 0x7f3e0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.509+0000 7f3e297fa700 1 -- 172.21.15.156:0/4270587297 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3e1c086120 con 0x7f3e18004d90 2023-11-28T13:09:52.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.509+0000 7f3e23fff700 1 --2- 172.21.15.156:0/4270587297 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3e0c0682f0 0x7f3e0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:52.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.510+0000 7f3e23fff700 1 --2- 172.21.15.156:0/4270587297 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3e0c0682f0 0x7f3e0c06a7a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3e14001050 tx=0x7f3e14000f80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:52.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.513+0000 7f3e297fa700 1 -- 172.21.15.156:0/4270587297 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3e1c052890 con 0x7f3e18004d90 2023-11-28T13:09:52.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.707+0000 7f3e325ff700 1 -- 172.21.15.156:0/4270587297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e2c0adde0 con 0x7f3e18004d90 2023-11-28T13:09:52.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.708+0000 7f3e297fa700 1 -- 172.21.15.156:0/4270587297 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+17982 (secure 0 0 0) 0x7f3e1c022090 con 0x7f3e18004d90 2023-11-28T13:09:52.712 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:52.713 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":46,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:09:52.244623+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:09:49.958043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T13:09:51.219302+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:09:52.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.711+0000 7f3e325ff700 1 -- 172.21.15.156:0/4270587297 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3e0c0682f0 msgr2=0x7f3e0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:52.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.711+0000 7f3e325ff700 1 --2- 172.21.15.156:0/4270587297 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3e0c0682f0 0x7f3e0c06a7a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3e14001050 tx=0x7f3e14000f80 comp rx=0 tx=0).stop 2023-11-28T13:09:52.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.711+0000 7f3e325ff700 1 -- 172.21.15.156:0/4270587297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e18004d90 msgr2=0x7f3e2c1f1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:52.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.711+0000 7f3e325ff700 1 --2- 172.21.15.156:0/4270587297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e18004d90 0x7f3e2c1f1db0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c011cd0 tx=0x7f3e1c009ed0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.711+0000 7f3e325ff700 1 -- 172.21.15.156:0/4270587297 shutdown_connections 2023-11-28T13:09:52.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.711+0000 7f3e325ff700 1 --2- 172.21.15.156:0/4270587297 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3e0c0682f0 0x7f3e0c06a7a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.711+0000 7f3e325ff700 1 --2- 172.21.15.156:0/4270587297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3e2c06f6f0 0x7f3e2c15f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.711+0000 7f3e325ff700 1 --2- 172.21.15.156:0/4270587297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e18004d90 0x7f3e2c1f1db0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.711+0000 7f3e325ff700 1 --2- 172.21.15.156:0/4270587297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e2c157180 0x7f3e2c06f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.711+0000 7f3e325ff700 1 -- 172.21.15.156:0/4270587297 >> 172.21.15.156:0/4270587297 conn(0x7f3e2c09a230 msgr2=0x7f3e2c0d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:52.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.711+0000 7f3e325ff700 1 -- 172.21.15.156:0/4270587297 shutdown_connections 2023-11-28T13:09:52.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.711+0000 7f3e325ff700 1 -- 172.21.15.156:0/4270587297 wait complete. 2023-11-28T13:09:52.731 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:09:52.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.929+0000 7fe683bff700 1 Processor -- start 2023-11-28T13:09:52.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.929+0000 7fe683bff700 1 -- start start 2023-11-28T13:09:52.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.930+0000 7fe683bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe67c161340 0x7fe67c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.930+0000 7fe683bff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe67c0d2320 0x7fe67c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.930+0000 7fe683bff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67c0d1950 con 0x7fe67c15c9b0 2023-11-28T13:09:52.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.930+0000 7fe683bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe67c0b5800 con 0x7fe67c161340 2023-11-28T13:09:52.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.930+0000 7fe683bff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe67c0b5300 con 0x7fe67c0d2320 2023-11-28T13:09:52.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.930+0000 7fe68199b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fe67c15c9b0 0x7fe67c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:59988/0 (socket says 172.21.15.156:59988) 2023-11-28T13:09:52.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.930+0000 7fe68199b700 1 -- 172.21.15.156:0/1526688558 learned_addr learned my addr 172.21.15.156:0/1526688558 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:52.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.930+0000 7fe68119a700 1 --2- 172.21.15.156:0/1526688558 >> v2:172.21.15.156:3300/0 conn(0x7fe67c161340 0x7fe67c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:52.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.930+0000 7fe68219c700 1 --2- 172.21.15.156:0/1526688558 >> v2:172.21.15.186:3300/0 conn(0x7fe67c0d2320 0x7fe67c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:52.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.931+0000 7fe680999700 1 -- 172.21.15.156:0/1526688558 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3206615480 0 0) 0x7fe67c0d1950 con 0x7fe67c15c9b0 2023-11-28T13:09:52.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.931+0000 7fe680999700 1 -- 172.21.15.156:0/1526688558 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6680035b0 con 0x7fe67c15c9b0 2023-11-28T13:09:52.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.931+0000 7fe68119a700 1 -- 172.21.15.156:0/1526688558 >> v2:172.21.15.186:3300/0 conn(0x7fe67c0d2320 msgr2=0x7fe67c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:52.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.931+0000 7fe68119a700 1 --2- 172.21.15.156:0/1526688558 >> v2:172.21.15.186:3300/0 conn(0x7fe67c0d2320 0x7fe67c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.931+0000 7fe68119a700 1 -- 172.21.15.156:0/1526688558 >> v1:172.21.15.156:6789/0 conn(0x7fe67c15c9b0 legacy=0x7fe67c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:52.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.931+0000 7fe68119a700 1 -- 172.21.15.156:0/1526688558 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c0b5520 con 0x7fe67c161340 2023-11-28T13:09:52.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.931+0000 7fe68119a700 1 --2- 172.21.15.156:0/1526688558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe67c161340 0x7fe67c0d1de0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe67c06c170 tx=0x7fe66c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4055e86a919f0c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:52.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.931+0000 7fe680999700 1 -- 172.21.15.156:0/1526688558 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c011830 con 0x7fe67c161340 2023-11-28T13:09:52.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.931+0000 7fe680999700 1 -- 172.21.15.156:0/1526688558 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66c011990 con 0x7fe67c161340 2023-11-28T13:09:52.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.931+0000 7fe680999700 1 -- 172.21.15.156:0/1526688558 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c011d20 con 0x7fe67c161340 2023-11-28T13:09:52.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.932+0000 7fe683bff700 1 -- 172.21.15.156:0/1526688558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe67c161340 msgr2=0x7fe67c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:52.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.932+0000 7fe683bff700 1 --2- 172.21.15.156:0/1526688558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe67c161340 0x7fe67c0d1de0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe67c06c170 tx=0x7fe66c009b50 comp rx=0 tx=0).stop 2023-11-28T13:09:52.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.932+0000 7fe683bff700 1 -- 172.21.15.156:0/1526688558 shutdown_connections 2023-11-28T13:09:52.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.932+0000 7fe683bff700 1 --2- 172.21.15.156:0/1526688558 >> v2:172.21.15.186:3300/0 conn(0x7fe67c0d2320 0x7fe67c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.932+0000 7fe683bff700 1 --2- 172.21.15.156:0/1526688558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe67c161340 0x7fe67c0d1de0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.932+0000 7fe683bff700 1 -- 172.21.15.156:0/1526688558 >> 172.21.15.156:0/1526688558 conn(0x7fe67c072600 msgr2=0x7fe67c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:52.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.932+0000 7fe683bff700 1 -- 172.21.15.156:0/1526688558 shutdown_connections 2023-11-28T13:09:52.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.932+0000 7fe683bff700 1 -- 172.21.15.156:0/1526688558 wait complete. 2023-11-28T13:09:52.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.934+0000 7fe683bff700 1 Processor -- start 2023-11-28T13:09:52.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.934+0000 7fe683bff700 1 -- start start 2023-11-28T13:09:52.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.935+0000 7fe683bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c0d2320 0x7fe67c15d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.935+0000 7fe683bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe67c15c9b0 0x7fe67c1517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.935+0000 7fe683bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe67c161340 0x7fe67c152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.935+0000 7fe683bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0c6ca0 con 0x7fe67c15c9b0 2023-11-28T13:09:52.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.935+0000 7fe683bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c15d8f0 con 0x7fe67c0d2320 2023-11-28T13:09:52.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.935+0000 7fe683bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c15dbd0 con 0x7fe67c161340 2023-11-28T13:09:52.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.935+0000 7fe68119a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe67c15c9b0 0x7fe67c1517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:52.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.935+0000 7fe68119a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe67c15c9b0 0x7fe67c1517f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38980/0 (socket says 172.21.15.156:38980) 2023-11-28T13:09:52.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.935+0000 7fe68119a700 1 -- 172.21.15.156:0/387805831 learned_addr learned my addr 172.21.15.156:0/387805831 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:52.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.935+0000 7fe68199b700 1 --2- 172.21.15.156:0/387805831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c0d2320 0x7fe67c15d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:52.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.936+0000 7fe68219c700 1 --2- 172.21.15.156:0/387805831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe67c161340 0x7fe67c152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:52.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.936+0000 7fe68119a700 1 -- 172.21.15.156:0/387805831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe67c161340 msgr2=0x7fe67c152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:52.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.936+0000 7fe68119a700 1 --2- 172.21.15.156:0/387805831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe67c161340 0x7fe67c152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.936+0000 7fe68119a700 1 -- 172.21.15.156:0/387805831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c0d2320 msgr2=0x7fe67c15d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:52.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.936+0000 7fe68119a700 1 --2- 172.21.15.156:0/387805831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c0d2320 0x7fe67c15d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:52.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.936+0000 7fe68119a700 1 -- 172.21.15.156:0/387805831 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c0094a0 con 0x7fe67c15c9b0 2023-11-28T13:09:52.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.936+0000 7fe68219c700 1 --2- 172.21.15.156:0/387805831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe67c161340 0x7fe67c152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:52.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.936+0000 7fe68119a700 1 --2- 172.21.15.156:0/387805831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe67c15c9b0 0x7fe67c1517f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0097c0 tx=0x7fe66c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:52.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.936+0000 7fe6727fc700 1 -- 172.21.15.156:0/387805831 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c0036a0 con 0x7fe67c15c9b0 2023-11-28T13:09:52.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.936+0000 7fe6727fc700 1 -- 172.21.15.156:0/387805831 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66c01e070 con 0x7fe67c15c9b0 2023-11-28T13:09:52.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.936+0000 7fe683bff700 1 -- 172.21.15.156:0/387805831 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67c0d15e0 con 0x7fe67c15c9b0 2023-11-28T13:09:52.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.936+0000 7fe6727fc700 1 -- 172.21.15.156:0/387805831 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c003a60 con 0x7fe67c15c9b0 2023-11-28T13:09:52.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.936+0000 7fe683bff700 1 -- 172.21.15.156:0/387805831 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67c0d1300 con 0x7fe67c15c9b0 2023-11-28T13:09:52.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.937+0000 7fe683bff700 1 -- 172.21.15.156:0/387805831 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67c15cd80 con 0x7fe67c15c9b0 2023-11-28T13:09:52.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.938+0000 7fe6727fc700 1 -- 172.21.15.156:0/387805831 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe66c00f050 con 0x7fe67c15c9b0 2023-11-28T13:09:52.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.939+0000 7fe6727fc700 1 --2- 172.21.15.156:0/387805831 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6600682f0 0x7fe66006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:52.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.939+0000 7fe6727fc700 1 -- 172.21.15.156:0/387805831 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe66c0867a0 con 0x7fe67c15c9b0 2023-11-28T13:09:52.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.939+0000 7fe68199b700 1 --2- 172.21.15.156:0/387805831 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6600682f0 0x7fe66006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:52.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.940+0000 7fe68199b700 1 --2- 172.21.15.156:0/387805831 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6600682f0 0x7fe66006a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe678001560 tx=0x7fe678009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:52.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:52.943+0000 7fe6727fc700 1 -- 172.21.15.156:0/387805831 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe66c053000 con 0x7fe67c15c9b0 2023-11-28T13:09:53.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.135+0000 7fe683bff700 1 -- 172.21.15.156:0/387805831 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe67c15e0c0 con 0x7fe67c15c9b0 2023-11-28T13:09:53.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.136+0000 7fe6727fc700 1 -- 172.21.15.156:0/387805831 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+17982 (secure 0 0 0) 0x7fe66c056620 con 0x7fe67c15c9b0 2023-11-28T13:09:53.138 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:53.138 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":46,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:09:52.244623+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:09:49.958043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T13:09:51.219302+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:09:53.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.138+0000 7fe683bff700 1 -- 172.21.15.156:0/387805831 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6600682f0 msgr2=0x7fe66006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:53.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.138+0000 7fe683bff700 1 --2- 172.21.15.156:0/387805831 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6600682f0 0x7fe66006a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe678001560 tx=0x7fe678009040 comp rx=0 tx=0).stop 2023-11-28T13:09:53.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.138+0000 7fe683bff700 1 -- 172.21.15.156:0/387805831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe67c15c9b0 msgr2=0x7fe67c1517f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:53.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.138+0000 7fe683bff700 1 --2- 172.21.15.156:0/387805831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe67c15c9b0 0x7fe67c1517f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0097c0 tx=0x7fe66c004750 comp rx=0 tx=0).stop 2023-11-28T13:09:53.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.138+0000 7fe683bff700 1 -- 172.21.15.156:0/387805831 shutdown_connections 2023-11-28T13:09:53.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.138+0000 7fe683bff700 1 --2- 172.21.15.156:0/387805831 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6600682f0 0x7fe66006a7a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:53.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.138+0000 7fe683bff700 1 --2- 172.21.15.156:0/387805831 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe67c161340 0x7fe67c152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:53.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.138+0000 7fe683bff700 1 --2- 172.21.15.156:0/387805831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c0d2320 0x7fe67c15d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:53.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.138+0000 7fe683bff700 1 --2- 172.21.15.156:0/387805831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe67c15c9b0 0x7fe67c1517f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:53.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.138+0000 7fe683bff700 1 -- 172.21.15.156:0/387805831 >> 172.21.15.156:0/387805831 conn(0x7fe67c072600 msgr2=0x7fe67c15a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:53.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.139+0000 7fe683bff700 1 -- 172.21.15.156:0/387805831 shutdown_connections 2023-11-28T13:09:53.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.139+0000 7fe683bff700 1 -- 172.21.15.156:0/387805831 wait complete. 2023-11-28T13:09:53.159 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T13:09:53.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.359+0000 7f748ffff700 1 Processor -- start 2023-11-28T13:09:53.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.359+0000 7f748ffff700 1 -- start start 2023-11-28T13:09:53.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.360+0000 7f748ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f748815c9b0 0x7f74880b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:53.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.360+0000 7f748ffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7488161340 0x7f74880d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:53.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.360+0000 7f748ffff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74880d1950 con 0x7f74880d2320 2023-11-28T13:09:53.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.360+0000 7f748ffff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74880b5800 con 0x7f748815c9b0 2023-11-28T13:09:53.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.360+0000 7f748ffff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74880b5520 con 0x7f7488161340 2023-11-28T13:09:53.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.360+0000 7f748dd9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f748815c9b0 0x7f74880b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:53.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.360+0000 7f748dd9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f748815c9b0 0x7f74880b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38990/0 (socket says 172.21.15.156:38990) 2023-11-28T13:09:53.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.360+0000 7f748dd9b700 1 -- 172.21.15.156:0/3619186397 learned_addr learned my addr 172.21.15.156:0/3619186397 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:53.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.361+0000 7f748d59a700 1 --2- 172.21.15.156:0/3619186397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7488161340 0x7f74880d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:53.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.361+0000 7f748dd9b700 1 -- 172.21.15.156:0/3619186397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7488161340 msgr2=0x7f74880d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:53.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.361+0000 7f748dd9b700 1 --2- 172.21.15.156:0/3619186397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7488161340 0x7f74880d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:53.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.361+0000 7f748dd9b700 1 -- 172.21.15.156:0/3619186397 >> v1:172.21.15.186:6789/0 conn(0x7f74880d2320 legacy=0x7f74880cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:53.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.361+0000 7f748dd9b700 1 -- 172.21.15.156:0/3619186397 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74880d1300 con 0x7f748815c9b0 2023-11-28T13:09:53.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.361+0000 7f748dd9b700 1 --2- 172.21.15.156:0/3619186397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748815c9b0 0x7f74880b4bf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f74840098f0 tx=0x7f7484009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47aa7b38f486cd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:53.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.361+0000 7f748cd99700 1 -- 172.21.15.156:0/3619186397 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748400b8a0 con 0x7f748815c9b0 2023-11-28T13:09:53.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.361+0000 7f748cd99700 1 -- 172.21.15.156:0/3619186397 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748400ba00 con 0x7f748815c9b0 2023-11-28T13:09:53.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.361+0000 7f748cd99700 1 -- 172.21.15.156:0/3619186397 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748400bd90 con 0x7f748815c9b0 2023-11-28T13:09:53.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.362+0000 7f748ffff700 1 -- 172.21.15.156:0/3619186397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748815c9b0 msgr2=0x7f74880b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:53.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.362+0000 7f748ffff700 1 --2- 172.21.15.156:0/3619186397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748815c9b0 0x7f74880b4bf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f74840098f0 tx=0x7f7484009c00 comp rx=0 tx=0).stop 2023-11-28T13:09:53.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.362+0000 7f748ffff700 1 -- 172.21.15.156:0/3619186397 shutdown_connections 2023-11-28T13:09:53.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.362+0000 7f748ffff700 1 --2- 172.21.15.156:0/3619186397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748815c9b0 0x7f74880b4bf0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:53.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.362+0000 7f748ffff700 1 --2- 172.21.15.156:0/3619186397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7488161340 0x7f74880d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:53.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.362+0000 7f748ffff700 1 -- 172.21.15.156:0/3619186397 >> 172.21.15.156:0/3619186397 conn(0x7f7488072600 msgr2=0x7f74880cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:53.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.362+0000 7f748ffff700 1 -- 172.21.15.156:0/3619186397 shutdown_connections 2023-11-28T13:09:53.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.362+0000 7f748ffff700 1 -- 172.21.15.156:0/3619186397 wait complete. 2023-11-28T13:09:53.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.364+0000 7f748ffff700 1 Processor -- start 2023-11-28T13:09:53.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.364+0000 7f748ffff700 1 -- start start 2023-11-28T13:09:53.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.365+0000 7f748ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74880d2320 0x7f7488151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:53.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.365+0000 7f748ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748815c9b0 0x7f7488153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:53.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.365+0000 7f748ffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7488161340 0x7f748806a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:53.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.365+0000 7f748ffff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74880b82b0 con 0x7f748815c9b0 2023-11-28T13:09:53.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.365+0000 7f748ffff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7488151550 con 0x7f74880d2320 2023-11-28T13:09:53.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.365+0000 7f748ffff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7488152380 con 0x7f7488161340 2023-11-28T13:09:53.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.365+0000 7f748d59a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748815c9b0 0x7f7488153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:53.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.365+0000 7f748d59a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748815c9b0 0x7f7488153420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39002/0 (socket says 172.21.15.156:39002) 2023-11-28T13:09:53.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.365+0000 7f748d59a700 1 -- 172.21.15.156:0/175492272 learned_addr learned my addr 172.21.15.156:0/175492272 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:53.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.366+0000 7f748dd9b700 1 --2- 172.21.15.156:0/175492272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74880d2320 0x7f7488151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:53.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.366+0000 7f748d59a700 1 -- 172.21.15.156:0/175492272 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7488161340 msgr2=0x7f748806a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:53.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.366+0000 7f748e59c700 1 --2- 172.21.15.156:0/175492272 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7488161340 0x7f748806a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:53.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.366+0000 7f748d59a700 1 --2- 172.21.15.156:0/175492272 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7488161340 0x7f748806a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:53.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.366+0000 7f748d59a700 1 -- 172.21.15.156:0/175492272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74880d2320 msgr2=0x7f7488151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:53.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.366+0000 7f748d59a700 1 --2- 172.21.15.156:0/175492272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74880d2320 0x7f7488151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:53.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.366+0000 7f748d59a700 1 -- 172.21.15.156:0/175492272 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7484009550 con 0x7f748815c9b0 2023-11-28T13:09:53.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.366+0000 7f748dd9b700 1 --2- 172.21.15.156:0/175492272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74880d2320 0x7f7488151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:53.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.366+0000 7f748e59c700 1 --2- 172.21.15.156:0/175492272 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7488161340 0x7f748806a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:53.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.366+0000 7f748d59a700 1 --2- 172.21.15.156:0/175492272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748815c9b0 0x7f7488153420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f747800e810 tx=0x7f747800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:53.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.366+0000 7f747e7fc700 1 -- 172.21.15.156:0/175492272 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478013610 con 0x7f748815c9b0 2023-11-28T13:09:53.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.366+0000 7f747e7fc700 1 -- 172.21.15.156:0/175492272 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7478013770 con 0x7f748815c9b0 2023-11-28T13:09:53.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.366+0000 7f747e7fc700 1 -- 172.21.15.156:0/175492272 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478013b20 con 0x7f748815c9b0 2023-11-28T13:09:53.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.366+0000 7f748ffff700 1 -- 172.21.15.156:0/175492272 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74880b5130 con 0x7f748815c9b0 2023-11-28T13:09:53.370 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.366+0000 7f748ffff700 1 -- 172.21.15.156:0/175492272 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74880d15e0 con 0x7f748815c9b0 2023-11-28T13:09:53.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.367+0000 7f748ffff700 1 -- 172.21.15.156:0/175492272 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74880b52c0 con 0x7f748815c9b0 2023-11-28T13:09:53.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.368+0000 7f747e7fc700 1 -- 172.21.15.156:0/175492272 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7478023070 con 0x7f748815c9b0 2023-11-28T13:09:53.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.369+0000 7f747e7fc700 1 --2- 172.21.15.156:0/175492272 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f746c0682f0 0x7f746c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:53.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.369+0000 7f747e7fc700 1 -- 172.21.15.156:0/175492272 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7478085b40 con 0x7f748815c9b0 2023-11-28T13:09:53.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.370+0000 7f748dd9b700 1 --2- 172.21.15.156:0/175492272 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f746c0682f0 0x7f746c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:53.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.370+0000 7f748dd9b700 1 --2- 172.21.15.156:0/175492272 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f746c0682f0 0x7f746c06a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7484014040 tx=0x7f7484008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:53.378 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.373+0000 7f747e7fc700 1 -- 172.21.15.156:0/175492272 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7478052220 con 0x7f748815c9b0 2023-11-28T13:09:53.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:53.614+0000 7f748ffff700 1 -- 172.21.15.156:0/175492272 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f74880745c0 con 0x7f748815c9b0 2023-11-28T13:09:54.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.252+0000 7f747e7fc700 1 -- 172.21.15.156:0/175492272 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7f74780170b0 con 0x7f748815c9b0 2023-11-28T13:09:54.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.254+0000 7f748ffff700 1 -- 172.21.15.156:0/175492272 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f746c0682f0 msgr2=0x7f746c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:54.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.254+0000 7f748ffff700 1 --2- 172.21.15.156:0/175492272 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f746c0682f0 0x7f746c06a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7484014040 tx=0x7f7484008030 comp rx=0 tx=0).stop 2023-11-28T13:09:54.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.254+0000 7f748ffff700 1 -- 172.21.15.156:0/175492272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748815c9b0 msgr2=0x7f7488153420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:54.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.254+0000 7f748ffff700 1 --2- 172.21.15.156:0/175492272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748815c9b0 0x7f7488153420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f747800e810 tx=0x7f747800ebd0 comp rx=0 tx=0).stop 2023-11-28T13:09:54.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.255+0000 7f748ffff700 1 -- 172.21.15.156:0/175492272 shutdown_connections 2023-11-28T13:09:54.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.255+0000 7f748ffff700 1 --2- 172.21.15.156:0/175492272 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f746c0682f0 0x7f746c06a7a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:54.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.255+0000 7f748ffff700 1 --2- 172.21.15.156:0/175492272 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7488161340 0x7f748806a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:54.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.255+0000 7f748ffff700 1 --2- 172.21.15.156:0/175492272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74880d2320 0x7f7488151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:54.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.255+0000 7f748ffff700 1 --2- 172.21.15.156:0/175492272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f748815c9b0 0x7f7488153420 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:54.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.255+0000 7f748ffff700 1 -- 172.21.15.156:0/175492272 >> 172.21.15.156:0/175492272 conn(0x7f7488072600 msgr2=0x7f74881582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:54.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.255+0000 7f748ffff700 1 -- 172.21.15.156:0/175492272 shutdown_connections 2023-11-28T13:09:54.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.255+0000 7f748ffff700 1 -- 172.21.15.156:0/175492272 wait complete. 2023-11-28T13:09:54.277 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T13:09:54.278 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T13:09:54.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.480+0000 7f07369ff700 1 Processor -- start 2023-11-28T13:09:54.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.480+0000 7f07369ff700 1 -- start start 2023-11-28T13:09:54.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.481+0000 7f07369ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0730161340 0x7f07300d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:54.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.481+0000 7f07369ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07300d1950 con 0x7f07300d2320 2023-11-28T13:09:54.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.482+0000 7f07369ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07300b5520 con 0x7f073015c9b0 2023-11-28T13:09:54.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.482+0000 7f07369ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07300b5800 con 0x7f0730161340 2023-11-28T13:09:54.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.482+0000 7f0734f9c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f07300d2320 0x7f07300cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:60004/0 (socket says 172.21.15.156:60004) 2023-11-28T13:09:54.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.482+0000 7f0734f9c700 1 -- 172.21.15.156:0/3112965732 learned_addr learned my addr 172.21.15.156:0/3112965732 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:54.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.482+0000 7f072f7fe700 1 --2- 172.21.15.156:0/3112965732 >> v2:172.21.15.156:3300/0 conn(0x7f0730161340 0x7f07300d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:54.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.482+0000 7f072f7fe700 1 -- 172.21.15.156:0/3112965732 >> v1:172.21.15.186:6789/0 conn(0x7f073015c9b0 legacy=0x7f07300b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:54.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.482+0000 7f072f7fe700 1 -- 172.21.15.156:0/3112965732 >> v1:172.21.15.156:6789/0 conn(0x7f07300d2320 legacy=0x7f07300cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:54.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.482+0000 7f072f7fe700 1 -- 172.21.15.156:0/3112965732 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07300d1300 con 0x7f0730161340 2023-11-28T13:09:54.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.482+0000 7f072f7fe700 1 --2- 172.21.15.156:0/3112965732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0730161340 0x7f07300d1de0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0724009840 tx=0x7f0724009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c6f02d0469d08a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:54.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.483+0000 7f072effd700 1 -- 172.21.15.156:0/3112965732 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0724004230 con 0x7f0730161340 2023-11-28T13:09:54.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.483+0000 7f072effd700 1 -- 172.21.15.156:0/3112965732 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0724004390 con 0x7f0730161340 2023-11-28T13:09:54.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.483+0000 7f072effd700 1 -- 172.21.15.156:0/3112965732 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07240036a0 con 0x7f0730161340 2023-11-28T13:09:54.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.483+0000 7f07369ff700 1 -- 172.21.15.156:0/3112965732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0730161340 msgr2=0x7f07300d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:54.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.483+0000 7f07369ff700 1 --2- 172.21.15.156:0/3112965732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0730161340 0x7f07300d1de0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0724009840 tx=0x7f0724009b50 comp rx=0 tx=0).stop 2023-11-28T13:09:54.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.483+0000 7f07369ff700 1 -- 172.21.15.156:0/3112965732 shutdown_connections 2023-11-28T13:09:54.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.483+0000 7f07369ff700 1 --2- 172.21.15.156:0/3112965732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0730161340 0x7f07300d1de0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:54.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.483+0000 7f07369ff700 1 -- 172.21.15.156:0/3112965732 >> 172.21.15.156:0/3112965732 conn(0x7f0730072600 msgr2=0x7f07300cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:54.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.483+0000 7f07369ff700 1 -- 172.21.15.156:0/3112965732 shutdown_connections 2023-11-28T13:09:54.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.484+0000 7f07369ff700 1 -- 172.21.15.156:0/3112965732 wait complete. 2023-11-28T13:09:54.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.485+0000 7f07369ff700 1 Processor -- start 2023-11-28T13:09:54.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.485+0000 7f07369ff700 1 -- start start 2023-11-28T13:09:54.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.486+0000 7f07369ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f07300d2320 0x7f07301e6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:54.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.486+0000 7f07369ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f073015c9b0 0x7f0730156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:54.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.486+0000 7f07369ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0730161340 0x7f0730157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:54.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.486+0000 7f07369ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07300c6a40 con 0x7f0730161340 2023-11-28T13:09:54.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.486+0000 7f07369ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07301e7930 con 0x7f073015c9b0 2023-11-28T13:09:54.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.486+0000 7f07369ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07301e7c10 con 0x7f07300d2320 2023-11-28T13:09:54.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.487+0000 7f0734f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0730161340 0x7f0730157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:54.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.487+0000 7f0734f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0730161340 0x7f0730157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39012/0 (socket says 172.21.15.156:39012) 2023-11-28T13:09:54.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.487+0000 7f0734f9c700 1 -- 172.21.15.156:0/2287761996 learned_addr learned my addr 172.21.15.156:0/2287761996 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:54.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.487+0000 7f072ffff700 1 --2- 172.21.15.156:0/2287761996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f07300d2320 0x7f07301e6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:54.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.487+0000 7f072f7fe700 1 --2- 172.21.15.156:0/2287761996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f073015c9b0 0x7f0730156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:54.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.487+0000 7f0734f9c700 1 -- 172.21.15.156:0/2287761996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f07300d2320 msgr2=0x7f07301e6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:54.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.487+0000 7f0734f9c700 1 --2- 172.21.15.156:0/2287761996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f07300d2320 0x7f07301e6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:54.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.487+0000 7f0734f9c700 1 -- 172.21.15.156:0/2287761996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f073015c9b0 msgr2=0x7f0730156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:54.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.487+0000 7f0734f9c700 1 --2- 172.21.15.156:0/2287761996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f073015c9b0 0x7f0730156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:54.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.487+0000 7f0734f9c700 1 -- 172.21.15.156:0/2287761996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07240094a0 con 0x7f0730161340 2023-11-28T13:09:54.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.487+0000 7f0734f9c700 1 --2- 172.21.15.156:0/2287761996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0730161340 0x7f0730157360 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f072800caa0 tx=0x7f072800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:54.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.488+0000 7f072cff9700 1 -- 172.21.15.156:0/2287761996 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072800a9e0 con 0x7f0730161340 2023-11-28T13:09:54.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.488+0000 7f072cff9700 1 -- 172.21.15.156:0/2287761996 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072800ab40 con 0x7f0730161340 2023-11-28T13:09:54.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.488+0000 7f07369ff700 1 -- 172.21.15.156:0/2287761996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073015cd80 con 0x7f0730161340 2023-11-28T13:09:54.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.488+0000 7f072cff9700 1 -- 172.21.15.156:0/2287761996 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728012400 con 0x7f0730161340 2023-11-28T13:09:54.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.488+0000 7f07369ff700 1 -- 172.21.15.156:0/2287761996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07300d15e0 con 0x7f0730161340 2023-11-28T13:09:54.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.489+0000 7f07369ff700 1 -- 172.21.15.156:0/2287761996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073015cf10 con 0x7f0730161340 2023-11-28T13:09:54.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.490+0000 7f072cff9700 1 -- 172.21.15.156:0/2287761996 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0728019030 con 0x7f0730161340 2023-11-28T13:09:54.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.491+0000 7f072cff9700 1 --2- 172.21.15.156:0/2287761996 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f071c0682f0 0x7f071c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:54.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.491+0000 7f072cff9700 1 -- 172.21.15.156:0/2287761996 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0728085510 con 0x7f0730161340 2023-11-28T13:09:54.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.491+0000 7f072ffff700 1 --2- 172.21.15.156:0/2287761996 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f071c0682f0 0x7f071c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:54.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.492+0000 7f072ffff700 1 --2- 172.21.15.156:0/2287761996 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f071c0682f0 0x7f071c06a7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0720007fd0 tx=0x7f0720007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:54.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.495+0000 7f072cff9700 1 -- 172.21.15.156:0/2287761996 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f072804b5e0 con 0x7f0730161340 2023-11-28T13:09:54.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:54.727+0000 7f07369ff700 1 -- 172.21.15.156:0/2287761996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f07300745c0 con 0x7f0730161340 2023-11-28T13:09:55.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.253+0000 7f072cff9700 1 -- 172.21.15.156:0/2287761996 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0728051ee0 con 0x7f0730161340 2023-11-28T13:09:55.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.257+0000 7f07369ff700 1 -- 172.21.15.156:0/2287761996 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f071c0682f0 msgr2=0x7f071c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:55.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.257+0000 7f07369ff700 1 --2- 172.21.15.156:0/2287761996 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f071c0682f0 0x7f071c06a7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0720007fd0 tx=0x7f0720007ec0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.257+0000 7f07369ff700 1 -- 172.21.15.156:0/2287761996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0730161340 msgr2=0x7f0730157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:55.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.257+0000 7f07369ff700 1 --2- 172.21.15.156:0/2287761996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0730161340 0x7f0730157360 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f072800caa0 tx=0x7f072800cdb0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.257+0000 7f07369ff700 1 -- 172.21.15.156:0/2287761996 shutdown_connections 2023-11-28T13:09:55.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.257+0000 7f07369ff700 1 --2- 172.21.15.156:0/2287761996 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f071c0682f0 0x7f071c06a7a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.257+0000 7f07369ff700 1 --2- 172.21.15.156:0/2287761996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0730161340 0x7f0730157360 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.257+0000 7f07369ff700 1 --2- 172.21.15.156:0/2287761996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f07300d2320 0x7f07301e6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.257+0000 7f07369ff700 1 --2- 172.21.15.156:0/2287761996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f073015c9b0 0x7f0730156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.257+0000 7f07369ff700 1 -- 172.21.15.156:0/2287761996 >> 172.21.15.156:0/2287761996 conn(0x7f0730072600 msgr2=0x7f07300b6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:55.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.257+0000 7f07369ff700 1 -- 172.21.15.156:0/2287761996 shutdown_connections 2023-11-28T13:09:55.260 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.257+0000 7f07369ff700 1 -- 172.21.15.156:0/2287761996 wait complete. 2023-11-28T13:09:55.278 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T13:09:55.279 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:09:55.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.475+0000 7f7b909ff700 1 Processor -- start 2023-11-28T13:09:55.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.476+0000 7f7b909ff700 1 -- start start 2023-11-28T13:09:55.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.477+0000 7f7b909ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7b8c15c9b0 0x7f7b8c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:55.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.477+0000 7f7b909ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7b8c0cd7c0 0x7f7b8c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:55.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.477+0000 7f7b909ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b8c0d1950 con 0x7f7b8c161340 2023-11-28T13:09:55.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.477+0000 7f7b909ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b8c0b5800 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.477+0000 7f7b909ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b8c0ce140 con 0x7f7b8c15c9b0 2023-11-28T13:09:55.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.477+0000 7f7b8ad9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7b8c0cd7c0 0x7f7b8c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:55.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.477+0000 7f7b8ad9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7b8c0cd7c0 0x7f7b8c0cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39024/0 (socket says 172.21.15.156:39024) 2023-11-28T13:09:55.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.477+0000 7f7b8ad9d700 1 -- 172.21.15.156:0/3776804493 learned_addr learned my addr 172.21.15.156:0/3776804493 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:55.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.477+0000 7f7b8a59c700 1 --2- 172.21.15.156:0/3776804493 >> v2:172.21.15.186:3300/0 conn(0x7f7b8c15c9b0 0x7f7b8c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:55.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.477+0000 7f7b8ad9d700 1 -- 172.21.15.156:0/3776804493 >> v2:172.21.15.186:3300/0 conn(0x7f7b8c15c9b0 msgr2=0x7f7b8c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:55.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.477+0000 7f7b8ad9d700 1 --2- 172.21.15.156:0/3776804493 >> v2:172.21.15.186:3300/0 conn(0x7f7b8c15c9b0 0x7f7b8c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.477+0000 7f7b8ad9d700 1 -- 172.21.15.156:0/3776804493 >> v1:172.21.15.186:6789/0 conn(0x7f7b8c161340 legacy=0x7f7b8c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:55.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.477+0000 7f7b8ad9d700 1 -- 172.21.15.156:0/3776804493 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c0d24f0 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.478+0000 7f7b8ad9d700 1 --2- 172.21.15.156:0/3776804493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b8c0cd7c0 0x7f7b8c0cdc00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c067460 tx=0x7f7b8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9312be1b1e063a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.478+0000 7f7b8959a700 1 -- 172.21.15.156:0/3776804493 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8400e070 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.478+0000 7f7b8959a700 1 -- 172.21.15.156:0/3776804493 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8400b9c0 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.478+0000 7f7b8959a700 1 -- 172.21.15.156:0/3776804493 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8400bd70 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.479+0000 7f7b909ff700 1 -- 172.21.15.156:0/3776804493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b8c0cd7c0 msgr2=0x7f7b8c0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.479+0000 7f7b909ff700 1 --2- 172.21.15.156:0/3776804493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b8c0cd7c0 0x7f7b8c0cdc00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c067460 tx=0x7f7b8400b330 comp rx=0 tx=0).stop 2023-11-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.479+0000 7f7b909ff700 1 -- 172.21.15.156:0/3776804493 shutdown_connections 2023-11-28T13:09:55.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.479+0000 7f7b909ff700 1 --2- 172.21.15.156:0/3776804493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b8c0cd7c0 0x7f7b8c0cdc00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.479+0000 7f7b909ff700 1 --2- 172.21.15.156:0/3776804493 >> v2:172.21.15.186:3300/0 conn(0x7f7b8c15c9b0 0x7f7b8c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.479+0000 7f7b909ff700 1 -- 172.21.15.156:0/3776804493 >> 172.21.15.156:0/3776804493 conn(0x7f7b8c072600 msgr2=0x7f7b8c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:55.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.479+0000 7f7b909ff700 1 -- 172.21.15.156:0/3776804493 shutdown_connections 2023-11-28T13:09:55.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.479+0000 7f7b909ff700 1 -- 172.21.15.156:0/3776804493 wait complete. 2023-11-28T13:09:55.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.480+0000 7f7b909ff700 1 Processor -- start 2023-11-28T13:09:55.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.481+0000 7f7b909ff700 1 -- start start 2023-11-28T13:09:55.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.481+0000 7f7b909ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b8c0cd7c0 0x7f7b8c156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:55.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.482+0000 7f7b909ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b8c15c9b0 0x7f7b8c14a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:55.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.482+0000 7f7b909ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7b8c161340 0x7f7b8c0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:55.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.482+0000 7f7b909ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c14a990 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.482+0000 7f7b909ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c0c7080 con 0x7f7b8c15c9b0 2023-11-28T13:09:55.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.482+0000 7f7b909ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c0c7360 con 0x7f7b8c161340 2023-11-28T13:09:55.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.482+0000 7f7b8a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b8c0cd7c0 0x7f7b8c156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:55.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.482+0000 7f7b8a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b8c0cd7c0 0x7f7b8c156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39028/0 (socket says 172.21.15.156:39028) 2023-11-28T13:09:55.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.482+0000 7f7b8a59c700 1 -- 172.21.15.156:0/3756374116 learned_addr learned my addr 172.21.15.156:0/3756374116 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:55.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.482+0000 7f7b8ad9d700 1 --2- 172.21.15.156:0/3756374116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7b8c161340 0x7f7b8c0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:55.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.482+0000 7f7b89d9b700 1 --2- 172.21.15.156:0/3756374116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b8c15c9b0 0x7f7b8c14a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:55.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.482+0000 7f7b8a59c700 1 -- 172.21.15.156:0/3756374116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7b8c161340 msgr2=0x7f7b8c0c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:55.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.482+0000 7f7b8a59c700 1 --2- 172.21.15.156:0/3756374116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7b8c161340 0x7f7b8c0c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.482+0000 7f7b8a59c700 1 -- 172.21.15.156:0/3756374116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b8c15c9b0 msgr2=0x7f7b8c14a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:55.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.482+0000 7f7b8a59c700 1 --2- 172.21.15.156:0/3756374116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b8c15c9b0 0x7f7b8c14a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.482+0000 7f7b8a59c700 1 -- 172.21.15.156:0/3756374116 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b84009b30 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.482+0000 7f7b89d9b700 1 --2- 172.21.15.156:0/3756374116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b8c15c9b0 0x7f7b8c14a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:55.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.483+0000 7f7b8a59c700 1 --2- 172.21.15.156:0/3756374116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b8c0cd7c0 0x7f7b8c156f90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00d800 tx=0x7f7b7c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:55.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.483+0000 7f7b7affd700 1 -- 172.21.15.156:0/3756374116 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0125f0 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.483+0000 7f7b7affd700 1 -- 172.21.15.156:0/3756374116 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7c012750 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.483+0000 7f7b7affd700 1 -- 172.21.15.156:0/3756374116 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c012b00 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.483+0000 7f7b909ff700 1 -- 172.21.15.156:0/3756374116 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b8c0d1300 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.483+0000 7f7b909ff700 1 -- 172.21.15.156:0/3756374116 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8c0b5520 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.484+0000 7f7b909ff700 1 -- 172.21.15.156:0/3756374116 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b8c0d15e0 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.485+0000 7f7b7affd700 1 -- 172.21.15.156:0/3756374116 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7b7c012c60 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.486+0000 7f7b7affd700 1 --2- 172.21.15.156:0/3756374116 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7b6c0682f0 0x7f7b6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:55.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.486+0000 7f7b7affd700 1 -- 172.21.15.156:0/3756374116 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7b7c086280 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.486+0000 7f7b89d9b700 1 --2- 172.21.15.156:0/3756374116 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7b6c0682f0 0x7f7b6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:55.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.487+0000 7f7b89d9b700 1 --2- 172.21.15.156:0/3756374116 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7b6c0682f0 0x7f7b6c06a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0cd790 tx=0x7f7b80009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:55.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.490+0000 7f7b7affd700 1 -- 172.21.15.156:0/3756374116 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7b7c0529f0 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.727+0000 7f7b909ff700 1 -- 172.21.15.156:0/3756374116 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b8c0745c0 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.727+0000 7f7b7affd700 1 -- 172.21.15.156:0/3756374116 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f7b7c056010 con 0x7f7b8c0cd7c0 2023-11-28T13:09:55.729 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:55.729 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3828141874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3828141874},{"type":"v1","addr":"172.21.15.156:6835","nonce":3828141874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1012489762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1012489762},{"type":"v1","addr":"172.21.15.186:6837","nonce":1012489762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2894670074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2894670074},{"type":"v1","addr":"172.21.15.186:6835","nonce":2894670074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:09:52.245032+0000","modified":"2023-11-28T13:09:55.251254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5481":{"gid":5481,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6834/1930782798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1930782798},{"type":"v1","addr":"172.21.15.156:6834","nonce":1930782798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:09:55.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.729+0000 7f7b909ff700 1 -- 172.21.15.156:0/3756374116 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7b6c0682f0 msgr2=0x7f7b6c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:55.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.729+0000 7f7b909ff700 1 --2- 172.21.15.156:0/3756374116 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7b6c0682f0 0x7f7b6c06a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0cd790 tx=0x7f7b80009040 comp rx=0 tx=0).stop 2023-11-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.729+0000 7f7b909ff700 1 -- 172.21.15.156:0/3756374116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b8c0cd7c0 msgr2=0x7f7b8c156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.729+0000 7f7b909ff700 1 --2- 172.21.15.156:0/3756374116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b8c0cd7c0 0x7f7b8c156f90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00d800 tx=0x7f7b7c00dbc0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.729+0000 7f7b909ff700 1 -- 172.21.15.156:0/3756374116 shutdown_connections 2023-11-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.729+0000 7f7b909ff700 1 --2- 172.21.15.156:0/3756374116 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7b6c0682f0 0x7f7b6c06a7a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.729+0000 7f7b909ff700 1 --2- 172.21.15.156:0/3756374116 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7b8c161340 0x7f7b8c0c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.729+0000 7f7b909ff700 1 --2- 172.21.15.156:0/3756374116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b8c0cd7c0 0x7f7b8c156f90 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.729+0000 7f7b909ff700 1 --2- 172.21.15.156:0/3756374116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b8c15c9b0 0x7f7b8c14a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.729+0000 7f7b909ff700 1 -- 172.21.15.156:0/3756374116 >> 172.21.15.156:0/3756374116 conn(0x7f7b8c072600 msgr2=0x7f7b8c159a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:55.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.730+0000 7f7b909ff700 1 -- 172.21.15.156:0/3756374116 shutdown_connections 2023-11-28T13:09:55.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.730+0000 7f7b909ff700 1 -- 172.21.15.156:0/3756374116 wait complete. 2023-11-28T13:09:55.733 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 23 2023-11-28T13:09:55.750 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:09:55.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.947+0000 7f69043ff700 1 Processor -- start 2023-11-28T13:09:55.950 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.948+0000 7f69043ff700 1 -- start start 2023-11-28T13:09:55.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.949+0000 7f69043ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f68fc161340 0x7f68fc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:55.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.949+0000 7f69043ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68fc0d1950 con 0x7f68fc15c9b0 2023-11-28T13:09:55.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.949+0000 7f69043ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68fc0b5520 con 0x7f68fc0d2320 2023-11-28T13:09:55.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.949+0000 7f69043ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68fc0b5800 con 0x7f68fc161340 2023-11-28T13:09:55.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.949+0000 7f690219b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f68fc15c9b0 0x7f68fc0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53322/0 (socket says 172.21.15.156:53322) 2023-11-28T13:09:55.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.949+0000 7f690219b700 1 -- 172.21.15.156:0/1741242408 learned_addr learned my addr 172.21.15.156:0/1741242408 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:55.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.949+0000 7f690199a700 1 --2- 172.21.15.156:0/1741242408 >> v2:172.21.15.186:3300/0 conn(0x7f68fc161340 0x7f68fc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:55.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.949+0000 7f6901199700 1 -- 172.21.15.156:0/1741242408 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1047155073 0 0) 0x7f68fc0d1950 con 0x7f68fc15c9b0 2023-11-28T13:09:55.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.949+0000 7f6901199700 1 -- 172.21.15.156:0/1741242408 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68e80035b0 con 0x7f68fc15c9b0 2023-11-28T13:09:55.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f6901199700 1 -- 172.21.15.156:0/1741242408 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3179780234 0 0) 0x7f68e80035b0 con 0x7f68fc15c9b0 2023-11-28T13:09:55.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f6901199700 1 -- 172.21.15.156:0/1741242408 >> v2:172.21.15.186:3300/0 conn(0x7f68fc161340 msgr2=0x7f68fc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:55.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f6901199700 1 --2- 172.21.15.156:0/1741242408 >> v2:172.21.15.186:3300/0 conn(0x7f68fc161340 0x7f68fc0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f6901199700 1 -- 172.21.15.156:0/1741242408 >> v1:172.21.15.186:6789/0 conn(0x7f68fc0d2320 legacy=0x7f68fc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:55.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f6901199700 1 -- 172.21.15.156:0/1741242408 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc0d1300 con 0x7f68fc15c9b0 2023-11-28T13:09:55.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f6901199700 1 -- 172.21.15.156:0/1741242408 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f68f8001670 con 0x7f68fc15c9b0 2023-11-28T13:09:55.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f6901199700 1 -- 172.21.15.156:0/1741242408 >> v1:172.21.15.156:6789/0 conn(0x7f68fc15c9b0 legacy=0x7f68fc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:55.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f6901199700 1 --2- 172.21.15.156:0/1741242408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68e8003a50 0x7f68e8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:55.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f6901199700 1 --2- 172.21.15.156:0/1741242408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68e8004850 0x7f68e8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:55.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f6901199700 1 --2- 172.21.15.156:0/1741242408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68e8005250 0x7f68e800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:55.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f6901199700 1 -- 172.21.15.156:0/1741242408 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0b5800 con 0x7f68e8003a50 2023-11-28T13:09:55.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f6901199700 1 -- 172.21.15.156:0/1741242408 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e800c160 con 0x7f68e8004850 2023-11-28T13:09:55.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f6901199700 1 -- 172.21.15.156:0/1741242408 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e800c320 con 0x7f68e8005250 2023-11-28T13:09:55.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f690219b700 1 --2- 172.21.15.156:0/1741242408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68e8003a50 0x7f68e8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:55.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f690199a700 1 --2- 172.21.15.156:0/1741242408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68e8004850 0x7f68e8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:55.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f690299c700 1 --2- 172.21.15.156:0/1741242408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68e8005250 0x7f68e800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:55.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f690219b700 1 -- 172.21.15.156:0/1741242408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68e8005250 msgr2=0x7f68e800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:55.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f690219b700 1 --2- 172.21.15.156:0/1741242408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68e8005250 0x7f68e800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f690219b700 1 -- 172.21.15.156:0/1741242408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68e8004850 msgr2=0x7f68e8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:55.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f690219b700 1 --2- 172.21.15.156:0/1741242408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68e8004850 0x7f68e8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.950+0000 7f690219b700 1 -- 172.21.15.156:0/1741242408 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e80035b0 con 0x7f68e8003a50 2023-11-28T13:09:55.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.951+0000 7f690299c700 1 -- 172.21.15.156:0/1741242408 reap_dead start 2023-11-28T13:09:55.965 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.951+0000 7f690219b700 1 --2- 172.21.15.156:0/1741242408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68e8003a50 0x7f68e8003e60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f68f800c160 tx=0x7f68e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbdfac4f43d851fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:55.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.951+0000 7f6901199700 1 -- 172.21.15.156:0/1741242408 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8007b70 con 0x7f68e8003a50 2023-11-28T13:09:55.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.951+0000 7f6901199700 1 -- 172.21.15.156:0/1741242408 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f800cd00 con 0x7f68e8003a50 2023-11-28T13:09:55.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.951+0000 7f6901199700 1 -- 172.21.15.156:0/1741242408 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8005ba0 con 0x7f68e8003a50 2023-11-28T13:09:55.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.951+0000 7f69043ff700 1 -- 172.21.15.156:0/1741242408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68e8003a50 msgr2=0x7f68e8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:55.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.951+0000 7f69043ff700 1 --2- 172.21.15.156:0/1741242408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68e8003a50 0x7f68e8003e60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f68f800c160 tx=0x7f68e8005880 comp rx=0 tx=0).stop 2023-11-28T13:09:55.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.952+0000 7f69043ff700 1 -- 172.21.15.156:0/1741242408 shutdown_connections 2023-11-28T13:09:55.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.952+0000 7f69043ff700 1 --2- 172.21.15.156:0/1741242408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68e8003a50 0x7f68e8003e60 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.952+0000 7f69043ff700 1 -- 172.21.15.156:0/1741242408 >> 172.21.15.156:0/1741242408 conn(0x7f68fc072600 msgr2=0x7f68fc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:55.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.952+0000 7f69043ff700 1 -- 172.21.15.156:0/1741242408 shutdown_connections 2023-11-28T13:09:55.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.952+0000 7f69043ff700 1 -- 172.21.15.156:0/1741242408 wait complete. 2023-11-28T13:09:55.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.953+0000 7f69043ff700 1 Processor -- start 2023-11-28T13:09:55.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.953+0000 7f69043ff700 1 -- start start 2023-11-28T13:09:55.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.954+0000 7f69043ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68e8003a50 0x7f68fc0b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:55.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.954+0000 7f69043ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68fc1e7dc0 0x7f68fc156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:55.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.954+0000 7f69043ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68fc06a900 0x7f68fc06ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:55.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.954+0000 7f69043ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc157af0 con 0x7f68fc06a900 2023-11-28T13:09:55.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.954+0000 7f69043ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc06b2b0 con 0x7f68e8003a50 2023-11-28T13:09:55.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.954+0000 7f69043ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc157040 con 0x7f68fc1e7dc0 2023-11-28T13:09:55.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.955+0000 7f690299c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68fc06a900 0x7f68fc06ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:55.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.955+0000 7f690299c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68fc06a900 0x7f68fc06ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46468/0 (socket says 172.21.15.156:46468) 2023-11-28T13:09:55.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.955+0000 7f690299c700 1 -- 172.21.15.156:0/2443881472 learned_addr learned my addr 172.21.15.156:0/2443881472 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:55.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.955+0000 7f690219b700 1 --2- 172.21.15.156:0/2443881472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68e8003a50 0x7f68fc0b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:55.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.955+0000 7f690199a700 1 --2- 172.21.15.156:0/2443881472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68fc1e7dc0 0x7f68fc156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:55.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.955+0000 7f690299c700 1 -- 172.21.15.156:0/2443881472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68fc1e7dc0 msgr2=0x7f68fc156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:55.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.955+0000 7f690299c700 1 --2- 172.21.15.156:0/2443881472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68fc1e7dc0 0x7f68fc156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.955+0000 7f690299c700 1 -- 172.21.15.156:0/2443881472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68e8003a50 msgr2=0x7f68fc0b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:55.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.955+0000 7f690299c700 1 --2- 172.21.15.156:0/2443881472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68e8003a50 0x7f68fc0b82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:55.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.955+0000 7f690299c700 1 -- 172.21.15.156:0/2443881472 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc15cd80 con 0x7f68fc06a900 2023-11-28T13:09:55.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.955+0000 7f690299c700 1 --2- 172.21.15.156:0/2443881472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68fc06a900 0x7f68fc06ad70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0cfbc0 tx=0x7f68f400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:55.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.955+0000 7f68f2ffd700 1 -- 172.21.15.156:0/2443881472 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4003c10 con 0x7f68fc06a900 2023-11-28T13:09:55.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.956+0000 7f68f2ffd700 1 -- 172.21.15.156:0/2443881472 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f4003d70 con 0x7f68fc06a900 2023-11-28T13:09:55.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.956+0000 7f68f2ffd700 1 -- 172.21.15.156:0/2443881472 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4005310 con 0x7f68fc06a900 2023-11-28T13:09:55.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.956+0000 7f69043ff700 1 -- 172.21.15.156:0/2443881472 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68fc0d15e0 con 0x7f68fc06a900 2023-11-28T13:09:55.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.956+0000 7f69043ff700 1 -- 172.21.15.156:0/2443881472 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68fc15cf10 con 0x7f68fc06a900 2023-11-28T13:09:55.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.957+0000 7f69043ff700 1 -- 172.21.15.156:0/2443881472 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68fc0b8890 con 0x7f68fc06a900 2023-11-28T13:09:55.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.957+0000 7f68f2ffd700 1 -- 172.21.15.156:0/2443881472 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f68f4005470 con 0x7f68fc06a900 2023-11-28T13:09:55.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.958+0000 7f68f2ffd700 1 --2- 172.21.15.156:0/2443881472 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f68e00682f0 0x7f68e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:55.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.958+0000 7f68f2ffd700 1 -- 172.21.15.156:0/2443881472 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f68f4085e90 con 0x7f68fc06a900 2023-11-28T13:09:55.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.959+0000 7f690219b700 1 --2- 172.21.15.156:0/2443881472 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f68e00682f0 0x7f68e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:55.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.959+0000 7f690219b700 1 --2- 172.21.15.156:0/2443881472 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f68e00682f0 0x7f68e006a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f68f8000c00 tx=0x7f68f8007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:55.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:55.962+0000 7f68f2ffd700 1 -- 172.21.15.156:0/2443881472 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f68f4052600 con 0x7f68fc06a900 2023-11-28T13:09:56.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.156+0000 7f69043ff700 1 -- 172.21.15.156:0/2443881472 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68fc0745c0 con 0x7f68fc06a900 2023-11-28T13:09:56.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.157+0000 7f68f2ffd700 1 -- 172.21.15.156:0/2443881472 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+17969 (secure 0 0 0) 0x7f68f4055c20 con 0x7f68fc06a900 2023-11-28T13:09:56.159 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:56.159 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":47,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:09:53.247402+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:09:49.958043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T13:09:51.219302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:09:56.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.160+0000 7f69043ff700 1 -- 172.21.15.156:0/2443881472 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f68e00682f0 msgr2=0x7f68e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:56.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.160+0000 7f69043ff700 1 --2- 172.21.15.156:0/2443881472 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f68e00682f0 0x7f68e006a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f68f8000c00 tx=0x7f68f8007000 comp rx=0 tx=0).stop 2023-11-28T13:09:56.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.160+0000 7f69043ff700 1 -- 172.21.15.156:0/2443881472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68fc06a900 msgr2=0x7f68fc06ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:56.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.160+0000 7f69043ff700 1 --2- 172.21.15.156:0/2443881472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68fc06a900 0x7f68fc06ad70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0cfbc0 tx=0x7f68f400ee00 comp rx=0 tx=0).stop 2023-11-28T13:09:56.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.160+0000 7f69043ff700 1 -- 172.21.15.156:0/2443881472 shutdown_connections 2023-11-28T13:09:56.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.160+0000 7f69043ff700 1 --2- 172.21.15.156:0/2443881472 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f68e00682f0 0x7f68e006a7a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.160+0000 7f69043ff700 1 --2- 172.21.15.156:0/2443881472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68fc06a900 0x7f68fc06ad70 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.160+0000 7f69043ff700 1 --2- 172.21.15.156:0/2443881472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68e8003a50 0x7f68fc0b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.160+0000 7f69043ff700 1 --2- 172.21.15.156:0/2443881472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68fc1e7dc0 0x7f68fc156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.160+0000 7f69043ff700 1 -- 172.21.15.156:0/2443881472 >> 172.21.15.156:0/2443881472 conn(0x7f68fc072600 msgr2=0x7f68fc0b6290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:56.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.160+0000 7f69043ff700 1 -- 172.21.15.156:0/2443881472 shutdown_connections 2023-11-28T13:09:56.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.160+0000 7f69043ff700 1 -- 172.21.15.156:0/2443881472 wait complete. 2023-11-28T13:09:56.180 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T13:09:56.181 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:09:56.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.377+0000 7fb34b1ff700 1 Processor -- start 2023-11-28T13:09:56.379 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.377+0000 7fb34b1ff700 1 -- start start 2023-11-28T13:09:56.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.378+0000 7fb34b1ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb344161340 0x7fb3440d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:56.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.378+0000 7fb34b1ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3440d1950 con 0x7fb34415c9b0 2023-11-28T13:09:56.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.378+0000 7fb34b1ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3440b5520 con 0x7fb3440d2320 2023-11-28T13:09:56.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.378+0000 7fb34b1ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3440b5800 con 0x7fb344161340 2023-11-28T13:09:56.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.378+0000 7fb348f9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb34415c9b0 0x7fb3440b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53336/0 (socket says 172.21.15.156:53336) 2023-11-28T13:09:56.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.378+0000 7fb348f9b700 1 -- 172.21.15.156:0/1253584376 learned_addr learned my addr 172.21.15.156:0/1253584376 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:56.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.379+0000 7fb343fff700 1 --2- 172.21.15.156:0/1253584376 >> v2:172.21.15.186:3300/0 conn(0x7fb344161340 0x7fb3440d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:56.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.379+0000 7fb3437fe700 1 -- 172.21.15.156:0/1253584376 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801021656 0 0) 0x7fb3440d1950 con 0x7fb34415c9b0 2023-11-28T13:09:56.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.379+0000 7fb3437fe700 1 -- 172.21.15.156:0/1253584376 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3340035b0 con 0x7fb34415c9b0 2023-11-28T13:09:56.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.379+0000 7fb3437fe700 1 -- 172.21.15.156:0/1253584376 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1435956033 0 0) 0x7fb3340035b0 con 0x7fb34415c9b0 2023-11-28T13:09:56.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.379+0000 7fb3437fe700 1 -- 172.21.15.156:0/1253584376 >> v2:172.21.15.186:3300/0 conn(0x7fb344161340 msgr2=0x7fb3440d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:56.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.379+0000 7fb3437fe700 1 --2- 172.21.15.156:0/1253584376 >> v2:172.21.15.186:3300/0 conn(0x7fb344161340 0x7fb3440d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.379+0000 7fb3437fe700 1 -- 172.21.15.156:0/1253584376 >> v1:172.21.15.186:6789/0 conn(0x7fb3440d2320 legacy=0x7fb3440cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:56.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.379+0000 7fb3437fe700 1 -- 172.21.15.156:0/1253584376 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3440d1300 con 0x7fb34415c9b0 2023-11-28T13:09:56.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.379+0000 7fb3437fe700 1 -- 172.21.15.156:0/1253584376 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7fb338001670 con 0x7fb34415c9b0 2023-11-28T13:09:56.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.379+0000 7fb3437fe700 1 -- 172.21.15.156:0/1253584376 >> v1:172.21.15.156:6789/0 conn(0x7fb34415c9b0 legacy=0x7fb3440b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:56.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.379+0000 7fb3437fe700 1 --2- 172.21.15.156:0/1253584376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb334003a50 0x7fb334003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:56.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.379+0000 7fb3437fe700 1 --2- 172.21.15.156:0/1253584376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb334004850 0x7fb334004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:56.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.379+0000 7fb3437fe700 1 --2- 172.21.15.156:0/1253584376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb334005250 0x7fb33400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:56.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.379+0000 7fb3437fe700 1 -- 172.21.15.156:0/1253584376 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440b5800 con 0x7fb334003a50 2023-11-28T13:09:56.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.379+0000 7fb3437fe700 1 -- 172.21.15.156:0/1253584376 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33400c160 con 0x7fb334004850 2023-11-28T13:09:56.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.379+0000 7fb3437fe700 1 -- 172.21.15.156:0/1253584376 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33400c320 con 0x7fb334005250 2023-11-28T13:09:56.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.379+0000 7fb348f9b700 1 --2- 172.21.15.156:0/1253584376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb334003a50 0x7fb334003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:56.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.380+0000 7fb343fff700 1 --2- 172.21.15.156:0/1253584376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb334004850 0x7fb334004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:56.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.380+0000 7fb34979c700 1 --2- 172.21.15.156:0/1253584376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb334005250 0x7fb33400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:56.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.380+0000 7fb348f9b700 1 -- 172.21.15.156:0/1253584376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb334005250 msgr2=0x7fb33400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:56.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.380+0000 7fb348f9b700 1 --2- 172.21.15.156:0/1253584376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb334005250 0x7fb33400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.380+0000 7fb348f9b700 1 -- 172.21.15.156:0/1253584376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb334004850 msgr2=0x7fb334004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:56.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.380+0000 7fb348f9b700 1 --2- 172.21.15.156:0/1253584376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb334004850 0x7fb334004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.380+0000 7fb348f9b700 1 -- 172.21.15.156:0/1253584376 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3340035b0 con 0x7fb334003a50 2023-11-28T13:09:56.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.380+0000 7fb34979c700 1 -- 172.21.15.156:0/1253584376 reap_dead start 2023-11-28T13:09:56.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.380+0000 7fb348f9b700 1 --2- 172.21.15.156:0/1253584376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb334003a50 0x7fb334003e60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb33800c160 tx=0x7fb334005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5d9d5eb899ebf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:56.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.380+0000 7fb3437fe700 1 -- 172.21.15.156:0/1253584376 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338007b70 con 0x7fb334003a50 2023-11-28T13:09:56.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.380+0000 7fb3437fe700 1 -- 172.21.15.156:0/1253584376 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33800cd20 con 0x7fb334003a50 2023-11-28T13:09:56.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.380+0000 7fb3437fe700 1 -- 172.21.15.156:0/1253584376 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338005ba0 con 0x7fb334003a50 2023-11-28T13:09:56.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.381+0000 7fb34b1ff700 1 -- 172.21.15.156:0/1253584376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb334003a50 msgr2=0x7fb334003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:56.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.381+0000 7fb34b1ff700 1 --2- 172.21.15.156:0/1253584376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb334003a50 0x7fb334003e60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb33800c160 tx=0x7fb334005880 comp rx=0 tx=0).stop 2023-11-28T13:09:56.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.381+0000 7fb34b1ff700 1 -- 172.21.15.156:0/1253584376 shutdown_connections 2023-11-28T13:09:56.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.381+0000 7fb34b1ff700 1 --2- 172.21.15.156:0/1253584376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb334003a50 0x7fb334003e60 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.381+0000 7fb34b1ff700 1 -- 172.21.15.156:0/1253584376 >> 172.21.15.156:0/1253584376 conn(0x7fb344072600 msgr2=0x7fb3440cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:56.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.381+0000 7fb34b1ff700 1 -- 172.21.15.156:0/1253584376 shutdown_connections 2023-11-28T13:09:56.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.381+0000 7fb34b1ff700 1 -- 172.21.15.156:0/1253584376 wait complete. 2023-11-28T13:09:56.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.383+0000 7fb34b1ff700 1 Processor -- start 2023-11-28T13:09:56.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.383+0000 7fb34b1ff700 1 -- start start 2023-11-28T13:09:56.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.384+0000 7fb34b1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb334003a50 0x7fb3441577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:56.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.384+0000 7fb34b1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3440c8e70 0x7fb3440c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:56.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.384+0000 7fb34b1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3441e6ff0 0x7fb3441e7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:56.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.384+0000 7fb34b1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440caa20 con 0x7fb334003a50 2023-11-28T13:09:56.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.384+0000 7fb34b1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440c9820 con 0x7fb3441e6ff0 2023-11-28T13:09:56.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.384+0000 7fb34b1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440ca6b0 con 0x7fb3440c8e70 2023-11-28T13:09:56.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.384+0000 7fb348f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb334003a50 0x7fb3441577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:56.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.384+0000 7fb348f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb334003a50 0x7fb3441577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46490/0 (socket says 172.21.15.156:46490) 2023-11-28T13:09:56.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.384+0000 7fb348f9b700 1 -- 172.21.15.156:0/1080765161 learned_addr learned my addr 172.21.15.156:0/1080765161 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:56.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.384+0000 7fb343fff700 1 --2- 172.21.15.156:0/1080765161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3440c8e70 0x7fb3440c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:56.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.384+0000 7fb34979c700 1 --2- 172.21.15.156:0/1080765161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3441e6ff0 0x7fb3441e7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:56.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.385+0000 7fb348f9b700 1 -- 172.21.15.156:0/1080765161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3440c8e70 msgr2=0x7fb3440c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:56.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.385+0000 7fb348f9b700 1 --2- 172.21.15.156:0/1080765161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3440c8e70 0x7fb3440c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.385+0000 7fb348f9b700 1 -- 172.21.15.156:0/1080765161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3441e6ff0 msgr2=0x7fb3441e7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:56.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.385+0000 7fb348f9b700 1 --2- 172.21.15.156:0/1080765161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3441e6ff0 0x7fb3441e7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.385+0000 7fb348f9b700 1 -- 172.21.15.156:0/1080765161 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34415cd80 con 0x7fb334003a50 2023-11-28T13:09:56.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.385+0000 7fb348f9b700 1 --2- 172.21.15.156:0/1080765161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb334003a50 0x7fb3441577c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb338007ca0 tx=0x7fb338001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:56.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.385+0000 7fb3417fa700 1 -- 172.21.15.156:0/1080765161 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338004c80 con 0x7fb334003a50 2023-11-28T13:09:56.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.385+0000 7fb3417fa700 1 -- 172.21.15.156:0/1080765161 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb338004e00 con 0x7fb334003a50 2023-11-28T13:09:56.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.385+0000 7fb34b1ff700 1 -- 172.21.15.156:0/1080765161 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3440d15e0 con 0x7fb334003a50 2023-11-28T13:09:56.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.385+0000 7fb3417fa700 1 -- 172.21.15.156:0/1080765161 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338007310 con 0x7fb334003a50 2023-11-28T13:09:56.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.385+0000 7fb34b1ff700 1 -- 172.21.15.156:0/1080765161 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34415cf10 con 0x7fb334003a50 2023-11-28T13:09:56.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.386+0000 7fb34b1ff700 1 -- 172.21.15.156:0/1080765161 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3440745c0 con 0x7fb334003a50 2023-11-28T13:09:56.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.387+0000 7fb3417fa700 1 -- 172.21.15.156:0/1080765161 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb338010070 con 0x7fb334003a50 2023-11-28T13:09:56.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.389+0000 7fb3417fa700 1 --2- 172.21.15.156:0/1080765161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3280682f0 0x7fb32806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:56.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.389+0000 7fb3417fa700 1 -- 172.21.15.156:0/1080765161 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb338086ba0 con 0x7fb334003a50 2023-11-28T13:09:56.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.390+0000 7fb343fff700 1 --2- 172.21.15.156:0/1080765161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3280682f0 0x7fb32806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:56.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.390+0000 7fb343fff700 1 --2- 172.21.15.156:0/1080765161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3280682f0 0x7fb32806a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb34415c0a0 tx=0x7fb330009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:56.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.393+0000 7fb3417fa700 1 -- 172.21.15.156:0/1080765161 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb338053400 con 0x7fb334003a50 2023-11-28T13:09:56.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.582+0000 7fb34b1ff700 1 -- 172.21.15.156:0/1080765161 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb3440d08a0 con 0x7fb3280682f0 2023-11-28T13:09:56.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.593+0000 7fb3417fa700 1 -- 172.21.15.156:0/1080765161 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312226 (secure 0 0 0) 0x7fb3440d08a0 con 0x7fb3280682f0 2023-11-28T13:09:56.600 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:56.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.598+0000 7fb34b1ff700 1 -- 172.21.15.156:0/1080765161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3280682f0 msgr2=0x7fb32806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:56.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.598+0000 7fb34b1ff700 1 --2- 172.21.15.156:0/1080765161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3280682f0 0x7fb32806a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb34415c0a0 tx=0x7fb330009210 comp rx=0 tx=0).stop 2023-11-28T13:09:56.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.598+0000 7fb34b1ff700 1 -- 172.21.15.156:0/1080765161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb334003a50 msgr2=0x7fb3441577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:56.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.598+0000 7fb34b1ff700 1 --2- 172.21.15.156:0/1080765161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb334003a50 0x7fb3441577c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb338007ca0 tx=0x7fb338001930 comp rx=0 tx=0).stop 2023-11-28T13:09:56.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.598+0000 7fb34b1ff700 1 -- 172.21.15.156:0/1080765161 shutdown_connections 2023-11-28T13:09:56.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.598+0000 7fb34b1ff700 1 --2- 172.21.15.156:0/1080765161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3280682f0 0x7fb32806a7a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.598+0000 7fb34b1ff700 1 --2- 172.21.15.156:0/1080765161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3441e6ff0 0x7fb3441e7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.598+0000 7fb34b1ff700 1 --2- 172.21.15.156:0/1080765161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb334003a50 0x7fb3441577c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.598+0000 7fb34b1ff700 1 --2- 172.21.15.156:0/1080765161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3440c8e70 0x7fb3440c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.598+0000 7fb34b1ff700 1 -- 172.21.15.156:0/1080765161 >> 172.21.15.156:0/1080765161 conn(0x7fb344072600 msgr2=0x7fb3440b6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:56.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.598+0000 7fb34b1ff700 1 -- 172.21.15.156:0/1080765161 shutdown_connections 2023-11-28T13:09:56.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.598+0000 7fb34b1ff700 1 -- 172.21.15.156:0/1080765161 wait complete. 2023-11-28T13:09:56.606 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T13:09:56.618 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2023-11-28T13:09:56.042192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":90468,"kb_used_data":5540,"kb_used_omap":0,"kb_used_meta":84864,"kb_avail":754884252,"statfs":{"total":773094113280,"available":773001474048,"internally_reserved":0,"allocated":5672960,"data_stored":3088346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":86900736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.300089"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471911+0000","last_change":"2023-11-28T13:09:52.274749+0000","last_active":"2023-11-28T13:09:53.471911+0000","last_peered":"2023-11-28T13:09:53.471911+0000","last_clean":"2023-11-28T13:09:53.471911+0000","last_became_active":"2023-11-28T13:09:52.274149+0000","last_became_peered":"2023-11-28T13:09:52.274149+0000","last_unstale":"2023-11-28T13:09:53.471911+0000","last_undegraded":"2023-11-28T13:09:53.471911+0000","last_fullsized":"2023-11-28T13:09:53.471911+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:10.991833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.254224+0000","last_change":"2023-11-28T13:09:52.275005+0000","last_active":"2023-11-28T13:09:53.254224+0000","last_peered":"2023-11-28T13:09:53.254224+0000","last_clean":"2023-11-28T13:09:53.254224+0000","last_became_active":"2023-11-28T13:09:52.274683+0000","last_became_peered":"2023-11-28T13:09:52.274683+0000","last_unstale":"2023-11-28T13:09:53.254224+0000","last_undegraded":"2023-11-28T13:09:53.254224+0000","last_fullsized":"2023-11-28T13:09:53.254224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:27:12.289547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472499+0000","last_change":"2023-11-28T13:09:52.282972+0000","last_active":"2023-11-28T13:09:53.472499+0000","last_peered":"2023-11-28T13:09:53.472499+0000","last_clean":"2023-11-28T13:09:53.472499+0000","last_became_active":"2023-11-28T13:09:52.282706+0000","last_became_peered":"2023-11-28T13:09:52.282706+0000","last_unstale":"2023-11-28T13:09:53.472499+0000","last_undegraded":"2023-11-28T13:09:53.472499+0000","last_fullsized":"2023-11-28T13:09:53.472499+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:07:18.517785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.254151+0000","last_change":"2023-11-28T13:09:52.268041+0000","last_active":"2023-11-28T13:09:53.254151+0000","last_peered":"2023-11-28T13:09:53.254151+0000","last_clean":"2023-11-28T13:09:53.254151+0000","last_became_active":"2023-11-28T13:09:52.267756+0000","last_became_peered":"2023-11-28T13:09:52.267756+0000","last_unstale":"2023-11-28T13:09:53.254151+0000","last_undegraded":"2023-11-28T13:09:53.254151+0000","last_fullsized":"2023-11-28T13:09:53.254151+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:24.470324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471163+0000","last_change":"2023-11-28T13:09:52.275511+0000","last_active":"2023-11-28T13:09:53.471163+0000","last_peered":"2023-11-28T13:09:53.471163+0000","last_clean":"2023-11-28T13:09:53.471163+0000","last_became_active":"2023-11-28T13:09:52.274367+0000","last_became_peered":"2023-11-28T13:09:52.274367+0000","last_unstale":"2023-11-28T13:09:53.471163+0000","last_undegraded":"2023-11-28T13:09:53.471163+0000","last_fullsized":"2023-11-28T13:09:53.471163+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:37:06.556688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471070+0000","last_change":"2023-11-28T13:09:52.266289+0000","last_active":"2023-11-28T13:09:53.471070+0000","last_peered":"2023-11-28T13:09:53.471070+0000","last_clean":"2023-11-28T13:09:53.471070+0000","last_became_active":"2023-11-28T13:09:52.265982+0000","last_became_peered":"2023-11-28T13:09:52.265982+0000","last_unstale":"2023-11-28T13:09:53.471070+0000","last_undegraded":"2023-11-28T13:09:53.471070+0000","last_fullsized":"2023-11-28T13:09:53.471070+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:47.315227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471189+0000","last_change":"2023-11-28T13:09:52.283865+0000","last_active":"2023-11-28T13:09:53.471189+0000","last_peered":"2023-11-28T13:09:53.471189+0000","last_clean":"2023-11-28T13:09:53.471189+0000","last_became_active":"2023-11-28T13:09:52.283229+0000","last_became_peered":"2023-11-28T13:09:52.283229+0000","last_unstale":"2023-11-28T13:09:53.471189+0000","last_undegraded":"2023-11-28T13:09:53.471189+0000","last_fullsized":"2023-11-28T13:09:53.471189+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:26:01.144694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470700+0000","last_change":"2023-11-28T13:09:52.275613+0000","last_active":"2023-11-28T13:09:53.470700+0000","last_peered":"2023-11-28T13:09:53.470700+0000","last_clean":"2023-11-28T13:09:53.470700+0000","last_became_active":"2023-11-28T13:09:52.275355+0000","last_became_peered":"2023-11-28T13:09:52.275355+0000","last_unstale":"2023-11-28T13:09:53.470700+0000","last_undegraded":"2023-11-28T13:09:53.470700+0000","last_fullsized":"2023-11-28T13:09:53.470700+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:45:45.645058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472702+0000","last_change":"2023-11-28T13:09:52.268360+0000","last_active":"2023-11-28T13:09:53.472702+0000","last_peered":"2023-11-28T13:09:53.472702+0000","last_clean":"2023-11-28T13:09:53.472702+0000","last_became_active":"2023-11-28T13:09:52.268108+0000","last_became_peered":"2023-11-28T13:09:52.268108+0000","last_unstale":"2023-11-28T13:09:53.472702+0000","last_undegraded":"2023-11-28T13:09:53.472702+0000","last_fullsized":"2023-11-28T13:09:53.472702+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:31.359124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472095+0000","last_change":"2023-11-28T13:09:52.276213+0000","last_active":"2023-11-28T13:09:53.472095+0000","last_peered":"2023-11-28T13:09:53.472095+0000","last_clean":"2023-11-28T13:09:53.472095+0000","last_became_active":"2023-11-28T13:09:52.275921+0000","last_became_peered":"2023-11-28T13:09:52.275921+0000","last_unstale":"2023-11-28T13:09:53.472095+0000","last_undegraded":"2023-11-28T13:09:53.472095+0000","last_fullsized":"2023-11-28T13:09:53.472095+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:39.366007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.251919+0000","last_change":"2023-11-28T13:09:52.264946+0000","last_active":"2023-11-28T13:09:53.251919+0000","last_peered":"2023-11-28T13:09:53.251919+0000","last_clean":"2023-11-28T13:09:53.251919+0000","last_became_active":"2023-11-28T13:09:52.264546+0000","last_became_peered":"2023-11-28T13:09:52.264546+0000","last_unstale":"2023-11-28T13:09:53.251919+0000","last_undegraded":"2023-11-28T13:09:53.251919+0000","last_fullsized":"2023-11-28T13:09:53.251919+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:24:53.209118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472126+0000","last_change":"2023-11-28T13:09:52.259187+0000","last_active":"2023-11-28T13:09:53.472126+0000","last_peered":"2023-11-28T13:09:53.472126+0000","last_clean":"2023-11-28T13:09:53.472126+0000","last_became_active":"2023-11-28T13:09:52.258825+0000","last_became_peered":"2023-11-28T13:09:52.258825+0000","last_unstale":"2023-11-28T13:09:53.472126+0000","last_undegraded":"2023-11-28T13:09:53.472126+0000","last_fullsized":"2023-11-28T13:09:53.472126+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:36:33.641644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471253+0000","last_change":"2023-11-28T13:09:52.267437+0000","last_active":"2023-11-28T13:09:53.471253+0000","last_peered":"2023-11-28T13:09:53.471253+0000","last_clean":"2023-11-28T13:09:53.471253+0000","last_became_active":"2023-11-28T13:09:52.267202+0000","last_became_peered":"2023-11-28T13:09:52.267202+0000","last_unstale":"2023-11-28T13:09:53.471253+0000","last_undegraded":"2023-11-28T13:09:53.471253+0000","last_fullsized":"2023-11-28T13:09:53.471253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:18.276083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471361+0000","last_change":"2023-11-28T13:09:52.283672+0000","last_active":"2023-11-28T13:09:53.471361+0000","last_peered":"2023-11-28T13:09:53.471361+0000","last_clean":"2023-11-28T13:09:53.471361+0000","last_became_active":"2023-11-28T13:09:52.283002+0000","last_became_peered":"2023-11-28T13:09:52.283002+0000","last_unstale":"2023-11-28T13:09:53.471361+0000","last_undegraded":"2023-11-28T13:09:53.471361+0000","last_fullsized":"2023-11-28T13:09:53.471361+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:44:38.788752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253911+0000","last_change":"2023-11-28T13:09:52.259704+0000","last_active":"2023-11-28T13:09:53.253911+0000","last_peered":"2023-11-28T13:09:53.253911+0000","last_clean":"2023-11-28T13:09:53.253911+0000","last_became_active":"2023-11-28T13:09:52.258872+0000","last_became_peered":"2023-11-28T13:09:52.258872+0000","last_unstale":"2023-11-28T13:09:53.253911+0000","last_undegraded":"2023-11-28T13:09:53.253911+0000","last_fullsized":"2023-11-28T13:09:53.253911+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:11:56.997320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470619+0000","last_change":"2023-11-28T13:09:52.271148+0000","last_active":"2023-11-28T13:09:53.470619+0000","last_peered":"2023-11-28T13:09:53.470619+0000","last_clean":"2023-11-28T13:09:53.470619+0000","last_became_active":"2023-11-28T13:09:52.270912+0000","last_became_peered":"2023-11-28T13:09:52.270912+0000","last_unstale":"2023-11-28T13:09:53.470619+0000","last_undegraded":"2023-11-28T13:09:53.470619+0000","last_fullsized":"2023-11-28T13:09:53.470619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:56:24.470434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472935+0000","last_change":"2023-11-28T13:09:52.277655+0000","last_active":"2023-11-28T13:09:53.472935+0000","last_peered":"2023-11-28T13:09:53.472935+0000","last_clean":"2023-11-28T13:09:53.472935+0000","last_became_active":"2023-11-28T13:09:52.277424+0000","last_became_peered":"2023-11-28T13:09:52.277424+0000","last_unstale":"2023-11-28T13:09:53.472935+0000","last_undegraded":"2023-11-28T13:09:53.472935+0000","last_fullsized":"2023-11-28T13:09:53.472935+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:18:42.555378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.254331+0000","last_change":"2023-11-28T13:09:52.273160+0000","last_active":"2023-11-28T13:09:53.254331+0000","last_peered":"2023-11-28T13:09:53.254331+0000","last_clean":"2023-11-28T13:09:53.254331+0000","last_became_active":"2023-11-28T13:09:52.272833+0000","last_became_peered":"2023-11-28T13:09:52.272833+0000","last_unstale":"2023-11-28T13:09:53.254331+0000","last_undegraded":"2023-11-28T13:09:53.254331+0000","last_fullsized":"2023-11-28T13:09:53.254331+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:40.623485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472591+0000","last_change":"2023-11-28T13:09:52.278452+0000","last_active":"2023-11-28T13:09:53.472591+0000","last_peered":"2023-11-28T13:09:53.472591+0000","last_clean":"2023-11-28T13:09:53.472591+0000","last_became_active":"2023-11-28T13:09:52.277877+0000","last_became_peered":"2023-11-28T13:09:52.277877+0000","last_unstale":"2023-11-28T13:09:53.472591+0000","last_undegraded":"2023-11-28T13:09:53.472591+0000","last_fullsized":"2023-11-28T13:09:53.472591+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:36:33.641644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471731+0000","last_change":"2023-11-28T13:09:52.262741+0000","last_active":"2023-11-28T13:09:53.471731+0000","last_peered":"2023-11-28T13:09:53.471731+0000","last_clean":"2023-11-28T13:09:53.471731+0000","last_became_active":"2023-11-28T13:09:52.262441+0000","last_became_peered":"2023-11-28T13:09:52.262441+0000","last_unstale":"2023-11-28T13:09:53.471731+0000","last_undegraded":"2023-11-28T13:09:53.471731+0000","last_fullsized":"2023-11-28T13:09:53.471731+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:41.248280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471764+0000","last_change":"2023-11-28T13:09:52.262739+0000","last_active":"2023-11-28T13:09:53.471764+0000","last_peered":"2023-11-28T13:09:53.471764+0000","last_clean":"2023-11-28T13:09:53.471764+0000","last_became_active":"2023-11-28T13:09:52.262390+0000","last_became_peered":"2023-11-28T13:09:52.262390+0000","last_unstale":"2023-11-28T13:09:53.471764+0000","last_undegraded":"2023-11-28T13:09:53.471764+0000","last_fullsized":"2023-11-28T13:09:53.471764+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:58:44.619275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470895+0000","last_change":"2023-11-28T13:09:52.269188+0000","last_active":"2023-11-28T13:09:53.470895+0000","last_peered":"2023-11-28T13:09:53.470895+0000","last_clean":"2023-11-28T13:09:53.470895+0000","last_became_active":"2023-11-28T13:09:52.268847+0000","last_became_peered":"2023-11-28T13:09:52.268847+0000","last_unstale":"2023-11-28T13:09:53.470895+0000","last_undegraded":"2023-11-28T13:09:53.470895+0000","last_fullsized":"2023-11-28T13:09:53.470895+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:08.028530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.251733+0000","last_change":"2023-11-28T13:09:52.272126+0000","last_active":"2023-11-28T13:09:53.251733+0000","last_peered":"2023-11-28T13:09:53.251733+0000","last_clean":"2023-11-28T13:09:53.251733+0000","last_became_active":"2023-11-28T13:09:52.271804+0000","last_became_peered":"2023-11-28T13:09:52.271804+0000","last_unstale":"2023-11-28T13:09:53.251733+0000","last_undegraded":"2023-11-28T13:09:53.251733+0000","last_fullsized":"2023-11-28T13:09:53.251733+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:35:12.078892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471047+0000","last_change":"2023-11-28T13:09:52.286298+0000","last_active":"2023-11-28T13:09:53.471047+0000","last_peered":"2023-11-28T13:09:53.471047+0000","last_clean":"2023-11-28T13:09:53.471047+0000","last_became_active":"2023-11-28T13:09:52.286019+0000","last_became_peered":"2023-11-28T13:09:52.286019+0000","last_unstale":"2023-11-28T13:09:53.471047+0000","last_undegraded":"2023-11-28T13:09:53.471047+0000","last_fullsized":"2023-11-28T13:09:53.471047+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:27:12.289547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472962+0000","last_change":"2023-11-28T13:09:52.274196+0000","last_active":"2023-11-28T13:09:53.472962+0000","last_peered":"2023-11-28T13:09:53.472962+0000","last_clean":"2023-11-28T13:09:53.472962+0000","last_became_active":"2023-11-28T13:09:52.273953+0000","last_became_peered":"2023-11-28T13:09:52.273953+0000","last_unstale":"2023-11-28T13:09:53.472962+0000","last_undegraded":"2023-11-28T13:09:53.472962+0000","last_fullsized":"2023-11-28T13:09:53.472962+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:06.374523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472475+0000","last_change":"2023-11-28T13:09:52.259203+0000","last_active":"2023-11-28T13:09:53.472475+0000","last_peered":"2023-11-28T13:09:53.472475+0000","last_clean":"2023-11-28T13:09:53.472475+0000","last_became_active":"2023-11-28T13:09:52.258910+0000","last_became_peered":"2023-11-28T13:09:52.258910+0000","last_unstale":"2023-11-28T13:09:53.472475+0000","last_undegraded":"2023-11-28T13:09:53.472475+0000","last_fullsized":"2023-11-28T13:09:53.472475+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:06.374523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471971+0000","last_change":"2023-11-28T13:09:52.273657+0000","last_active":"2023-11-28T13:09:53.471971+0000","last_peered":"2023-11-28T13:09:53.471971+0000","last_clean":"2023-11-28T13:09:53.471971+0000","last_became_active":"2023-11-28T13:09:52.273357+0000","last_became_peered":"2023-11-28T13:09:52.273357+0000","last_unstale":"2023-11-28T13:09:53.471971+0000","last_undegraded":"2023-11-28T13:09:53.471971+0000","last_fullsized":"2023-11-28T13:09:53.471971+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:02.778153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471940+0000","last_change":"2023-11-28T13:09:52.266326+0000","last_active":"2023-11-28T13:09:53.471940+0000","last_peered":"2023-11-28T13:09:53.471940+0000","last_clean":"2023-11-28T13:09:53.471940+0000","last_became_active":"2023-11-28T13:09:52.265933+0000","last_became_peered":"2023-11-28T13:09:52.265933+0000","last_unstale":"2023-11-28T13:09:53.471940+0000","last_undegraded":"2023-11-28T13:09:53.471940+0000","last_fullsized":"2023-11-28T13:09:53.471940+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:59.167162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471159+0000","last_change":"2023-11-28T13:09:52.270557+0000","last_active":"2023-11-28T13:09:53.471159+0000","last_peered":"2023-11-28T13:09:53.471159+0000","last_clean":"2023-11-28T13:09:53.471159+0000","last_became_active":"2023-11-28T13:09:52.270296+0000","last_became_peered":"2023-11-28T13:09:52.270296+0000","last_unstale":"2023-11-28T13:09:53.471159+0000","last_undegraded":"2023-11-28T13:09:53.471159+0000","last_fullsized":"2023-11-28T13:09:53.471159+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:31.359124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253956+0000","last_change":"2023-11-28T13:09:52.267157+0000","last_active":"2023-11-28T13:09:53.253956+0000","last_peered":"2023-11-28T13:09:53.253956+0000","last_clean":"2023-11-28T13:09:53.253956+0000","last_became_active":"2023-11-28T13:09:52.266905+0000","last_became_peered":"2023-11-28T13:09:52.266905+0000","last_unstale":"2023-11-28T13:09:53.253956+0000","last_undegraded":"2023-11-28T13:09:53.253956+0000","last_fullsized":"2023-11-28T13:09:53.253956+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:47.315227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471419+0000","last_change":"2023-11-28T13:09:52.276980+0000","last_active":"2023-11-28T13:09:53.471419+0000","last_peered":"2023-11-28T13:09:53.471419+0000","last_clean":"2023-11-28T13:09:53.471419+0000","last_became_active":"2023-11-28T13:09:52.276682+0000","last_became_peered":"2023-11-28T13:09:52.276682+0000","last_unstale":"2023-11-28T13:09:53.471419+0000","last_undegraded":"2023-11-28T13:09:53.471419+0000","last_fullsized":"2023-11-28T13:09:53.471419+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:24.470324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253564+0000","last_change":"2023-11-28T13:09:52.278591+0000","last_active":"2023-11-28T13:09:53.253564+0000","last_peered":"2023-11-28T13:09:53.253564+0000","last_clean":"2023-11-28T13:09:53.253564+0000","last_became_active":"2023-11-28T13:09:52.278278+0000","last_became_peered":"2023-11-28T13:09:52.278278+0000","last_unstale":"2023-11-28T13:09:53.253564+0000","last_undegraded":"2023-11-28T13:09:53.253564+0000","last_fullsized":"2023-11-28T13:09:53.253564+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:02:26.898065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.252704+0000","last_change":"2023-11-28T13:09:52.270918+0000","last_active":"2023-11-28T13:09:53.252704+0000","last_peered":"2023-11-28T13:09:53.252704+0000","last_clean":"2023-11-28T13:09:53.252704+0000","last_became_active":"2023-11-28T13:09:52.270608+0000","last_became_peered":"2023-11-28T13:09:52.270608+0000","last_unstale":"2023-11-28T13:09:53.252704+0000","last_undegraded":"2023-11-28T13:09:53.252704+0000","last_fullsized":"2023-11-28T13:09:53.252704+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:02.778153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"46'4","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472095+0000","last_change":"2023-11-28T13:09:51.757952+0000","last_active":"2023-11-28T13:09:53.472095+0000","last_peered":"2023-11-28T13:09:53.472095+0000","last_clean":"2023-11-28T13:09:53.472095+0000","last_became_active":"2023-11-28T13:09:51.757399+0000","last_became_peered":"2023-11-28T13:09:51.757399+0000","last_unstale":"2023-11-28T13:09:53.472095+0000","last_undegraded":"2023-11-28T13:09:53.472095+0000","last_fullsized":"2023-11-28T13:09:53.472095+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:52.078719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253288+0000","last_change":"2023-11-28T13:09:52.278144+0000","last_active":"2023-11-28T13:09:53.253288+0000","last_peered":"2023-11-28T13:09:53.253288+0000","last_clean":"2023-11-28T13:09:53.253288+0000","last_became_active":"2023-11-28T13:09:52.277904+0000","last_became_peered":"2023-11-28T13:09:52.277904+0000","last_unstale":"2023-11-28T13:09:53.253288+0000","last_undegraded":"2023-11-28T13:09:53.253288+0000","last_fullsized":"2023-11-28T13:09:53.253288+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:31.359124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472182+0000","last_change":"2023-11-28T13:09:51.772021+0000","last_active":"2023-11-28T13:09:53.472182+0000","last_peered":"2023-11-28T13:09:53.472182+0000","last_clean":"2023-11-28T13:09:53.472182+0000","last_became_active":"2023-11-28T13:09:51.771729+0000","last_became_peered":"2023-11-28T13:09:51.771729+0000","last_unstale":"2023-11-28T13:09:53.472182+0000","last_undegraded":"2023-11-28T13:09:53.472182+0000","last_fullsized":"2023-11-28T13:09:53.472182+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:14:28.072677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471749+0000","last_change":"2023-11-28T13:09:52.262889+0000","last_active":"2023-11-28T13:09:53.471749+0000","last_peered":"2023-11-28T13:09:53.471749+0000","last_clean":"2023-11-28T13:09:53.471749+0000","last_became_active":"2023-11-28T13:09:52.262592+0000","last_became_peered":"2023-11-28T13:09:52.262592+0000","last_unstale":"2023-11-28T13:09:53.471749+0000","last_undegraded":"2023-11-28T13:09:53.471749+0000","last_fullsized":"2023-11-28T13:09:53.471749+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:27:17.350629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472011+0000","last_change":"2023-11-28T13:09:51.766854+0000","last_active":"2023-11-28T13:09:53.472011+0000","last_peered":"2023-11-28T13:09:53.472011+0000","last_clean":"2023-11-28T13:09:53.472011+0000","last_became_active":"2023-11-28T13:09:51.765671+0000","last_became_peered":"2023-11-28T13:09:51.765671+0000","last_unstale":"2023-11-28T13:09:53.472011+0000","last_undegraded":"2023-11-28T13:09:53.472011+0000","last_fullsized":"2023-11-28T13:09:53.472011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:24:52.203192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471566+0000","last_change":"2023-11-28T13:09:52.259260+0000","last_active":"2023-11-28T13:09:53.471566+0000","last_peered":"2023-11-28T13:09:53.471566+0000","last_clean":"2023-11-28T13:09:53.471566+0000","last_became_active":"2023-11-28T13:09:52.258978+0000","last_became_peered":"2023-11-28T13:09:52.258978+0000","last_unstale":"2023-11-28T13:09:53.471566+0000","last_undegraded":"2023-11-28T13:09:53.471566+0000","last_fullsized":"2023-11-28T13:09:53.471566+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:31.670491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253411+0000","last_change":"2023-11-28T13:09:51.773821+0000","last_active":"2023-11-28T13:09:53.253411+0000","last_peered":"2023-11-28T13:09:53.253411+0000","last_clean":"2023-11-28T13:09:53.253411+0000","last_became_active":"2023-11-28T13:09:51.773477+0000","last_became_peered":"2023-11-28T13:09:51.773477+0000","last_unstale":"2023-11-28T13:09:53.253411+0000","last_undegraded":"2023-11-28T13:09:53.253411+0000","last_fullsized":"2023-11-28T13:09:53.253411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:11:55.991394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471395+0000","last_change":"2023-11-28T13:09:52.272811+0000","last_active":"2023-11-28T13:09:53.471395+0000","last_peered":"2023-11-28T13:09:53.471395+0000","last_clean":"2023-11-28T13:09:53.471395+0000","last_became_active":"2023-11-28T13:09:52.271614+0000","last_became_peered":"2023-11-28T13:09:52.271614+0000","last_unstale":"2023-11-28T13:09:53.471395+0000","last_undegraded":"2023-11-28T13:09:53.471395+0000","last_fullsized":"2023-11-28T13:09:53.471395+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:30:01.228085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471582+0000","last_change":"2023-11-28T13:09:51.764543+0000","last_active":"2023-11-28T13:09:53.471582+0000","last_peered":"2023-11-28T13:09:53.471582+0000","last_clean":"2023-11-28T13:09:53.471582+0000","last_became_active":"2023-11-28T13:09:51.763869+0000","last_became_peered":"2023-11-28T13:09:51.763869+0000","last_unstale":"2023-11-28T13:09:53.471582+0000","last_undegraded":"2023-11-28T13:09:53.471582+0000","last_fullsized":"2023-11-28T13:09:53.471582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:17.270157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.254228+0000","last_change":"2023-11-28T13:09:52.280099+0000","last_active":"2023-11-28T13:09:53.254228+0000","last_peered":"2023-11-28T13:09:53.254228+0000","last_clean":"2023-11-28T13:09:53.254228+0000","last_became_active":"2023-11-28T13:09:52.277480+0000","last_became_peered":"2023-11-28T13:09:52.277480+0000","last_unstale":"2023-11-28T13:09:53.254228+0000","last_undegraded":"2023-11-28T13:09:53.254228+0000","last_fullsized":"2023-11-28T13:09:53.254228+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:05.536831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471113+0000","last_change":"2023-11-28T13:09:51.756324+0000","last_active":"2023-11-28T13:09:53.471113+0000","last_peered":"2023-11-28T13:09:53.471113+0000","last_clean":"2023-11-28T13:09:53.471113+0000","last_became_active":"2023-11-28T13:09:51.755970+0000","last_became_peered":"2023-11-28T13:09:51.755970+0000","last_unstale":"2023-11-28T13:09:53.471113+0000","last_undegraded":"2023-11-28T13:09:53.471113+0000","last_fullsized":"2023-11-28T13:09:53.471113+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:04.530905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471978+0000","last_change":"2023-11-28T13:09:52.266103+0000","last_active":"2023-11-28T13:09:53.471978+0000","last_peered":"2023-11-28T13:09:53.471978+0000","last_clean":"2023-11-28T13:09:53.471978+0000","last_became_active":"2023-11-28T13:09:52.265835+0000","last_became_peered":"2023-11-28T13:09:52.265835+0000","last_unstale":"2023-11-28T13:09:53.471978+0000","last_undegraded":"2023-11-28T13:09:53.471978+0000","last_fullsized":"2023-11-28T13:09:53.471978+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:18:42.555378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471587+0000","last_change":"2023-11-28T13:09:51.766581+0000","last_active":"2023-11-28T13:09:53.471587+0000","last_peered":"2023-11-28T13:09:53.471587+0000","last_clean":"2023-11-28T13:09:53.471587+0000","last_became_active":"2023-11-28T13:09:51.766256+0000","last_became_peered":"2023-11-28T13:09:51.766256+0000","last_unstale":"2023-11-28T13:09:53.471587+0000","last_undegraded":"2023-11-28T13:09:53.471587+0000","last_fullsized":"2023-11-28T13:09:53.471587+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:55.487936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.254291+0000","last_change":"2023-11-28T13:09:52.268480+0000","last_active":"2023-11-28T13:09:53.254291+0000","last_peered":"2023-11-28T13:09:53.254291+0000","last_clean":"2023-11-28T13:09:53.254291+0000","last_became_active":"2023-11-28T13:09:52.268101+0000","last_became_peered":"2023-11-28T13:09:52.268101+0000","last_unstale":"2023-11-28T13:09:53.254291+0000","last_undegraded":"2023-11-28T13:09:53.254291+0000","last_fullsized":"2023-11-28T13:09:53.254291+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:07.534480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471213+0000","last_change":"2023-11-28T13:09:51.761072+0000","last_active":"2023-11-28T13:09:53.471213+0000","last_peered":"2023-11-28T13:09:53.471213+0000","last_clean":"2023-11-28T13:09:53.471213+0000","last_became_active":"2023-11-28T13:09:51.760838+0000","last_became_peered":"2023-11-28T13:09:51.760838+0000","last_unstale":"2023-11-28T13:09:53.471213+0000","last_undegraded":"2023-11-28T13:09:53.471213+0000","last_fullsized":"2023-11-28T13:09:53.471213+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:56:23.464508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470963+0000","last_change":"2023-11-28T13:09:52.272851+0000","last_active":"2023-11-28T13:09:53.470963+0000","last_peered":"2023-11-28T13:09:53.470963+0000","last_clean":"2023-11-28T13:09:53.470963+0000","last_became_active":"2023-11-28T13:09:52.272341+0000","last_became_peered":"2023-11-28T13:09:52.272341+0000","last_unstale":"2023-11-28T13:09:53.470963+0000","last_undegraded":"2023-11-28T13:09:53.470963+0000","last_fullsized":"2023-11-28T13:09:53.470963+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:02:26.898065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.254271+0000","last_change":"2023-11-28T13:09:51.770173+0000","last_active":"2023-11-28T13:09:53.254271+0000","last_peered":"2023-11-28T13:09:53.254271+0000","last_clean":"2023-11-28T13:09:53.254271+0000","last_became_active":"2023-11-28T13:09:51.769889+0000","last_became_peered":"2023-11-28T13:09:51.769889+0000","last_unstale":"2023-11-28T13:09:53.254271+0000","last_undegraded":"2023-11-28T13:09:53.254271+0000","last_fullsized":"2023-11-28T13:09:53.254271+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:46:55.192689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253153+0000","last_change":"2023-11-28T13:09:52.269030+0000","last_active":"2023-11-28T13:09:53.253153+0000","last_peered":"2023-11-28T13:09:53.253153+0000","last_clean":"2023-11-28T13:09:53.253153+0000","last_became_active":"2023-11-28T13:09:52.268452+0000","last_became_peered":"2023-11-28T13:09:52.268452+0000","last_unstale":"2023-11-28T13:09:53.253153+0000","last_undegraded":"2023-11-28T13:09:53.253153+0000","last_fullsized":"2023-11-28T13:09:53.253153+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:37:06.556688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471495+0000","last_change":"2023-11-28T13:09:51.769161+0000","last_active":"2023-11-28T13:09:53.471495+0000","last_peered":"2023-11-28T13:09:53.471495+0000","last_clean":"2023-11-28T13:09:53.471495+0000","last_became_active":"2023-11-28T13:09:51.768815+0000","last_became_peered":"2023-11-28T13:09:51.768815+0000","last_unstale":"2023-11-28T13:09:53.471495+0000","last_undegraded":"2023-11-28T13:09:53.471495+0000","last_fullsized":"2023-11-28T13:09:53.471495+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:39.617559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471466+0000","last_change":"2023-11-28T13:09:52.272577+0000","last_active":"2023-11-28T13:09:53.471466+0000","last_peered":"2023-11-28T13:09:53.471466+0000","last_clean":"2023-11-28T13:09:53.471466+0000","last_became_active":"2023-11-28T13:09:52.272250+0000","last_became_peered":"2023-11-28T13:09:52.272250+0000","last_unstale":"2023-11-28T13:09:53.471466+0000","last_undegraded":"2023-11-28T13:09:53.471466+0000","last_fullsized":"2023-11-28T13:09:53.471466+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:56:47.727803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"46'3","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.254242+0000","last_change":"2023-11-28T13:09:51.773763+0000","last_active":"2023-11-28T13:09:53.254242+0000","last_peered":"2023-11-28T13:09:53.254242+0000","last_clean":"2023-11-28T13:09:53.254242+0000","last_became_active":"2023-11-28T13:09:51.773505+0000","last_became_peered":"2023-11-28T13:09:51.773505+0000","last_unstale":"2023-11-28T13:09:53.254242+0000","last_undegraded":"2023-11-28T13:09:53.254242+0000","last_fullsized":"2023-11-28T13:09:53.254242+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:47:34.022920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471188+0000","last_change":"2023-11-28T13:09:52.275851+0000","last_active":"2023-11-28T13:09:53.471188+0000","last_peered":"2023-11-28T13:09:53.471188+0000","last_clean":"2023-11-28T13:09:53.471188+0000","last_became_active":"2023-11-28T13:09:52.275574+0000","last_became_peered":"2023-11-28T13:09:52.275574+0000","last_unstale":"2023-11-28T13:09:53.471188+0000","last_undegraded":"2023-11-28T13:09:53.471188+0000","last_fullsized":"2023-11-28T13:09:53.471188+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:59.756502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"46'6","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472035+0000","last_change":"2023-11-28T13:09:51.769242+0000","last_active":"2023-11-28T13:09:53.472035+0000","last_peered":"2023-11-28T13:09:53.472035+0000","last_clean":"2023-11-28T13:09:53.472035+0000","last_became_active":"2023-11-28T13:09:51.768806+0000","last_became_peered":"2023-11-28T13:09:51.768806+0000","last_unstale":"2023-11-28T13:09:53.472035+0000","last_undegraded":"2023-11-28T13:09:53.472035+0000","last_fullsized":"2023-11-28T13:09:53.472035+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:04.530905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253061+0000","last_change":"2023-11-28T13:09:52.277681+0000","last_active":"2023-11-28T13:09:53.253061+0000","last_peered":"2023-11-28T13:09:53.253061+0000","last_clean":"2023-11-28T13:09:53.253061+0000","last_became_active":"2023-11-28T13:09:52.277362+0000","last_became_peered":"2023-11-28T13:09:52.277362+0000","last_unstale":"2023-11-28T13:09:53.253061+0000","last_undegraded":"2023-11-28T13:09:53.253061+0000","last_fullsized":"2023-11-28T13:09:53.253061+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:07:18.517785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471691+0000","last_change":"2023-11-28T13:09:51.767042+0000","last_active":"2023-11-28T13:09:53.471691+0000","last_peered":"2023-11-28T13:09:53.471691+0000","last_clean":"2023-11-28T13:09:53.471691+0000","last_became_active":"2023-11-28T13:09:51.766763+0000","last_became_peered":"2023-11-28T13:09:51.766763+0000","last_unstale":"2023-11-28T13:09:53.471691+0000","last_undegraded":"2023-11-28T13:09:53.471691+0000","last_fullsized":"2023-11-28T13:09:53.471691+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:10:46.983683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472498+0000","last_change":"2023-11-28T13:09:52.274829+0000","last_active":"2023-11-28T13:09:53.472498+0000","last_peered":"2023-11-28T13:09:53.472498+0000","last_clean":"2023-11-28T13:09:53.472498+0000","last_became_active":"2023-11-28T13:09:52.274475+0000","last_became_peered":"2023-11-28T13:09:52.274475+0000","last_unstale":"2023-11-28T13:09:53.472498+0000","last_undegraded":"2023-11-28T13:09:53.472498+0000","last_fullsized":"2023-11-28T13:09:53.472498+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:45:45.645058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471721+0000","last_change":"2023-11-28T13:09:51.767895+0000","last_active":"2023-11-28T13:09:53.471721+0000","last_peered":"2023-11-28T13:09:53.471721+0000","last_clean":"2023-11-28T13:09:53.471721+0000","last_became_active":"2023-11-28T13:09:51.767590+0000","last_became_peered":"2023-11-28T13:09:51.767590+0000","last_unstale":"2023-11-28T13:09:53.471721+0000","last_undegraded":"2023-11-28T13:09:53.471721+0000","last_fullsized":"2023-11-28T13:09:53.471721+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:39.586756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471886+0000","last_change":"2023-11-28T13:09:52.265869+0000","last_active":"2023-11-28T13:09:53.471886+0000","last_peered":"2023-11-28T13:09:53.471886+0000","last_clean":"2023-11-28T13:09:53.471886+0000","last_became_active":"2023-11-28T13:09:52.265623+0000","last_became_peered":"2023-11-28T13:09:52.265623+0000","last_unstale":"2023-11-28T13:09:53.471886+0000","last_undegraded":"2023-11-28T13:09:53.471886+0000","last_fullsized":"2023-11-28T13:09:53.471886+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:44:38.788752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253900+0000","last_change":"2023-11-28T13:09:51.766137+0000","last_active":"2023-11-28T13:09:53.253900+0000","last_peered":"2023-11-28T13:09:53.253900+0000","last_clean":"2023-11-28T13:09:53.253900+0000","last_became_active":"2023-11-28T13:09:51.765780+0000","last_became_peered":"2023-11-28T13:09:51.765780+0000","last_unstale":"2023-11-28T13:09:53.253900+0000","last_undegraded":"2023-11-28T13:09:53.253900+0000","last_fullsized":"2023-11-28T13:09:53.253900+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:40.242354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472870+0000","last_change":"2023-11-28T13:09:52.279558+0000","last_active":"2023-11-28T13:09:53.472870+0000","last_peered":"2023-11-28T13:09:53.472870+0000","last_clean":"2023-11-28T13:09:53.472870+0000","last_became_active":"2023-11-28T13:09:52.278752+0000","last_became_peered":"2023-11-28T13:09:52.278752+0000","last_unstale":"2023-11-28T13:09:53.472870+0000","last_undegraded":"2023-11-28T13:09:53.472870+0000","last_fullsized":"2023-11-28T13:09:53.472870+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:43.979876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"46'8","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470888+0000","last_change":"2023-11-28T13:09:51.766711+0000","last_active":"2023-11-28T13:09:53.470888+0000","last_peered":"2023-11-28T13:09:53.470888+0000","last_clean":"2023-11-28T13:09:53.470888+0000","last_became_active":"2023-11-28T13:09:51.766155+0000","last_became_peered":"2023-11-28T13:09:51.766155+0000","last_unstale":"2023-11-28T13:09:53.470888+0000","last_undegraded":"2023-11-28T13:09:53.470888+0000","last_fullsized":"2023-11-28T13:09:53.470888+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:01.772227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470794+0000","last_change":"2023-11-28T13:09:52.270571+0000","last_active":"2023-11-28T13:09:53.470794+0000","last_peered":"2023-11-28T13:09:53.470794+0000","last_clean":"2023-11-28T13:09:53.470794+0000","last_became_active":"2023-11-28T13:09:52.270286+0000","last_became_peered":"2023-11-28T13:09:52.270286+0000","last_unstale":"2023-11-28T13:09:53.470794+0000","last_undegraded":"2023-11-28T13:09:53.470794+0000","last_fullsized":"2023-11-28T13:09:53.470794+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:02:26.898065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472848+0000","last_change":"2023-11-28T13:09:51.771962+0000","last_active":"2023-11-28T13:09:53.472848+0000","last_peered":"2023-11-28T13:09:53.472848+0000","last_clean":"2023-11-28T13:09:53.472848+0000","last_became_active":"2023-11-28T13:09:51.771654+0000","last_became_peered":"2023-11-28T13:09:51.771654+0000","last_unstale":"2023-11-28T13:09:53.472848+0000","last_undegraded":"2023-11-28T13:09:53.472848+0000","last_fullsized":"2023-11-28T13:09:53.472848+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:55:01.616495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470780+0000","last_change":"2023-11-28T13:09:52.273705+0000","last_active":"2023-11-28T13:09:53.470780+0000","last_peered":"2023-11-28T13:09:53.470780+0000","last_clean":"2023-11-28T13:09:53.470780+0000","last_became_active":"2023-11-28T13:09:52.273380+0000","last_became_peered":"2023-11-28T13:09:52.273380+0000","last_unstale":"2023-11-28T13:09:53.470780+0000","last_undegraded":"2023-11-28T13:09:53.470780+0000","last_fullsized":"2023-11-28T13:09:53.470780+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:11:56.997320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.252097+0000","last_change":"2023-11-28T13:09:51.760087+0000","last_active":"2023-11-28T13:09:53.252097+0000","last_peered":"2023-11-28T13:09:53.252097+0000","last_clean":"2023-11-28T13:09:53.252097+0000","last_became_active":"2023-11-28T13:09:51.759804+0000","last_became_peered":"2023-11-28T13:09:51.759804+0000","last_unstale":"2023-11-28T13:09:53.252097+0000","last_undegraded":"2023-11-28T13:09:53.252097+0000","last_fullsized":"2023-11-28T13:09:53.252097+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:52.078719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470998+0000","last_change":"2023-11-28T13:09:52.273571+0000","last_active":"2023-11-28T13:09:53.470998+0000","last_peered":"2023-11-28T13:09:53.470998+0000","last_clean":"2023-11-28T13:09:53.470998+0000","last_became_active":"2023-11-28T13:09:52.273294+0000","last_became_peered":"2023-11-28T13:09:52.273294+0000","last_unstale":"2023-11-28T13:09:53.470998+0000","last_undegraded":"2023-11-28T13:09:53.470998+0000","last_fullsized":"2023-11-28T13:09:53.470998+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:48.180966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472021+0000","last_change":"2023-11-28T13:09:51.761533+0000","last_active":"2023-11-28T13:09:53.472021+0000","last_peered":"2023-11-28T13:09:53.472021+0000","last_clean":"2023-11-28T13:09:53.472021+0000","last_became_active":"2023-11-28T13:09:51.761238+0000","last_became_peered":"2023-11-28T13:09:51.761238+0000","last_unstale":"2023-11-28T13:09:53.472021+0000","last_undegraded":"2023-11-28T13:09:53.472021+0000","last_fullsized":"2023-11-28T13:09:53.472021+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:12:18.204614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471357+0000","last_change":"2023-11-28T13:09:52.276438+0000","last_active":"2023-11-28T13:09:53.471357+0000","last_peered":"2023-11-28T13:09:53.471357+0000","last_clean":"2023-11-28T13:09:53.471357+0000","last_became_active":"2023-11-28T13:09:52.275736+0000","last_became_peered":"2023-11-28T13:09:52.275736+0000","last_unstale":"2023-11-28T13:09:53.471357+0000","last_undegraded":"2023-11-28T13:09:53.471357+0000","last_fullsized":"2023-11-28T13:09:53.471357+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:05.536831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471877+0000","last_change":"2023-11-28T13:09:51.773430+0000","last_active":"2023-11-28T13:09:53.471877+0000","last_peered":"2023-11-28T13:09:53.471877+0000","last_clean":"2023-11-28T13:09:53.471877+0000","last_became_active":"2023-11-28T13:09:51.773189+0000","last_became_peered":"2023-11-28T13:09:51.773189+0000","last_unstale":"2023-11-28T13:09:53.471877+0000","last_undegraded":"2023-11-28T13:09:53.471877+0000","last_fullsized":"2023-11-28T13:09:53.471877+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:59:47.972951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253519+0000","last_change":"2023-11-28T13:09:52.275372+0000","last_active":"2023-11-28T13:09:53.253519+0000","last_peered":"2023-11-28T13:09:53.253519+0000","last_clean":"2023-11-28T13:09:53.253519+0000","last_became_active":"2023-11-28T13:09:52.275092+0000","last_became_peered":"2023-11-28T13:09:52.275092+0000","last_unstale":"2023-11-28T13:09:53.253519+0000","last_undegraded":"2023-11-28T13:09:53.253519+0000","last_fullsized":"2023-11-28T13:09:53.253519+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:48.180966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471385+0000","last_change":"2023-11-28T13:09:51.765086+0000","last_active":"2023-11-28T13:09:53.471385+0000","last_peered":"2023-11-28T13:09:53.471385+0000","last_clean":"2023-11-28T13:09:53.471385+0000","last_became_active":"2023-11-28T13:09:51.764788+0000","last_became_peered":"2023-11-28T13:09:51.764788+0000","last_unstale":"2023-11-28T13:09:53.471385+0000","last_undegraded":"2023-11-28T13:09:53.471385+0000","last_fullsized":"2023-11-28T13:09:53.471385+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:09:30.031005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472009+0000","last_change":"2023-11-28T13:09:52.276116+0000","last_active":"2023-11-28T13:09:53.472009+0000","last_peered":"2023-11-28T13:09:53.472009+0000","last_clean":"2023-11-28T13:09:53.472009+0000","last_became_active":"2023-11-28T13:09:52.275232+0000","last_became_peered":"2023-11-28T13:09:52.275232+0000","last_unstale":"2023-11-28T13:09:53.472009+0000","last_undegraded":"2023-11-28T13:09:53.472009+0000","last_fullsized":"2023-11-28T13:09:53.472009+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:47:35.028846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470974+0000","last_change":"2023-11-28T13:09:51.771687+0000","last_active":"2023-11-28T13:09:53.470974+0000","last_peered":"2023-11-28T13:09:53.470974+0000","last_clean":"2023-11-28T13:09:53.470974+0000","last_became_active":"2023-11-28T13:09:51.771432+0000","last_became_peered":"2023-11-28T13:09:51.771432+0000","last_unstale":"2023-11-28T13:09:53.470974+0000","last_undegraded":"2023-11-28T13:09:53.470974+0000","last_fullsized":"2023-11-28T13:09:53.470974+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:11:55.991394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472695+0000","last_change":"2023-11-28T13:09:52.270971+0000","last_active":"2023-11-28T13:09:53.472695+0000","last_peered":"2023-11-28T13:09:53.472695+0000","last_clean":"2023-11-28T13:09:53.472695+0000","last_became_active":"2023-11-28T13:09:52.270718+0000","last_became_peered":"2023-11-28T13:09:52.270718+0000","last_unstale":"2023-11-28T13:09:53.472695+0000","last_undegraded":"2023-11-28T13:09:53.472695+0000","last_fullsized":"2023-11-28T13:09:53.472695+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:27:10.518404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471273+0000","last_change":"2023-11-28T13:09:51.762882+0000","last_active":"2023-11-28T13:09:53.471273+0000","last_peered":"2023-11-28T13:09:53.471273+0000","last_clean":"2023-11-28T13:09:53.471273+0000","last_became_active":"2023-11-28T13:09:51.761003+0000","last_became_peered":"2023-11-28T13:09:51.761003+0000","last_unstale":"2023-11-28T13:09:53.471273+0000","last_undegraded":"2023-11-28T13:09:53.471273+0000","last_fullsized":"2023-11-28T13:09:53.471273+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:20:52.814374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472671+0000","last_change":"2023-11-28T13:09:52.271718+0000","last_active":"2023-11-28T13:09:53.472671+0000","last_peered":"2023-11-28T13:09:53.472671+0000","last_clean":"2023-11-28T13:09:53.472671+0000","last_became_active":"2023-11-28T13:09:52.271448+0000","last_became_peered":"2023-11-28T13:09:52.271448+0000","last_unstale":"2023-11-28T13:09:53.472671+0000","last_undegraded":"2023-11-28T13:09:53.472671+0000","last_fullsized":"2023-11-28T13:09:53.472671+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:40.623485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.251689+0000","last_change":"2023-11-28T13:09:51.765728+0000","last_active":"2023-11-28T13:09:53.251689+0000","last_peered":"2023-11-28T13:09:53.251689+0000","last_clean":"2023-11-28T13:09:53.251689+0000","last_became_active":"2023-11-28T13:09:51.765433+0000","last_became_peered":"2023-11-28T13:09:51.765433+0000","last_unstale":"2023-11-28T13:09:53.251689+0000","last_undegraded":"2023-11-28T13:09:53.251689+0000","last_fullsized":"2023-11-28T13:09:53.251689+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:27:55.476924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472384+0000","last_change":"2023-11-28T13:09:52.268049+0000","last_active":"2023-11-28T13:09:53.472384+0000","last_peered":"2023-11-28T13:09:53.472384+0000","last_clean":"2023-11-28T13:09:53.472384+0000","last_became_active":"2023-11-28T13:09:52.267818+0000","last_became_peered":"2023-11-28T13:09:52.267818+0000","last_unstale":"2023-11-28T13:09:53.472384+0000","last_undegraded":"2023-11-28T13:09:53.472384+0000","last_fullsized":"2023-11-28T13:09:53.472384+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:31.670491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471785+0000","last_change":"2023-11-28T13:09:51.769416+0000","last_active":"2023-11-28T13:09:53.471785+0000","last_peered":"2023-11-28T13:09:53.471785+0000","last_clean":"2023-11-28T13:09:53.471785+0000","last_became_active":"2023-11-28T13:09:51.768508+0000","last_became_peered":"2023-11-28T13:09:51.768508+0000","last_unstale":"2023-11-28T13:09:53.471785+0000","last_undegraded":"2023-11-28T13:09:53.471785+0000","last_fullsized":"2023-11-28T13:09:53.471785+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:39.617559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":39,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.267364+0000","last_change":"2023-11-28T13:07:56.890378+0000","last_active":"2023-11-28T13:09:52.267364+0000","last_peered":"2023-11-28T13:09:52.267364+0000","last_clean":"2023-11-28T13:09:52.267364+0000","last_became_active":"2023-11-28T13:07:56.890058+0000","last_became_peered":"2023-11-28T13:07:56.890058+0000","last_unstale":"2023-11-28T13:09:52.267364+0000","last_undegraded":"2023-11-28T13:09:52.267364+0000","last_fullsized":"2023-11-28T13:09:52.267364+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:05.344602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.252933+0000","last_change":"2023-11-28T13:09:52.277672+0000","last_active":"2023-11-28T13:09:53.252933+0000","last_peered":"2023-11-28T13:09:53.252933+0000","last_clean":"2023-11-28T13:09:53.252933+0000","last_became_active":"2023-11-28T13:09:52.277374+0000","last_became_peered":"2023-11-28T13:09:52.277374+0000","last_unstale":"2023-11-28T13:09:53.252933+0000","last_undegraded":"2023-11-28T13:09:53.252933+0000","last_fullsized":"2023-11-28T13:09:53.252933+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:02.353727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471555+0000","last_change":"2023-11-28T13:09:51.771155+0000","last_active":"2023-11-28T13:09:53.471555+0000","last_peered":"2023-11-28T13:09:53.471555+0000","last_clean":"2023-11-28T13:09:53.471555+0000","last_became_active":"2023-11-28T13:09:51.770148+0000","last_became_peered":"2023-11-28T13:09:51.770148+0000","last_unstale":"2023-11-28T13:09:53.471555+0000","last_undegraded":"2023-11-28T13:09:53.471555+0000","last_fullsized":"2023-11-28T13:09:53.471555+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:46.309301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.250199+0000","last_change":"2023-11-28T13:07:56.844509+0000","last_active":"2023-11-28T13:09:52.250199+0000","last_peered":"2023-11-28T13:09:52.250199+0000","last_clean":"2023-11-28T13:09:52.250199+0000","last_became_active":"2023-11-28T13:07:56.844081+0000","last_became_peered":"2023-11-28T13:07:56.844081+0000","last_unstale":"2023-11-28T13:09:52.250199+0000","last_undegraded":"2023-11-28T13:09:52.250199+0000","last_fullsized":"2023-11-28T13:09:52.250199+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471239+0000","last_change":"2023-11-28T13:09:52.277425+0000","last_active":"2023-11-28T13:09:53.471239+0000","last_peered":"2023-11-28T13:09:53.471239+0000","last_clean":"2023-11-28T13:09:53.471239+0000","last_became_active":"2023-11-28T13:09:52.277195+0000","last_became_peered":"2023-11-28T13:09:52.277195+0000","last_unstale":"2023-11-28T13:09:53.471239+0000","last_undegraded":"2023-11-28T13:09:53.471239+0000","last_fullsized":"2023-11-28T13:09:53.471239+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:55:02.622421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.254322+0000","last_change":"2023-11-28T13:09:51.768271+0000","last_active":"2023-11-28T13:09:53.254322+0000","last_peered":"2023-11-28T13:09:53.254322+0000","last_clean":"2023-11-28T13:09:53.254322+0000","last_became_active":"2023-11-28T13:09:51.768022+0000","last_became_peered":"2023-11-28T13:09:51.768022+0000","last_unstale":"2023-11-28T13:09:53.254322+0000","last_undegraded":"2023-11-28T13:09:53.254322+0000","last_fullsized":"2023-11-28T13:09:53.254322+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:47:08.021309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":36,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.257680+0000","last_change":"2023-11-28T13:07:56.844699+0000","last_active":"2023-11-28T13:09:52.257680+0000","last_peered":"2023-11-28T13:09:52.257680+0000","last_clean":"2023-11-28T13:09:52.257680+0000","last_became_active":"2023-11-28T13:07:56.844391+0000","last_became_peered":"2023-11-28T13:07:56.844391+0000","last_unstale":"2023-11-28T13:09:52.257680+0000","last_undegraded":"2023-11-28T13:09:52.257680+0000","last_fullsized":"2023-11-28T13:09:52.257680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":186,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.254631+0000","last_change":"2023-11-28T13:07:57.844020+0000","last_active":"2023-11-28T13:09:52.254631+0000","last_peered":"2023-11-28T13:09:52.254631+0000","last_clean":"2023-11-28T13:09:52.254631+0000","last_became_active":"2023-11-28T13:07:57.842981+0000","last_became_peered":"2023-11-28T13:07:57.842981+0000","last_unstale":"2023-11-28T13:09:52.254631+0000","last_undegraded":"2023-11-28T13:09:52.254631+0000","last_fullsized":"2023-11-28T13:09:52.254631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_clean_scrub_stamp":"2023-11-28T13:07:56.825711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:44.973218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471683+0000","last_change":"2023-11-28T13:09:52.263490+0000","last_active":"2023-11-28T13:09:53.471683+0000","last_peered":"2023-11-28T13:09:53.471683+0000","last_clean":"2023-11-28T13:09:53.471683+0000","last_became_active":"2023-11-28T13:09:52.262892+0000","last_became_peered":"2023-11-28T13:09:52.262892+0000","last_unstale":"2023-11-28T13:09:53.471683+0000","last_undegraded":"2023-11-28T13:09:53.471683+0000","last_fullsized":"2023-11-28T13:09:53.471683+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:56:47.727803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471827+0000","last_change":"2023-11-28T13:09:51.771254+0000","last_active":"2023-11-28T13:09:53.471827+0000","last_peered":"2023-11-28T13:09:53.471827+0000","last_clean":"2023-11-28T13:09:53.471827+0000","last_became_active":"2023-11-28T13:09:51.768113+0000","last_became_peered":"2023-11-28T13:09:51.768113+0000","last_unstale":"2023-11-28T13:09:53.471827+0000","last_undegraded":"2023-11-28T13:09:53.471827+0000","last_fullsized":"2023-11-28T13:09:53.471827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:38.360081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":37,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.254750+0000","last_change":"2023-11-28T13:07:57.306318+0000","last_active":"2023-11-28T13:09:52.254750+0000","last_peered":"2023-11-28T13:09:52.254750+0000","last_clean":"2023-11-28T13:09:52.254750+0000","last_became_active":"2023-11-28T13:07:57.305937+0000","last_became_peered":"2023-11-28T13:07:57.305937+0000","last_unstale":"2023-11-28T13:09:52.254750+0000","last_undegraded":"2023-11-28T13:09:52.254750+0000","last_fullsized":"2023-11-28T13:09:52.254750+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471852+0000","last_change":"2023-11-28T13:09:52.270817+0000","last_active":"2023-11-28T13:09:53.471852+0000","last_peered":"2023-11-28T13:09:53.471852+0000","last_clean":"2023-11-28T13:09:53.471852+0000","last_became_active":"2023-11-28T13:09:52.270002+0000","last_became_peered":"2023-11-28T13:09:52.270002+0000","last_unstale":"2023-11-28T13:09:53.471852+0000","last_undegraded":"2023-11-28T13:09:53.471852+0000","last_fullsized":"2023-11-28T13:09:53.471852+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:20:53.820300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472684+0000","last_change":"2023-11-28T13:09:51.772898+0000","last_active":"2023-11-28T13:09:53.472684+0000","last_peered":"2023-11-28T13:09:53.472684+0000","last_clean":"2023-11-28T13:09:53.472684+0000","last_became_active":"2023-11-28T13:09:51.772613+0000","last_became_peered":"2023-11-28T13:09:51.772613+0000","last_unstale":"2023-11-28T13:09:53.472684+0000","last_undegraded":"2023-11-28T13:09:53.472684+0000","last_fullsized":"2023-11-28T13:09:53.472684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:58:43.613349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":37,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.254529+0000","last_change":"2023-11-28T13:07:57.306277+0000","last_active":"2023-11-28T13:09:52.254529+0000","last_peered":"2023-11-28T13:09:52.254529+0000","last_clean":"2023-11-28T13:09:52.254529+0000","last_became_active":"2023-11-28T13:07:57.305983+0000","last_became_peered":"2023-11-28T13:07:57.305983+0000","last_unstale":"2023-11-28T13:09:52.254529+0000","last_undegraded":"2023-11-28T13:09:52.254529+0000","last_fullsized":"2023-11-28T13:09:52.254529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253055+0000","last_change":"2023-11-28T13:09:52.272965+0000","last_active":"2023-11-28T13:09:53.253055+0000","last_peered":"2023-11-28T13:09:53.253055+0000","last_clean":"2023-11-28T13:09:53.253055+0000","last_became_active":"2023-11-28T13:09:52.272665+0000","last_became_peered":"2023-11-28T13:09:52.272665+0000","last_unstale":"2023-11-28T13:09:53.253055+0000","last_undegraded":"2023-11-28T13:09:53.253055+0000","last_fullsized":"2023-11-28T13:09:53.253055+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:30:01.228085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472096+0000","last_change":"2023-11-28T13:09:51.772702+0000","last_active":"2023-11-28T13:09:53.472096+0000","last_peered":"2023-11-28T13:09:53.472096+0000","last_clean":"2023-11-28T13:09:53.472096+0000","last_became_active":"2023-11-28T13:09:51.772471+0000","last_became_peered":"2023-11-28T13:09:51.772471+0000","last_unstale":"2023-11-28T13:09:53.472096+0000","last_undegraded":"2023-11-28T13:09:53.472096+0000","last_fullsized":"2023-11-28T13:09:53.472096+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:27:55.476924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.270529+0000","last_change":"2023-11-28T13:07:57.306177+0000","last_active":"2023-11-28T13:09:52.270529+0000","last_peered":"2023-11-28T13:09:52.270529+0000","last_clean":"2023-11-28T13:09:52.270529+0000","last_became_active":"2023-11-28T13:07:57.305853+0000","last_became_peered":"2023-11-28T13:07:57.305853+0000","last_unstale":"2023-11-28T13:09:52.270529+0000","last_undegraded":"2023-11-28T13:09:52.270529+0000","last_fullsized":"2023-11-28T13:09:52.270529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472792+0000","last_change":"2023-11-28T13:09:52.264312+0000","last_active":"2023-11-28T13:09:53.472792+0000","last_peered":"2023-11-28T13:09:53.472792+0000","last_clean":"2023-11-28T13:09:53.472792+0000","last_became_active":"2023-11-28T13:09:52.264048+0000","last_became_peered":"2023-11-28T13:09:52.264048+0000","last_unstale":"2023-11-28T13:09:53.472792+0000","last_undegraded":"2023-11-28T13:09:53.472792+0000","last_fullsized":"2023-11-28T13:09:53.472792+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:43.979876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253173+0000","last_change":"2023-11-28T13:09:51.762790+0000","last_active":"2023-11-28T13:09:53.253173+0000","last_peered":"2023-11-28T13:09:53.253173+0000","last_clean":"2023-11-28T13:09:53.253173+0000","last_became_active":"2023-11-28T13:09:51.762518+0000","last_became_peered":"2023-11-28T13:09:51.762518+0000","last_unstale":"2023-11-28T13:09:53.253173+0000","last_undegraded":"2023-11-28T13:09:53.253173+0000","last_fullsized":"2023-11-28T13:09:53.253173+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:39.617559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.255543+0000","last_change":"2023-11-28T13:07:57.306321+0000","last_active":"2023-11-28T13:09:52.255543+0000","last_peered":"2023-11-28T13:09:52.255543+0000","last_clean":"2023-11-28T13:09:52.255543+0000","last_became_active":"2023-11-28T13:07:57.305923+0000","last_became_peered":"2023-11-28T13:07:57.305923+0000","last_unstale":"2023-11-28T13:09:52.255543+0000","last_undegraded":"2023-11-28T13:09:52.255543+0000","last_fullsized":"2023-11-28T13:09:52.255543+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:41.104679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472596+0000","last_change":"2023-11-28T13:09:52.282788+0000","last_active":"2023-11-28T13:09:53.472596+0000","last_peered":"2023-11-28T13:09:53.472596+0000","last_clean":"2023-11-28T13:09:53.472596+0000","last_became_active":"2023-11-28T13:09:52.282515+0000","last_became_peered":"2023-11-28T13:09:52.282515+0000","last_unstale":"2023-11-28T13:09:53.472596+0000","last_undegraded":"2023-11-28T13:09:53.472596+0000","last_fullsized":"2023-11-28T13:09:53.472596+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:54.003015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471033+0000","last_change":"2023-11-28T13:09:51.770776+0000","last_active":"2023-11-28T13:09:53.471033+0000","last_peered":"2023-11-28T13:09:53.471033+0000","last_clean":"2023-11-28T13:09:53.471033+0000","last_became_active":"2023-11-28T13:09:51.770461+0000","last_became_peered":"2023-11-28T13:09:51.770461+0000","last_unstale":"2023-11-28T13:09:53.471033+0000","last_undegraded":"2023-11-28T13:09:53.471033+0000","last_fullsized":"2023-11-28T13:09:53.471033+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:46.309301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":38,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.256116+0000","last_change":"2023-11-28T13:07:56.844305+0000","last_active":"2023-11-28T13:09:52.256116+0000","last_peered":"2023-11-28T13:09:52.256116+0000","last_clean":"2023-11-28T13:09:52.256116+0000","last_became_active":"2023-11-28T13:07:56.843980+0000","last_became_peered":"2023-11-28T13:07:56.843980+0000","last_unstale":"2023-11-28T13:09:52.256116+0000","last_undegraded":"2023-11-28T13:09:52.256116+0000","last_fullsized":"2023-11-28T13:09:52.256116+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10160,"kb_used_data":1128,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361680,"statfs":{"total":96636764160,"available":96626360320,"internally_reserved":0,"allocated":1155072,"data_stored":828484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.78800000000000003}]},{"osd":1,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.84899999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.86899999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.91700000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.60199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.67300000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.70299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8812,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363028,"statfs":{"total":96636764160,"available":96627740672,"internally_reserved":0,"allocated":561152,"data_stored":238628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.64100000000000001}]},{"osd":1,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.57699999999999996}]},{"osd":2,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58799999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.60899999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.63},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.74099999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.67700000000000005}]},{"osd":7,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.71599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14128,"kb_used_data":1128,"kb_used_omap":0,"kb_used_meta":12992,"kb_avail":94357712,"statfs":{"total":96636764160,"available":96622297088,"internally_reserved":0,"allocated":1155072,"data_stored":828520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13303808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.45500000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.434}]},{"osd":4,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.96299999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.93000000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.0860000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.995}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13096,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":12544,"kb_avail":94358744,"statfs":{"total":96636764160,"available":96623353856,"internally_reserved":0,"allocated":557056,"data_stored":237062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12845056},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.5},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.63400000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.68000000000000005}]},{"osd":3,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.70599999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.91800000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.85199999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.94199999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12976,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":12416,"kb_avail":94358864,"statfs":{"total":96636764160,"available":96623476736,"internally_reserved":0,"allocated":565248,"data_stored":239540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12713984},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.49099999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.438}]},{"osd":3,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.53100000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.76200000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.91400000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.77000000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.83899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13236,"kb_used_data":556,"kb_used_omap":0,"kb_used_meta":12672,"kb_avail":94358604,"statfs":{"total":96636764160,"available":96623210496,"internally_reserved":0,"allocated":569344,"data_stored":239562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12976128},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.48799999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.54600000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.58499999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.78200000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.91400000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.80000000000000004}]},{"osd":7,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.75900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9064,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362776,"statfs":{"total":96636764160,"available":96627482624,"internally_reserved":0,"allocated":557056,"data_stored":238558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.81000000000000005}]},{"osd":1,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.54300000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.75800000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.71599999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.78400000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.83999999999999997}]},{"osd":7,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.85799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8996,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362844,"statfs":{"total":96636764160,"available":96627552256,"internally_reserved":0,"allocated":552960,"data_stored":237992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.627},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48599999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.52600000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.69599999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.64700000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.71799999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.505}]},{"osd":7,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:09:56.626 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:09:56.729 INFO:tasks.ceph.mds.c.smithi156.stderr:2023-11-28T13:09:56.727+0000 7fa1217f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:09:56.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.854+0000 7f82421ff700 1 Processor -- start 2023-11-28T13:09:56.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.854+0000 7f82421ff700 1 -- start start 2023-11-28T13:09:56.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.855+0000 7f82421ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f823c161340 0x7f823c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:56.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.855+0000 7f82421ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f823c0d2320 0x7f823c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:56.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.856+0000 7f82421ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823c0d1950 con 0x7f823c15c9b0 2023-11-28T13:09:56.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.856+0000 7f82421ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823c0b5800 con 0x7f823c161340 2023-11-28T13:09:56.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.856+0000 7f82421ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823c0b5300 con 0x7f823c0d2320 2023-11-28T13:09:56.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.856+0000 7f8233fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f823c161340 0x7f823c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:56.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.856+0000 7f8233fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f823c161340 0x7f823c0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46492/0 (socket says 172.21.15.156:46492) 2023-11-28T13:09:56.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.856+0000 7f8233fff700 1 -- 172.21.15.156:0/2640541634 learned_addr learned my addr 172.21.15.156:0/2640541634 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:56.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.856+0000 7f823bfff700 1 --2- 172.21.15.156:0/2640541634 >> v2:172.21.15.186:3300/0 conn(0x7f823c0d2320 0x7f823c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:56.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.856+0000 7f8233fff700 1 -- 172.21.15.156:0/2640541634 >> v2:172.21.15.186:3300/0 conn(0x7f823c0d2320 msgr2=0x7f823c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:56.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.856+0000 7f8233fff700 1 --2- 172.21.15.156:0/2640541634 >> v2:172.21.15.186:3300/0 conn(0x7f823c0d2320 0x7f823c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.856+0000 7f8233fff700 1 -- 172.21.15.156:0/2640541634 >> v1:172.21.15.186:6789/0 conn(0x7f823c15c9b0 legacy=0x7f823c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:56.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.856+0000 7f8233fff700 1 -- 172.21.15.156:0/2640541634 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c0b5520 con 0x7f823c161340 2023-11-28T13:09:56.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.857+0000 7f8233fff700 1 --2- 172.21.15.156:0/2640541634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f823c161340 0x7f823c0d1de0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f823c06c170 tx=0x7f8224009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b8f2c3a9b18834 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:56.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.857+0000 7f823affd700 1 -- 172.21.15.156:0/2640541634 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224011830 con 0x7f823c161340 2023-11-28T13:09:56.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.857+0000 7f823affd700 1 -- 172.21.15.156:0/2640541634 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8224011990 con 0x7f823c161340 2023-11-28T13:09:56.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.857+0000 7f823affd700 1 -- 172.21.15.156:0/2640541634 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224011d20 con 0x7f823c161340 2023-11-28T13:09:56.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.857+0000 7f82421ff700 1 -- 172.21.15.156:0/2640541634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f823c161340 msgr2=0x7f823c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:56.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.857+0000 7f82421ff700 1 --2- 172.21.15.156:0/2640541634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f823c161340 0x7f823c0d1de0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f823c06c170 tx=0x7f8224009c00 comp rx=0 tx=0).stop 2023-11-28T13:09:56.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.857+0000 7f82421ff700 1 -- 172.21.15.156:0/2640541634 shutdown_connections 2023-11-28T13:09:56.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.857+0000 7f82421ff700 1 --2- 172.21.15.156:0/2640541634 >> v2:172.21.15.186:3300/0 conn(0x7f823c0d2320 0x7f823c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.857+0000 7f82421ff700 1 --2- 172.21.15.156:0/2640541634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f823c161340 0x7f823c0d1de0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.857+0000 7f82421ff700 1 -- 172.21.15.156:0/2640541634 >> 172.21.15.156:0/2640541634 conn(0x7f823c072600 msgr2=0x7f823c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:56.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.858+0000 7f82421ff700 1 -- 172.21.15.156:0/2640541634 shutdown_connections 2023-11-28T13:09:56.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.858+0000 7f82421ff700 1 -- 172.21.15.156:0/2640541634 wait complete. 2023-11-28T13:09:56.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.860+0000 7f82421ff700 1 Processor -- start 2023-11-28T13:09:56.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.860+0000 7f82421ff700 1 -- start start 2023-11-28T13:09:56.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.861+0000 7f82421ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f823c0d2320 0x7f823c150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:56.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.861+0000 7f82421ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f823c15c9b0 0x7f823c1e7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:56.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.861+0000 7f82421ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f823c161340 0x7f823c156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:56.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.861+0000 7f82421ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c1547e0 con 0x7f823c0d2320 2023-11-28T13:09:56.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.861+0000 7f82421ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c1e6460 con 0x7f823c15c9b0 2023-11-28T13:09:56.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.861+0000 7f82421ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823c1e72c0 con 0x7f823c161340 2023-11-28T13:09:56.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.862+0000 7f823b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f823c0d2320 0x7f823c150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:56.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.862+0000 7f823b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f823c0d2320 0x7f823c150cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46508/0 (socket says 172.21.15.156:46508) 2023-11-28T13:09:56.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.862+0000 7f823b7fe700 1 -- 172.21.15.156:0/1180031515 learned_addr learned my addr 172.21.15.156:0/1180031515 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:56.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.862+0000 7f8233fff700 1 --2- 172.21.15.156:0/1180031515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f823c15c9b0 0x7f823c1e7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:56.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.862+0000 7f823bfff700 1 --2- 172.21.15.156:0/1180031515 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f823c161340 0x7f823c156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:56.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.862+0000 7f823b7fe700 1 -- 172.21.15.156:0/1180031515 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f823c161340 msgr2=0x7f823c156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:56.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.862+0000 7f823b7fe700 1 --2- 172.21.15.156:0/1180031515 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f823c161340 0x7f823c156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.862+0000 7f823b7fe700 1 -- 172.21.15.156:0/1180031515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f823c15c9b0 msgr2=0x7f823c1e7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:56.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.862+0000 7f823b7fe700 1 --2- 172.21.15.156:0/1180031515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f823c15c9b0 0x7f823c1e7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:56.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.862+0000 7f823b7fe700 1 -- 172.21.15.156:0/1180031515 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8224009550 con 0x7f823c0d2320 2023-11-28T13:09:56.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.862+0000 7f823b7fe700 1 --2- 172.21.15.156:0/1180031515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f823c0d2320 0x7f823c150cc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f822c00c480 tx=0x7f822c00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:56.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.862+0000 7f8238ff9700 1 -- 172.21.15.156:0/1180031515 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c0052d0 con 0x7f823c0d2320 2023-11-28T13:09:56.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.863+0000 7f8238ff9700 1 -- 172.21.15.156:0/1180031515 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c005430 con 0x7f823c0d2320 2023-11-28T13:09:56.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.863+0000 7f8238ff9700 1 -- 172.21.15.156:0/1180031515 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c0038c0 con 0x7f823c0d2320 2023-11-28T13:09:56.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.863+0000 7f82421ff700 1 -- 172.21.15.156:0/1180031515 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823c0d15e0 con 0x7f823c0d2320 2023-11-28T13:09:56.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.863+0000 7f82421ff700 1 -- 172.21.15.156:0/1180031515 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823c0d1300 con 0x7f823c0d2320 2023-11-28T13:09:56.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.864+0000 7f82421ff700 1 -- 172.21.15.156:0/1180031515 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823c15cd80 con 0x7f823c0d2320 2023-11-28T13:09:56.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.865+0000 7f8238ff9700 1 -- 172.21.15.156:0/1180031515 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f822c019030 con 0x7f823c0d2320 2023-11-28T13:09:56.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.865+0000 7f8238ff9700 1 --2- 172.21.15.156:0/1180031515 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f821c0682f0 0x7f821c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:56.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.865+0000 7f8238ff9700 1 -- 172.21.15.156:0/1180031515 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f822c084e70 con 0x7f823c0d2320 2023-11-28T13:09:56.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.866+0000 7f8233fff700 1 --2- 172.21.15.156:0/1180031515 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f821c0682f0 0x7f821c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:56.868 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.867+0000 7f8233fff700 1 --2- 172.21.15.156:0/1180031515 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f821c0682f0 0x7f821c06a7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8224009870 tx=0x7f8224008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:56.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:56.869+0000 7f8238ff9700 1 -- 172.21.15.156:0/1180031515 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f822c051670 con 0x7f823c0d2320 2023-11-28T13:09:57.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.055+0000 7f82421ff700 1 -- 172.21.15.156:0/1180031515 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f823c1e7400 con 0x7f821c0682f0 2023-11-28T13:09:57.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.064+0000 7f8238ff9700 1 -- 172.21.15.156:0/1180031515 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312226 (secure 0 0 0) 0x7f8224011b00 con 0x7f821c0682f0 2023-11-28T13:09:57.070 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:57.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.068+0000 7f82421ff700 1 -- 172.21.15.156:0/1180031515 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f821c0682f0 msgr2=0x7f821c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:57.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.068+0000 7f82421ff700 1 --2- 172.21.15.156:0/1180031515 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f821c0682f0 0x7f821c06a7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8224009870 tx=0x7f8224008030 comp rx=0 tx=0).stop 2023-11-28T13:09:57.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.069+0000 7f82421ff700 1 -- 172.21.15.156:0/1180031515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f823c0d2320 msgr2=0x7f823c150cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:57.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.069+0000 7f82421ff700 1 --2- 172.21.15.156:0/1180031515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f823c0d2320 0x7f823c150cc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f822c00c480 tx=0x7f822c00c790 comp rx=0 tx=0).stop 2023-11-28T13:09:57.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.069+0000 7f82421ff700 1 -- 172.21.15.156:0/1180031515 shutdown_connections 2023-11-28T13:09:57.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.069+0000 7f82421ff700 1 --2- 172.21.15.156:0/1180031515 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f821c0682f0 0x7f821c06a7a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:57.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.069+0000 7f82421ff700 1 --2- 172.21.15.156:0/1180031515 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f823c161340 0x7f823c156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:57.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.069+0000 7f82421ff700 1 --2- 172.21.15.156:0/1180031515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f823c0d2320 0x7f823c150cc0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:57.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.069+0000 7f82421ff700 1 --2- 172.21.15.156:0/1180031515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f823c15c9b0 0x7f823c1e7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:57.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.069+0000 7f82421ff700 1 -- 172.21.15.156:0/1180031515 >> 172.21.15.156:0/1180031515 conn(0x7f823c072600 msgr2=0x7f823c0b6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:57.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.069+0000 7f82421ff700 1 -- 172.21.15.156:0/1180031515 shutdown_connections 2023-11-28T13:09:57.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.069+0000 7f82421ff700 1 -- 172.21.15.156:0/1180031515 wait complete. 2023-11-28T13:09:57.073 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T13:09:57.089 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2023-11-28T13:09:56.042192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":90468,"kb_used_data":5540,"kb_used_omap":0,"kb_used_meta":84864,"kb_avail":754884252,"statfs":{"total":773094113280,"available":773001474048,"internally_reserved":0,"allocated":5672960,"data_stored":3088346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":86900736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.300089"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471911+0000","last_change":"2023-11-28T13:09:52.274749+0000","last_active":"2023-11-28T13:09:53.471911+0000","last_peered":"2023-11-28T13:09:53.471911+0000","last_clean":"2023-11-28T13:09:53.471911+0000","last_became_active":"2023-11-28T13:09:52.274149+0000","last_became_peered":"2023-11-28T13:09:52.274149+0000","last_unstale":"2023-11-28T13:09:53.471911+0000","last_undegraded":"2023-11-28T13:09:53.471911+0000","last_fullsized":"2023-11-28T13:09:53.471911+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:10.991833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.254224+0000","last_change":"2023-11-28T13:09:52.275005+0000","last_active":"2023-11-28T13:09:53.254224+0000","last_peered":"2023-11-28T13:09:53.254224+0000","last_clean":"2023-11-28T13:09:53.254224+0000","last_became_active":"2023-11-28T13:09:52.274683+0000","last_became_peered":"2023-11-28T13:09:52.274683+0000","last_unstale":"2023-11-28T13:09:53.254224+0000","last_undegraded":"2023-11-28T13:09:53.254224+0000","last_fullsized":"2023-11-28T13:09:53.254224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:27:12.289547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472499+0000","last_change":"2023-11-28T13:09:52.282972+0000","last_active":"2023-11-28T13:09:53.472499+0000","last_peered":"2023-11-28T13:09:53.472499+0000","last_clean":"2023-11-28T13:09:53.472499+0000","last_became_active":"2023-11-28T13:09:52.282706+0000","last_became_peered":"2023-11-28T13:09:52.282706+0000","last_unstale":"2023-11-28T13:09:53.472499+0000","last_undegraded":"2023-11-28T13:09:53.472499+0000","last_fullsized":"2023-11-28T13:09:53.472499+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:07:18.517785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.254151+0000","last_change":"2023-11-28T13:09:52.268041+0000","last_active":"2023-11-28T13:09:53.254151+0000","last_peered":"2023-11-28T13:09:53.254151+0000","last_clean":"2023-11-28T13:09:53.254151+0000","last_became_active":"2023-11-28T13:09:52.267756+0000","last_became_peered":"2023-11-28T13:09:52.267756+0000","last_unstale":"2023-11-28T13:09:53.254151+0000","last_undegraded":"2023-11-28T13:09:53.254151+0000","last_fullsized":"2023-11-28T13:09:53.254151+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:24.470324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471163+0000","last_change":"2023-11-28T13:09:52.275511+0000","last_active":"2023-11-28T13:09:53.471163+0000","last_peered":"2023-11-28T13:09:53.471163+0000","last_clean":"2023-11-28T13:09:53.471163+0000","last_became_active":"2023-11-28T13:09:52.274367+0000","last_became_peered":"2023-11-28T13:09:52.274367+0000","last_unstale":"2023-11-28T13:09:53.471163+0000","last_undegraded":"2023-11-28T13:09:53.471163+0000","last_fullsized":"2023-11-28T13:09:53.471163+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:37:06.556688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471070+0000","last_change":"2023-11-28T13:09:52.266289+0000","last_active":"2023-11-28T13:09:53.471070+0000","last_peered":"2023-11-28T13:09:53.471070+0000","last_clean":"2023-11-28T13:09:53.471070+0000","last_became_active":"2023-11-28T13:09:52.265982+0000","last_became_peered":"2023-11-28T13:09:52.265982+0000","last_unstale":"2023-11-28T13:09:53.471070+0000","last_undegraded":"2023-11-28T13:09:53.471070+0000","last_fullsized":"2023-11-28T13:09:53.471070+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:47.315227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471189+0000","last_change":"2023-11-28T13:09:52.283865+0000","last_active":"2023-11-28T13:09:53.471189+0000","last_peered":"2023-11-28T13:09:53.471189+0000","last_clean":"2023-11-28T13:09:53.471189+0000","last_became_active":"2023-11-28T13:09:52.283229+0000","last_became_peered":"2023-11-28T13:09:52.283229+0000","last_unstale":"2023-11-28T13:09:53.471189+0000","last_undegraded":"2023-11-28T13:09:53.471189+0000","last_fullsized":"2023-11-28T13:09:53.471189+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:26:01.144694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470700+0000","last_change":"2023-11-28T13:09:52.275613+0000","last_active":"2023-11-28T13:09:53.470700+0000","last_peered":"2023-11-28T13:09:53.470700+0000","last_clean":"2023-11-28T13:09:53.470700+0000","last_became_active":"2023-11-28T13:09:52.275355+0000","last_became_peered":"2023-11-28T13:09:52.275355+0000","last_unstale":"2023-11-28T13:09:53.470700+0000","last_undegraded":"2023-11-28T13:09:53.470700+0000","last_fullsized":"2023-11-28T13:09:53.470700+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:45:45.645058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472702+0000","last_change":"2023-11-28T13:09:52.268360+0000","last_active":"2023-11-28T13:09:53.472702+0000","last_peered":"2023-11-28T13:09:53.472702+0000","last_clean":"2023-11-28T13:09:53.472702+0000","last_became_active":"2023-11-28T13:09:52.268108+0000","last_became_peered":"2023-11-28T13:09:52.268108+0000","last_unstale":"2023-11-28T13:09:53.472702+0000","last_undegraded":"2023-11-28T13:09:53.472702+0000","last_fullsized":"2023-11-28T13:09:53.472702+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:31.359124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472095+0000","last_change":"2023-11-28T13:09:52.276213+0000","last_active":"2023-11-28T13:09:53.472095+0000","last_peered":"2023-11-28T13:09:53.472095+0000","last_clean":"2023-11-28T13:09:53.472095+0000","last_became_active":"2023-11-28T13:09:52.275921+0000","last_became_peered":"2023-11-28T13:09:52.275921+0000","last_unstale":"2023-11-28T13:09:53.472095+0000","last_undegraded":"2023-11-28T13:09:53.472095+0000","last_fullsized":"2023-11-28T13:09:53.472095+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:39.366007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.251919+0000","last_change":"2023-11-28T13:09:52.264946+0000","last_active":"2023-11-28T13:09:53.251919+0000","last_peered":"2023-11-28T13:09:53.251919+0000","last_clean":"2023-11-28T13:09:53.251919+0000","last_became_active":"2023-11-28T13:09:52.264546+0000","last_became_peered":"2023-11-28T13:09:52.264546+0000","last_unstale":"2023-11-28T13:09:53.251919+0000","last_undegraded":"2023-11-28T13:09:53.251919+0000","last_fullsized":"2023-11-28T13:09:53.251919+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:24:53.209118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472126+0000","last_change":"2023-11-28T13:09:52.259187+0000","last_active":"2023-11-28T13:09:53.472126+0000","last_peered":"2023-11-28T13:09:53.472126+0000","last_clean":"2023-11-28T13:09:53.472126+0000","last_became_active":"2023-11-28T13:09:52.258825+0000","last_became_peered":"2023-11-28T13:09:52.258825+0000","last_unstale":"2023-11-28T13:09:53.472126+0000","last_undegraded":"2023-11-28T13:09:53.472126+0000","last_fullsized":"2023-11-28T13:09:53.472126+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:36:33.641644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471253+0000","last_change":"2023-11-28T13:09:52.267437+0000","last_active":"2023-11-28T13:09:53.471253+0000","last_peered":"2023-11-28T13:09:53.471253+0000","last_clean":"2023-11-28T13:09:53.471253+0000","last_became_active":"2023-11-28T13:09:52.267202+0000","last_became_peered":"2023-11-28T13:09:52.267202+0000","last_unstale":"2023-11-28T13:09:53.471253+0000","last_undegraded":"2023-11-28T13:09:53.471253+0000","last_fullsized":"2023-11-28T13:09:53.471253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:18.276083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471361+0000","last_change":"2023-11-28T13:09:52.283672+0000","last_active":"2023-11-28T13:09:53.471361+0000","last_peered":"2023-11-28T13:09:53.471361+0000","last_clean":"2023-11-28T13:09:53.471361+0000","last_became_active":"2023-11-28T13:09:52.283002+0000","last_became_peered":"2023-11-28T13:09:52.283002+0000","last_unstale":"2023-11-28T13:09:53.471361+0000","last_undegraded":"2023-11-28T13:09:53.471361+0000","last_fullsized":"2023-11-28T13:09:53.471361+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:44:38.788752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253911+0000","last_change":"2023-11-28T13:09:52.259704+0000","last_active":"2023-11-28T13:09:53.253911+0000","last_peered":"2023-11-28T13:09:53.253911+0000","last_clean":"2023-11-28T13:09:53.253911+0000","last_became_active":"2023-11-28T13:09:52.258872+0000","last_became_peered":"2023-11-28T13:09:52.258872+0000","last_unstale":"2023-11-28T13:09:53.253911+0000","last_undegraded":"2023-11-28T13:09:53.253911+0000","last_fullsized":"2023-11-28T13:09:53.253911+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:11:56.997320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470619+0000","last_change":"2023-11-28T13:09:52.271148+0000","last_active":"2023-11-28T13:09:53.470619+0000","last_peered":"2023-11-28T13:09:53.470619+0000","last_clean":"2023-11-28T13:09:53.470619+0000","last_became_active":"2023-11-28T13:09:52.270912+0000","last_became_peered":"2023-11-28T13:09:52.270912+0000","last_unstale":"2023-11-28T13:09:53.470619+0000","last_undegraded":"2023-11-28T13:09:53.470619+0000","last_fullsized":"2023-11-28T13:09:53.470619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:56:24.470434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472935+0000","last_change":"2023-11-28T13:09:52.277655+0000","last_active":"2023-11-28T13:09:53.472935+0000","last_peered":"2023-11-28T13:09:53.472935+0000","last_clean":"2023-11-28T13:09:53.472935+0000","last_became_active":"2023-11-28T13:09:52.277424+0000","last_became_peered":"2023-11-28T13:09:52.277424+0000","last_unstale":"2023-11-28T13:09:53.472935+0000","last_undegraded":"2023-11-28T13:09:53.472935+0000","last_fullsized":"2023-11-28T13:09:53.472935+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:18:42.555378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.254331+0000","last_change":"2023-11-28T13:09:52.273160+0000","last_active":"2023-11-28T13:09:53.254331+0000","last_peered":"2023-11-28T13:09:53.254331+0000","last_clean":"2023-11-28T13:09:53.254331+0000","last_became_active":"2023-11-28T13:09:52.272833+0000","last_became_peered":"2023-11-28T13:09:52.272833+0000","last_unstale":"2023-11-28T13:09:53.254331+0000","last_undegraded":"2023-11-28T13:09:53.254331+0000","last_fullsized":"2023-11-28T13:09:53.254331+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:40.623485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472591+0000","last_change":"2023-11-28T13:09:52.278452+0000","last_active":"2023-11-28T13:09:53.472591+0000","last_peered":"2023-11-28T13:09:53.472591+0000","last_clean":"2023-11-28T13:09:53.472591+0000","last_became_active":"2023-11-28T13:09:52.277877+0000","last_became_peered":"2023-11-28T13:09:52.277877+0000","last_unstale":"2023-11-28T13:09:53.472591+0000","last_undegraded":"2023-11-28T13:09:53.472591+0000","last_fullsized":"2023-11-28T13:09:53.472591+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:36:33.641644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471731+0000","last_change":"2023-11-28T13:09:52.262741+0000","last_active":"2023-11-28T13:09:53.471731+0000","last_peered":"2023-11-28T13:09:53.471731+0000","last_clean":"2023-11-28T13:09:53.471731+0000","last_became_active":"2023-11-28T13:09:52.262441+0000","last_became_peered":"2023-11-28T13:09:52.262441+0000","last_unstale":"2023-11-28T13:09:53.471731+0000","last_undegraded":"2023-11-28T13:09:53.471731+0000","last_fullsized":"2023-11-28T13:09:53.471731+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:41.248280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471764+0000","last_change":"2023-11-28T13:09:52.262739+0000","last_active":"2023-11-28T13:09:53.471764+0000","last_peered":"2023-11-28T13:09:53.471764+0000","last_clean":"2023-11-28T13:09:53.471764+0000","last_became_active":"2023-11-28T13:09:52.262390+0000","last_became_peered":"2023-11-28T13:09:52.262390+0000","last_unstale":"2023-11-28T13:09:53.471764+0000","last_undegraded":"2023-11-28T13:09:53.471764+0000","last_fullsized":"2023-11-28T13:09:53.471764+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:58:44.619275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470895+0000","last_change":"2023-11-28T13:09:52.269188+0000","last_active":"2023-11-28T13:09:53.470895+0000","last_peered":"2023-11-28T13:09:53.470895+0000","last_clean":"2023-11-28T13:09:53.470895+0000","last_became_active":"2023-11-28T13:09:52.268847+0000","last_became_peered":"2023-11-28T13:09:52.268847+0000","last_unstale":"2023-11-28T13:09:53.470895+0000","last_undegraded":"2023-11-28T13:09:53.470895+0000","last_fullsized":"2023-11-28T13:09:53.470895+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:08.028530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.251733+0000","last_change":"2023-11-28T13:09:52.272126+0000","last_active":"2023-11-28T13:09:53.251733+0000","last_peered":"2023-11-28T13:09:53.251733+0000","last_clean":"2023-11-28T13:09:53.251733+0000","last_became_active":"2023-11-28T13:09:52.271804+0000","last_became_peered":"2023-11-28T13:09:52.271804+0000","last_unstale":"2023-11-28T13:09:53.251733+0000","last_undegraded":"2023-11-28T13:09:53.251733+0000","last_fullsized":"2023-11-28T13:09:53.251733+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:35:12.078892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471047+0000","last_change":"2023-11-28T13:09:52.286298+0000","last_active":"2023-11-28T13:09:53.471047+0000","last_peered":"2023-11-28T13:09:53.471047+0000","last_clean":"2023-11-28T13:09:53.471047+0000","last_became_active":"2023-11-28T13:09:52.286019+0000","last_became_peered":"2023-11-28T13:09:52.286019+0000","last_unstale":"2023-11-28T13:09:53.471047+0000","last_undegraded":"2023-11-28T13:09:53.471047+0000","last_fullsized":"2023-11-28T13:09:53.471047+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:27:12.289547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472962+0000","last_change":"2023-11-28T13:09:52.274196+0000","last_active":"2023-11-28T13:09:53.472962+0000","last_peered":"2023-11-28T13:09:53.472962+0000","last_clean":"2023-11-28T13:09:53.472962+0000","last_became_active":"2023-11-28T13:09:52.273953+0000","last_became_peered":"2023-11-28T13:09:52.273953+0000","last_unstale":"2023-11-28T13:09:53.472962+0000","last_undegraded":"2023-11-28T13:09:53.472962+0000","last_fullsized":"2023-11-28T13:09:53.472962+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:06.374523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472475+0000","last_change":"2023-11-28T13:09:52.259203+0000","last_active":"2023-11-28T13:09:53.472475+0000","last_peered":"2023-11-28T13:09:53.472475+0000","last_clean":"2023-11-28T13:09:53.472475+0000","last_became_active":"2023-11-28T13:09:52.258910+0000","last_became_peered":"2023-11-28T13:09:52.258910+0000","last_unstale":"2023-11-28T13:09:53.472475+0000","last_undegraded":"2023-11-28T13:09:53.472475+0000","last_fullsized":"2023-11-28T13:09:53.472475+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:06.374523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471971+0000","last_change":"2023-11-28T13:09:52.273657+0000","last_active":"2023-11-28T13:09:53.471971+0000","last_peered":"2023-11-28T13:09:53.471971+0000","last_clean":"2023-11-28T13:09:53.471971+0000","last_became_active":"2023-11-28T13:09:52.273357+0000","last_became_peered":"2023-11-28T13:09:52.273357+0000","last_unstale":"2023-11-28T13:09:53.471971+0000","last_undegraded":"2023-11-28T13:09:53.471971+0000","last_fullsized":"2023-11-28T13:09:53.471971+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:02.778153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471940+0000","last_change":"2023-11-28T13:09:52.266326+0000","last_active":"2023-11-28T13:09:53.471940+0000","last_peered":"2023-11-28T13:09:53.471940+0000","last_clean":"2023-11-28T13:09:53.471940+0000","last_became_active":"2023-11-28T13:09:52.265933+0000","last_became_peered":"2023-11-28T13:09:52.265933+0000","last_unstale":"2023-11-28T13:09:53.471940+0000","last_undegraded":"2023-11-28T13:09:53.471940+0000","last_fullsized":"2023-11-28T13:09:53.471940+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:59.167162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471159+0000","last_change":"2023-11-28T13:09:52.270557+0000","last_active":"2023-11-28T13:09:53.471159+0000","last_peered":"2023-11-28T13:09:53.471159+0000","last_clean":"2023-11-28T13:09:53.471159+0000","last_became_active":"2023-11-28T13:09:52.270296+0000","last_became_peered":"2023-11-28T13:09:52.270296+0000","last_unstale":"2023-11-28T13:09:53.471159+0000","last_undegraded":"2023-11-28T13:09:53.471159+0000","last_fullsized":"2023-11-28T13:09:53.471159+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:31.359124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253956+0000","last_change":"2023-11-28T13:09:52.267157+0000","last_active":"2023-11-28T13:09:53.253956+0000","last_peered":"2023-11-28T13:09:53.253956+0000","last_clean":"2023-11-28T13:09:53.253956+0000","last_became_active":"2023-11-28T13:09:52.266905+0000","last_became_peered":"2023-11-28T13:09:52.266905+0000","last_unstale":"2023-11-28T13:09:53.253956+0000","last_undegraded":"2023-11-28T13:09:53.253956+0000","last_fullsized":"2023-11-28T13:09:53.253956+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:47.315227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471419+0000","last_change":"2023-11-28T13:09:52.276980+0000","last_active":"2023-11-28T13:09:53.471419+0000","last_peered":"2023-11-28T13:09:53.471419+0000","last_clean":"2023-11-28T13:09:53.471419+0000","last_became_active":"2023-11-28T13:09:52.276682+0000","last_became_peered":"2023-11-28T13:09:52.276682+0000","last_unstale":"2023-11-28T13:09:53.471419+0000","last_undegraded":"2023-11-28T13:09:53.471419+0000","last_fullsized":"2023-11-28T13:09:53.471419+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:24.470324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253564+0000","last_change":"2023-11-28T13:09:52.278591+0000","last_active":"2023-11-28T13:09:53.253564+0000","last_peered":"2023-11-28T13:09:53.253564+0000","last_clean":"2023-11-28T13:09:53.253564+0000","last_became_active":"2023-11-28T13:09:52.278278+0000","last_became_peered":"2023-11-28T13:09:52.278278+0000","last_unstale":"2023-11-28T13:09:53.253564+0000","last_undegraded":"2023-11-28T13:09:53.253564+0000","last_fullsized":"2023-11-28T13:09:53.253564+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:02:26.898065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.252704+0000","last_change":"2023-11-28T13:09:52.270918+0000","last_active":"2023-11-28T13:09:53.252704+0000","last_peered":"2023-11-28T13:09:53.252704+0000","last_clean":"2023-11-28T13:09:53.252704+0000","last_became_active":"2023-11-28T13:09:52.270608+0000","last_became_peered":"2023-11-28T13:09:52.270608+0000","last_unstale":"2023-11-28T13:09:53.252704+0000","last_undegraded":"2023-11-28T13:09:53.252704+0000","last_fullsized":"2023-11-28T13:09:53.252704+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:02.778153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"46'4","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472095+0000","last_change":"2023-11-28T13:09:51.757952+0000","last_active":"2023-11-28T13:09:53.472095+0000","last_peered":"2023-11-28T13:09:53.472095+0000","last_clean":"2023-11-28T13:09:53.472095+0000","last_became_active":"2023-11-28T13:09:51.757399+0000","last_became_peered":"2023-11-28T13:09:51.757399+0000","last_unstale":"2023-11-28T13:09:53.472095+0000","last_undegraded":"2023-11-28T13:09:53.472095+0000","last_fullsized":"2023-11-28T13:09:53.472095+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:52.078719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253288+0000","last_change":"2023-11-28T13:09:52.278144+0000","last_active":"2023-11-28T13:09:53.253288+0000","last_peered":"2023-11-28T13:09:53.253288+0000","last_clean":"2023-11-28T13:09:53.253288+0000","last_became_active":"2023-11-28T13:09:52.277904+0000","last_became_peered":"2023-11-28T13:09:52.277904+0000","last_unstale":"2023-11-28T13:09:53.253288+0000","last_undegraded":"2023-11-28T13:09:53.253288+0000","last_fullsized":"2023-11-28T13:09:53.253288+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:31.359124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472182+0000","last_change":"2023-11-28T13:09:51.772021+0000","last_active":"2023-11-28T13:09:53.472182+0000","last_peered":"2023-11-28T13:09:53.472182+0000","last_clean":"2023-11-28T13:09:53.472182+0000","last_became_active":"2023-11-28T13:09:51.771729+0000","last_became_peered":"2023-11-28T13:09:51.771729+0000","last_unstale":"2023-11-28T13:09:53.472182+0000","last_undegraded":"2023-11-28T13:09:53.472182+0000","last_fullsized":"2023-11-28T13:09:53.472182+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:14:28.072677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471749+0000","last_change":"2023-11-28T13:09:52.262889+0000","last_active":"2023-11-28T13:09:53.471749+0000","last_peered":"2023-11-28T13:09:53.471749+0000","last_clean":"2023-11-28T13:09:53.471749+0000","last_became_active":"2023-11-28T13:09:52.262592+0000","last_became_peered":"2023-11-28T13:09:52.262592+0000","last_unstale":"2023-11-28T13:09:53.471749+0000","last_undegraded":"2023-11-28T13:09:53.471749+0000","last_fullsized":"2023-11-28T13:09:53.471749+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:27:17.350629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472011+0000","last_change":"2023-11-28T13:09:51.766854+0000","last_active":"2023-11-28T13:09:53.472011+0000","last_peered":"2023-11-28T13:09:53.472011+0000","last_clean":"2023-11-28T13:09:53.472011+0000","last_became_active":"2023-11-28T13:09:51.765671+0000","last_became_peered":"2023-11-28T13:09:51.765671+0000","last_unstale":"2023-11-28T13:09:53.472011+0000","last_undegraded":"2023-11-28T13:09:53.472011+0000","last_fullsized":"2023-11-28T13:09:53.472011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:24:52.203192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471566+0000","last_change":"2023-11-28T13:09:52.259260+0000","last_active":"2023-11-28T13:09:53.471566+0000","last_peered":"2023-11-28T13:09:53.471566+0000","last_clean":"2023-11-28T13:09:53.471566+0000","last_became_active":"2023-11-28T13:09:52.258978+0000","last_became_peered":"2023-11-28T13:09:52.258978+0000","last_unstale":"2023-11-28T13:09:53.471566+0000","last_undegraded":"2023-11-28T13:09:53.471566+0000","last_fullsized":"2023-11-28T13:09:53.471566+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:31.670491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253411+0000","last_change":"2023-11-28T13:09:51.773821+0000","last_active":"2023-11-28T13:09:53.253411+0000","last_peered":"2023-11-28T13:09:53.253411+0000","last_clean":"2023-11-28T13:09:53.253411+0000","last_became_active":"2023-11-28T13:09:51.773477+0000","last_became_peered":"2023-11-28T13:09:51.773477+0000","last_unstale":"2023-11-28T13:09:53.253411+0000","last_undegraded":"2023-11-28T13:09:53.253411+0000","last_fullsized":"2023-11-28T13:09:53.253411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:11:55.991394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471395+0000","last_change":"2023-11-28T13:09:52.272811+0000","last_active":"2023-11-28T13:09:53.471395+0000","last_peered":"2023-11-28T13:09:53.471395+0000","last_clean":"2023-11-28T13:09:53.471395+0000","last_became_active":"2023-11-28T13:09:52.271614+0000","last_became_peered":"2023-11-28T13:09:52.271614+0000","last_unstale":"2023-11-28T13:09:53.471395+0000","last_undegraded":"2023-11-28T13:09:53.471395+0000","last_fullsized":"2023-11-28T13:09:53.471395+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:30:01.228085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471582+0000","last_change":"2023-11-28T13:09:51.764543+0000","last_active":"2023-11-28T13:09:53.471582+0000","last_peered":"2023-11-28T13:09:53.471582+0000","last_clean":"2023-11-28T13:09:53.471582+0000","last_became_active":"2023-11-28T13:09:51.763869+0000","last_became_peered":"2023-11-28T13:09:51.763869+0000","last_unstale":"2023-11-28T13:09:53.471582+0000","last_undegraded":"2023-11-28T13:09:53.471582+0000","last_fullsized":"2023-11-28T13:09:53.471582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:17.270157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.254228+0000","last_change":"2023-11-28T13:09:52.280099+0000","last_active":"2023-11-28T13:09:53.254228+0000","last_peered":"2023-11-28T13:09:53.254228+0000","last_clean":"2023-11-28T13:09:53.254228+0000","last_became_active":"2023-11-28T13:09:52.277480+0000","last_became_peered":"2023-11-28T13:09:52.277480+0000","last_unstale":"2023-11-28T13:09:53.254228+0000","last_undegraded":"2023-11-28T13:09:53.254228+0000","last_fullsized":"2023-11-28T13:09:53.254228+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:05.536831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471113+0000","last_change":"2023-11-28T13:09:51.756324+0000","last_active":"2023-11-28T13:09:53.471113+0000","last_peered":"2023-11-28T13:09:53.471113+0000","last_clean":"2023-11-28T13:09:53.471113+0000","last_became_active":"2023-11-28T13:09:51.755970+0000","last_became_peered":"2023-11-28T13:09:51.755970+0000","last_unstale":"2023-11-28T13:09:53.471113+0000","last_undegraded":"2023-11-28T13:09:53.471113+0000","last_fullsized":"2023-11-28T13:09:53.471113+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:04.530905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471978+0000","last_change":"2023-11-28T13:09:52.266103+0000","last_active":"2023-11-28T13:09:53.471978+0000","last_peered":"2023-11-28T13:09:53.471978+0000","last_clean":"2023-11-28T13:09:53.471978+0000","last_became_active":"2023-11-28T13:09:52.265835+0000","last_became_peered":"2023-11-28T13:09:52.265835+0000","last_unstale":"2023-11-28T13:09:53.471978+0000","last_undegraded":"2023-11-28T13:09:53.471978+0000","last_fullsized":"2023-11-28T13:09:53.471978+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:18:42.555378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471587+0000","last_change":"2023-11-28T13:09:51.766581+0000","last_active":"2023-11-28T13:09:53.471587+0000","last_peered":"2023-11-28T13:09:53.471587+0000","last_clean":"2023-11-28T13:09:53.471587+0000","last_became_active":"2023-11-28T13:09:51.766256+0000","last_became_peered":"2023-11-28T13:09:51.766256+0000","last_unstale":"2023-11-28T13:09:53.471587+0000","last_undegraded":"2023-11-28T13:09:53.471587+0000","last_fullsized":"2023-11-28T13:09:53.471587+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:55.487936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.254291+0000","last_change":"2023-11-28T13:09:52.268480+0000","last_active":"2023-11-28T13:09:53.254291+0000","last_peered":"2023-11-28T13:09:53.254291+0000","last_clean":"2023-11-28T13:09:53.254291+0000","last_became_active":"2023-11-28T13:09:52.268101+0000","last_became_peered":"2023-11-28T13:09:52.268101+0000","last_unstale":"2023-11-28T13:09:53.254291+0000","last_undegraded":"2023-11-28T13:09:53.254291+0000","last_fullsized":"2023-11-28T13:09:53.254291+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:07.534480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471213+0000","last_change":"2023-11-28T13:09:51.761072+0000","last_active":"2023-11-28T13:09:53.471213+0000","last_peered":"2023-11-28T13:09:53.471213+0000","last_clean":"2023-11-28T13:09:53.471213+0000","last_became_active":"2023-11-28T13:09:51.760838+0000","last_became_peered":"2023-11-28T13:09:51.760838+0000","last_unstale":"2023-11-28T13:09:53.471213+0000","last_undegraded":"2023-11-28T13:09:53.471213+0000","last_fullsized":"2023-11-28T13:09:53.471213+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:56:23.464508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470963+0000","last_change":"2023-11-28T13:09:52.272851+0000","last_active":"2023-11-28T13:09:53.470963+0000","last_peered":"2023-11-28T13:09:53.470963+0000","last_clean":"2023-11-28T13:09:53.470963+0000","last_became_active":"2023-11-28T13:09:52.272341+0000","last_became_peered":"2023-11-28T13:09:52.272341+0000","last_unstale":"2023-11-28T13:09:53.470963+0000","last_undegraded":"2023-11-28T13:09:53.470963+0000","last_fullsized":"2023-11-28T13:09:53.470963+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:02:26.898065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.254271+0000","last_change":"2023-11-28T13:09:51.770173+0000","last_active":"2023-11-28T13:09:53.254271+0000","last_peered":"2023-11-28T13:09:53.254271+0000","last_clean":"2023-11-28T13:09:53.254271+0000","last_became_active":"2023-11-28T13:09:51.769889+0000","last_became_peered":"2023-11-28T13:09:51.769889+0000","last_unstale":"2023-11-28T13:09:53.254271+0000","last_undegraded":"2023-11-28T13:09:53.254271+0000","last_fullsized":"2023-11-28T13:09:53.254271+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:46:55.192689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253153+0000","last_change":"2023-11-28T13:09:52.269030+0000","last_active":"2023-11-28T13:09:53.253153+0000","last_peered":"2023-11-28T13:09:53.253153+0000","last_clean":"2023-11-28T13:09:53.253153+0000","last_became_active":"2023-11-28T13:09:52.268452+0000","last_became_peered":"2023-11-28T13:09:52.268452+0000","last_unstale":"2023-11-28T13:09:53.253153+0000","last_undegraded":"2023-11-28T13:09:53.253153+0000","last_fullsized":"2023-11-28T13:09:53.253153+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:37:06.556688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471495+0000","last_change":"2023-11-28T13:09:51.769161+0000","last_active":"2023-11-28T13:09:53.471495+0000","last_peered":"2023-11-28T13:09:53.471495+0000","last_clean":"2023-11-28T13:09:53.471495+0000","last_became_active":"2023-11-28T13:09:51.768815+0000","last_became_peered":"2023-11-28T13:09:51.768815+0000","last_unstale":"2023-11-28T13:09:53.471495+0000","last_undegraded":"2023-11-28T13:09:53.471495+0000","last_fullsized":"2023-11-28T13:09:53.471495+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:39.617559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471466+0000","last_change":"2023-11-28T13:09:52.272577+0000","last_active":"2023-11-28T13:09:53.471466+0000","last_peered":"2023-11-28T13:09:53.471466+0000","last_clean":"2023-11-28T13:09:53.471466+0000","last_became_active":"2023-11-28T13:09:52.272250+0000","last_became_peered":"2023-11-28T13:09:52.272250+0000","last_unstale":"2023-11-28T13:09:53.471466+0000","last_undegraded":"2023-11-28T13:09:53.471466+0000","last_fullsized":"2023-11-28T13:09:53.471466+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:56:47.727803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"46'3","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.254242+0000","last_change":"2023-11-28T13:09:51.773763+0000","last_active":"2023-11-28T13:09:53.254242+0000","last_peered":"2023-11-28T13:09:53.254242+0000","last_clean":"2023-11-28T13:09:53.254242+0000","last_became_active":"2023-11-28T13:09:51.773505+0000","last_became_peered":"2023-11-28T13:09:51.773505+0000","last_unstale":"2023-11-28T13:09:53.254242+0000","last_undegraded":"2023-11-28T13:09:53.254242+0000","last_fullsized":"2023-11-28T13:09:53.254242+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:47:34.022920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471188+0000","last_change":"2023-11-28T13:09:52.275851+0000","last_active":"2023-11-28T13:09:53.471188+0000","last_peered":"2023-11-28T13:09:53.471188+0000","last_clean":"2023-11-28T13:09:53.471188+0000","last_became_active":"2023-11-28T13:09:52.275574+0000","last_became_peered":"2023-11-28T13:09:52.275574+0000","last_unstale":"2023-11-28T13:09:53.471188+0000","last_undegraded":"2023-11-28T13:09:53.471188+0000","last_fullsized":"2023-11-28T13:09:53.471188+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:59.756502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"46'6","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472035+0000","last_change":"2023-11-28T13:09:51.769242+0000","last_active":"2023-11-28T13:09:53.472035+0000","last_peered":"2023-11-28T13:09:53.472035+0000","last_clean":"2023-11-28T13:09:53.472035+0000","last_became_active":"2023-11-28T13:09:51.768806+0000","last_became_peered":"2023-11-28T13:09:51.768806+0000","last_unstale":"2023-11-28T13:09:53.472035+0000","last_undegraded":"2023-11-28T13:09:53.472035+0000","last_fullsized":"2023-11-28T13:09:53.472035+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:04.530905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253061+0000","last_change":"2023-11-28T13:09:52.277681+0000","last_active":"2023-11-28T13:09:53.253061+0000","last_peered":"2023-11-28T13:09:53.253061+0000","last_clean":"2023-11-28T13:09:53.253061+0000","last_became_active":"2023-11-28T13:09:52.277362+0000","last_became_peered":"2023-11-28T13:09:52.277362+0000","last_unstale":"2023-11-28T13:09:53.253061+0000","last_undegraded":"2023-11-28T13:09:53.253061+0000","last_fullsized":"2023-11-28T13:09:53.253061+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:07:18.517785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471691+0000","last_change":"2023-11-28T13:09:51.767042+0000","last_active":"2023-11-28T13:09:53.471691+0000","last_peered":"2023-11-28T13:09:53.471691+0000","last_clean":"2023-11-28T13:09:53.471691+0000","last_became_active":"2023-11-28T13:09:51.766763+0000","last_became_peered":"2023-11-28T13:09:51.766763+0000","last_unstale":"2023-11-28T13:09:53.471691+0000","last_undegraded":"2023-11-28T13:09:53.471691+0000","last_fullsized":"2023-11-28T13:09:53.471691+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:10:46.983683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472498+0000","last_change":"2023-11-28T13:09:52.274829+0000","last_active":"2023-11-28T13:09:53.472498+0000","last_peered":"2023-11-28T13:09:53.472498+0000","last_clean":"2023-11-28T13:09:53.472498+0000","last_became_active":"2023-11-28T13:09:52.274475+0000","last_became_peered":"2023-11-28T13:09:52.274475+0000","last_unstale":"2023-11-28T13:09:53.472498+0000","last_undegraded":"2023-11-28T13:09:53.472498+0000","last_fullsized":"2023-11-28T13:09:53.472498+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:45:45.645058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471721+0000","last_change":"2023-11-28T13:09:51.767895+0000","last_active":"2023-11-28T13:09:53.471721+0000","last_peered":"2023-11-28T13:09:53.471721+0000","last_clean":"2023-11-28T13:09:53.471721+0000","last_became_active":"2023-11-28T13:09:51.767590+0000","last_became_peered":"2023-11-28T13:09:51.767590+0000","last_unstale":"2023-11-28T13:09:53.471721+0000","last_undegraded":"2023-11-28T13:09:53.471721+0000","last_fullsized":"2023-11-28T13:09:53.471721+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:39.586756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471886+0000","last_change":"2023-11-28T13:09:52.265869+0000","last_active":"2023-11-28T13:09:53.471886+0000","last_peered":"2023-11-28T13:09:53.471886+0000","last_clean":"2023-11-28T13:09:53.471886+0000","last_became_active":"2023-11-28T13:09:52.265623+0000","last_became_peered":"2023-11-28T13:09:52.265623+0000","last_unstale":"2023-11-28T13:09:53.471886+0000","last_undegraded":"2023-11-28T13:09:53.471886+0000","last_fullsized":"2023-11-28T13:09:53.471886+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:44:38.788752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253900+0000","last_change":"2023-11-28T13:09:51.766137+0000","last_active":"2023-11-28T13:09:53.253900+0000","last_peered":"2023-11-28T13:09:53.253900+0000","last_clean":"2023-11-28T13:09:53.253900+0000","last_became_active":"2023-11-28T13:09:51.765780+0000","last_became_peered":"2023-11-28T13:09:51.765780+0000","last_unstale":"2023-11-28T13:09:53.253900+0000","last_undegraded":"2023-11-28T13:09:53.253900+0000","last_fullsized":"2023-11-28T13:09:53.253900+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:40.242354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472870+0000","last_change":"2023-11-28T13:09:52.279558+0000","last_active":"2023-11-28T13:09:53.472870+0000","last_peered":"2023-11-28T13:09:53.472870+0000","last_clean":"2023-11-28T13:09:53.472870+0000","last_became_active":"2023-11-28T13:09:52.278752+0000","last_became_peered":"2023-11-28T13:09:52.278752+0000","last_unstale":"2023-11-28T13:09:53.472870+0000","last_undegraded":"2023-11-28T13:09:53.472870+0000","last_fullsized":"2023-11-28T13:09:53.472870+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:43.979876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"46'8","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470888+0000","last_change":"2023-11-28T13:09:51.766711+0000","last_active":"2023-11-28T13:09:53.470888+0000","last_peered":"2023-11-28T13:09:53.470888+0000","last_clean":"2023-11-28T13:09:53.470888+0000","last_became_active":"2023-11-28T13:09:51.766155+0000","last_became_peered":"2023-11-28T13:09:51.766155+0000","last_unstale":"2023-11-28T13:09:53.470888+0000","last_undegraded":"2023-11-28T13:09:53.470888+0000","last_fullsized":"2023-11-28T13:09:53.470888+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:01.772227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470794+0000","last_change":"2023-11-28T13:09:52.270571+0000","last_active":"2023-11-28T13:09:53.470794+0000","last_peered":"2023-11-28T13:09:53.470794+0000","last_clean":"2023-11-28T13:09:53.470794+0000","last_became_active":"2023-11-28T13:09:52.270286+0000","last_became_peered":"2023-11-28T13:09:52.270286+0000","last_unstale":"2023-11-28T13:09:53.470794+0000","last_undegraded":"2023-11-28T13:09:53.470794+0000","last_fullsized":"2023-11-28T13:09:53.470794+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:02:26.898065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472848+0000","last_change":"2023-11-28T13:09:51.771962+0000","last_active":"2023-11-28T13:09:53.472848+0000","last_peered":"2023-11-28T13:09:53.472848+0000","last_clean":"2023-11-28T13:09:53.472848+0000","last_became_active":"2023-11-28T13:09:51.771654+0000","last_became_peered":"2023-11-28T13:09:51.771654+0000","last_unstale":"2023-11-28T13:09:53.472848+0000","last_undegraded":"2023-11-28T13:09:53.472848+0000","last_fullsized":"2023-11-28T13:09:53.472848+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:55:01.616495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470780+0000","last_change":"2023-11-28T13:09:52.273705+0000","last_active":"2023-11-28T13:09:53.470780+0000","last_peered":"2023-11-28T13:09:53.470780+0000","last_clean":"2023-11-28T13:09:53.470780+0000","last_became_active":"2023-11-28T13:09:52.273380+0000","last_became_peered":"2023-11-28T13:09:52.273380+0000","last_unstale":"2023-11-28T13:09:53.470780+0000","last_undegraded":"2023-11-28T13:09:53.470780+0000","last_fullsized":"2023-11-28T13:09:53.470780+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:11:56.997320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.252097+0000","last_change":"2023-11-28T13:09:51.760087+0000","last_active":"2023-11-28T13:09:53.252097+0000","last_peered":"2023-11-28T13:09:53.252097+0000","last_clean":"2023-11-28T13:09:53.252097+0000","last_became_active":"2023-11-28T13:09:51.759804+0000","last_became_peered":"2023-11-28T13:09:51.759804+0000","last_unstale":"2023-11-28T13:09:53.252097+0000","last_undegraded":"2023-11-28T13:09:53.252097+0000","last_fullsized":"2023-11-28T13:09:53.252097+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:52.078719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470998+0000","last_change":"2023-11-28T13:09:52.273571+0000","last_active":"2023-11-28T13:09:53.470998+0000","last_peered":"2023-11-28T13:09:53.470998+0000","last_clean":"2023-11-28T13:09:53.470998+0000","last_became_active":"2023-11-28T13:09:52.273294+0000","last_became_peered":"2023-11-28T13:09:52.273294+0000","last_unstale":"2023-11-28T13:09:53.470998+0000","last_undegraded":"2023-11-28T13:09:53.470998+0000","last_fullsized":"2023-11-28T13:09:53.470998+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:48.180966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472021+0000","last_change":"2023-11-28T13:09:51.761533+0000","last_active":"2023-11-28T13:09:53.472021+0000","last_peered":"2023-11-28T13:09:53.472021+0000","last_clean":"2023-11-28T13:09:53.472021+0000","last_became_active":"2023-11-28T13:09:51.761238+0000","last_became_peered":"2023-11-28T13:09:51.761238+0000","last_unstale":"2023-11-28T13:09:53.472021+0000","last_undegraded":"2023-11-28T13:09:53.472021+0000","last_fullsized":"2023-11-28T13:09:53.472021+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:12:18.204614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471357+0000","last_change":"2023-11-28T13:09:52.276438+0000","last_active":"2023-11-28T13:09:53.471357+0000","last_peered":"2023-11-28T13:09:53.471357+0000","last_clean":"2023-11-28T13:09:53.471357+0000","last_became_active":"2023-11-28T13:09:52.275736+0000","last_became_peered":"2023-11-28T13:09:52.275736+0000","last_unstale":"2023-11-28T13:09:53.471357+0000","last_undegraded":"2023-11-28T13:09:53.471357+0000","last_fullsized":"2023-11-28T13:09:53.471357+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:05.536831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471877+0000","last_change":"2023-11-28T13:09:51.773430+0000","last_active":"2023-11-28T13:09:53.471877+0000","last_peered":"2023-11-28T13:09:53.471877+0000","last_clean":"2023-11-28T13:09:53.471877+0000","last_became_active":"2023-11-28T13:09:51.773189+0000","last_became_peered":"2023-11-28T13:09:51.773189+0000","last_unstale":"2023-11-28T13:09:53.471877+0000","last_undegraded":"2023-11-28T13:09:53.471877+0000","last_fullsized":"2023-11-28T13:09:53.471877+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:59:47.972951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253519+0000","last_change":"2023-11-28T13:09:52.275372+0000","last_active":"2023-11-28T13:09:53.253519+0000","last_peered":"2023-11-28T13:09:53.253519+0000","last_clean":"2023-11-28T13:09:53.253519+0000","last_became_active":"2023-11-28T13:09:52.275092+0000","last_became_peered":"2023-11-28T13:09:52.275092+0000","last_unstale":"2023-11-28T13:09:53.253519+0000","last_undegraded":"2023-11-28T13:09:53.253519+0000","last_fullsized":"2023-11-28T13:09:53.253519+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:48.180966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471385+0000","last_change":"2023-11-28T13:09:51.765086+0000","last_active":"2023-11-28T13:09:53.471385+0000","last_peered":"2023-11-28T13:09:53.471385+0000","last_clean":"2023-11-28T13:09:53.471385+0000","last_became_active":"2023-11-28T13:09:51.764788+0000","last_became_peered":"2023-11-28T13:09:51.764788+0000","last_unstale":"2023-11-28T13:09:53.471385+0000","last_undegraded":"2023-11-28T13:09:53.471385+0000","last_fullsized":"2023-11-28T13:09:53.471385+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:09:30.031005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472009+0000","last_change":"2023-11-28T13:09:52.276116+0000","last_active":"2023-11-28T13:09:53.472009+0000","last_peered":"2023-11-28T13:09:53.472009+0000","last_clean":"2023-11-28T13:09:53.472009+0000","last_became_active":"2023-11-28T13:09:52.275232+0000","last_became_peered":"2023-11-28T13:09:52.275232+0000","last_unstale":"2023-11-28T13:09:53.472009+0000","last_undegraded":"2023-11-28T13:09:53.472009+0000","last_fullsized":"2023-11-28T13:09:53.472009+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:47:35.028846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.470974+0000","last_change":"2023-11-28T13:09:51.771687+0000","last_active":"2023-11-28T13:09:53.470974+0000","last_peered":"2023-11-28T13:09:53.470974+0000","last_clean":"2023-11-28T13:09:53.470974+0000","last_became_active":"2023-11-28T13:09:51.771432+0000","last_became_peered":"2023-11-28T13:09:51.771432+0000","last_unstale":"2023-11-28T13:09:53.470974+0000","last_undegraded":"2023-11-28T13:09:53.470974+0000","last_fullsized":"2023-11-28T13:09:53.470974+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:11:55.991394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472695+0000","last_change":"2023-11-28T13:09:52.270971+0000","last_active":"2023-11-28T13:09:53.472695+0000","last_peered":"2023-11-28T13:09:53.472695+0000","last_clean":"2023-11-28T13:09:53.472695+0000","last_became_active":"2023-11-28T13:09:52.270718+0000","last_became_peered":"2023-11-28T13:09:52.270718+0000","last_unstale":"2023-11-28T13:09:53.472695+0000","last_undegraded":"2023-11-28T13:09:53.472695+0000","last_fullsized":"2023-11-28T13:09:53.472695+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:27:10.518404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471273+0000","last_change":"2023-11-28T13:09:51.762882+0000","last_active":"2023-11-28T13:09:53.471273+0000","last_peered":"2023-11-28T13:09:53.471273+0000","last_clean":"2023-11-28T13:09:53.471273+0000","last_became_active":"2023-11-28T13:09:51.761003+0000","last_became_peered":"2023-11-28T13:09:51.761003+0000","last_unstale":"2023-11-28T13:09:53.471273+0000","last_undegraded":"2023-11-28T13:09:53.471273+0000","last_fullsized":"2023-11-28T13:09:53.471273+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:20:52.814374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472671+0000","last_change":"2023-11-28T13:09:52.271718+0000","last_active":"2023-11-28T13:09:53.472671+0000","last_peered":"2023-11-28T13:09:53.472671+0000","last_clean":"2023-11-28T13:09:53.472671+0000","last_became_active":"2023-11-28T13:09:52.271448+0000","last_became_peered":"2023-11-28T13:09:52.271448+0000","last_unstale":"2023-11-28T13:09:53.472671+0000","last_undegraded":"2023-11-28T13:09:53.472671+0000","last_fullsized":"2023-11-28T13:09:53.472671+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:40.623485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.251689+0000","last_change":"2023-11-28T13:09:51.765728+0000","last_active":"2023-11-28T13:09:53.251689+0000","last_peered":"2023-11-28T13:09:53.251689+0000","last_clean":"2023-11-28T13:09:53.251689+0000","last_became_active":"2023-11-28T13:09:51.765433+0000","last_became_peered":"2023-11-28T13:09:51.765433+0000","last_unstale":"2023-11-28T13:09:53.251689+0000","last_undegraded":"2023-11-28T13:09:53.251689+0000","last_fullsized":"2023-11-28T13:09:53.251689+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:27:55.476924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472384+0000","last_change":"2023-11-28T13:09:52.268049+0000","last_active":"2023-11-28T13:09:53.472384+0000","last_peered":"2023-11-28T13:09:53.472384+0000","last_clean":"2023-11-28T13:09:53.472384+0000","last_became_active":"2023-11-28T13:09:52.267818+0000","last_became_peered":"2023-11-28T13:09:52.267818+0000","last_unstale":"2023-11-28T13:09:53.472384+0000","last_undegraded":"2023-11-28T13:09:53.472384+0000","last_fullsized":"2023-11-28T13:09:53.472384+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:31.670491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471785+0000","last_change":"2023-11-28T13:09:51.769416+0000","last_active":"2023-11-28T13:09:53.471785+0000","last_peered":"2023-11-28T13:09:53.471785+0000","last_clean":"2023-11-28T13:09:53.471785+0000","last_became_active":"2023-11-28T13:09:51.768508+0000","last_became_peered":"2023-11-28T13:09:51.768508+0000","last_unstale":"2023-11-28T13:09:53.471785+0000","last_undegraded":"2023-11-28T13:09:53.471785+0000","last_fullsized":"2023-11-28T13:09:53.471785+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:39.617559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":39,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.267364+0000","last_change":"2023-11-28T13:07:56.890378+0000","last_active":"2023-11-28T13:09:52.267364+0000","last_peered":"2023-11-28T13:09:52.267364+0000","last_clean":"2023-11-28T13:09:52.267364+0000","last_became_active":"2023-11-28T13:07:56.890058+0000","last_became_peered":"2023-11-28T13:07:56.890058+0000","last_unstale":"2023-11-28T13:09:52.267364+0000","last_undegraded":"2023-11-28T13:09:52.267364+0000","last_fullsized":"2023-11-28T13:09:52.267364+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:05.344602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.252933+0000","last_change":"2023-11-28T13:09:52.277672+0000","last_active":"2023-11-28T13:09:53.252933+0000","last_peered":"2023-11-28T13:09:53.252933+0000","last_clean":"2023-11-28T13:09:53.252933+0000","last_became_active":"2023-11-28T13:09:52.277374+0000","last_became_peered":"2023-11-28T13:09:52.277374+0000","last_unstale":"2023-11-28T13:09:53.252933+0000","last_undegraded":"2023-11-28T13:09:53.252933+0000","last_fullsized":"2023-11-28T13:09:53.252933+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:02.353727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471555+0000","last_change":"2023-11-28T13:09:51.771155+0000","last_active":"2023-11-28T13:09:53.471555+0000","last_peered":"2023-11-28T13:09:53.471555+0000","last_clean":"2023-11-28T13:09:53.471555+0000","last_became_active":"2023-11-28T13:09:51.770148+0000","last_became_peered":"2023-11-28T13:09:51.770148+0000","last_unstale":"2023-11-28T13:09:53.471555+0000","last_undegraded":"2023-11-28T13:09:53.471555+0000","last_fullsized":"2023-11-28T13:09:53.471555+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:46.309301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.250199+0000","last_change":"2023-11-28T13:07:56.844509+0000","last_active":"2023-11-28T13:09:52.250199+0000","last_peered":"2023-11-28T13:09:52.250199+0000","last_clean":"2023-11-28T13:09:52.250199+0000","last_became_active":"2023-11-28T13:07:56.844081+0000","last_became_peered":"2023-11-28T13:07:56.844081+0000","last_unstale":"2023-11-28T13:09:52.250199+0000","last_undegraded":"2023-11-28T13:09:52.250199+0000","last_fullsized":"2023-11-28T13:09:52.250199+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471239+0000","last_change":"2023-11-28T13:09:52.277425+0000","last_active":"2023-11-28T13:09:53.471239+0000","last_peered":"2023-11-28T13:09:53.471239+0000","last_clean":"2023-11-28T13:09:53.471239+0000","last_became_active":"2023-11-28T13:09:52.277195+0000","last_became_peered":"2023-11-28T13:09:52.277195+0000","last_unstale":"2023-11-28T13:09:53.471239+0000","last_undegraded":"2023-11-28T13:09:53.471239+0000","last_fullsized":"2023-11-28T13:09:53.471239+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:55:02.622421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.254322+0000","last_change":"2023-11-28T13:09:51.768271+0000","last_active":"2023-11-28T13:09:53.254322+0000","last_peered":"2023-11-28T13:09:53.254322+0000","last_clean":"2023-11-28T13:09:53.254322+0000","last_became_active":"2023-11-28T13:09:51.768022+0000","last_became_peered":"2023-11-28T13:09:51.768022+0000","last_unstale":"2023-11-28T13:09:53.254322+0000","last_undegraded":"2023-11-28T13:09:53.254322+0000","last_fullsized":"2023-11-28T13:09:53.254322+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:47:08.021309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":36,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.257680+0000","last_change":"2023-11-28T13:07:56.844699+0000","last_active":"2023-11-28T13:09:52.257680+0000","last_peered":"2023-11-28T13:09:52.257680+0000","last_clean":"2023-11-28T13:09:52.257680+0000","last_became_active":"2023-11-28T13:07:56.844391+0000","last_became_peered":"2023-11-28T13:07:56.844391+0000","last_unstale":"2023-11-28T13:09:52.257680+0000","last_undegraded":"2023-11-28T13:09:52.257680+0000","last_fullsized":"2023-11-28T13:09:52.257680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":186,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.254631+0000","last_change":"2023-11-28T13:07:57.844020+0000","last_active":"2023-11-28T13:09:52.254631+0000","last_peered":"2023-11-28T13:09:52.254631+0000","last_clean":"2023-11-28T13:09:52.254631+0000","last_became_active":"2023-11-28T13:07:57.842981+0000","last_became_peered":"2023-11-28T13:07:57.842981+0000","last_unstale":"2023-11-28T13:09:52.254631+0000","last_undegraded":"2023-11-28T13:09:52.254631+0000","last_fullsized":"2023-11-28T13:09:52.254631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_clean_scrub_stamp":"2023-11-28T13:07:56.825711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:44.973218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471683+0000","last_change":"2023-11-28T13:09:52.263490+0000","last_active":"2023-11-28T13:09:53.471683+0000","last_peered":"2023-11-28T13:09:53.471683+0000","last_clean":"2023-11-28T13:09:53.471683+0000","last_became_active":"2023-11-28T13:09:52.262892+0000","last_became_peered":"2023-11-28T13:09:52.262892+0000","last_unstale":"2023-11-28T13:09:53.471683+0000","last_undegraded":"2023-11-28T13:09:53.471683+0000","last_fullsized":"2023-11-28T13:09:53.471683+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:56:47.727803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471827+0000","last_change":"2023-11-28T13:09:51.771254+0000","last_active":"2023-11-28T13:09:53.471827+0000","last_peered":"2023-11-28T13:09:53.471827+0000","last_clean":"2023-11-28T13:09:53.471827+0000","last_became_active":"2023-11-28T13:09:51.768113+0000","last_became_peered":"2023-11-28T13:09:51.768113+0000","last_unstale":"2023-11-28T13:09:53.471827+0000","last_undegraded":"2023-11-28T13:09:53.471827+0000","last_fullsized":"2023-11-28T13:09:53.471827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:38.360081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":37,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.254750+0000","last_change":"2023-11-28T13:07:57.306318+0000","last_active":"2023-11-28T13:09:52.254750+0000","last_peered":"2023-11-28T13:09:52.254750+0000","last_clean":"2023-11-28T13:09:52.254750+0000","last_became_active":"2023-11-28T13:07:57.305937+0000","last_became_peered":"2023-11-28T13:07:57.305937+0000","last_unstale":"2023-11-28T13:09:52.254750+0000","last_undegraded":"2023-11-28T13:09:52.254750+0000","last_fullsized":"2023-11-28T13:09:52.254750+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471852+0000","last_change":"2023-11-28T13:09:52.270817+0000","last_active":"2023-11-28T13:09:53.471852+0000","last_peered":"2023-11-28T13:09:53.471852+0000","last_clean":"2023-11-28T13:09:53.471852+0000","last_became_active":"2023-11-28T13:09:52.270002+0000","last_became_peered":"2023-11-28T13:09:52.270002+0000","last_unstale":"2023-11-28T13:09:53.471852+0000","last_undegraded":"2023-11-28T13:09:53.471852+0000","last_fullsized":"2023-11-28T13:09:53.471852+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:20:53.820300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472684+0000","last_change":"2023-11-28T13:09:51.772898+0000","last_active":"2023-11-28T13:09:53.472684+0000","last_peered":"2023-11-28T13:09:53.472684+0000","last_clean":"2023-11-28T13:09:53.472684+0000","last_became_active":"2023-11-28T13:09:51.772613+0000","last_became_peered":"2023-11-28T13:09:51.772613+0000","last_unstale":"2023-11-28T13:09:53.472684+0000","last_undegraded":"2023-11-28T13:09:53.472684+0000","last_fullsized":"2023-11-28T13:09:53.472684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:58:43.613349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":37,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.254529+0000","last_change":"2023-11-28T13:07:57.306277+0000","last_active":"2023-11-28T13:09:52.254529+0000","last_peered":"2023-11-28T13:09:52.254529+0000","last_clean":"2023-11-28T13:09:52.254529+0000","last_became_active":"2023-11-28T13:07:57.305983+0000","last_became_peered":"2023-11-28T13:07:57.305983+0000","last_unstale":"2023-11-28T13:09:52.254529+0000","last_undegraded":"2023-11-28T13:09:52.254529+0000","last_fullsized":"2023-11-28T13:09:52.254529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253055+0000","last_change":"2023-11-28T13:09:52.272965+0000","last_active":"2023-11-28T13:09:53.253055+0000","last_peered":"2023-11-28T13:09:53.253055+0000","last_clean":"2023-11-28T13:09:53.253055+0000","last_became_active":"2023-11-28T13:09:52.272665+0000","last_became_peered":"2023-11-28T13:09:52.272665+0000","last_unstale":"2023-11-28T13:09:53.253055+0000","last_undegraded":"2023-11-28T13:09:53.253055+0000","last_fullsized":"2023-11-28T13:09:53.253055+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:30:01.228085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472096+0000","last_change":"2023-11-28T13:09:51.772702+0000","last_active":"2023-11-28T13:09:53.472096+0000","last_peered":"2023-11-28T13:09:53.472096+0000","last_clean":"2023-11-28T13:09:53.472096+0000","last_became_active":"2023-11-28T13:09:51.772471+0000","last_became_peered":"2023-11-28T13:09:51.772471+0000","last_unstale":"2023-11-28T13:09:53.472096+0000","last_undegraded":"2023-11-28T13:09:53.472096+0000","last_fullsized":"2023-11-28T13:09:53.472096+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:27:55.476924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.270529+0000","last_change":"2023-11-28T13:07:57.306177+0000","last_active":"2023-11-28T13:09:52.270529+0000","last_peered":"2023-11-28T13:09:52.270529+0000","last_clean":"2023-11-28T13:09:52.270529+0000","last_became_active":"2023-11-28T13:07:57.305853+0000","last_became_peered":"2023-11-28T13:07:57.305853+0000","last_unstale":"2023-11-28T13:09:52.270529+0000","last_undegraded":"2023-11-28T13:09:52.270529+0000","last_fullsized":"2023-11-28T13:09:52.270529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472792+0000","last_change":"2023-11-28T13:09:52.264312+0000","last_active":"2023-11-28T13:09:53.472792+0000","last_peered":"2023-11-28T13:09:53.472792+0000","last_clean":"2023-11-28T13:09:53.472792+0000","last_became_active":"2023-11-28T13:09:52.264048+0000","last_became_peered":"2023-11-28T13:09:52.264048+0000","last_unstale":"2023-11-28T13:09:53.472792+0000","last_undegraded":"2023-11-28T13:09:53.472792+0000","last_fullsized":"2023-11-28T13:09:53.472792+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:43.979876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.253173+0000","last_change":"2023-11-28T13:09:51.762790+0000","last_active":"2023-11-28T13:09:53.253173+0000","last_peered":"2023-11-28T13:09:53.253173+0000","last_clean":"2023-11-28T13:09:53.253173+0000","last_became_active":"2023-11-28T13:09:51.762518+0000","last_became_peered":"2023-11-28T13:09:51.762518+0000","last_unstale":"2023-11-28T13:09:53.253173+0000","last_undegraded":"2023-11-28T13:09:53.253173+0000","last_fullsized":"2023-11-28T13:09:53.253173+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:39.617559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.255543+0000","last_change":"2023-11-28T13:07:57.306321+0000","last_active":"2023-11-28T13:09:52.255543+0000","last_peered":"2023-11-28T13:09:52.255543+0000","last_clean":"2023-11-28T13:09:52.255543+0000","last_became_active":"2023-11-28T13:07:57.305923+0000","last_became_peered":"2023-11-28T13:07:57.305923+0000","last_unstale":"2023-11-28T13:09:52.255543+0000","last_undegraded":"2023-11-28T13:09:52.255543+0000","last_fullsized":"2023-11-28T13:09:52.255543+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:41.104679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.472596+0000","last_change":"2023-11-28T13:09:52.282788+0000","last_active":"2023-11-28T13:09:53.472596+0000","last_peered":"2023-11-28T13:09:53.472596+0000","last_clean":"2023-11-28T13:09:53.472596+0000","last_became_active":"2023-11-28T13:09:52.282515+0000","last_became_peered":"2023-11-28T13:09:52.282515+0000","last_unstale":"2023-11-28T13:09:53.472596+0000","last_undegraded":"2023-11-28T13:09:53.472596+0000","last_fullsized":"2023-11-28T13:09:53.472596+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:51.737535+0000","last_clean_scrub_stamp":"2023-11-28T13:09:51.737535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:54.003015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-28T13:09:53.471033+0000","last_change":"2023-11-28T13:09:51.770776+0000","last_active":"2023-11-28T13:09:53.471033+0000","last_peered":"2023-11-28T13:09:53.471033+0000","last_clean":"2023-11-28T13:09:53.471033+0000","last_became_active":"2023-11-28T13:09:51.770461+0000","last_became_peered":"2023-11-28T13:09:51.770461+0000","last_unstale":"2023-11-28T13:09:53.471033+0000","last_undegraded":"2023-11-28T13:09:53.471033+0000","last_fullsized":"2023-11-28T13:09:53.471033+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:09:50.731609+0000","last_clean_scrub_stamp":"2023-11-28T13:09:50.731609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:46.309301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":38,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T13:09:52.256116+0000","last_change":"2023-11-28T13:07:56.844305+0000","last_active":"2023-11-28T13:09:52.256116+0000","last_peered":"2023-11-28T13:09:52.256116+0000","last_clean":"2023-11-28T13:09:52.256116+0000","last_became_active":"2023-11-28T13:07:56.843980+0000","last_became_peered":"2023-11-28T13:07:56.843980+0000","last_unstale":"2023-11-28T13:09:52.256116+0000","last_undegraded":"2023-11-28T13:09:52.256116+0000","last_fullsized":"2023-11-28T13:09:52.256116+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10160,"kb_used_data":1128,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361680,"statfs":{"total":96636764160,"available":96626360320,"internally_reserved":0,"allocated":1155072,"data_stored":828484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.78800000000000003}]},{"osd":1,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.84899999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.86899999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.91700000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.60199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.67300000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.70299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8812,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363028,"statfs":{"total":96636764160,"available":96627740672,"internally_reserved":0,"allocated":561152,"data_stored":238628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.64100000000000001}]},{"osd":1,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.57699999999999996}]},{"osd":2,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58799999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.60899999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.63},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.74099999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.67700000000000005}]},{"osd":7,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.71599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14128,"kb_used_data":1128,"kb_used_omap":0,"kb_used_meta":12992,"kb_avail":94357712,"statfs":{"total":96636764160,"available":96622297088,"internally_reserved":0,"allocated":1155072,"data_stored":828520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13303808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.45500000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.434}]},{"osd":4,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.96299999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.93000000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.0860000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.995}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13096,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":12544,"kb_avail":94358744,"statfs":{"total":96636764160,"available":96623353856,"internally_reserved":0,"allocated":557056,"data_stored":237062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12845056},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.5},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.63400000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.68000000000000005}]},{"osd":3,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.70599999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.91800000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.85199999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.94199999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12976,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":12416,"kb_avail":94358864,"statfs":{"total":96636764160,"available":96623476736,"internally_reserved":0,"allocated":565248,"data_stored":239540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12713984},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.49099999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.438}]},{"osd":3,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.53100000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.76200000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.91400000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.77000000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.83899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13236,"kb_used_data":556,"kb_used_omap":0,"kb_used_meta":12672,"kb_avail":94358604,"statfs":{"total":96636764160,"available":96623210496,"internally_reserved":0,"allocated":569344,"data_stored":239562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12976128},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.48799999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.54600000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.58499999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.78200000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.91400000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.80000000000000004}]},{"osd":7,"last update":"Tue Nov 28 13:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.75900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9064,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362776,"statfs":{"total":96636764160,"available":96627482624,"internally_reserved":0,"allocated":557056,"data_stored":238558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.81000000000000005}]},{"osd":1,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.54300000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.75800000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.71599999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.78400000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.83999999999999997}]},{"osd":7,"last update":"Tue Nov 28 13:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.85799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8996,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362844,"statfs":{"total":96636764160,"available":96627552256,"internally_reserved":0,"allocated":552960,"data_stored":237992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.627},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48599999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.52600000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.69599999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.64700000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.71799999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.505}]},{"osd":7,"last update":"Tue Nov 28 13:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:09:57.094 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T13:09:57.095 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T13:09:57.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.332+0000 7f56b37ff700 1 Processor -- start 2023-11-28T13:09:57.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.333+0000 7f56b37ff700 1 -- start start 2023-11-28T13:09:57.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.334+0000 7f56b37ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56ac15c9b0 0x7f56ac0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:57.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.334+0000 7f56b37ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f56ac0cd7c0 0x7f56ac0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:57.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.334+0000 7f56b37ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ac0ce160 con 0x7f56ac161340 2023-11-28T13:09:57.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.334+0000 7f56b37ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ac0b5800 con 0x7f56ac0cd7c0 2023-11-28T13:09:57.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.334+0000 7f56b37ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac0d1950 con 0x7f56ac15c9b0 2023-11-28T13:09:57.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.334+0000 7f56b0d9a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f56ac161340 0x7f56ac0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53338/0 (socket says 172.21.15.156:53338) 2023-11-28T13:09:57.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.334+0000 7f56b0d9a700 1 -- 172.21.15.156:0/3084160427 learned_addr learned my addr 172.21.15.156:0/3084160427 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:57.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.334+0000 7f56b1d9c700 1 --2- 172.21.15.156:0/3084160427 >> v2:172.21.15.186:3300/0 conn(0x7f56ac0cd7c0 0x7f56ac0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:57.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.334+0000 7f56b159b700 1 --2- 172.21.15.156:0/3084160427 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56ac15c9b0 0x7f56ac0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:57.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.334+0000 7f56a3fff700 1 -- 172.21.15.156:0/3084160427 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 702321529 0 0) 0x7f56ac0ce160 con 0x7f56ac161340 2023-11-28T13:09:57.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56a3fff700 1 -- 172.21.15.156:0/3084160427 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56980035b0 con 0x7f56ac161340 2023-11-28T13:09:57.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56a3fff700 1 -- 172.21.15.156:0/3084160427 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3720121950 0 0) 0x7f56980035b0 con 0x7f56ac161340 2023-11-28T13:09:57.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56a3fff700 1 -- 172.21.15.156:0/3084160427 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56ac15c9b0 msgr2=0x7f56ac0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:57.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56a3fff700 1 --2- 172.21.15.156:0/3084160427 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56ac15c9b0 0x7f56ac0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:57.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56a3fff700 1 -- 172.21.15.156:0/3084160427 >> v2:172.21.15.186:3300/0 conn(0x7f56ac0cd7c0 msgr2=0x7f56ac0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:57.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56a3fff700 1 --2- 172.21.15.156:0/3084160427 >> v2:172.21.15.186:3300/0 conn(0x7f56ac0cd7c0 0x7f56ac0cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:57.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56a3fff700 1 -- 172.21.15.156:0/3084160427 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac0d24f0 con 0x7f56ac161340 2023-11-28T13:09:57.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56a3fff700 1 -- 172.21.15.156:0/3084160427 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f569c001670 con 0x7f56ac161340 2023-11-28T13:09:57.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56b1d9c700 1 --2- 172.21.15.156:0/3084160427 >> v2:172.21.15.186:3300/0 conn(0x7f56ac0cd7c0 0x7f56ac0cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:09:57.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56a3fff700 1 -- 172.21.15.156:0/3084160427 >> v1:172.21.15.156:6789/0 conn(0x7f56ac161340 legacy=0x7f56ac0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:57.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56a3fff700 1 --2- 172.21.15.156:0/3084160427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5698003a00 0x7f5698003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:57.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56a3fff700 1 --2- 172.21.15.156:0/3084160427 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698004430 0x7f56980048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:57.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56a3fff700 1 --2- 172.21.15.156:0/3084160427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5698004fb0 0x7f569800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:57.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56a3fff700 1 -- 172.21.15.156:0/3084160427 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac0b5800 con 0x7f5698003a00 2023-11-28T13:09:57.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56a3fff700 1 -- 172.21.15.156:0/3084160427 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac0d1950 con 0x7f5698004fb0 2023-11-28T13:09:57.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56a3fff700 1 -- 172.21.15.156:0/3084160427 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569800bf50 con 0x7f5698004430 2023-11-28T13:09:57.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56b159b700 1 --2- 172.21.15.156:0/3084160427 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f56ac15c9b0 0x7f56ac0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:09:57.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56b0d9a700 1 --2- 172.21.15.156:0/3084160427 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698004430 0x7f56980048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:57.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56b159b700 1 --2- 172.21.15.156:0/3084160427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5698003a00 0x7f5698003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:57.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.335+0000 7f56b1d9c700 1 --2- 172.21.15.156:0/3084160427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5698004fb0 0x7f569800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:57.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.336+0000 7f56b159b700 1 -- 172.21.15.156:0/3084160427 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698004430 msgr2=0x7f56980048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:57.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.336+0000 7f56b159b700 1 --2- 172.21.15.156:0/3084160427 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698004430 0x7f56980048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:57.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.336+0000 7f56b159b700 1 -- 172.21.15.156:0/3084160427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5698004fb0 msgr2=0x7f569800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:57.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.336+0000 7f56b159b700 1 --2- 172.21.15.156:0/3084160427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5698004fb0 0x7f569800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:57.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.336+0000 7f56b159b700 1 -- 172.21.15.156:0/3084160427 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56980035b0 con 0x7f5698003a00 2023-11-28T13:09:57.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.336+0000 7f56b0d9a700 1 --2- 172.21.15.156:0/3084160427 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698004430 0x7f56980048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:09:57.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.336+0000 7f56b159b700 1 --2- 172.21.15.156:0/3084160427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5698003a00 0x7f5698003e10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0d4270 tx=0x7f5698005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352a169f67bee270 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:57.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.336+0000 7f56a3fff700 1 -- 172.21.15.156:0/3084160427 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8004030 con 0x7f5698003a00 2023-11-28T13:09:57.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.336+0000 7f56a3fff700 1 -- 172.21.15.156:0/3084160427 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a8004190 con 0x7f5698003a00 2023-11-28T13:09:57.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.336+0000 7f56a3fff700 1 -- 172.21.15.156:0/3084160427 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8004540 con 0x7f5698003a00 2023-11-28T13:09:57.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.337+0000 7f56b37ff700 1 -- 172.21.15.156:0/3084160427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5698003a00 msgr2=0x7f5698003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:57.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.337+0000 7f56b37ff700 1 --2- 172.21.15.156:0/3084160427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5698003a00 0x7f5698003e10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0d4270 tx=0x7f5698005880 comp rx=0 tx=0).stop 2023-11-28T13:09:57.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.337+0000 7f56b1d9c700 1 -- 172.21.15.156:0/3084160427 reap_dead start 2023-11-28T13:09:57.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.337+0000 7f56b37ff700 1 -- 172.21.15.156:0/3084160427 shutdown_connections 2023-11-28T13:09:57.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.337+0000 7f56b37ff700 1 -- 172.21.15.156:0/3084160427 >> 172.21.15.156:0/3084160427 conn(0x7f56ac072600 msgr2=0x7f56ac0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:57.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.337+0000 7f56b37ff700 1 -- 172.21.15.156:0/3084160427 shutdown_connections 2023-11-28T13:09:57.344 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.337+0000 7f56b37ff700 1 -- 172.21.15.156:0/3084160427 wait complete. 2023-11-28T13:09:57.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.339+0000 7f56b37ff700 1 Processor -- start 2023-11-28T13:09:57.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.339+0000 7f56b37ff700 1 -- start start 2023-11-28T13:09:57.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.340+0000 7f56b37ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698003a00 0x7f56ac0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:57.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.340+0000 7f56b37ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0c8280 0x7f56ac0b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:57.347 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.340+0000 7f56b37ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56ac0b64e0 0x7f56ac149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:57.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.340+0000 7f56b37ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac0b6980 con 0x7f56ac0c8280 2023-11-28T13:09:57.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.340+0000 7f56b37ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac1576d0 con 0x7f56ac0b64e0 2023-11-28T13:09:57.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.340+0000 7f56b37ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac14a270 con 0x7f5698003a00 2023-11-28T13:09:57.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.340+0000 7f56b0d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0c8280 0x7f56ac0b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:57.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.340+0000 7f56b0d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0c8280 0x7f56ac0b5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46528/0 (socket says 172.21.15.156:46528) 2023-11-28T13:09:57.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.340+0000 7f56b0d9a700 1 -- 172.21.15.156:0/1740377496 learned_addr learned my addr 172.21.15.156:0/1740377496 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:57.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.340+0000 7f56b159b700 1 --2- 172.21.15.156:0/1740377496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698003a00 0x7f56ac0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:57.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.340+0000 7f56b1d9c700 1 --2- 172.21.15.156:0/1740377496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56ac0b64e0 0x7f56ac149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:57.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.340+0000 7f56b0d9a700 1 -- 172.21.15.156:0/1740377496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698003a00 msgr2=0x7f56ac0cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:57.348 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.340+0000 7f56b0d9a700 1 --2- 172.21.15.156:0/1740377496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698003a00 0x7f56ac0cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:57.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.340+0000 7f56b0d9a700 1 -- 172.21.15.156:0/1740377496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56ac0b64e0 msgr2=0x7f56ac149360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:57.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.340+0000 7f56b0d9a700 1 --2- 172.21.15.156:0/1740377496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56ac0b64e0 0x7f56ac149360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:57.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.340+0000 7f56b0d9a700 1 -- 172.21.15.156:0/1740377496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac0d1300 con 0x7f56ac0c8280 2023-11-28T13:09:57.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.341+0000 7f56b0d9a700 1 --2- 172.21.15.156:0/1740377496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0c8280 0x7f56ac0b5fa0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f569c007340 tx=0x7f569c007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:57.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.341+0000 7f56a1ffb700 1 -- 172.21.15.156:0/1740377496 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c0077e0 con 0x7f56ac0c8280 2023-11-28T13:09:57.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.341+0000 7f56a1ffb700 1 -- 172.21.15.156:0/1740377496 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569c00f870 con 0x7f56ac0c8280 2023-11-28T13:09:57.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.341+0000 7f56b37ff700 1 -- 172.21.15.156:0/1740377496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ac0b5520 con 0x7f56ac0c8280 2023-11-28T13:09:57.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.341+0000 7f56a1ffb700 1 -- 172.21.15.156:0/1740377496 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c00fbe0 con 0x7f56ac0c8280 2023-11-28T13:09:57.349 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.341+0000 7f56b37ff700 1 -- 172.21.15.156:0/1740377496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ac0d15e0 con 0x7f56ac0c8280 2023-11-28T13:09:57.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.342+0000 7f56b37ff700 1 -- 172.21.15.156:0/1740377496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ac0745c0 con 0x7f56ac0c8280 2023-11-28T13:09:57.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.343+0000 7f56a1ffb700 1 -- 172.21.15.156:0/1740377496 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f569c01c030 con 0x7f56ac0c8280 2023-11-28T13:09:57.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.344+0000 7f56a1ffb700 1 --2- 172.21.15.156:0/1740377496 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56900682f0 0x7f569006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:57.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.344+0000 7f56a1ffb700 1 -- 172.21.15.156:0/1740377496 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f569c085650 con 0x7f56ac0c8280 2023-11-28T13:09:57.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.344+0000 7f56b159b700 1 --2- 172.21.15.156:0/1740377496 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56900682f0 0x7f569006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:57.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.345+0000 7f56b159b700 1 --2- 172.21.15.156:0/1740377496 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56900682f0 0x7f569006a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f56a8016040 tx=0x7f56a801f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:57.350 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.348+0000 7f56a1ffb700 1 -- 172.21.15.156:0/1740377496 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f569c051eb0 con 0x7f56ac0c8280 2023-11-28T13:09:57.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.615+0000 7f56b37ff700 1 -- 172.21.15.156:0/1740377496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f56ac0d08a0 con 0x7f56ac0c8280 2023-11-28T13:09:57.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.619+0000 7f56a1ffb700 1 -- 172.21.15.156:0/1740377496 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 184+0+0 (secure 0 0 0) 0x7f569c0554d0 con 0x7f56ac0c8280 2023-11-28T13:09:57.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.621+0000 7f56b37ff700 1 -- 172.21.15.156:0/1740377496 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56900682f0 msgr2=0x7f569006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:57.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.621+0000 7f56b37ff700 1 --2- 172.21.15.156:0/1740377496 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56900682f0 0x7f569006a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f56a8016040 tx=0x7f56a801f040 comp rx=0 tx=0).stop 2023-11-28T13:09:57.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.621+0000 7f56b37ff700 1 -- 172.21.15.156:0/1740377496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0c8280 msgr2=0x7f56ac0b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:57.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.621+0000 7f56b37ff700 1 --2- 172.21.15.156:0/1740377496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0c8280 0x7f56ac0b5fa0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f569c007340 tx=0x7f569c007650 comp rx=0 tx=0).stop 2023-11-28T13:09:57.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.621+0000 7f56b37ff700 1 -- 172.21.15.156:0/1740377496 shutdown_connections 2023-11-28T13:09:57.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.621+0000 7f56b37ff700 1 --2- 172.21.15.156:0/1740377496 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f56900682f0 0x7f569006a7a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:57.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.621+0000 7f56b37ff700 1 --2- 172.21.15.156:0/1740377496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56ac0b64e0 0x7f56ac149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:57.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.621+0000 7f56b37ff700 1 --2- 172.21.15.156:0/1740377496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5698003a00 0x7f56ac0cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:57.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.621+0000 7f56b37ff700 1 --2- 172.21.15.156:0/1740377496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56ac0c8280 0x7f56ac0b5fa0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:57.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.621+0000 7f56b37ff700 1 -- 172.21.15.156:0/1740377496 >> 172.21.15.156:0/1740377496 conn(0x7f56ac072600 msgr2=0x7f56ac073190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:57.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.621+0000 7f56b37ff700 1 -- 172.21.15.156:0/1740377496 shutdown_connections 2023-11-28T13:09:57.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.621+0000 7f56b37ff700 1 -- 172.21.15.156:0/1740377496 wait complete. 2023-11-28T13:09:57.624 INFO:teuthology.orchestra.run.smithi156.stderr:updated caps for client.0 2023-11-28T13:09:57.642 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:09:57.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.837+0000 7fd8237ff700 1 Processor -- start 2023-11-28T13:09:57.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.838+0000 7fd8237ff700 1 -- start start 2023-11-28T13:09:57.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.839+0000 7fd8237ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd81c0cd7c0 0x7fd81c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:57.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.839+0000 7fd8237ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd81c0d1950 con 0x7fd81c15c9b0 2023-11-28T13:09:57.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.839+0000 7fd8237ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd81c0ce140 con 0x7fd81c161340 2023-11-28T13:09:57.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.839+0000 7fd8237ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0b5800 con 0x7fd81c0cd7c0 2023-11-28T13:09:57.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.839+0000 7fd82159b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd81c15c9b0 0x7fd81c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53342/0 (socket says 172.21.15.156:53342) 2023-11-28T13:09:57.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.839+0000 7fd82159b700 1 -- 172.21.15.156:0/2608139889 learned_addr learned my addr 172.21.15.156:0/2608139889 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:57.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.839+0000 7fd821d9c700 1 --2- 172.21.15.156:0/2608139889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd81c0cd7c0 0x7fd81c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:57.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.839+0000 7fd813fff700 1 -- 172.21.15.156:0/2608139889 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3681044439 0 0) 0x7fd81c0d1950 con 0x7fd81c15c9b0 2023-11-28T13:09:57.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.839+0000 7fd813fff700 1 -- 172.21.15.156:0/2608139889 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8080035b0 con 0x7fd81c15c9b0 2023-11-28T13:09:57.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.840+0000 7fd813fff700 1 -- 172.21.15.156:0/2608139889 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022891234 0 0) 0x7fd81c0ce140 con 0x7fd81c161340 2023-11-28T13:09:57.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.840+0000 7fd813fff700 1 -- 172.21.15.156:0/2608139889 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd81c0d1950 con 0x7fd81c161340 2023-11-28T13:09:57.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.840+0000 7fd821d9c700 1 -- 172.21.15.156:0/2608139889 >> v1:172.21.15.186:6789/0 conn(0x7fd81c161340 legacy=0x7fd81c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:57.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.840+0000 7fd821d9c700 1 -- 172.21.15.156:0/2608139889 >> v1:172.21.15.156:6789/0 conn(0x7fd81c15c9b0 legacy=0x7fd81c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:57.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.840+0000 7fd821d9c700 1 -- 172.21.15.156:0/2608139889 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c0d24f0 con 0x7fd81c0cd7c0 2023-11-28T13:09:57.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.840+0000 7fd821d9c700 1 --2- 172.21.15.156:0/2608139889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd81c0cd7c0 0x7fd81c0cdc00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd814009dc0 tx=0x7fd81400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fae6beff1a73c7db server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:57.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.840+0000 7fd813fff700 1 -- 172.21.15.156:0/2608139889 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81400e070 con 0x7fd81c0cd7c0 2023-11-28T13:09:57.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.840+0000 7fd813fff700 1 -- 172.21.15.156:0/2608139889 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81400bb20 con 0x7fd81c0cd7c0 2023-11-28T13:09:57.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.840+0000 7fd813fff700 1 -- 172.21.15.156:0/2608139889 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81400be90 con 0x7fd81c0cd7c0 2023-11-28T13:09:57.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.841+0000 7fd8237ff700 1 -- 172.21.15.156:0/2608139889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd81c0cd7c0 msgr2=0x7fd81c0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:57.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.841+0000 7fd8237ff700 1 --2- 172.21.15.156:0/2608139889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd81c0cd7c0 0x7fd81c0cdc00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd814009dc0 tx=0x7fd81400b330 comp rx=0 tx=0).stop 2023-11-28T13:09:57.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.841+0000 7fd8237ff700 1 -- 172.21.15.156:0/2608139889 shutdown_connections 2023-11-28T13:09:57.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.841+0000 7fd8237ff700 1 --2- 172.21.15.156:0/2608139889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd81c0cd7c0 0x7fd81c0cdc00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:57.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.841+0000 7fd8237ff700 1 -- 172.21.15.156:0/2608139889 >> 172.21.15.156:0/2608139889 conn(0x7fd81c072600 msgr2=0x7fd81c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:57.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.841+0000 7fd8237ff700 1 -- 172.21.15.156:0/2608139889 shutdown_connections 2023-11-28T13:09:57.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.841+0000 7fd8237ff700 1 -- 172.21.15.156:0/2608139889 wait complete. 2023-11-28T13:09:57.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.843+0000 7fd8237ff700 1 Processor -- start 2023-11-28T13:09:57.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.843+0000 7fd8237ff700 1 -- start start 2023-11-28T13:09:57.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.844+0000 7fd8237ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd81c0cd7c0 0x7fd81c1570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:57.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.844+0000 7fd8237ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd81c15c9b0 0x7fd81c0cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:57.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.844+0000 7fd8237ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd81c161340 0x7fd81c1568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:57.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.844+0000 7fd8237ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0cc1a0 con 0x7fd81c15c9b0 2023-11-28T13:09:57.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.844+0000 7fd8237ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0cb190 con 0x7fd81c0cd7c0 2023-11-28T13:09:57.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.844+0000 7fd8237ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0cb590 con 0x7fd81c161340 2023-11-28T13:09:57.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.844+0000 7fd820d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd81c15c9b0 0x7fd81c0cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:57.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.844+0000 7fd820d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd81c15c9b0 0x7fd81c0cac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46544/0 (socket says 172.21.15.156:46544) 2023-11-28T13:09:57.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.844+0000 7fd820d9a700 1 -- 172.21.15.156:0/1864715768 learned_addr learned my addr 172.21.15.156:0/1864715768 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:57.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.844+0000 7fd820d9a700 1 -- 172.21.15.156:0/1864715768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd81c161340 msgr2=0x7fd81c1568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:57.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.844+0000 7fd820d9a700 1 --2- 172.21.15.156:0/1864715768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd81c161340 0x7fd81c1568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:57.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.844+0000 7fd820d9a700 1 -- 172.21.15.156:0/1864715768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd81c0cd7c0 msgr2=0x7fd81c1570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:57.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.844+0000 7fd820d9a700 1 --2- 172.21.15.156:0/1864715768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd81c0cd7c0 0x7fd81c1570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:57.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.844+0000 7fd820d9a700 1 -- 172.21.15.156:0/1864715768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd814009a70 con 0x7fd81c15c9b0 2023-11-28T13:09:57.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.845+0000 7fd820d9a700 1 --2- 172.21.15.156:0/1864715768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd81c15c9b0 0x7fd81c0cac50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00c530 tx=0x7fd80c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:57.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.845+0000 7fd811ffb700 1 -- 172.21.15.156:0/1864715768 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c0052d0 con 0x7fd81c15c9b0 2023-11-28T13:09:57.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.845+0000 7fd811ffb700 1 -- 172.21.15.156:0/1864715768 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80c005430 con 0x7fd81c15c9b0 2023-11-28T13:09:57.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.845+0000 7fd8237ff700 1 -- 172.21.15.156:0/1864715768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd81c0d1300 con 0x7fd81c15c9b0 2023-11-28T13:09:57.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.845+0000 7fd811ffb700 1 -- 172.21.15.156:0/1864715768 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c0038c0 con 0x7fd81c15c9b0 2023-11-28T13:09:57.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.845+0000 7fd8237ff700 1 -- 172.21.15.156:0/1864715768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81c0b5520 con 0x7fd81c15c9b0 2023-11-28T13:09:57.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.846+0000 7fd8237ff700 1 -- 172.21.15.156:0/1864715768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81c0d15e0 con 0x7fd81c15c9b0 2023-11-28T13:09:57.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.847+0000 7fd811ffb700 1 -- 172.21.15.156:0/1864715768 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd80c019030 con 0x7fd81c15c9b0 2023-11-28T13:09:57.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.848+0000 7fd811ffb700 1 --2- 172.21.15.156:0/1864715768 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd8000682f0 0x7fd80006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:57.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.848+0000 7fd811ffb700 1 -- 172.21.15.156:0/1864715768 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd80c085150 con 0x7fd81c15c9b0 2023-11-28T13:09:57.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.849+0000 7fd82159b700 1 --2- 172.21.15.156:0/1864715768 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd8000682f0 0x7fd80006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:57.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.849+0000 7fd82159b700 1 --2- 172.21.15.156:0/1864715768 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd8000682f0 0x7fd80006a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd818001610 tx=0x7fd8180079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:57.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:57.852+0000 7fd811ffb700 1 -- 172.21.15.156:0/1864715768 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd80c0518c0 con 0x7fd81c15c9b0 2023-11-28T13:09:58.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.086+0000 7fd8237ff700 1 -- 172.21.15.156:0/1864715768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd81c0745c0 con 0x7fd81c15c9b0 2023-11-28T13:09:58.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.087+0000 7fd811ffb700 1 -- 172.21.15.156:0/1864715768 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fd80c054ee0 con 0x7fd81c15c9b0 2023-11-28T13:09:58.089 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:09:58.089 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3828141874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3828141874},{"type":"v1","addr":"172.21.15.156:6835","nonce":3828141874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1012489762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1012489762},{"type":"v1","addr":"172.21.15.186:6837","nonce":1012489762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2894670074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2894670074},{"type":"v1","addr":"172.21.15.186:6835","nonce":2894670074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:09:52.245032+0000","modified":"2023-11-28T13:09:55.251254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5481":{"gid":5481,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6834/1930782798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1930782798},{"type":"v1","addr":"172.21.15.156:6834","nonce":1930782798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:09:58.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.089+0000 7fd8237ff700 1 -- 172.21.15.156:0/1864715768 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd8000682f0 msgr2=0x7fd80006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:58.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.089+0000 7fd8237ff700 1 --2- 172.21.15.156:0/1864715768 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd8000682f0 0x7fd80006a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd818001610 tx=0x7fd8180079c0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.089+0000 7fd8237ff700 1 -- 172.21.15.156:0/1864715768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd81c15c9b0 msgr2=0x7fd81c0cac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:58.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.089+0000 7fd8237ff700 1 --2- 172.21.15.156:0/1864715768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd81c15c9b0 0x7fd81c0cac50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00c530 tx=0x7fd80c00c8f0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.089+0000 7fd8237ff700 1 -- 172.21.15.156:0/1864715768 shutdown_connections 2023-11-28T13:09:58.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.089+0000 7fd8237ff700 1 --2- 172.21.15.156:0/1864715768 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd8000682f0 0x7fd80006a7a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.089+0000 7fd8237ff700 1 --2- 172.21.15.156:0/1864715768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd81c161340 0x7fd81c1568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.089+0000 7fd8237ff700 1 --2- 172.21.15.156:0/1864715768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd81c0cd7c0 0x7fd81c1570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.089+0000 7fd8237ff700 1 --2- 172.21.15.156:0/1864715768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd81c15c9b0 0x7fd81c0cac50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.089+0000 7fd8237ff700 1 -- 172.21.15.156:0/1864715768 >> 172.21.15.156:0/1864715768 conn(0x7fd81c072600 msgr2=0x7fd81c0b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:58.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.089+0000 7fd8237ff700 1 -- 172.21.15.156:0/1864715768 shutdown_connections 2023-11-28T13:09:58.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.089+0000 7fd8237ff700 1 -- 172.21.15.156:0/1864715768 wait complete. 2023-11-28T13:09:58.093 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 23 2023-11-28T13:09:58.512 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T13:09:52.245032+0000', 'modified': '2023-11-28T13:09:55.251254+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5481}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5481': {'gid': 5481, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.156:6834/1930782798', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6832', 'nonce': 1930782798}, {'type': 'v1', 'addr': '172.21.15.156:6834', 'nonce': 1930782798}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T13:09:58.512 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T13:09:58.512 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T13:09:58.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.712+0000 7eff2e1ff700 1 Processor -- start 2023-11-28T13:09:58.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.712+0000 7eff2e1ff700 1 -- start start 2023-11-28T13:09:58.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.713+0000 7eff2e1ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7eff2815c9b0 0x7eff280b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.713+0000 7eff2e1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff28161340 0x7eff280d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.713+0000 7eff2e1ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff280d1950 con 0x7eff280d2320 2023-11-28T13:09:58.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.713+0000 7eff2e1ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff280b5800 con 0x7eff2815c9b0 2023-11-28T13:09:58.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.713+0000 7eff2e1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff280b5520 con 0x7eff28161340 2023-11-28T13:09:58.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.714+0000 7eff277fe700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7eff2815c9b0 0x7eff280b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.714+0000 7eff26ffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff28161340 0x7eff280d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.714+0000 7eff277fe700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7eff2815c9b0 0x7eff280b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:50884/0 (socket says 172.21.15.156:50884) 2023-11-28T13:09:58.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.714+0000 7eff277fe700 1 -- 172.21.15.156:0/1583125020 learned_addr learned my addr 172.21.15.156:0/1583125020 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:58.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.714+0000 7eff277fe700 1 -- 172.21.15.156:0/1583125020 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff28161340 msgr2=0x7eff280d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:58.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.714+0000 7eff277fe700 1 --2- 172.21.15.156:0/1583125020 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff28161340 0x7eff280d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.714+0000 7eff277fe700 1 -- 172.21.15.156:0/1583125020 >> v1:172.21.15.186:6789/0 conn(0x7eff280d2320 legacy=0x7eff280cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:58.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.715+0000 7eff277fe700 1 -- 172.21.15.156:0/1583125020 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff280d1300 con 0x7eff2815c9b0 2023-11-28T13:09:58.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.715+0000 7eff277fe700 1 --2- 172.21.15.156:0/1583125020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff2815c9b0 0x7eff280b4bf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7eff180098f0 tx=0x7eff18009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d582dc972eb8b5cb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:58.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.715+0000 7eff267fc700 1 -- 172.21.15.156:0/1583125020 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18011830 con 0x7eff2815c9b0 2023-11-28T13:09:58.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.715+0000 7eff267fc700 1 -- 172.21.15.156:0/1583125020 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff18011990 con 0x7eff2815c9b0 2023-11-28T13:09:58.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.715+0000 7eff267fc700 1 -- 172.21.15.156:0/1583125020 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18011d20 con 0x7eff2815c9b0 2023-11-28T13:09:58.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.716+0000 7eff2e1ff700 1 -- 172.21.15.156:0/1583125020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff2815c9b0 msgr2=0x7eff280b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:58.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.716+0000 7eff2e1ff700 1 --2- 172.21.15.156:0/1583125020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff2815c9b0 0x7eff280b4bf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7eff180098f0 tx=0x7eff18009c00 comp rx=0 tx=0).stop 2023-11-28T13:09:58.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.716+0000 7eff2e1ff700 1 -- 172.21.15.156:0/1583125020 shutdown_connections 2023-11-28T13:09:58.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.716+0000 7eff2e1ff700 1 --2- 172.21.15.156:0/1583125020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff2815c9b0 0x7eff280b4bf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.716+0000 7eff2e1ff700 1 --2- 172.21.15.156:0/1583125020 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff28161340 0x7eff280d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.716+0000 7eff2e1ff700 1 -- 172.21.15.156:0/1583125020 >> 172.21.15.156:0/1583125020 conn(0x7eff28072600 msgr2=0x7eff280cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:58.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.716+0000 7eff2e1ff700 1 -- 172.21.15.156:0/1583125020 shutdown_connections 2023-11-28T13:09:58.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.716+0000 7eff2e1ff700 1 -- 172.21.15.156:0/1583125020 wait complete. 2023-11-28T13:09:58.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.718+0000 7eff2e1ff700 1 Processor -- start 2023-11-28T13:09:58.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.718+0000 7eff2e1ff700 1 -- start start 2023-11-28T13:09:58.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.719+0000 7eff2e1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff280d2320 0x7eff28151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.719+0000 7eff2e1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff2815c9b0 0x7eff28150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.719+0000 7eff2e1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff28161340 0x7eff281e8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.719+0000 7eff2e1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff28150760 con 0x7eff28161340 2023-11-28T13:09:58.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.719+0000 7eff2e1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff28154af0 con 0x7eff2815c9b0 2023-11-28T13:09:58.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.719+0000 7eff2e1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff28154dd0 con 0x7eff280d2320 2023-11-28T13:09:58.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.719+0000 7eff27fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff28161340 0x7eff281e8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.719+0000 7eff27fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff28161340 0x7eff281e8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46558/0 (socket says 172.21.15.156:46558) 2023-11-28T13:09:58.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.719+0000 7eff27fff700 1 -- 172.21.15.156:0/2745528838 learned_addr learned my addr 172.21.15.156:0/2745528838 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:58.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.719+0000 7eff26ffd700 1 --2- 172.21.15.156:0/2745528838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff2815c9b0 0x7eff28150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.719+0000 7eff277fe700 1 --2- 172.21.15.156:0/2745528838 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff280d2320 0x7eff28151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.719+0000 7eff27fff700 1 -- 172.21.15.156:0/2745528838 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff280d2320 msgr2=0x7eff28151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.719+0000 7eff27fff700 1 --2- 172.21.15.156:0/2745528838 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff280d2320 0x7eff28151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.719+0000 7eff27fff700 1 -- 172.21.15.156:0/2745528838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff2815c9b0 msgr2=0x7eff28150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.719+0000 7eff27fff700 1 --2- 172.21.15.156:0/2745528838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff2815c9b0 0x7eff28150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.719+0000 7eff27fff700 1 -- 172.21.15.156:0/2745528838 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff18009550 con 0x7eff28161340 2023-11-28T13:09:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.720+0000 7eff27fff700 1 --2- 172.21.15.156:0/2745528838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff28161340 0x7eff281e8430 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7eff2000cb50 tx=0x7eff2000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.720+0000 7eff0ffff700 1 -- 172.21.15.156:0/2745528838 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff20003c00 con 0x7eff28161340 2023-11-28T13:09:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.720+0000 7eff0ffff700 1 -- 172.21.15.156:0/2745528838 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff20003d60 con 0x7eff28161340 2023-11-28T13:09:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.720+0000 7eff2e1ff700 1 -- 172.21.15.156:0/2745528838 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff280b5130 con 0x7eff28161340 2023-11-28T13:09:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.720+0000 7eff0ffff700 1 -- 172.21.15.156:0/2745528838 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2000aa20 con 0x7eff28161340 2023-11-28T13:09:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.720+0000 7eff2e1ff700 1 -- 172.21.15.156:0/2745528838 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff280d15e0 con 0x7eff28161340 2023-11-28T13:09:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.722+0000 7eff0ffff700 1 -- 172.21.15.156:0/2745528838 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7eff20010050 con 0x7eff28161340 2023-11-28T13:09:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.723+0000 7eff0ffff700 1 --2- 172.21.15.156:0/2745528838 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7eff140682f0 0x7eff1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.723+0000 7eff0ffff700 1 -- 172.21.15.156:0/2745528838 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7eff20085b00 con 0x7eff28161340 2023-11-28T13:09:58.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.723+0000 7eff277fe700 1 --2- 172.21.15.156:0/2745528838 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7eff140682f0 0x7eff1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.724+0000 7eff277fe700 1 --2- 172.21.15.156:0/2745528838 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7eff140682f0 0x7eff1406a7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7eff18009c40 tx=0x7eff18008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:58.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.725+0000 7eff41a2bb80 1 Processor -- start 2023-11-28T13:09:58.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.725+0000 7eff41a2bb80 1 -- start start 2023-11-28T13:09:58.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.726+0000 7eff41a2bb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b5b810 0x55cdf1b31a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.726+0000 7eff41a2bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b59130 0x55cdf1b5bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.726+0000 7eff41a2bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5c120 0x55cdf1b77770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.726+0000 7eff41a2bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf1a1bbc0 con 0x55cdf1b5c120 2023-11-28T13:09:58.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.726+0000 7eff41a2bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf1a8a350 con 0x55cdf1b59130 2023-11-28T13:09:58.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.726+0000 7eff41a2bb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdf1a83880 con 0x55cdf1b5b810 2023-11-28T13:09:58.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.727+0000 7eff27fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5c120 0x55cdf1b77770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.727+0000 7eff27fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5c120 0x55cdf1b77770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46564/0 (socket says 172.21.15.156:46564) 2023-11-28T13:09:58.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.727+0000 7eff27fff700 1 -- 172.21.15.156:0/360213854 learned_addr learned my addr 172.21.15.156:0/360213854 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:58.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.727+0000 7eff27fff700 1 -- 172.21.15.156:0/360213854 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b5b810 msgr2=0x55cdf1b31a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:58.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.727+0000 7eff27fff700 1 --2- 172.21.15.156:0/360213854 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b5b810 0x55cdf1b31a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.727+0000 7eff27fff700 1 -- 172.21.15.156:0/360213854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b59130 msgr2=0x55cdf1b5bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:58.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.727+0000 7eff27fff700 1 --2- 172.21.15.156:0/360213854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b59130 0x55cdf1b5bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.727+0000 7eff27fff700 1 -- 172.21.15.156:0/360213854 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdf19055e0 con 0x55cdf1b5c120 2023-11-28T13:09:58.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.727+0000 7eff27fff700 1 --2- 172.21.15.156:0/360213854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5c120 0x55cdf1b77770 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7eff2005e1b0 tx=0x7eff2005e1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fe49f05d5b3c68 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:58.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.727+0000 7eff0d7fa700 1 -- 172.21.15.156:0/360213854 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff200544e0 con 0x55cdf1b5c120 2023-11-28T13:09:58.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.728+0000 7eff0d7fa700 1 -- 172.21.15.156:0/360213854 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2006d1c0 con 0x55cdf1b5c120 2023-11-28T13:09:58.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.728+0000 7eff0d7fa700 1 -- 172.21.15.156:0/360213854 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2005de20 con 0x55cdf1b5c120 2023-11-28T13:09:58.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.728+0000 7eff41a2bb80 1 -- 172.21.15.156:0/360213854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5c120 msgr2=0x55cdf1b77770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:58.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.728+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/360213854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5c120 0x55cdf1b77770 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7eff2005e1b0 tx=0x7eff2005e1e0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.728+0000 7eff41a2bb80 1 -- 172.21.15.156:0/360213854 shutdown_connections 2023-11-28T13:09:58.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.728+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/360213854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5c120 0x55cdf1b77770 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.728+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/360213854 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b5b810 0x55cdf1b31a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.728+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/360213854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b59130 0x55cdf1b5bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.728+0000 7eff41a2bb80 1 -- 172.21.15.156:0/360213854 >> 172.21.15.156:0/360213854 conn(0x55cdf1b2ed60 msgr2=0x55cdf1b2f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:58.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.728+0000 7eff41a2bb80 1 -- 172.21.15.156:0/360213854 shutdown_connections 2023-11-28T13:09:58.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.728+0000 7eff41a2bb80 1 -- 172.21.15.156:0/360213854 wait complete. 2023-11-28T13:09:58.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.729+0000 7eff41a2bb80 1 -- start start 2023-11-28T13:09:58.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.729+0000 7eff41a2bb80 1 Processor -- start 2023-11-28T13:09:58.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.730+0000 7eff41a2bb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b59130 0x55cdf1b408d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.730+0000 7eff41a2bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1b40e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.730+0000 7eff41a2bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b5c120 0x55cdf1b5a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.730+0000 7eff41a2bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf19078d0 con 0x55cdf1b5b810 2023-11-28T13:09:58.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.730+0000 7eff41a2bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf1b418c0 con 0x55cdf1b5c120 2023-11-28T13:09:58.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.730+0000 7eff41a2bb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdf1b6fb10 con 0x55cdf1b59130 2023-11-28T13:09:58.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.730+0000 7eff41a2bb80 10 client.0 _ll_register_callbacks cb 0x55cdf1929240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:09:58.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.731+0000 7eff26ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1b40e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.731+0000 7eff26ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1b40e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46578/0 (socket says 172.21.15.156:46578) 2023-11-28T13:09:58.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.731+0000 7eff26ffd700 1 -- 172.21.15.156:0/4286723872 learned_addr learned my addr 172.21.15.156:0/4286723872 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:58.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.731+0000 7eff27fff700 1 --2- 172.21.15.156:0/4286723872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b5c120 0x55cdf1b5a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.731+0000 7eff277fe700 1 --2- 172.21.15.156:0/4286723872 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b59130 0x55cdf1b408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.731+0000 7eff26ffd700 1 -- 172.21.15.156:0/4286723872 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b59130 msgr2=0x55cdf1b408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:58.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.731+0000 7eff26ffd700 1 --2- 172.21.15.156:0/4286723872 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b59130 0x55cdf1b408d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.731+0000 7eff26ffd700 1 -- 172.21.15.156:0/4286723872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b5c120 msgr2=0x55cdf1b5a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:58.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.731+0000 7eff26ffd700 1 --2- 172.21.15.156:0/4286723872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b5c120 0x55cdf1b5a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.731+0000 7eff26ffd700 1 -- 172.21.15.156:0/4286723872 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff280d15e0 con 0x55cdf1b5b810 2023-11-28T13:09:58.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.731+0000 7eff277fe700 1 --2- 172.21.15.156:0/4286723872 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b59130 0x55cdf1b408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:09:58.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.731+0000 7eff26ffd700 1 --2- 172.21.15.156:0/4286723872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1b40e10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7eff1c011590 tx=0x7eff1c0118a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7a0bf18d65b27d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:58.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.731+0000 7eff067fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:09:58.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.731+0000 7eff067fc700 1 -- 172.21.15.156:0/4286723872 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c004240 con 0x55cdf1b5b810 2023-11-28T13:09:58.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.731+0000 7eff067fc700 1 -- 172.21.15.156:0/4286723872 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7eff1c0043a0 con 0x55cdf1b5b810 2023-11-28T13:09:58.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.732+0000 7eff41a2bb80 1 -- 172.21.15.156:0/4286723872 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdf1a51140 con 0x55cdf1b5b810 2023-11-28T13:09:58.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.732+0000 7eff067fc700 1 -- 172.21.15.156:0/4286723872 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff1c004560 con 0x55cdf1b5b810 2023-11-28T13:09:58.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.732+0000 7eff41a2bb80 10 client.5625 fetch_fsmap learned FSMap version 23 2023-11-28T13:09:58.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.732+0000 7eff41a2bb80 1 -- 172.21.15.156:0/4286723872 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55cdf18b1d90 con 0x55cdf1b5b810 2023-11-28T13:09:58.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.732+0000 7eff067fc700 1 -- 172.21.15.156:0/4286723872 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cdf1a51140 con 0x55cdf1b5b810 2023-11-28T13:09:58.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.732+0000 7eff067fc700 1 -- 172.21.15.156:0/4286723872 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7eff1c0036a0 con 0x55cdf1b5b810 2023-11-28T13:09:58.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.732+0000 7eff41a2bb80 10 client.5625 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T13:09:58.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.732+0000 7eff41a2bb80 10 client.5625 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:09:58.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.732+0000 7eff41a2bb80 20 client.5625 populate_metadata read hostname 'smithi156' 2023-11-28T13:09:58.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.732+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/4286723872 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55cdf1b7acd0 0x55cdf1b70360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.732+0000 7eff41a2bb80 4 client.5625 mds_command: new command op to 5481 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:09:58.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.732+0000 7eff41a2bb80 1 -- 172.21.15.156:0/4286723872 --> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdf1907f10 con 0x55cdf1b7acd0 2023-11-28T13:09:58.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.732+0000 7eff277fe700 1 --2- 172.21.15.156:0/4286723872 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55cdf1b7acd0 0x55cdf1b70360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.733+0000 7eff277fe700 1 --2- 172.21.15.156:0/4286723872 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55cdf1b7acd0 0x55cdf1b70360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bba84fb95f75c5cd server_cookie=3d9908a31da8ec3f in_seq=0 out_seq=0 2023-11-28T13:09:58.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.733+0000 7eff067fc700 10 client.5625 ms_handle_connect on v2:172.21.15.156:6832/1930782798 2023-11-28T13:09:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.734+0000 7eff067fc700 1 -- 172.21.15.156:0/4286723872 <== mds.0 v2:172.21.15.156:6832/1930782798 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55cdf1907f10 con 0x55cdf1b7acd0 2023-11-28T13:09:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.734+0000 7eff067fc700 10 client.5625 handle_command_reply: tid=0 2023-11-28T13:09:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.734+0000 7eff41a2bb80 1 client.5625 shutdown 2023-11-28T13:09:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.734+0000 7eff41a2bb80 1 -- 172.21.15.156:0/4286723872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 msgr2=0x55cdf1b40e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.734+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/4286723872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1b40e10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7eff1c011590 tx=0x7eff1c0118a0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.734+0000 7eff41a2bb80 1 -- 172.21.15.156:0/4286723872 shutdown_connections 2023-11-28T13:09:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.734+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/4286723872 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55cdf1b7acd0 0x55cdf1b70360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.734+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/4286723872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b5c120 0x55cdf1b5a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.734+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/4286723872 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b59130 0x55cdf1b408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.734+0000 7eff067fc700 0 client.5625 ms_handle_reset on v2:172.21.15.156:6832/1930782798 2023-11-28T13:09:58.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.734+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/4286723872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1b40e10 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.734+0000 7eff41a2bb80 1 -- 172.21.15.156:0/4286723872 >> 172.21.15.156:0/4286723872 conn(0x55cdf1b2ed60 msgr2=0x55cdf1b2f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:58.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.734+0000 7eff41a2bb80 1 -- 172.21.15.156:0/4286723872 shutdown_connections 2023-11-28T13:09:58.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.734+0000 7eff41a2bb80 1 -- 172.21.15.156:0/4286723872 wait complete. 2023-11-28T13:09:58.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.734+0000 7eff41a2bb80 20 client.5625 trim_cache size 0 max 16384 2023-11-28T13:09:58.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.752+0000 7eff41a2bb80 1 Processor -- start 2023-11-28T13:09:58.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.752+0000 7eff41a2bb80 1 -- start start 2023-11-28T13:09:58.755 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.753+0000 7eff41a2bb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b59130 0x55cdf1b5a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.755 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.753+0000 7eff41a2bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1b40a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.753+0000 7eff41a2bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b5c120 0x55cdf1b2c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.753+0000 7eff41a2bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf1b7d040 con 0x55cdf1b5b810 2023-11-28T13:09:58.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.753+0000 7eff41a2bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf1b2c780 con 0x55cdf1b5c120 2023-11-28T13:09:58.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.753+0000 7eff41a2bb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdf1b51de0 con 0x55cdf1b59130 2023-11-28T13:09:58.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.753+0000 7eff26ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1b40a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.753+0000 7eff26ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1b40a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46582/0 (socket says 172.21.15.156:46582) 2023-11-28T13:09:58.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.753+0000 7eff26ffd700 1 -- 172.21.15.156:0/2247134697 learned_addr learned my addr 172.21.15.156:0/2247134697 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:58.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.753+0000 7eff27fff700 1 --2- 172.21.15.156:0/2247134697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b5c120 0x55cdf1b2c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.753+0000 7eff277fe700 1 --2- 172.21.15.156:0/2247134697 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b59130 0x55cdf1b5a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.754+0000 7eff26ffd700 1 -- 172.21.15.156:0/2247134697 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b59130 msgr2=0x55cdf1b5a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:58.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.754+0000 7eff26ffd700 1 --2- 172.21.15.156:0/2247134697 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b59130 0x55cdf1b5a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.754+0000 7eff26ffd700 1 -- 172.21.15.156:0/2247134697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b5c120 msgr2=0x55cdf1b2c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:58.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.754+0000 7eff26ffd700 1 --2- 172.21.15.156:0/2247134697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b5c120 0x55cdf1b2c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.754+0000 7eff26ffd700 1 -- 172.21.15.156:0/2247134697 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdf1a4a140 con 0x55cdf1b5b810 2023-11-28T13:09:58.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.754+0000 7eff26ffd700 1 --2- 172.21.15.156:0/2247134697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1b40a70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7eff1c006c90 tx=0x7eff1c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1226ee26ab1edee6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:58.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.754+0000 7eff0dffb700 1 -- 172.21.15.156:0/2247134697 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c003c00 con 0x55cdf1b5b810 2023-11-28T13:09:58.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.754+0000 7eff0dffb700 1 -- 172.21.15.156:0/2247134697 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1c0036a0 con 0x55cdf1b5b810 2023-11-28T13:09:58.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.754+0000 7eff0dffb700 1 -- 172.21.15.156:0/2247134697 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c004440 con 0x55cdf1b5b810 2023-11-28T13:09:58.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.755+0000 7eff41a2bb80 1 -- 172.21.15.156:0/2247134697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 msgr2=0x55cdf1b40a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:58.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.755+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/2247134697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1b40a70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7eff1c006c90 tx=0x7eff1c016000 comp rx=0 tx=0).stop 2023-11-28T13:09:58.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.755+0000 7eff41a2bb80 1 -- 172.21.15.156:0/2247134697 shutdown_connections 2023-11-28T13:09:58.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.755+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/2247134697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b5c120 0x55cdf1b2c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.755+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/2247134697 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b59130 0x55cdf1b5a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.755+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/2247134697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1b40a70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.755+0000 7eff41a2bb80 1 -- 172.21.15.156:0/2247134697 >> 172.21.15.156:0/2247134697 conn(0x55cdf1b2ed60 msgr2=0x55cdf1b70360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:58.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.755+0000 7eff41a2bb80 1 -- 172.21.15.156:0/2247134697 shutdown_connections 2023-11-28T13:09:58.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.755+0000 7eff41a2bb80 1 -- 172.21.15.156:0/2247134697 wait complete. 2023-11-28T13:09:58.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.755+0000 7eff41a2bb80 1 -- start start 2023-11-28T13:09:58.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.756+0000 7eff41a2bb80 1 Processor -- start 2023-11-28T13:09:58.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff41a2bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b59130 0x55cdf1b5e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff41a2bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1bdb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff41a2bb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b5c120 0x55cdf1b6ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff41a2bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf1b5f790 con 0x55cdf1b5b810 2023-11-28T13:09:58.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff41a2bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf1b5f150 con 0x55cdf1b59130 2023-11-28T13:09:58.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff41a2bb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdf1b79470 con 0x55cdf1b5c120 2023-11-28T13:09:58.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff41a2bb80 10 client.0 _ll_register_callbacks cb 0x55cdf1929240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:09:58.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff26ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1bdb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff26ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1bdb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46590/0 (socket says 172.21.15.156:46590) 2023-11-28T13:09:58.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff277fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b59130 0x55cdf1b5e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff26ffd700 1 -- 172.21.15.156:0/1021144371 learned_addr learned my addr 172.21.15.156:0/1021144371 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:58.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff27fff700 1 --2- 172.21.15.156:0/1021144371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b5c120 0x55cdf1b6ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff26ffd700 1 -- 172.21.15.156:0/1021144371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b5c120 msgr2=0x55cdf1b6ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:58.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff26ffd700 1 --2- 172.21.15.156:0/1021144371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b5c120 0x55cdf1b6ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff26ffd700 1 -- 172.21.15.156:0/1021144371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b59130 msgr2=0x55cdf1b5e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:09:58.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff26ffd700 1 --2- 172.21.15.156:0/1021144371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b59130 0x55cdf1b5e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff26ffd700 1 -- 172.21.15.156:0/1021144371 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cdf18b1d90 con 0x55cdf1b5b810 2023-11-28T13:09:58.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.757+0000 7eff277fe700 1 --2- 172.21.15.156:0/1021144371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b59130 0x55cdf1b5e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:09:58.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.758+0000 7eff26ffd700 1 --2- 172.21.15.156:0/1021144371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1bdb760 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7eff1c012010 tx=0x7eff1c009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321ac1692cb55a0b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:58.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.758+0000 7eff067fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:09:58.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.758+0000 7eff067fc700 1 -- 172.21.15.156:0/1021144371 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c01e640 con 0x55cdf1b5b810 2023-11-28T13:09:58.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.758+0000 7eff41a2bb80 1 -- 172.21.15.156:0/1021144371 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdf1a41160 con 0x55cdf1b5b810 2023-11-28T13:09:58.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.758+0000 7eff067fc700 1 -- 172.21.15.156:0/1021144371 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7eff1c000e40 con 0x55cdf1b5b810 2023-11-28T13:09:58.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.758+0000 7eff067fc700 1 -- 172.21.15.156:0/1021144371 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff1c003c00 con 0x55cdf1b5b810 2023-11-28T13:09:58.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.758+0000 7eff41a2bb80 10 client.5631 fetch_fsmap learned FSMap version 23 2023-11-28T13:09:58.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.758+0000 7eff41a2bb80 1 -- 172.21.15.156:0/1021144371 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55cdf18d8200 con 0x55cdf1b5b810 2023-11-28T13:09:58.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.758+0000 7eff067fc700 1 -- 172.21.15.156:0/1021144371 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cdf1a41160 con 0x55cdf1b5b810 2023-11-28T13:09:58.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.758+0000 7eff067fc700 1 -- 172.21.15.156:0/1021144371 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7eff1c0119c0 con 0x55cdf1b5b810 2023-11-28T13:09:58.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.758+0000 7eff41a2bb80 10 client.5631 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T13:09:58.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.758+0000 7eff41a2bb80 10 client.5631 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:09:58.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.758+0000 7eff41a2bb80 20 client.5631 populate_metadata read hostname 'smithi156' 2023-11-28T13:09:58.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.759+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/1021144371 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55cdf1b7e090 0x55cdf1b7e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.759+0000 7eff41a2bb80 4 client.5631 mds_command: new command op to 5481 tid=0[{"prefix": "status"}] 2023-11-28T13:09:58.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.759+0000 7eff41a2bb80 1 -- 172.21.15.156:0/1021144371 --> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cdf1a2b170 con 0x55cdf1b7e090 2023-11-28T13:09:58.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.759+0000 7eff277fe700 1 --2- 172.21.15.156:0/1021144371 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55cdf1b7e090 0x55cdf1b7e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.759+0000 7eff277fe700 1 --2- 172.21.15.156:0/1021144371 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55cdf1b7e090 0x55cdf1b7e460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0ed891c31d5b8c4 server_cookie=d4a983ddb77e22cd in_seq=0 out_seq=0 2023-11-28T13:09:58.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.759+0000 7eff067fc700 10 client.5631 ms_handle_connect on v2:172.21.15.156:6832/1930782798 2023-11-28T13:09:58.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.759+0000 7eff067fc700 1 -- 172.21.15.156:0/1021144371 <== mds.0 v2:172.21.15.156:6832/1930782798 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55cdf1a2b170 con 0x55cdf1b7e090 2023-11-28T13:09:58.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.759+0000 7eff067fc700 10 client.5631 handle_command_reply: tid=0 2023-11-28T13:09:58.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.759+0000 7eff41a2bb80 1 client.5631 shutdown 2023-11-28T13:09:58.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.760+0000 7eff41a2bb80 1 -- 172.21.15.156:0/1021144371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 msgr2=0x55cdf1bdb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:58.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.760+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/1021144371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1bdb760 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7eff1c012010 tx=0x7eff1c009210 comp rx=0 tx=0).stop 2023-11-28T13:09:58.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.760+0000 7eff41a2bb80 1 -- 172.21.15.156:0/1021144371 shutdown_connections 2023-11-28T13:09:58.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.760+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/1021144371 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55cdf1b7e090 0x55cdf1b7e460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.760+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/1021144371 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55cdf1b5c120 0x55cdf1b6ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.760+0000 7eff067fc700 0 client.5631 ms_handle_reset on v2:172.21.15.156:6832/1930782798 2023-11-28T13:09:58.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.760+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/1021144371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cdf1b59130 0x55cdf1b5e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.760+0000 7eff41a2bb80 1 --2- 172.21.15.156:0/1021144371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdf1b5b810 0x55cdf1bdb760 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.760+0000 7eff41a2bb80 1 -- 172.21.15.156:0/1021144371 >> 172.21.15.156:0/1021144371 conn(0x55cdf1b2ed60 msgr2=0x55cdf1b5aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:58.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.760+0000 7eff41a2bb80 1 -- 172.21.15.156:0/1021144371 shutdown_connections 2023-11-28T13:09:58.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.760+0000 7eff41a2bb80 1 -- 172.21.15.156:0/1021144371 wait complete. 2023-11-28T13:09:58.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.760+0000 7eff41a2bb80 20 client.5631 trim_cache size 0 max 16384 2023-11-28T13:09:58.774 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:09:58.774 INFO:teuthology.orchestra.run.smithi156.stdout: "cluster_fsid": "12d9e849-dc60-46f0-8e8e-9e291633f61b", 2023-11-28T13:09:58.774 INFO:teuthology.orchestra.run.smithi156.stdout: "whoami": 0, 2023-11-28T13:09:58.774 INFO:teuthology.orchestra.run.smithi156.stdout: "id": 5481, 2023-11-28T13:09:58.774 INFO:teuthology.orchestra.run.smithi156.stdout: "want_state": "up:active", 2023-11-28T13:09:58.774 INFO:teuthology.orchestra.run.smithi156.stdout: "state": "up:active", 2023-11-28T13:09:58.774 INFO:teuthology.orchestra.run.smithi156.stdout: "fs_name": "cephfs", 2023-11-28T13:09:58.775 INFO:teuthology.orchestra.run.smithi156.stdout: "rank_uptime": 6.5132157399999997, 2023-11-28T13:09:58.775 INFO:teuthology.orchestra.run.smithi156.stdout: "mdsmap_epoch": 23, 2023-11-28T13:09:58.775 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_epoch": 47, 2023-11-28T13:09:58.775 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_epoch_barrier": 47, 2023-11-28T13:09:58.775 INFO:teuthology.orchestra.run.smithi156.stdout: "uptime": 12.522681534 2023-11-28T13:09:58.775 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:09:58.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.761+0000 7eff2e1ff700 1 -- 172.21.15.156:0/2745528838 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7eff140682f0 msgr2=0x7eff1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:58.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.761+0000 7eff2e1ff700 1 --2- 172.21.15.156:0/2745528838 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7eff140682f0 0x7eff1406a7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7eff18009c40 tx=0x7eff18008030 comp rx=0 tx=0).stop 2023-11-28T13:09:58.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.761+0000 7eff2e1ff700 1 -- 172.21.15.156:0/2745528838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff28161340 msgr2=0x7eff281e8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:58.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.761+0000 7eff2e1ff700 1 --2- 172.21.15.156:0/2745528838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff28161340 0x7eff281e8430 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7eff2000cb50 tx=0x7eff2000ce60 comp rx=0 tx=0).stop 2023-11-28T13:09:58.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.761+0000 7eff2e1ff700 1 -- 172.21.15.156:0/2745528838 shutdown_connections 2023-11-28T13:09:58.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.761+0000 7eff2e1ff700 1 --2- 172.21.15.156:0/2745528838 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7eff140682f0 0x7eff1406a7a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.761+0000 7eff2e1ff700 1 --2- 172.21.15.156:0/2745528838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff28161340 0x7eff281e8430 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.761+0000 7eff2e1ff700 1 --2- 172.21.15.156:0/2745528838 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7eff280d2320 0x7eff28151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.761+0000 7eff2e1ff700 1 --2- 172.21.15.156:0/2745528838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff2815c9b0 0x7eff28150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.761+0000 7eff2e1ff700 1 -- 172.21.15.156:0/2745528838 >> 172.21.15.156:0/2745528838 conn(0x7eff28072600 msgr2=0x7eff28160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:58.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.761+0000 7eff2e1ff700 1 -- 172.21.15.156:0/2745528838 shutdown_connections 2023-11-28T13:09:58.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.761+0000 7eff2e1ff700 1 -- 172.21.15.156:0/2745528838 wait complete. 2023-11-28T13:09:58.780 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:09:58.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.984+0000 7f4d3b7ff700 1 Processor -- start 2023-11-28T13:09:58.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.985+0000 7f4d3b7ff700 1 -- start start 2023-11-28T13:09:58.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.986+0000 7f4d3b7ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4d34161340 0x7f4d340d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.986+0000 7f4d3b7ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d340d1950 con 0x7f4d3415c9b0 2023-11-28T13:09:58.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.986+0000 7f4d3b7ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d340b5520 con 0x7f4d340d2320 2023-11-28T13:09:58.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.986+0000 7f4d3b7ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d340b5800 con 0x7f4d34161340 2023-11-28T13:09:58.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.986+0000 7f4d3959b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4d3415c9b0 0x7f4d340b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53356/0 (socket says 172.21.15.156:53356) 2023-11-28T13:09:58.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.986+0000 7f4d3959b700 1 -- 172.21.15.156:0/3915286053 learned_addr learned my addr 172.21.15.156:0/3915286053 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:09:58.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.986+0000 7f4d38d9a700 1 --2- 172.21.15.156:0/3915286053 >> v2:172.21.15.156:3300/0 conn(0x7f4d34161340 0x7f4d340d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.987+0000 7f4d38d9a700 1 -- 172.21.15.156:0/3915286053 >> v1:172.21.15.186:6789/0 conn(0x7f4d340d2320 legacy=0x7f4d340cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:58.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.987+0000 7f4d38d9a700 1 -- 172.21.15.156:0/3915286053 >> v1:172.21.15.156:6789/0 conn(0x7f4d3415c9b0 legacy=0x7f4d340b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:58.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.987+0000 7f4d38d9a700 1 -- 172.21.15.156:0/3915286053 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d340d1300 con 0x7f4d34161340 2023-11-28T13:09:58.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.987+0000 7f4d38d9a700 1 --2- 172.21.15.156:0/3915286053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d34161340 0x7f4d340d1de0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009840 tx=0x7f4d24009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0a42f3873d444 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:58.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.987+0000 7f4d2bfff700 1 -- 172.21.15.156:0/3915286053 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24011830 con 0x7f4d34161340 2023-11-28T13:09:58.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.987+0000 7f4d2bfff700 1 -- 172.21.15.156:0/3915286053 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d24011990 con 0x7f4d34161340 2023-11-28T13:09:58.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.987+0000 7f4d2bfff700 1 -- 172.21.15.156:0/3915286053 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24011d20 con 0x7f4d34161340 2023-11-28T13:09:58.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.988+0000 7f4d3b7ff700 1 -- 172.21.15.156:0/3915286053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d34161340 msgr2=0x7f4d340d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:58.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.988+0000 7f4d3b7ff700 1 --2- 172.21.15.156:0/3915286053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d34161340 0x7f4d340d1de0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009840 tx=0x7f4d24009b50 comp rx=0 tx=0).stop 2023-11-28T13:09:58.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.988+0000 7f4d3b7ff700 1 -- 172.21.15.156:0/3915286053 shutdown_connections 2023-11-28T13:09:58.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.988+0000 7f4d3b7ff700 1 --2- 172.21.15.156:0/3915286053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d34161340 0x7f4d340d1de0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.988+0000 7f4d3b7ff700 1 -- 172.21.15.156:0/3915286053 >> 172.21.15.156:0/3915286053 conn(0x7f4d34072600 msgr2=0x7f4d340cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:58.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.988+0000 7f4d3b7ff700 1 -- 172.21.15.156:0/3915286053 shutdown_connections 2023-11-28T13:09:58.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.988+0000 7f4d3b7ff700 1 -- 172.21.15.156:0/3915286053 wait complete. 2023-11-28T13:09:58.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.989+0000 7f4d3b7ff700 1 Processor -- start 2023-11-28T13:09:58.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.990+0000 7f4d3b7ff700 1 -- start start 2023-11-28T13:09:58.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.990+0000 7f4d3b7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d340d2320 0x7f4d340b8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.990+0000 7f4d3b7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d3415c9b0 0x7f4d341e8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.990+0000 7f4d3b7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34161340 0x7f4d3414a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:58.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.990+0000 7f4d3b7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d340c6d40 con 0x7f4d3415c9b0 2023-11-28T13:09:58.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.990+0000 7f4d3b7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d340c7020 con 0x7f4d34161340 2023-11-28T13:09:58.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.990+0000 7f4d3b7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d340c7300 con 0x7f4d340d2320 2023-11-28T13:09:58.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.991+0000 7f4d38d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d3415c9b0 0x7f4d341e8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.991+0000 7f4d38d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d3415c9b0 0x7f4d341e8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46608/0 (socket says 172.21.15.156:46608) 2023-11-28T13:09:58.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.991+0000 7f4d38d9a700 1 -- 172.21.15.156:0/635588542 learned_addr learned my addr 172.21.15.156:0/635588542 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:09:58.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.991+0000 7f4d38d9a700 1 -- 172.21.15.156:0/635588542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d340d2320 msgr2=0x7f4d340b8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:58.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.991+0000 7f4d38d9a700 1 --2- 172.21.15.156:0/635588542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d340d2320 0x7f4d340b8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.991+0000 7f4d39d9c700 1 --2- 172.21.15.156:0/635588542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34161340 0x7f4d3414a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:58.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.991+0000 7f4d38d9a700 1 -- 172.21.15.156:0/635588542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34161340 msgr2=0x7f4d3414a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:58.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.991+0000 7f4d38d9a700 1 --2- 172.21.15.156:0/635588542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34161340 0x7f4d3414a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:58.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.991+0000 7f4d38d9a700 1 -- 172.21.15.156:0/635588542 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d240094a0 con 0x7f4d3415c9b0 2023-11-28T13:09:58.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.991+0000 7f4d39d9c700 1 --2- 172.21.15.156:0/635588542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34161340 0x7f4d3414a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:09:58.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.991+0000 7f4d38d9a700 1 --2- 172.21.15.156:0/635588542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d3415c9b0 0x7f4d341e8e20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4d24000c00 tx=0x7f4d24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:58.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.992+0000 7f4d29ffb700 1 -- 172.21.15.156:0/635588542 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d240038a0 con 0x7f4d3415c9b0 2023-11-28T13:09:58.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.992+0000 7f4d29ffb700 1 -- 172.21.15.156:0/635588542 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2401e070 con 0x7f4d3415c9b0 2023-11-28T13:09:58.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.992+0000 7f4d3b7ff700 1 -- 172.21.15.156:0/635588542 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3415cd80 con 0x7f4d3415c9b0 2023-11-28T13:09:58.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.992+0000 7f4d29ffb700 1 -- 172.21.15.156:0/635588542 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24003c60 con 0x7f4d3415c9b0 2023-11-28T13:09:58.999 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.992+0000 7f4d3b7ff700 1 -- 172.21.15.156:0/635588542 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d340d15e0 con 0x7f4d3415c9b0 2023-11-28T13:09:59.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.993+0000 7f4d3b7ff700 1 -- 172.21.15.156:0/635588542 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3415cf10 con 0x7f4d3415c9b0 2023-11-28T13:09:59.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.994+0000 7f4d29ffb700 1 -- 172.21.15.156:0/635588542 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4d2400f050 con 0x7f4d3415c9b0 2023-11-28T13:09:59.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.994+0000 7f4d29ffb700 1 --2- 172.21.15.156:0/635588542 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d180682f0 0x7f4d1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:09:59.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.994+0000 7f4d29ffb700 1 -- 172.21.15.156:0/635588542 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d24086f10 con 0x7f4d3415c9b0 2023-11-28T13:09:59.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.995+0000 7f4d3959b700 1 --2- 172.21.15.156:0/635588542 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d180682f0 0x7f4d1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:09:59.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.996+0000 7f4d3959b700 1 --2- 172.21.15.156:0/635588542 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d180682f0 0x7f4d1806a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4d34157e80 tx=0x7f4d30007ad0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:09:59.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:58.999+0000 7f4d29ffb700 1 -- 172.21.15.156:0/635588542 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4d24053680 con 0x7f4d3415c9b0 2023-11-28T13:09:59.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:59.232+0000 7f4d3b7ff700 1 -- 172.21.15.156:0/635588542 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4d340745c0 con 0x7f4d3415c9b0 2023-11-28T13:09:59.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:59.233+0000 7f4d29ffb700 1 -- 172.21.15.156:0/635588542 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4d24015070 con 0x7f4d3415c9b0 2023-11-28T13:09:59.234 INFO:teuthology.orchestra.run.smithi156.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T13:09:59.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:59.235+0000 7f4d3b7ff700 1 -- 172.21.15.156:0/635588542 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d180682f0 msgr2=0x7f4d1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:59.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:59.235+0000 7f4d3b7ff700 1 --2- 172.21.15.156:0/635588542 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d180682f0 0x7f4d1806a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4d34157e80 tx=0x7f4d30007ad0 comp rx=0 tx=0).stop 2023-11-28T13:09:59.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:59.235+0000 7f4d3b7ff700 1 -- 172.21.15.156:0/635588542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d3415c9b0 msgr2=0x7f4d341e8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:09:59.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:59.235+0000 7f4d3b7ff700 1 --2- 172.21.15.156:0/635588542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d3415c9b0 0x7f4d341e8e20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4d24000c00 tx=0x7f4d24004750 comp rx=0 tx=0).stop 2023-11-28T13:09:59.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:59.235+0000 7f4d3b7ff700 1 -- 172.21.15.156:0/635588542 shutdown_connections 2023-11-28T13:09:59.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:59.235+0000 7f4d3b7ff700 1 --2- 172.21.15.156:0/635588542 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4d180682f0 0x7f4d1806a7a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:59.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:59.235+0000 7f4d3b7ff700 1 --2- 172.21.15.156:0/635588542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34161340 0x7f4d3414a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:59.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:59.235+0000 7f4d3b7ff700 1 --2- 172.21.15.156:0/635588542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d340d2320 0x7f4d340b8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:59.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:59.235+0000 7f4d3b7ff700 1 --2- 172.21.15.156:0/635588542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d3415c9b0 0x7f4d341e8e20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:09:59.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:59.235+0000 7f4d3b7ff700 1 -- 172.21.15.156:0/635588542 >> 172.21.15.156:0/635588542 conn(0x7f4d34072600 msgr2=0x7f4d340b6240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:09:59.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:59.235+0000 7f4d3b7ff700 1 -- 172.21.15.156:0/635588542 shutdown_connections 2023-11-28T13:09:59.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:09:59.235+0000 7f4d3b7ff700 1 -- 172.21.15.156:0/635588542 wait complete. 2023-11-28T13:09:59.255 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T13:09:59.255 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:09:59.256 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-11-28T13:09:59.256 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T13:09:59.256 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T13:09:59.256 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T13:09:59.256 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T13:09:59.256 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:09:59.256 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:09:59.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:59.257 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-11-28T13:09:59.302 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:09:59.302 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:09:59.302 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:09:59.302 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:09:59.302 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-28T13:09:59.302 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:09:59.302 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:09:59.302 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:09:59.303 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f0 2023-11-28T13:09:59.303 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:09:59.303 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:09:59.303 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f1 2023-11-28T13:09:59.303 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:09:59.303 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-11-28T13:09:59.303 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:09:59.303 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-11-28T13:09:59.303 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T13:09:59.303 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42064sec preferred_lft 42064sec 2023-11-28T13:09:59.303 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::d27c:fc8f:2674:dd68/64 scope link noprefixroute 2023-11-28T13:09:59.304 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:09:59.304 INFO:teuthology.orchestra.run.smithi156.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:09:59.304 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 3e:ff:33:55:aa:40 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:09:59.304 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T13:09:59.304 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:09:59.304 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::4497:b0ff:fe27:d641/64 scope link 2023-11-28T13:09:59.304 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:09:59.304 INFO:teuthology.orchestra.run.smithi156.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:09:59.304 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 96:cd:43:a7:29:10 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:09:59.304 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::94cd:43ff:fea7:2910/64 scope link 2023-11-28T13:09:59.304 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:09:59.304 INFO:teuthology.orchestra.run.smithi156.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:09:59.305 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 3e:ff:33:55:aa:40 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T13:09:59.305 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::3cff:33ff:fe55:aa40/64 scope link 2023-11-28T13:09:59.305 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:09:59.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:59.305 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:09:59.409 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-28T13:09:59.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:59.413 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-11-28T13:09:59.459 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-11-28T13:09:59.459 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:09:59.460 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T13:09:59.460 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T13:09:59.460 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:09:59.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:59.460 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T13:09:59.460 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T13:09:59.460 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T13:09:59.461 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T13:09:59.461 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T13:09:59.461 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T13:09:59.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:59.634 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T13:09:59.677 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T13:09:59.677 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T13:09:59.678 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:09:59.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:59.678 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.0 up 2023-11-28T13:09:59.792 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:09:59.793 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T13:09:59.837 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:09:59.837 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:09:59.837 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:09:59.920 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T13:09:59.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:09:59.921 DEBUG:teuthology.orchestra.run.smithi156:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:10:00.076 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:10:00.076 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:10:00.077 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T13:10:00.095 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:10:00.095 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:10:00.096 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T13:10:00.096 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T13:10:00.096 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:10:00.118 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:10:00.118 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:10:00.118 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T13:10:00.118 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T13:10:00.118 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:10:00.118 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790" 2023-11-28T13:10:00.119 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2023-11-28T13:10:00.119 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=12d9e849-dc60-46f0-8e8e-9e291633f61b" will pass to kernel 2023-11-28T13:10:00.123 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T13:10:00.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:10:00.123 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:10:00.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:10:00.194 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:10:00.294 INFO:teuthology.orchestra.run.smithi156.stdout:client5416 2023-11-28T13:10:00.295 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:10:00.295 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:10:00.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:10:00.339 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2023-11-28T13:10:00.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:10:00.428 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2023-11-28T13:10:00.475 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:10:00.475 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-11-28T13:10:00.475 DEBUG:teuthology.orchestra.run.smithi156:> import stat 2023-11-28T13:10:00.475 DEBUG:teuthology.orchestra.run.smithi156:> 2023-11-28T13:10:00.475 DEBUG:teuthology.orchestra.run.smithi156:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2023-11-28T13:10:00.475 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-11-28T13:10:00.628 INFO:teuthology.orchestra.run.smithi156.stdout:1099511627777 2023-11-28T13:10:00.812 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:10:01.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.048+0000 7fba253ff700 1 Processor -- start 2023-11-28T13:10:01.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.048+0000 7fba253ff700 1 -- start start 2023-11-28T13:10:01.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.049+0000 7fba253ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fba20159490 0x7fba200b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.049+0000 7fba253ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba20159860 0x7fba200b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.049+0000 7fba253ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fba200cdc80 0x7fba200c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.049+0000 7fba253ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba200ccc00 con 0x7fba20159490 2023-11-28T13:10:01.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.049+0000 7fba253ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba200ccec0 con 0x7fba200cdc80 2023-11-28T13:10:01.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.049+0000 7fba253ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba200cd1a0 con 0x7fba20159860 2023-11-28T13:10:01.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.049+0000 7fba1effd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fba20159490 0x7fba200b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.049+0000 7fba1effd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fba20159490 0x7fba200b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46614/0 (socket says 172.21.15.156:46614) 2023-11-28T13:10:01.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.049+0000 7fba1effd700 1 -- 172.21.15.156:0/3646546212 learned_addr learned my addr 172.21.15.156:0/3646546212 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:01.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.050+0000 7fba1e7fc700 1 --2- 172.21.15.156:0/3646546212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba20159860 0x7fba200b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.050+0000 7fba1f7fe700 1 --2- 172.21.15.156:0/3646546212 >> v2:172.21.15.186:3300/0 conn(0x7fba200cdc80 0x7fba200c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.050+0000 7fba1effd700 1 -- 172.21.15.156:0/3646546212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba20159860 msgr2=0x7fba200b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:01.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.050+0000 7fba1effd700 1 --2- 172.21.15.156:0/3646546212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba20159860 0x7fba200b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.050+0000 7fba1effd700 1 -- 172.21.15.156:0/3646546212 >> v2:172.21.15.186:3300/0 conn(0x7fba200cdc80 msgr2=0x7fba200c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:01.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.050+0000 7fba1effd700 1 --2- 172.21.15.156:0/3646546212 >> v2:172.21.15.186:3300/0 conn(0x7fba200cdc80 0x7fba200c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.050+0000 7fba1effd700 1 -- 172.21.15.156:0/3646546212 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba200cd480 con 0x7fba20159490 2023-11-28T13:10:01.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.050+0000 7fba1effd700 1 --2- 172.21.15.156:0/3646546212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba20159490 0x7fba200b0670 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fba200b4b20 tx=0x7fba10009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3319cabb7833446a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:01.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.050+0000 7fba1dffb700 1 -- 172.21.15.156:0/3646546212 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1000b870 con 0x7fba20159490 2023-11-28T13:10:01.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.050+0000 7fba1dffb700 1 -- 172.21.15.156:0/3646546212 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1000b9d0 con 0x7fba20159490 2023-11-28T13:10:01.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.050+0000 7fba1dffb700 1 -- 172.21.15.156:0/3646546212 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1000bd60 con 0x7fba20159490 2023-11-28T13:10:01.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.051+0000 7fba253ff700 1 -- 172.21.15.156:0/3646546212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba20159490 msgr2=0x7fba200b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:01.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.051+0000 7fba253ff700 1 --2- 172.21.15.156:0/3646546212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba20159490 0x7fba200b0670 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fba200b4b20 tx=0x7fba10009bd0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.051+0000 7fba253ff700 1 -- 172.21.15.156:0/3646546212 shutdown_connections 2023-11-28T13:10:01.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.051+0000 7fba253ff700 1 --2- 172.21.15.156:0/3646546212 >> v2:172.21.15.186:3300/0 conn(0x7fba200cdc80 0x7fba200c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.051+0000 7fba253ff700 1 --2- 172.21.15.156:0/3646546212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba20159490 0x7fba200b0670 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.051+0000 7fba253ff700 1 --2- 172.21.15.156:0/3646546212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba20159860 0x7fba200b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.051+0000 7fba253ff700 1 -- 172.21.15.156:0/3646546212 >> 172.21.15.156:0/3646546212 conn(0x7fba200cb670 msgr2=0x7fba200cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:01.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.051+0000 7fba253ff700 1 -- 172.21.15.156:0/3646546212 shutdown_connections 2023-11-28T13:10:01.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.051+0000 7fba253ff700 1 -- 172.21.15.156:0/3646546212 wait complete. 2023-11-28T13:10:01.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.053+0000 7fba253ff700 1 Processor -- start 2023-11-28T13:10:01.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.053+0000 7fba253ff700 1 -- start start 2023-11-28T13:10:01.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.054+0000 7fba253ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba200cdc80 0x7fba20154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.054+0000 7fba253ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba20159490 0x7fba20154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.054+0000 7fba253ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba200c7690 0x7fba200c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.054+0000 7fba253ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba200c7c50 con 0x7fba20159490 2023-11-28T13:10:01.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.054+0000 7fba253ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba200c7f30 con 0x7fba200cdc80 2023-11-28T13:10:01.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.054+0000 7fba253ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba200c8210 con 0x7fba200c7690 2023-11-28T13:10:01.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.054+0000 7fba1e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba20159490 0x7fba20154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.054+0000 7fba1e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba20159490 0x7fba20154ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46630/0 (socket says 172.21.15.156:46630) 2023-11-28T13:10:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.054+0000 7fba1e7fc700 1 -- 172.21.15.156:0/323243486 learned_addr learned my addr 172.21.15.156:0/323243486 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.054+0000 7fba1f7fe700 1 --2- 172.21.15.156:0/323243486 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba200c7690 0x7fba200c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.054+0000 7fba1effd700 1 --2- 172.21.15.156:0/323243486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba200cdc80 0x7fba20154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.055+0000 7fba1e7fc700 1 -- 172.21.15.156:0/323243486 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba200c7690 msgr2=0x7fba200c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.055+0000 7fba1e7fc700 1 --2- 172.21.15.156:0/323243486 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba200c7690 0x7fba200c3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.055+0000 7fba1e7fc700 1 -- 172.21.15.156:0/323243486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba200cdc80 msgr2=0x7fba20154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.055+0000 7fba1e7fc700 1 --2- 172.21.15.156:0/323243486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba200cdc80 0x7fba20154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.055+0000 7fba1e7fc700 1 -- 172.21.15.156:0/323243486 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba10009550 con 0x7fba20159490 2023-11-28T13:10:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.055+0000 7fba1e7fc700 1 --2- 172.21.15.156:0/323243486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba20159490 0x7fba20154ed0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fba1400d800 tx=0x7fba1400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.055+0000 7fba077fe700 1 -- 172.21.15.156:0/323243486 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba140125f0 con 0x7fba20159490 2023-11-28T13:10:01.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.055+0000 7fba077fe700 1 -- 172.21.15.156:0/323243486 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba14012750 con 0x7fba20159490 2023-11-28T13:10:01.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.055+0000 7fba253ff700 1 -- 172.21.15.156:0/323243486 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba200c9a00 con 0x7fba20159490 2023-11-28T13:10:01.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.055+0000 7fba077fe700 1 -- 172.21.15.156:0/323243486 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14012b00 con 0x7fba20159490 2023-11-28T13:10:01.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.055+0000 7fba253ff700 1 -- 172.21.15.156:0/323243486 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba200cd760 con 0x7fba20159490 2023-11-28T13:10:01.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.056+0000 7fba253ff700 1 -- 172.21.15.156:0/323243486 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba200c9b90 con 0x7fba20159490 2023-11-28T13:10:01.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.057+0000 7fba077fe700 1 -- 172.21.15.156:0/323243486 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fba14012c60 con 0x7fba20159490 2023-11-28T13:10:01.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.058+0000 7fba077fe700 1 --2- 172.21.15.156:0/323243486 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fba0c0682f0 0x7fba0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.058+0000 7fba077fe700 1 -- 172.21.15.156:0/323243486 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fba14085e40 con 0x7fba20159490 2023-11-28T13:10:01.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.058+0000 7fba1effd700 1 --2- 172.21.15.156:0/323243486 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fba0c0682f0 0x7fba0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.059+0000 7fba1effd700 1 --2- 172.21.15.156:0/323243486 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fba0c0682f0 0x7fba0c06a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fba10000c00 tx=0x7fba10008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:01.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.062+0000 7fba077fe700 1 -- 172.21.15.156:0/323243486 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fba140525b0 con 0x7fba20159490 2023-11-28T13:10:01.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.295+0000 7fba253ff700 1 -- 172.21.15.156:0/323243486 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba200c5500 con 0x7fba20159490 2023-11-28T13:10:01.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.296+0000 7fba077fe700 1 -- 172.21.15.156:0/323243486 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fba14055bd0 con 0x7fba20159490 2023-11-28T13:10:01.298 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:01.298 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3828141874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3828141874},{"type":"v1","addr":"172.21.15.156:6835","nonce":3828141874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1012489762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1012489762},{"type":"v1","addr":"172.21.15.186:6837","nonce":1012489762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2894670074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2894670074},{"type":"v1","addr":"172.21.15.186:6835","nonce":2894670074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:09:52.245032+0000","modified":"2023-11-28T13:09:55.251254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5481":{"gid":5481,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6834/1930782798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1930782798},{"type":"v1","addr":"172.21.15.156:6834","nonce":1930782798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:10:01.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.298+0000 7fba253ff700 1 -- 172.21.15.156:0/323243486 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fba0c0682f0 msgr2=0x7fba0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:01.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.298+0000 7fba253ff700 1 --2- 172.21.15.156:0/323243486 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fba0c0682f0 0x7fba0c06a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fba10000c00 tx=0x7fba10008030 comp rx=0 tx=0).stop 2023-11-28T13:10:01.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.298+0000 7fba253ff700 1 -- 172.21.15.156:0/323243486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba20159490 msgr2=0x7fba20154ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:01.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.298+0000 7fba253ff700 1 --2- 172.21.15.156:0/323243486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba20159490 0x7fba20154ed0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fba1400d800 tx=0x7fba1400db10 comp rx=0 tx=0).stop 2023-11-28T13:10:01.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.298+0000 7fba253ff700 1 -- 172.21.15.156:0/323243486 shutdown_connections 2023-11-28T13:10:01.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.298+0000 7fba253ff700 1 --2- 172.21.15.156:0/323243486 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fba0c0682f0 0x7fba0c06a7a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.298+0000 7fba253ff700 1 --2- 172.21.15.156:0/323243486 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba200c7690 0x7fba200c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.298+0000 7fba253ff700 1 --2- 172.21.15.156:0/323243486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba200cdc80 0x7fba20154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.298+0000 7fba253ff700 1 --2- 172.21.15.156:0/323243486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba20159490 0x7fba20154ed0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.298+0000 7fba253ff700 1 -- 172.21.15.156:0/323243486 >> 172.21.15.156:0/323243486 conn(0x7fba200cb670 msgr2=0x7fba200b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:01.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.298+0000 7fba253ff700 1 -- 172.21.15.156:0/323243486 shutdown_connections 2023-11-28T13:10:01.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.298+0000 7fba253ff700 1 -- 172.21.15.156:0/323243486 wait complete. 2023-11-28T13:10:01.301 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 23 2023-11-28T13:10:01.319 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-11-28T13:10:01.544 INFO:teuthology.orchestra.run.smithi156.stdout:{"message":"","return_code":0} 2023-11-28T13:10:01.544 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T13:10:01.545 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T13:10:01.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.629+0000 7f056ec63080 1 Processor -- start 2023-11-28T13:10:01.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.629+0000 7f056ec63080 1 -- start start 2023-11-28T13:10:01.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.630+0000 7f056ec63080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a8b0102920 0x55a8b018f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.630+0000 7f056ec63080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a8b0192250 0x55a8b018c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.630+0000 7f056ec63080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8b0192620 0x55a8b00e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.630+0000 7f056ec63080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8afff4fe0 con 0x55a8b0192620 2023-11-28T13:10:01.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.630+0000 7f056ec63080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b00e9010 con 0x55a8b0192250 2023-11-28T13:10:01.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.630+0000 7f056ec63080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8b018ec10 con 0x55a8b0102920 2023-11-28T13:10:01.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.630+0000 7f0560b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8b0192620 0x55a8b00e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.630+0000 7f0560b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8b0192620 0x55a8b00e68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46652/0 (socket says 172.21.15.156:46652) 2023-11-28T13:10:01.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.630+0000 7f0560b9c700 1 -- 172.21.15.156:0/1991399309 learned_addr learned my addr 172.21.15.156:0/1991399309 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:01.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.630+0000 7f055fb9a700 1 --2- 172.21.15.156:0/1991399309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a8b0192250 0x55a8b018c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.631+0000 7f056039b700 1 --2- 172.21.15.156:0/1991399309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a8b0102920 0x55a8b018f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.631+0000 7f0560b9c700 1 -- 172.21.15.156:0/1991399309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a8b0102920 msgr2=0x55a8b018f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:01.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.631+0000 7f0560b9c700 1 --2- 172.21.15.156:0/1991399309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a8b0102920 0x55a8b018f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.631+0000 7f0560b9c700 1 -- 172.21.15.156:0/1991399309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a8b0192250 msgr2=0x55a8b018c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:01.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.631+0000 7f0560b9c700 1 --2- 172.21.15.156:0/1991399309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a8b0192250 0x55a8b018c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.631+0000 7f0560b9c700 1 -- 172.21.15.156:0/1991399309 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0554005d20 con 0x55a8b0192620 2023-11-28T13:10:01.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.631+0000 7f0560b9c700 1 --2- 172.21.15.156:0/1991399309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8b0192620 0x55a8b00e68d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f05580014a0 tx=0x7f05580017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619175f42369a0db server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:01.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.631+0000 7f055eb98700 1 -- 172.21.15.156:0/1991399309 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558006310 con 0x55a8b0192620 2023-11-28T13:10:01.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.631+0000 7f055eb98700 1 -- 172.21.15.156:0/1991399309 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05580020c0 con 0x55a8b0192620 2023-11-28T13:10:01.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.631+0000 7f055eb98700 1 -- 172.21.15.156:0/1991399309 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558004740 con 0x55a8b0192620 2023-11-28T13:10:01.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.632+0000 7f056ec63080 1 -- 172.21.15.156:0/1991399309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8b0192620 msgr2=0x55a8b00e68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:01.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.632+0000 7f056ec63080 1 --2- 172.21.15.156:0/1991399309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8b0192620 0x55a8b00e68d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f05580014a0 tx=0x7f05580017b0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.632+0000 7f056ec63080 1 -- 172.21.15.156:0/1991399309 shutdown_connections 2023-11-28T13:10:01.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.632+0000 7f056ec63080 1 --2- 172.21.15.156:0/1991399309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8b0192620 0x55a8b00e68d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.632+0000 7f056ec63080 1 --2- 172.21.15.156:0/1991399309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a8b0102920 0x55a8b018f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.632+0000 7f056ec63080 1 --2- 172.21.15.156:0/1991399309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a8b0192250 0x55a8b018c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.632+0000 7f056ec63080 1 -- 172.21.15.156:0/1991399309 >> 172.21.15.156:0/1991399309 conn(0x55a8b01069d0 msgr2=0x55a8b01912d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:01.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.632+0000 7f056ec63080 1 -- 172.21.15.156:0/1991399309 shutdown_connections 2023-11-28T13:10:01.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.632+0000 7f056ec63080 1 -- 172.21.15.156:0/1991399309 wait complete. 2023-11-28T13:10:01.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.633+0000 7f056ec63080 1 Processor -- start 2023-11-28T13:10:01.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.633+0000 7f056ec63080 1 -- start start 2023-11-28T13:10:01.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.634+0000 7f056ec63080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8b0102920 0x55a8b0188fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.634+0000 7f056ec63080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a8b0192250 0x55a8b0190e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.634+0000 7f056ec63080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a8b0192620 0x55a8b0191360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.634+0000 7f056ec63080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b0195f40 con 0x55a8b0102920 2023-11-28T13:10:01.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.634+0000 7f056ec63080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b00fdb20 con 0x55a8b0192250 2023-11-28T13:10:01.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.634+0000 7f056ec63080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8b00fde00 con 0x55a8b0192620 2023-11-28T13:10:01.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.634+0000 7f056039b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8b0102920 0x55a8b0188fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.634+0000 7f056039b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8b0102920 0x55a8b0188fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46654/0 (socket says 172.21.15.156:46654) 2023-11-28T13:10:01.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.634+0000 7f056039b700 1 -- 172.21.15.156:0/3955946249 learned_addr learned my addr 172.21.15.156:0/3955946249 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:01.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.634+0000 7f055fb9a700 1 --2- 172.21.15.156:0/3955946249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a8b0192250 0x55a8b0190e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.634+0000 7f0560b9c700 1 --2- 172.21.15.156:0/3955946249 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a8b0192620 0x55a8b0191360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.635+0000 7f056039b700 1 -- 172.21.15.156:0/3955946249 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a8b0192620 msgr2=0x55a8b0191360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:01.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.635+0000 7f056039b700 1 --2- 172.21.15.156:0/3955946249 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a8b0192620 0x55a8b0191360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.635+0000 7f056039b700 1 -- 172.21.15.156:0/3955946249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a8b0192250 msgr2=0x55a8b0190e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:01.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.635+0000 7f056039b700 1 --2- 172.21.15.156:0/3955946249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a8b0192250 0x55a8b0190e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.635+0000 7f056039b700 1 -- 172.21.15.156:0/3955946249 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8b00004a0 con 0x55a8b0102920 2023-11-28T13:10:01.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.635+0000 7f056039b700 1 --2- 172.21.15.156:0/3955946249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8b0102920 0x55a8b0188fa0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0550008810 tx=0x7f0550008b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:01.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.635+0000 7f055db96700 1 -- 172.21.15.156:0/3955946249 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055000f040 con 0x55a8b0102920 2023-11-28T13:10:01.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.635+0000 7f055db96700 1 -- 172.21.15.156:0/3955946249 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05500147e0 con 0x55a8b0102920 2023-11-28T13:10:01.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.635+0000 7f056ec63080 1 -- 172.21.15.156:0/3955946249 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a8b000d120 con 0x55a8b0102920 2023-11-28T13:10:01.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.635+0000 7f055db96700 1 -- 172.21.15.156:0/3955946249 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550014b90 con 0x55a8b0102920 2023-11-28T13:10:01.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.635+0000 7f056ec63080 1 -- 172.21.15.156:0/3955946249 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8b0000830 con 0x55a8b0102920 2023-11-28T13:10:01.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.637+0000 7f055db96700 1 -- 172.21.15.156:0/3955946249 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0550014cf0 con 0x55a8b0102920 2023-11-28T13:10:01.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.638+0000 7f055db96700 1 --2- 172.21.15.156:0/3955946249 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f054c066c10 0x7f054c0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.646 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.638+0000 7f055db96700 1 -- 172.21.15.156:0/3955946249 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f05500856c0 con 0x55a8b0102920 2023-11-28T13:10:01.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.638+0000 7f056ec63080 1 --2- 172.21.15.156:0/3955946249 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x55a8b0190520 0x55a8b00ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.638+0000 7f056ec63080 1 -- 172.21.15.156:0/3955946249 --> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x55a8b0194d10 con 0x55a8b0190520 2023-11-28T13:10:01.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.638+0000 7f055fb9a700 1 --2- 172.21.15.156:0/3955946249 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f054c066c10 0x7f054c0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.638+0000 7f0560b9c700 1 --2- 172.21.15.156:0/3955946249 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x55a8b0190520 0x55a8b00ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.639+0000 7f0560b9c700 1 --2- 172.21.15.156:0/3955946249 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x55a8b0190520 0x55a8b00ffef0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:01.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.639+0000 7f0560b9c700 1 -- 172.21.15.156:0/3955946249 <== osd.1 v2:172.21.15.156:6800/132479 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f0558004520 con 0x55a8b0190520 2023-11-28T13:10:01.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.639+0000 7f055fb9a700 1 --2- 172.21.15.156:0/3955946249 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f054c066c10 0x7f054c0690c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0554000c00 tx=0x7f055400f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:01.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.639+0000 7f056ec63080 1 -- 172.21.15.156:0/3955946249 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x55a8b0190520 msgr2=0x55a8b00ffef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:01.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.639+0000 7f056ec63080 1 --2- 172.21.15.156:0/3955946249 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x55a8b0190520 0x55a8b00ffef0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.639+0000 7f056ec63080 1 -- 172.21.15.156:0/3955946249 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f054c066c10 msgr2=0x7f054c0690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:01.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.639+0000 7f056ec63080 1 --2- 172.21.15.156:0/3955946249 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f054c066c10 0x7f054c0690c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0554000c00 tx=0x7f055400f000 comp rx=0 tx=0).stop 2023-11-28T13:10:01.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.639+0000 7f056ec63080 1 -- 172.21.15.156:0/3955946249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8b0102920 msgr2=0x55a8b0188fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:01.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.639+0000 7f056ec63080 1 --2- 172.21.15.156:0/3955946249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a8b0102920 0x55a8b0188fa0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0550008810 tx=0x7f0550008b20 comp rx=0 tx=0).stop 2023-11-28T13:10:01.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.639+0000 7f0560b9c700 1 -- 172.21.15.156:0/3955946249 reap_dead start 2023-11-28T13:10:01.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.640+0000 7f056ec63080 1 -- 172.21.15.156:0/3955946249 shutdown_connections 2023-11-28T13:10:01.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.640+0000 7f056ec63080 1 -- 172.21.15.156:0/3955946249 >> 172.21.15.156:0/3955946249 conn(0x55a8b01069d0 msgr2=0x55a8b018cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:01.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.640+0000 7f056ec63080 1 -- 172.21.15.156:0/3955946249 shutdown_connections 2023-11-28T13:10:01.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.640+0000 7f056ec63080 1 -- 172.21.15.156:0/3955946249 wait complete. 2023-11-28T13:10:01.649 INFO:teuthology.orchestra.run.smithi156.stdout:Yalphaparent_a 2023-11-28T13:10:01.649 DEBUG:teuthology.orchestra.run.smithi156:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T13:10:01.811 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:10:01.811 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 1099511627777, 2023-11-28T13:10:01.811 INFO:teuthology.orchestra.run.smithi156.stdout: "ancestors": [ 2023-11-28T13:10:01.811 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:01.811 INFO:teuthology.orchestra.run.smithi156.stdout: "dirino": 1099511627776, 2023-11-28T13:10:01.812 INFO:teuthology.orchestra.run.smithi156.stdout: "dname": "alpha", 2023-11-28T13:10:01.812 INFO:teuthology.orchestra.run.smithi156.stdout: "version": 2 2023-11-28T13:10:01.812 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:01.813 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:01.813 INFO:teuthology.orchestra.run.smithi156.stdout: "dirino": 1, 2023-11-28T13:10:01.813 INFO:teuthology.orchestra.run.smithi156.stdout: "dname": "parent_a", 2023-11-28T13:10:01.813 INFO:teuthology.orchestra.run.smithi156.stdout: "version": 4 2023-11-28T13:10:01.814 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:01.814 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2023-11-28T13:10:01.814 INFO:teuthology.orchestra.run.smithi156.stdout: "pool": 8, 2023-11-28T13:10:01.814 INFO:teuthology.orchestra.run.smithi156.stdout: "old_pools": [] 2023-11-28T13:10:01.814 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:10:01.815 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:01.816 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2023-11-28T13:10:01.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.900+0000 7f3fbabba080 1 Processor -- start 2023-11-28T13:10:01.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.900+0000 7f3fbabba080 1 -- start start 2023-11-28T13:10:01.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.901+0000 7f3fbabba080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fdc6c80930 0x55fdc6d10a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.901+0000 7f3fbabba080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdc6d10260 0x55fdc6c64e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.901+0000 7f3fbabba080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdc6d10630 0x55fdc6d0f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.901+0000 7f3fbabba080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdc6d0cb40 con 0x55fdc6d10260 2023-11-28T13:10:01.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.901+0000 7f3fbabba080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdc6d0cd90 con 0x55fdc6d10630 2023-11-28T13:10:01.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.901+0000 7f3fbabba080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdc6d0e4f0 con 0x55fdc6c80930 2023-11-28T13:10:01.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.901+0000 7f3fab99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdc6d10260 0x55fdc6c64e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.901+0000 7f3fab99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdc6d10260 0x55fdc6c64e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46670/0 (socket says 172.21.15.156:46670) 2023-11-28T13:10:01.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.901+0000 7f3fab99a700 1 -- 172.21.15.156:0/192865426 learned_addr learned my addr 172.21.15.156:0/192865426 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:01.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.901+0000 7f3fac19b700 1 --2- 172.21.15.156:0/192865426 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fdc6c80930 0x55fdc6d10a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.901+0000 7f3fac99c700 1 --2- 172.21.15.156:0/192865426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdc6d10630 0x55fdc6d0f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.902+0000 7f3fab99a700 1 -- 172.21.15.156:0/192865426 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fdc6c80930 msgr2=0x55fdc6d10a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:01.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.902+0000 7f3fab99a700 1 --2- 172.21.15.156:0/192865426 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fdc6c80930 0x55fdc6d10a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.902+0000 7f3fab99a700 1 -- 172.21.15.156:0/192865426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdc6d10630 msgr2=0x55fdc6d0f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:01.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.902+0000 7f3fab99a700 1 --2- 172.21.15.156:0/192865426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdc6d10630 0x55fdc6d0f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.902+0000 7f3fab99a700 1 -- 172.21.15.156:0/192865426 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa0005de0 con 0x55fdc6d10260 2023-11-28T13:10:01.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.902+0000 7f3fab99a700 1 --2- 172.21.15.156:0/192865426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdc6d10260 0x55fdc6c64e30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0015040 tx=0x7f3fa000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f3ade8586ef438 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:01.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.902+0000 7f3faa998700 1 -- 172.21.15.156:0/192865426 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0008940 con 0x55fdc6d10260 2023-11-28T13:10:01.906 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.902+0000 7f3faa998700 1 -- 172.21.15.156:0/192865426 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa0008aa0 con 0x55fdc6d10260 2023-11-28T13:10:01.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.902+0000 7f3faa998700 1 -- 172.21.15.156:0/192865426 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0008e50 con 0x55fdc6d10260 2023-11-28T13:10:01.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.903+0000 7f3fbabba080 1 -- 172.21.15.156:0/192865426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdc6d10260 msgr2=0x55fdc6c64e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:01.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.903+0000 7f3fbabba080 1 --2- 172.21.15.156:0/192865426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdc6d10260 0x55fdc6c64e30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0015040 tx=0x7f3fa000d000 comp rx=0 tx=0).stop 2023-11-28T13:10:01.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.903+0000 7f3fbabba080 1 -- 172.21.15.156:0/192865426 shutdown_connections 2023-11-28T13:10:01.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.903+0000 7f3fbabba080 1 --2- 172.21.15.156:0/192865426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdc6d10630 0x55fdc6d0f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.903+0000 7f3fbabba080 1 --2- 172.21.15.156:0/192865426 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fdc6c80930 0x55fdc6d10a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.903+0000 7f3fbabba080 1 --2- 172.21.15.156:0/192865426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdc6d10260 0x55fdc6c64e30 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.903+0000 7f3fbabba080 1 -- 172.21.15.156:0/192865426 >> 172.21.15.156:0/192865426 conn(0x55fdc6c849e0 msgr2=0x55fdc6d0d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:01.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.903+0000 7f3fbabba080 1 -- 172.21.15.156:0/192865426 shutdown_connections 2023-11-28T13:10:01.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.903+0000 7f3fbabba080 1 -- 172.21.15.156:0/192865426 wait complete. 2023-11-28T13:10:01.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.904+0000 7f3fbabba080 1 Processor -- start 2023-11-28T13:10:01.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.904+0000 7f3fbabba080 1 -- start start 2023-11-28T13:10:01.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.905+0000 7f3fbabba080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdc6c80930 0x55fdc6d13070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.905+0000 7f3fbabba080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdc6d10260 0x55fdc6c784f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.905+0000 7f3fbabba080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fdc6d10630 0x55fdc6d046f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.905+0000 7f3fbabba080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdc6d0f8c0 con 0x55fdc6d10260 2023-11-28T13:10:01.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.905+0000 7f3fbabba080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdc6d12be0 con 0x55fdc6c80930 2023-11-28T13:10:01.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.905+0000 7f3fbabba080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdc6d11fd0 con 0x55fdc6d10630 2023-11-28T13:10:01.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.905+0000 7f3fab99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdc6d10260 0x55fdc6c784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.905+0000 7f3fab99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdc6d10260 0x55fdc6c784f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46686/0 (socket says 172.21.15.156:46686) 2023-11-28T13:10:01.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.905+0000 7f3fab99a700 1 -- 172.21.15.156:0/3810790982 learned_addr learned my addr 172.21.15.156:0/3810790982 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:01.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.905+0000 7f3fac19b700 1 --2- 172.21.15.156:0/3810790982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdc6c80930 0x55fdc6d13070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.905+0000 7f3fac99c700 1 --2- 172.21.15.156:0/3810790982 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fdc6d10630 0x55fdc6d046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.905+0000 7f3fab99a700 1 -- 172.21.15.156:0/3810790982 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fdc6d10630 msgr2=0x55fdc6d046f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:01.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.905+0000 7f3fab99a700 1 --2- 172.21.15.156:0/3810790982 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55fdc6d10630 0x55fdc6d046f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.905+0000 7f3fab99a700 1 -- 172.21.15.156:0/3810790982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdc6c80930 msgr2=0x55fdc6d13070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:01.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.905+0000 7f3fab99a700 1 --2- 172.21.15.156:0/3810790982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fdc6c80930 0x55fdc6d13070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.905+0000 7f3fab99a700 1 -- 172.21.15.156:0/3810790982 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdc6b7e4a0 con 0x55fdc6d10260 2023-11-28T13:10:01.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.906+0000 7f3fab99a700 1 --2- 172.21.15.156:0/3810790982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdc6d10260 0x55fdc6c784f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000a8c0 tx=0x7f3fa0003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:01.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.906+0000 7f3fa9996700 1 -- 172.21.15.156:0/3810790982 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa001e860 con 0x55fdc6d10260 2023-11-28T13:10:01.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.906+0000 7f3fa9996700 1 -- 172.21.15.156:0/3810790982 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa001e9e0 con 0x55fdc6d10260 2023-11-28T13:10:01.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.906+0000 7f3fa9996700 1 -- 172.21.15.156:0/3810790982 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa001ed90 con 0x55fdc6d10260 2023-11-28T13:10:01.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.906+0000 7f3fbabba080 1 -- 172.21.15.156:0/3810790982 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fdc6ba54f0 con 0x55fdc6d10260 2023-11-28T13:10:01.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.906+0000 7f3fbabba080 1 -- 172.21.15.156:0/3810790982 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdc6b8b120 con 0x55fdc6d10260 2023-11-28T13:10:01.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.908+0000 7f3fa9996700 1 -- 172.21.15.156:0/3810790982 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3fa0039070 con 0x55fdc6d10260 2023-11-28T13:10:01.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.909+0000 7f3fa9996700 1 --2- 172.21.15.156:0/3810790982 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3f98066c10 0x7f3f980690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.909+0000 7f3fa9996700 1 -- 172.21.15.156:0/3810790982 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3fa0087050 con 0x55fdc6d10260 2023-11-28T13:10:01.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.909+0000 7f3fac19b700 1 --2- 172.21.15.156:0/3810790982 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3f98066c10 0x7f3f980690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.909+0000 7f3fbabba080 1 --2- 172.21.15.156:0/3810790982 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x55fdc6d04c30 0x55fdc6db2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:01.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.909+0000 7f3fbabba080 1 -- 172.21.15.156:0/3810790982 --> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v8 -- 0x55fdc6db3000 con 0x55fdc6d04c30 2023-11-28T13:10:01.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.909+0000 7f3fac99c700 1 --2- 172.21.15.156:0/3810790982 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x55fdc6d04c30 0x55fdc6db2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:01.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.910+0000 7f3fac19b700 1 --2- 172.21.15.156:0/3810790982 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3f98066c10 0x7f3f980690c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c005fd0 tx=0x7f3f9c00e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:01.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.910+0000 7f3fac99c700 1 --2- 172.21.15.156:0/3810790982 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x55fdc6d04c30 0x55fdc6db2ac0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:01.914 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.911+0000 7f3fac99c700 1 -- 172.21.15.156:0/3810790982 <== osd.1 v2:172.21.15.156:6800/132479 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v47'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3fa400f7d0 con 0x55fdc6d04c30 2023-11-28T13:10:01.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.911+0000 7f3fbabba080 1 -- 172.21.15.156:0/3810790982 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x55fdc6d04c30 msgr2=0x55fdc6db2ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:01.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.911+0000 7f3fbabba080 1 --2- 172.21.15.156:0/3810790982 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x55fdc6d04c30 0x55fdc6db2ac0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:01.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.911+0000 7f3fbabba080 1 -- 172.21.15.156:0/3810790982 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3f98066c10 msgr2=0x7f3f980690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:01.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.911+0000 7f3fbabba080 1 --2- 172.21.15.156:0/3810790982 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3f98066c10 0x7f3f980690c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c005fd0 tx=0x7f3f9c00e040 comp rx=0 tx=0).stop 2023-11-28T13:10:01.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.911+0000 7f3fbabba080 1 -- 172.21.15.156:0/3810790982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdc6d10260 msgr2=0x55fdc6c784f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:01.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.911+0000 7f3fbabba080 1 --2- 172.21.15.156:0/3810790982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdc6d10260 0x55fdc6c784f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000a8c0 tx=0x7f3fa0003020 comp rx=0 tx=0).stop 2023-11-28T13:10:01.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.911+0000 7f3fac99c700 1 -- 172.21.15.156:0/3810790982 reap_dead start 2023-11-28T13:10:01.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.911+0000 7f3fbabba080 1 -- 172.21.15.156:0/3810790982 shutdown_connections 2023-11-28T13:10:01.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.911+0000 7f3fbabba080 1 -- 172.21.15.156:0/3810790982 >> 172.21.15.156:0/3810790982 conn(0x55fdc6c849e0 msgr2=0x55fdc6d0d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:01.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.911+0000 7f3fbabba080 1 -- 172.21.15.156:0/3810790982 shutdown_connections 2023-11-28T13:10:01.915 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:01.912+0000 7f3fbabba080 1 -- 172.21.15.156:0/3810790982 wait complete. 2023-11-28T13:10:01.917 DEBUG:teuthology.orchestra.run.smithi156:> sudo daemon-helper kill ceph --cluster ceph -w 2023-11-28T13:10:01.921 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2023-11-28T13:10:02.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.125+0000 7f238cfff700 1 Processor -- start 2023-11-28T13:10:02.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.126+0000 7f238cfff700 1 -- start start 2023-11-28T13:10:02.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.127+0000 7f238cfff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f238815cd30 0x7f2388161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.127+0000 7f238cfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23880cfda0 0x7f23880d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.127+0000 7f238cfff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23880d0840 con 0x7f238806a270 2023-11-28T13:10:02.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.127+0000 7f238cfff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23880d0b20 con 0x7f238815cd30 2023-11-28T13:10:02.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.127+0000 7f238cfff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2388160670 con 0x7f23880cfda0 2023-11-28T13:10:02.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.127+0000 7f237ffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23880cfda0 0x7f23880d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.127+0000 7f2386d9d700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f238806a270 0x7f2388070de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:43346/0 (socket says 172.21.15.156:43346) 2023-11-28T13:10:02.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.127+0000 7f238659c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f238815cd30 0x7f2388161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.127+0000 7f2386d9d700 1 -- 172.21.15.156:0/3656024436 learned_addr learned my addr 172.21.15.156:0/3656024436 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:02.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.128+0000 7f2385d9b700 1 -- 172.21.15.156:0/3656024436 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969204579 0 0) 0x7f23880d0840 con 0x7f238806a270 2023-11-28T13:10:02.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.128+0000 7f237ffff700 1 -- 172.21.15.156:0/3656024436 >> v2:172.21.15.186:3300/0 conn(0x7f238815cd30 msgr2=0x7f2388161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.128+0000 7f237ffff700 1 --2- 172.21.15.156:0/3656024436 >> v2:172.21.15.186:3300/0 conn(0x7f238815cd30 0x7f2388161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.128+0000 7f237ffff700 1 -- 172.21.15.156:0/3656024436 >> v1:172.21.15.186:6789/0 conn(0x7f238806a270 legacy=0x7f2388070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.128+0000 7f237ffff700 1 -- 172.21.15.156:0/3656024436 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2388160950 con 0x7f23880cfda0 2023-11-28T13:10:02.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.128+0000 7f238659c700 1 --2- 172.21.15.156:0/3656024436 >> v2:172.21.15.186:3300/0 conn(0x7f238815cd30 0x7f2388161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:10:02.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.128+0000 7f237ffff700 1 --2- 172.21.15.156:0/3656024436 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23880cfda0 0x7f23880d0210 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2370009840 tx=0x7f2370009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13bbd30fd14c8885 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:02.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.128+0000 7f2385d9b700 1 -- 172.21.15.156:0/3656024436 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2370011830 con 0x7f23880cfda0 2023-11-28T13:10:02.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.128+0000 7f2385d9b700 1 -- 172.21.15.156:0/3656024436 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2370011990 con 0x7f23880cfda0 2023-11-28T13:10:02.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.128+0000 7f2385d9b700 1 -- 172.21.15.156:0/3656024436 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2370011d20 con 0x7f23880cfda0 2023-11-28T13:10:02.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.129+0000 7f238cfff700 1 -- 172.21.15.156:0/3656024436 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23880cfda0 msgr2=0x7f23880d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.129+0000 7f238cfff700 1 --2- 172.21.15.156:0/3656024436 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23880cfda0 0x7f23880d0210 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2370009840 tx=0x7f2370009b50 comp rx=0 tx=0).stop 2023-11-28T13:10:02.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.129+0000 7f238cfff700 1 -- 172.21.15.156:0/3656024436 shutdown_connections 2023-11-28T13:10:02.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.129+0000 7f238cfff700 1 --2- 172.21.15.156:0/3656024436 >> v2:172.21.15.186:3300/0 conn(0x7f238815cd30 0x7f2388161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.129+0000 7f238cfff700 1 --2- 172.21.15.156:0/3656024436 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23880cfda0 0x7f23880d0210 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.129+0000 7f238cfff700 1 -- 172.21.15.156:0/3656024436 >> 172.21.15.156:0/3656024436 conn(0x7f238806b380 msgr2=0x7f238806b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:02.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.129+0000 7f238cfff700 1 -- 172.21.15.156:0/3656024436 shutdown_connections 2023-11-28T13:10:02.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.129+0000 7f238cfff700 1 -- 172.21.15.156:0/3656024436 wait complete. 2023-11-28T13:10:02.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.131+0000 7f238cfff700 1 Processor -- start 2023-11-28T13:10:02.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.131+0000 7f238cfff700 1 -- start start 2023-11-28T13:10:02.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.132+0000 7f238cfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f238806a270 0x7f23880c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.132+0000 7f238cfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23880cfda0 0x7f23880c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.132+0000 7f238cfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f238815cd30 0x7f23880b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.132+0000 7f238cfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23880c7450 con 0x7f238806a270 2023-11-28T13:10:02.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.132+0000 7f238cfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23880b3970 con 0x7f238815cd30 2023-11-28T13:10:02.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.132+0000 7f238cfff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23880b3c50 con 0x7f23880cfda0 2023-11-28T13:10:02.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.132+0000 7f238659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f238806a270 0x7f23880c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.132+0000 7f238659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f238806a270 0x7f23880c6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46696/0 (socket says 172.21.15.156:46696) 2023-11-28T13:10:02.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.132+0000 7f238659c700 1 -- 172.21.15.156:0/4019771693 learned_addr learned my addr 172.21.15.156:0/4019771693 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:02.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.133+0000 7f237ffff700 1 --2- 172.21.15.156:0/4019771693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23880cfda0 0x7f23880c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.133+0000 7f2386d9d700 1 --2- 172.21.15.156:0/4019771693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f238815cd30 0x7f23880b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.133+0000 7f238659c700 1 -- 172.21.15.156:0/4019771693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23880cfda0 msgr2=0x7f23880c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.133+0000 7f238659c700 1 --2- 172.21.15.156:0/4019771693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23880cfda0 0x7f23880c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.133+0000 7f238659c700 1 -- 172.21.15.156:0/4019771693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f238815cd30 msgr2=0x7f23880b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.133+0000 7f238659c700 1 --2- 172.21.15.156:0/4019771693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f238815cd30 0x7f23880b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.133+0000 7f238659c700 1 -- 172.21.15.156:0/4019771693 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23780094a0 con 0x7f238806a270 2023-11-28T13:10:02.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.133+0000 7f238659c700 1 --2- 172.21.15.156:0/4019771693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f238806a270 0x7f23880c6f10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f237800ebe0 tx=0x7f237800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:02.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.133+0000 7f237effd700 1 -- 172.21.15.156:0/4019771693 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378009760 con 0x7f238806a270 2023-11-28T13:10:02.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.134+0000 7f237effd700 1 -- 172.21.15.156:0/4019771693 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237800c850 con 0x7f238806a270 2023-11-28T13:10:02.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.134+0000 7f238cfff700 1 -- 172.21.15.156:0/4019771693 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23700094a0 con 0x7f238806a270 2023-11-28T13:10:02.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.134+0000 7f237effd700 1 -- 172.21.15.156:0/4019771693 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237800cbe0 con 0x7f238806a270 2023-11-28T13:10:02.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.134+0000 7f238cfff700 1 -- 172.21.15.156:0/4019771693 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23880cc280 con 0x7f238806a270 2023-11-28T13:10:02.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.136+0000 7f237effd700 1 -- 172.21.15.156:0/4019771693 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2378019030 con 0x7f238806a270 2023-11-28T13:10:02.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.137+0000 7f237effd700 1 --2- 172.21.15.156:0/4019771693 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f23680682f0 0x7f236806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.137+0000 7f237effd700 1 -- 172.21.15.156:0/4019771693 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2378085c10 con 0x7f238806a270 2023-11-28T13:10:02.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.137+0000 7f237ffff700 1 --2- 172.21.15.156:0/4019771693 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f23680682f0 0x7f236806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.139 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.138+0000 7f237ffff700 1 --2- 172.21.15.156:0/4019771693 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f23680682f0 0x7f236806a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2370009fd0 tx=0x7f2370008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:02.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.139+0000 7f23a0a0cb80 1 Processor -- start 2023-11-28T13:10:02.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.139+0000 7f23a0a0cb80 1 -- start start 2023-11-28T13:10:02.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f23a0a0cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900618e0 0x55f590037af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f23a0a0cb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f59005f200 0x55f590061cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f23a0a0cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5900621f0 0x55f59007d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f23a0a0cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58ff21c90 con 0x55f5900618e0 2023-11-28T13:10:02.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f23a0a0cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58ff90420 con 0x55f5900621f0 2023-11-28T13:10:02.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f23a0a0cb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f58ff89950 con 0x55f59005f200 2023-11-28T13:10:02.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f238659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900618e0 0x55f590037af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f238659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900618e0 0x55f590037af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46698/0 (socket says 172.21.15.156:46698) 2023-11-28T13:10:02.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f238659c700 1 -- 172.21.15.156:0/260297678 learned_addr learned my addr 172.21.15.156:0/260297678 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:02.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f237ffff700 1 --2- 172.21.15.156:0/260297678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f59005f200 0x55f590061cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f2386d9d700 1 --2- 172.21.15.156:0/260297678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5900621f0 0x55f59007d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f238659c700 1 -- 172.21.15.156:0/260297678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f59005f200 msgr2=0x55f590061cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f238659c700 1 --2- 172.21.15.156:0/260297678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f59005f200 0x55f590061cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f238659c700 1 -- 172.21.15.156:0/260297678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5900621f0 msgr2=0x55f59007d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f238659c700 1 --2- 172.21.15.156:0/260297678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5900621f0 0x55f59007d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f238659c700 1 -- 172.21.15.156:0/260297678 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f58fe0b6b0 con 0x55f5900618e0 2023-11-28T13:10:02.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f237ffff700 1 --2- 172.21.15.156:0/260297678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f59005f200 0x55f590061cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:02.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.140+0000 7f2386d9d700 1 --2- 172.21.15.156:0/260297678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5900621f0 0x55f59007d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:02.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.141+0000 7f238659c700 1 --2- 172.21.15.156:0/260297678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900618e0 0x55f590037af0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f237805e1d0 tx=0x7f237805e200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7749d7b244ba73b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:02.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.141+0000 7f2367fff700 1 -- 172.21.15.156:0/260297678 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378055530 con 0x55f5900618e0 2023-11-28T13:10:02.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.141+0000 7f2367fff700 1 -- 172.21.15.156:0/260297678 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2378055690 con 0x55f5900618e0 2023-11-28T13:10:02.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.141+0000 7f2367fff700 1 -- 172.21.15.156:0/260297678 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378055800 con 0x55f5900618e0 2023-11-28T13:10:02.149 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.141+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/260297678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900618e0 msgr2=0x55f590037af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.141+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/260297678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900618e0 0x55f590037af0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f237805e1d0 tx=0x7f237805e200 comp rx=0 tx=0).stop 2023-11-28T13:10:02.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.142+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/260297678 shutdown_connections 2023-11-28T13:10:02.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.142+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/260297678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5900621f0 0x55f59007d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.142+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/260297678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900618e0 0x55f590037af0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.142+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/260297678 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f59005f200 0x55f590061cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.142+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/260297678 >> 172.21.15.156:0/260297678 conn(0x55f590034e30 msgr2=0x55f5900357c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:02.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.142+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/260297678 shutdown_connections 2023-11-28T13:10:02.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.142+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/260297678 wait complete. 2023-11-28T13:10:02.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.143+0000 7f23a0a0cb80 1 -- start start 2023-11-28T13:10:02.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.143+0000 7f23a0a0cb80 1 Processor -- start 2023-11-28T13:10:02.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f23a0a0cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59005f200 0x55f590065640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f23a0a0cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5900618e0 0x55f590050430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f23a0a0cb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5900621f0 0x55f590050ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f23a0a0cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58fe0d9a0 con 0x55f59005f200 2023-11-28T13:10:02.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f23a0a0cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f590073910 con 0x55f5900618e0 2023-11-28T13:10:02.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f23a0a0cb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f590060960 con 0x55f5900621f0 2023-11-28T13:10:02.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f23a0a0cb80 10 client.0 _ll_register_callbacks cb 0x55f58fe2f310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:10:02.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f238659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59005f200 0x55f590065640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f238659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59005f200 0x55f590065640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46702/0 (socket says 172.21.15.156:46702) 2023-11-28T13:10:02.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f238659c700 1 -- 172.21.15.156:0/62147807 learned_addr learned my addr 172.21.15.156:0/62147807 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:02.151 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f237ffff700 1 --2- 172.21.15.156:0/62147807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5900618e0 0x55f590050430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f2386d9d700 1 --2- 172.21.15.156:0/62147807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5900621f0 0x55f590050ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f238659c700 1 -- 172.21.15.156:0/62147807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5900621f0 msgr2=0x55f590050ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f238659c700 1 --2- 172.21.15.156:0/62147807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5900621f0 0x55f590050ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f238659c700 1 -- 172.21.15.156:0/62147807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5900618e0 msgr2=0x55f590050430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f238659c700 1 --2- 172.21.15.156:0/62147807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5900618e0 0x55f590050430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f238659c700 1 -- 172.21.15.156:0/62147807 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23880cc280 con 0x55f59005f200 2023-11-28T13:10:02.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.144+0000 7f237ffff700 1 --2- 172.21.15.156:0/62147807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5900618e0 0x55f590050430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:02.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.145+0000 7f238659c700 1 --2- 172.21.15.156:0/62147807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59005f200 0x55f590065640 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f237805b010 tx=0x7f2378004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47bd72edae941156 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:02.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.145+0000 7f23657fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:10:02.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.145+0000 7f23657fa700 1 -- 172.21.15.156:0/62147807 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378051e30 con 0x55f59005f200 2023-11-28T13:10:02.152 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.145+0000 7f23657fa700 1 -- 172.21.15.156:0/62147807 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2378051f90 con 0x55f59005f200 2023-11-28T13:10:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.145+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/62147807 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f58ff57210 con 0x55f59005f200 2023-11-28T13:10:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.145+0000 7f23657fa700 1 -- 172.21.15.156:0/62147807 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2378052150 con 0x55f59005f200 2023-11-28T13:10:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.145+0000 7f23a0a0cb80 10 client.5676 fetch_fsmap learned FSMap version 23 2023-11-28T13:10:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.145+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/62147807 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55f58fdb7e60 con 0x55f59005f200 2023-11-28T13:10:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.145+0000 7f23657fa700 1 -- 172.21.15.156:0/62147807 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f58ff57210 con 0x55f59005f200 2023-11-28T13:10:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.145+0000 7f23657fa700 1 -- 172.21.15.156:0/62147807 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f2378067020 con 0x55f59005f200 2023-11-28T13:10:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.145+0000 7f23a0a0cb80 10 client.5676 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T13:10:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.145+0000 7f23a0a0cb80 10 client.5676 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T13:10:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.145+0000 7f23a0a0cb80 20 client.5676 populate_metadata read hostname 'smithi156' 2023-11-28T13:10:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.145+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/62147807 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55f590053e50 0x55f590054220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.146+0000 7f23a0a0cb80 4 client.5676 mds_command: new command op to 5481 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:10:02.153 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.146+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/62147807 --> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f58fe0dfe0 con 0x55f590053e50 2023-11-28T13:10:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.146+0000 7f237ffff700 1 --2- 172.21.15.156:0/62147807 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55f590053e50 0x55f590054220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.146+0000 7f23657fa700 10 client.5676 ms_handle_connect on v2:172.21.15.156:6832/1930782798 2023-11-28T13:10:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.146+0000 7f237ffff700 1 --2- 172.21.15.156:0/62147807 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55f590053e50 0x55f590054220 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a12e16932ce27203 server_cookie=16b4631b5edc1164 in_seq=0 out_seq=0 2023-11-28T13:10:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.147+0000 7f23657fa700 1 -- 172.21.15.156:0/62147807 <== mds.0 v2:172.21.15.156:6832/1930782798 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55f58fe0dfe0 con 0x55f590053e50 2023-11-28T13:10:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.147+0000 7f23657fa700 10 client.5676 handle_command_reply: tid=0 2023-11-28T13:10:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.147+0000 7f23a0a0cb80 1 client.5676 shutdown 2023-11-28T13:10:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.147+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/62147807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59005f200 msgr2=0x55f590065640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.147+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/62147807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59005f200 0x55f590065640 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f237805b010 tx=0x7f2378004750 comp rx=0 tx=0).stop 2023-11-28T13:10:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.147+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/62147807 shutdown_connections 2023-11-28T13:10:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.147+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/62147807 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55f590053e50 0x55f590054220 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.147+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/62147807 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5900621f0 0x55f590050ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.154 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.147+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/62147807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f59005f200 0x55f590065640 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.147+0000 7f23657fa700 0 client.5676 ms_handle_reset on v2:172.21.15.156:6832/1930782798 2023-11-28T13:10:02.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.147+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/62147807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5900618e0 0x55f590050430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.147+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/62147807 >> 172.21.15.156:0/62147807 conn(0x55f590034e30 msgr2=0x55f590037a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:02.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.148+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/62147807 shutdown_connections 2023-11-28T13:10:02.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.148+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/62147807 wait complete. 2023-11-28T13:10:02.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.148+0000 7f23a0a0cb80 20 client.5676 trim_cache size 0 max 16384 2023-11-28T13:10:02.155 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.153+0000 7f920bdff700 1 Processor -- start 2023-11-28T13:10:02.156 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.154+0000 7f920bdff700 1 -- start start 2023-11-28T13:10:02.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.155+0000 7f920bdff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f920415c9f0 0x7f92040d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.155+0000 7f920bdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92040d2b30 0x7f92040d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.155+0000 7f920bdff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92040b6810 con 0x7f92040b40c0 2023-11-28T13:10:02.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.155+0000 7f920bdff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92040b6a90 con 0x7f920415c9f0 2023-11-28T13:10:02.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.155+0000 7f920bdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92040b6d70 con 0x7f92040d2b30 2023-11-28T13:10:02.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.155+0000 7f9209b9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f920415c9f0 0x7f92040d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.155+0000 7f920a39c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f92040b40c0 0x7f9204160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53374/0 (socket says 172.21.15.156:53374) 2023-11-28T13:10:02.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.155+0000 7f920a39c700 1 -- 172.21.15.156:0/3287936366 learned_addr learned my addr 172.21.15.156:0/3287936366 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:02.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.155+0000 7f920939a700 1 --2- 172.21.15.156:0/3287936366 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92040d2b30 0x7f92040d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.156+0000 7f9209b9b700 1 -- 172.21.15.156:0/3287936366 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92040d2b30 msgr2=0x7f92040d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.156+0000 7f9209b9b700 1 --2- 172.21.15.156:0/3287936366 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92040d2b30 0x7f92040d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.156+0000 7f9209b9b700 1 -- 172.21.15.156:0/3287936366 >> v1:172.21.15.156:6789/0 conn(0x7f92040b40c0 legacy=0x7f9204160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.156+0000 7f9209b9b700 1 -- 172.21.15.156:0/3287936366 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92040b7050 con 0x7f920415c9f0 2023-11-28T13:10:02.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.156+0000 7f9209b9b700 1 --2- 172.21.15.156:0/3287936366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f920415c9f0 0x7f92040d0bb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9200009840 tx=0x7f9200009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2945346a57d41245 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:02.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.156+0000 7f9208b99700 1 -- 172.21.15.156:0/3287936366 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200011830 con 0x7f920415c9f0 2023-11-28T13:10:02.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.156+0000 7f9208b99700 1 -- 172.21.15.156:0/3287936366 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9200011990 con 0x7f920415c9f0 2023-11-28T13:10:02.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.156+0000 7f9208b99700 1 -- 172.21.15.156:0/3287936366 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200011d20 con 0x7f920415c9f0 2023-11-28T13:10:02.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.157+0000 7f920bdff700 1 -- 172.21.15.156:0/3287936366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f920415c9f0 msgr2=0x7f92040d0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.157+0000 7f920bdff700 1 --2- 172.21.15.156:0/3287936366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f920415c9f0 0x7f92040d0bb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9200009840 tx=0x7f9200009b50 comp rx=0 tx=0).stop 2023-11-28T13:10:02.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.157+0000 7f920bdff700 1 -- 172.21.15.156:0/3287936366 shutdown_connections 2023-11-28T13:10:02.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.157+0000 7f920bdff700 1 --2- 172.21.15.156:0/3287936366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f920415c9f0 0x7f92040d0bb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.157+0000 7f920bdff700 1 --2- 172.21.15.156:0/3287936366 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92040d2b30 0x7f92040d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.157+0000 7f920bdff700 1 -- 172.21.15.156:0/3287936366 >> 172.21.15.156:0/3287936366 conn(0x7f920409c350 msgr2=0x7f92040d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:02.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.157+0000 7f920bdff700 1 -- 172.21.15.156:0/3287936366 shutdown_connections 2023-11-28T13:10:02.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.157+0000 7f920bdff700 1 -- 172.21.15.156:0/3287936366 wait complete. 2023-11-28T13:10:02.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.158+0000 7f920bdff700 1 Processor -- start 2023-11-28T13:10:02.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.159+0000 7f920bdff700 1 -- start start 2023-11-28T13:10:02.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.159+0000 7f920bdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92040b40c0 0x7f92041564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.159+0000 7f920bdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92040d2b30 0x7f920409e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.160+0000 7f920bdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f920415c9f0 0x7f920409ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.160+0000 7f920bdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92040b2210 con 0x7f920415c9f0 2023-11-28T13:10:02.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.160+0000 7f920bdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920409d680 con 0x7f92040b40c0 2023-11-28T13:10:02.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.160+0000 7f920bdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920409d960 con 0x7f92040d2b30 2023-11-28T13:10:02.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.160+0000 7f920a39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f920415c9f0 0x7f920409ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.160+0000 7f920a39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f920415c9f0 0x7f920409ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46716/0 (socket says 172.21.15.156:46716) 2023-11-28T13:10:02.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.160+0000 7f920a39c700 1 -- 172.21.15.156:0/1882351353 learned_addr learned my addr 172.21.15.156:0/1882351353 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:02.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.160+0000 7f9209b9b700 1 --2- 172.21.15.156:0/1882351353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92040b40c0 0x7f92041564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.160+0000 7f920939a700 1 --2- 172.21.15.156:0/1882351353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92040d2b30 0x7f920409e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.160+0000 7f920a39c700 1 -- 172.21.15.156:0/1882351353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92040d2b30 msgr2=0x7f920409e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.160+0000 7f920a39c700 1 --2- 172.21.15.156:0/1882351353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92040d2b30 0x7f920409e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.160+0000 7f920a39c700 1 -- 172.21.15.156:0/1882351353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92040b40c0 msgr2=0x7f92041564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.160+0000 7f920a39c700 1 --2- 172.21.15.156:0/1882351353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92040b40c0 0x7f92041564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.160+0000 7f920a39c700 1 -- 172.21.15.156:0/1882351353 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92000094a0 con 0x7f920415c9f0 2023-11-28T13:10:02.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.160+0000 7f920939a700 1 --2- 172.21.15.156:0/1882351353 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92040d2b30 0x7f920409e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:02.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.160+0000 7f920a39c700 1 --2- 172.21.15.156:0/1882351353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f920415c9f0 0x7f920409ccc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00caf0 tx=0x7f91fc00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:02.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.161+0000 7f91fa7fc700 1 -- 172.21.15.156:0/1882351353 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc00aaa0 con 0x7f920415c9f0 2023-11-28T13:10:02.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.161+0000 7f91fa7fc700 1 -- 172.21.15.156:0/1882351353 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91fc00ac00 con 0x7f920415c9f0 2023-11-28T13:10:02.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.161+0000 7f91fa7fc700 1 -- 172.21.15.156:0/1882351353 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc012400 con 0x7f920415c9f0 2023-11-28T13:10:02.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.161+0000 7f920bdff700 1 -- 172.21.15.156:0/1882351353 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92040b7610 con 0x7f920415c9f0 2023-11-28T13:10:02.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.161+0000 7f920bdff700 1 -- 172.21.15.156:0/1882351353 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92040b7330 con 0x7f920415c9f0 2023-11-28T13:10:02.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.161+0000 7f920bdff700 1 -- 172.21.15.156:0/1882351353 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f92040b46f0 con 0x7f920415c9f0 2023-11-28T13:10:02.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.163+0000 7f91fa7fc700 1 -- 172.21.15.156:0/1882351353 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f91fc019030 con 0x7f920415c9f0 2023-11-28T13:10:02.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.164+0000 7f91fa7fc700 1 --2- 172.21.15.156:0/1882351353 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f91e80682f0 0x7f91e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.164+0000 7f91fa7fc700 1 -- 172.21.15.156:0/1882351353 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f91fc085f50 con 0x7f920415c9f0 2023-11-28T13:10:02.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.164+0000 7f91fa7fc700 1 -- 172.21.15.156:0/1882351353 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+502 (secure 0 0 0) 0x7f91fc086300 con 0x7f920415c9f0 2023-11-28T13:10:02.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.164+0000 7f9209b9b700 1 --2- 172.21.15.156:0/1882351353 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f91e80682f0 0x7f91e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.164+0000 7f920bdff700 1 -- 172.21.15.156:0/1882351353 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f92040b2380 con 0x7f920415c9f0 2023-11-28T13:10:02.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.165+0000 7f9209b9b700 1 --2- 172.21.15.156:0/1882351353 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f91e80682f0 0x7f91e806a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f92000090b0 tx=0x7f920001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:02.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.167+0000 7f23a0a0cb80 1 Processor -- start 2023-11-28T13:10:02.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.167+0000 7f23a0a0cb80 1 -- start start 2023-11-28T13:10:02.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.168+0000 7f23a0a0cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f59005f200 0x55f5900548f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.168+0000 7f23a0a0cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900618e0 0x55f59009d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.168+0000 7f23a0a0cb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5900621f0 0x55f590072080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.168+0000 7f23a0a0cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f590050500 con 0x55f5900618e0 2023-11-28T13:10:02.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.168+0000 7f23a0a0cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f590076130 con 0x55f59005f200 2023-11-28T13:10:02.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.168+0000 7f23a0a0cb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f590035f60 con 0x55f5900621f0 2023-11-28T13:10:02.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.168+0000 7f237ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900618e0 0x55f59009d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.168+0000 7f237ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900618e0 0x55f59009d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46730/0 (socket says 172.21.15.156:46730) 2023-11-28T13:10:02.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.168+0000 7f237ffff700 1 -- 172.21.15.156:0/4147868630 learned_addr learned my addr 172.21.15.156:0/4147868630 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:02.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.168+0000 7f238659c700 1 --2- 172.21.15.156:0/4147868630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f59005f200 0x55f5900548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.168+0000 7f2386d9d700 1 --2- 172.21.15.156:0/4147868630 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5900621f0 0x55f590072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.169+0000 7f237ffff700 1 -- 172.21.15.156:0/4147868630 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5900621f0 msgr2=0x55f590072080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.169+0000 7f237ffff700 1 --2- 172.21.15.156:0/4147868630 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5900621f0 0x55f590072080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.169+0000 7f237ffff700 1 -- 172.21.15.156:0/4147868630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f59005f200 msgr2=0x55f5900548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.169+0000 7f237ffff700 1 --2- 172.21.15.156:0/4147868630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f59005f200 0x55f5900548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.169+0000 7f237ffff700 1 -- 172.21.15.156:0/4147868630 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f58ff50210 con 0x55f5900618e0 2023-11-28T13:10:02.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.169+0000 7f237ffff700 1 --2- 172.21.15.156:0/4147868630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900618e0 0x55f59009d820 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2370013000 tx=0x7f2370010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a12f28dbb9a2a0a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:02.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.169+0000 7f237cff9700 1 -- 172.21.15.156:0/4147868630 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237002c040 con 0x55f5900618e0 2023-11-28T13:10:02.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.169+0000 7f237cff9700 1 -- 172.21.15.156:0/4147868630 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2370033480 con 0x55f5900618e0 2023-11-28T13:10:02.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.169+0000 7f237cff9700 1 -- 172.21.15.156:0/4147868630 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2370020070 con 0x55f5900618e0 2023-11-28T13:10:02.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.170+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/4147868630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900618e0 msgr2=0x55f59009d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.170+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/4147868630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900618e0 0x55f59009d820 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2370013000 tx=0x7f2370010000 comp rx=0 tx=0).stop 2023-11-28T13:10:02.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.170+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/4147868630 shutdown_connections 2023-11-28T13:10:02.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.170+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/4147868630 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f5900621f0 0x55f590072080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.170+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/4147868630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f59005f200 0x55f5900548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.170+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/4147868630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900618e0 0x55f59009d820 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.170+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/4147868630 >> 172.21.15.156:0/4147868630 conn(0x55f590034e30 msgr2=0x55f5900529a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:02.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.170+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/4147868630 shutdown_connections 2023-11-28T13:10:02.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.170+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/4147868630 wait complete. 2023-11-28T13:10:02.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.171+0000 7f23a0a0cb80 1 -- start start 2023-11-28T13:10:02.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.171+0000 7f23a0a0cb80 1 Processor -- start 2023-11-28T13:10:02.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.172+0000 7f23a0a0cb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f59005f200 0x55f590094df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.172+0000 7f23a0a0cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5900618e0 0x55f5900cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.172+0000 7f23a0a0cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900621f0 0x55f590056900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.172+0000 7f23a0a0cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5900cb4f0 con 0x55f5900621f0 2023-11-28T13:10:02.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.172+0000 7f23a0a0cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5900573f0 con 0x55f5900618e0 2023-11-28T13:10:02.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.172+0000 7f23a0a0cb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f590031370 con 0x55f59005f200 2023-11-28T13:10:02.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.172+0000 7f23a0a0cb80 10 client.0 _ll_register_callbacks cb 0x55f58fe2f310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:10:02.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.172+0000 7f2386d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900621f0 0x55f590056900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.172+0000 7f2386d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900621f0 0x55f590056900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46734/0 (socket says 172.21.15.156:46734) 2023-11-28T13:10:02.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.172+0000 7f2386d9d700 1 -- 172.21.15.156:0/620548857 learned_addr learned my addr 172.21.15.156:0/620548857 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:02.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.172+0000 7f237ffff700 1 --2- 172.21.15.156:0/620548857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5900618e0 0x55f5900cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.172+0000 7f238659c700 1 --2- 172.21.15.156:0/620548857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f59005f200 0x55f590094df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.173+0000 7f2386d9d700 1 -- 172.21.15.156:0/620548857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f59005f200 msgr2=0x55f590094df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.173+0000 7f2386d9d700 1 --2- 172.21.15.156:0/620548857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f59005f200 0x55f590094df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.173+0000 7f2386d9d700 1 -- 172.21.15.156:0/620548857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5900618e0 msgr2=0x55f5900cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.173+0000 7f2386d9d700 1 --2- 172.21.15.156:0/620548857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5900618e0 0x55f5900cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.173+0000 7f2386d9d700 1 -- 172.21.15.156:0/620548857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2388160950 con 0x55f5900621f0 2023-11-28T13:10:02.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.173+0000 7f238659c700 1 --2- 172.21.15.156:0/620548857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f59005f200 0x55f590094df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:02.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.173+0000 7f2386d9d700 1 --2- 172.21.15.156:0/620548857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900621f0 0x55f590056900 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2380001560 tx=0x7f2380015630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe8f33a6dcea763 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:02.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.173+0000 7f23657fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:10:02.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.173+0000 7f23657fa700 1 -- 172.21.15.156:0/620548857 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2380002070 con 0x55f5900621f0 2023-11-28T13:10:02.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.173+0000 7f23657fa700 1 -- 172.21.15.156:0/620548857 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2380015dd0 con 0x55f5900621f0 2023-11-28T13:10:02.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.173+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/620548857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f58ff47230 con 0x55f5900621f0 2023-11-28T13:10:02.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.173+0000 7f23657fa700 1 -- 172.21.15.156:0/620548857 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2380017430 con 0x55f5900621f0 2023-11-28T13:10:02.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.173+0000 7f23a0a0cb80 10 client.5691 fetch_fsmap learned FSMap version 23 2023-11-28T13:10:02.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.173+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/620548857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55f58fdde2d0 con 0x55f5900621f0 2023-11-28T13:10:02.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.173+0000 7f23657fa700 1 -- 172.21.15.156:0/620548857 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f58ff47230 con 0x55f5900621f0 2023-11-28T13:10:02.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.174+0000 7f23657fa700 1 -- 172.21.15.156:0/620548857 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f2380017590 con 0x55f5900621f0 2023-11-28T13:10:02.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.174+0000 7f23a0a0cb80 10 client.5691 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T13:10:02.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.174+0000 7f23a0a0cb80 10 client.5691 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T13:10:02.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.174+0000 7f23a0a0cb80 20 client.5691 populate_metadata read hostname 'smithi156' 2023-11-28T13:10:02.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.174+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/620548857 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55f590030ec0 0x55f59007f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.174+0000 7f23a0a0cb80 4 client.5691 mds_command: new command op to 5481 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-28T13:10:02.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.174+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/620548857 --> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55f58ff31240 con 0x55f590030ec0 2023-11-28T13:10:02.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.174+0000 7f238659c700 1 --2- 172.21.15.156:0/620548857 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55f590030ec0 0x55f59007f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.174+0000 7f238659c700 1 --2- 172.21.15.156:0/620548857 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55f590030ec0 0x55f59007f4e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=933406bc2c7e88e1 server_cookie=9baab19ad83c1306 in_seq=0 out_seq=0 2023-11-28T13:10:02.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.174+0000 7f23657fa700 10 client.5691 ms_handle_connect on v2:172.21.15.156:6832/1930782798 2023-11-28T13:10:02.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.175+0000 7f23657fa700 1 -- 172.21.15.156:0/620548857 <== mds.0 v2:172.21.15.156:6832/1930782798 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f58ff31240 con 0x55f590030ec0 2023-11-28T13:10:02.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.175+0000 7f23657fa700 10 client.5691 handle_command_reply: tid=0 2023-11-28T13:10:02.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.175+0000 7f23a0a0cb80 1 client.5691 shutdown 2023-11-28T13:10:02.181 INFO:tasks.ceph.mds.c.smithi156.stderr:2023-11-28T13:10:02.175+0000 7fa11dfea700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5416=0-4194304@1} caps={5416=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5416 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x559ff5783700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T13:10:02.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.175+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/620548857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900621f0 msgr2=0x55f590056900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.175+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/620548857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900621f0 0x55f590056900 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2380001560 tx=0x7f2380015630 comp rx=0 tx=0).stop 2023-11-28T13:10:02.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.176+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/620548857 shutdown_connections 2023-11-28T13:10:02.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.176+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/620548857 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55f590030ec0 0x55f59007f4e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.176+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/620548857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5900621f0 0x55f590056900 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.176+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/620548857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55f59005f200 0x55f590094df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.176+0000 7f23657fa700 0 client.5691 ms_handle_reset on v2:172.21.15.156:6832/1930782798 2023-11-28T13:10:02.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.176+0000 7f23a0a0cb80 1 --2- 172.21.15.156:0/620548857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5900618e0 0x55f5900cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.176+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/620548857 >> 172.21.15.156:0/620548857 conn(0x55f590034e30 msgr2=0x55f590074b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:02.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.176+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/620548857 shutdown_connections 2023-11-28T13:10:02.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.176+0000 7f23a0a0cb80 1 -- 172.21.15.156:0/620548857 wait complete. 2023-11-28T13:10:02.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.176+0000 7f23a0a0cb80 20 client.5691 trim_cache size 0 max 16384 2023-11-28T13:10:02.183 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:10:02.183 INFO:teuthology.orchestra.run.smithi156.stdout: "return_code": 0, 2023-11-28T13:10:02.183 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_tag": "bc42f3c3-a0e0-4bec-921d-b1eb75085cd1", 2023-11-28T13:10:02.183 INFO:teuthology.orchestra.run.smithi156.stdout: "mode": "asynchronous" 2023-11-28T13:10:02.183 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:10:02.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.177+0000 7f238cfff700 1 -- 172.21.15.156:0/4019771693 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f23680682f0 msgr2=0x7f236806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.177+0000 7f238cfff700 1 --2- 172.21.15.156:0/4019771693 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f23680682f0 0x7f236806a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2370009fd0 tx=0x7f2370008030 comp rx=0 tx=0).stop 2023-11-28T13:10:02.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.177+0000 7f238cfff700 1 -- 172.21.15.156:0/4019771693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f238806a270 msgr2=0x7f23880c6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.177+0000 7f238cfff700 1 --2- 172.21.15.156:0/4019771693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f238806a270 0x7f23880c6f10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f237800ebe0 tx=0x7f237800eef0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.177+0000 7f238cfff700 1 -- 172.21.15.156:0/4019771693 shutdown_connections 2023-11-28T13:10:02.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.177+0000 7f238cfff700 1 --2- 172.21.15.156:0/4019771693 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f23680682f0 0x7f236806a7a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.177+0000 7f238cfff700 1 --2- 172.21.15.156:0/4019771693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f238815cd30 0x7f23880b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.177+0000 7f238cfff700 1 --2- 172.21.15.156:0/4019771693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f238806a270 0x7f23880c6f10 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.177+0000 7f238cfff700 1 --2- 172.21.15.156:0/4019771693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23880cfda0 0x7f23880c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.177+0000 7f238cfff700 1 -- 172.21.15.156:0/4019771693 >> 172.21.15.156:0/4019771693 conn(0x7f238806b380 msgr2=0x7f2388158230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:02.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.177+0000 7f238cfff700 1 -- 172.21.15.156:0/4019771693 shutdown_connections 2023-11-28T13:10:02.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.177+0000 7f238cfff700 1 -- 172.21.15.156:0/4019771693 wait complete. 2023-11-28T13:10:02.196 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T13:10:02.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.393+0000 7f85d29ff700 1 Processor -- start 2023-11-28T13:10:02.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.393+0000 7f85d29ff700 1 -- start start 2023-11-28T13:10:02.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.395+0000 7f85d29ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85cc15c9b0 0x7f85cc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.395+0000 7f85d29ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f85cc0cd7c0 0x7f85cc0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.395+0000 7f85d29ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85cc0ce160 con 0x7f85cc161340 2023-11-28T13:10:02.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.395+0000 7f85d29ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85cc0b5800 con 0x7f85cc0cd7c0 2023-11-28T13:10:02.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.395+0000 7f85d29ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc0d1950 con 0x7f85cc15c9b0 2023-11-28T13:10:02.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.395+0000 7f85cbfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85cc15c9b0 0x7f85cc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.395+0000 7f85d0f9c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f85cc0cd7c0 0x7f85cc0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.395+0000 7f85cb7fe700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f85cc161340 0x7f85cc0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:43356/0 (socket says 172.21.15.156:43356) 2023-11-28T13:10:02.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.395+0000 7f85cb7fe700 1 -- 172.21.15.156:0/507562917 learned_addr learned my addr 172.21.15.156:0/507562917 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:02.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.396+0000 7f85d0f9c700 1 -- 172.21.15.156:0/507562917 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85cc15c9b0 msgr2=0x7f85cc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.396+0000 7f85d0f9c700 1 --2- 172.21.15.156:0/507562917 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85cc15c9b0 0x7f85cc0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.396+0000 7f85d0f9c700 1 -- 172.21.15.156:0/507562917 >> v1:172.21.15.186:6789/0 conn(0x7f85cc161340 legacy=0x7f85cc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.396+0000 7f85d0f9c700 1 -- 172.21.15.156:0/507562917 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85cc0d24f0 con 0x7f85cc0cd7c0 2023-11-28T13:10:02.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.396+0000 7f85d0f9c700 1 --2- 172.21.15.156:0/507562917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85cc0cd7c0 0x7f85cc0cdc20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009d90 tx=0x7f85c400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f17b0eb5c61d2c72 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:02.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.396+0000 7f85caffd700 1 -- 172.21.15.156:0/507562917 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c400e070 con 0x7f85cc0cd7c0 2023-11-28T13:10:02.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.396+0000 7f85caffd700 1 -- 172.21.15.156:0/507562917 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c400bb20 con 0x7f85cc0cd7c0 2023-11-28T13:10:02.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.396+0000 7f85caffd700 1 -- 172.21.15.156:0/507562917 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c4012430 con 0x7f85cc0cd7c0 2023-11-28T13:10:02.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.397+0000 7f85d29ff700 1 -- 172.21.15.156:0/507562917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85cc0cd7c0 msgr2=0x7f85cc0cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.397+0000 7f85d29ff700 1 --2- 172.21.15.156:0/507562917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85cc0cd7c0 0x7f85cc0cdc20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009d90 tx=0x7f85c400b330 comp rx=0 tx=0).stop 2023-11-28T13:10:02.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.397+0000 7f85d29ff700 1 -- 172.21.15.156:0/507562917 shutdown_connections 2023-11-28T13:10:02.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.397+0000 7f85d29ff700 1 --2- 172.21.15.156:0/507562917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85cc0cd7c0 0x7f85cc0cdc20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.397+0000 7f85d29ff700 1 --2- 172.21.15.156:0/507562917 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85cc15c9b0 0x7f85cc0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.397+0000 7f85d29ff700 1 -- 172.21.15.156:0/507562917 >> 172.21.15.156:0/507562917 conn(0x7f85cc072600 msgr2=0x7f85cc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:02.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.397+0000 7f85d29ff700 1 -- 172.21.15.156:0/507562917 shutdown_connections 2023-11-28T13:10:02.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.397+0000 7f85d29ff700 1 -- 172.21.15.156:0/507562917 wait complete. 2023-11-28T13:10:02.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.399+0000 7f85d29ff700 1 Processor -- start 2023-11-28T13:10:02.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.400+0000 7f85d29ff700 1 -- start start 2023-11-28T13:10:02.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.401+0000 7f85d29ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85cc0cd7c0 0x7f85cc156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.401+0000 7f85d29ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85cc15c9b0 0x7f85cc0c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.401+0000 7f85d29ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85cc161340 0x7f85cc15e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.401+0000 7f85d29ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc0c91d0 con 0x7f85cc0cd7c0 2023-11-28T13:10:02.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.401+0000 7f85d29ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc0c8ef0 con 0x7f85cc161340 2023-11-28T13:10:02.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.401+0000 7f85d29ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc0c6dd0 con 0x7f85cc15c9b0 2023-11-28T13:10:02.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.401+0000 7f85cbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85cc0cd7c0 0x7f85cc156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.401+0000 7f85cbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85cc0cd7c0 0x7f85cc156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46750/0 (socket says 172.21.15.156:46750) 2023-11-28T13:10:02.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.401+0000 7f85cbfff700 1 -- 172.21.15.156:0/4129235824 learned_addr learned my addr 172.21.15.156:0/4129235824 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:02.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.401+0000 7f85d0f9c700 1 --2- 172.21.15.156:0/4129235824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85cc161340 0x7f85cc15e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.401+0000 7f85cb7fe700 1 --2- 172.21.15.156:0/4129235824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85cc15c9b0 0x7f85cc0c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.402+0000 7f85cbfff700 1 -- 172.21.15.156:0/4129235824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85cc15c9b0 msgr2=0x7f85cc0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.402+0000 7f85cbfff700 1 --2- 172.21.15.156:0/4129235824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85cc15c9b0 0x7f85cc0c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.402+0000 7f85cbfff700 1 -- 172.21.15.156:0/4129235824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85cc161340 msgr2=0x7f85cc15e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.402+0000 7f85cbfff700 1 --2- 172.21.15.156:0/4129235824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85cc161340 0x7f85cc15e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.402+0000 7f85cbfff700 1 -- 172.21.15.156:0/4129235824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c4009a70 con 0x7f85cc0cd7c0 2023-11-28T13:10:02.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.402+0000 7f85cbfff700 1 --2- 172.21.15.156:0/4129235824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85cc0cd7c0 0x7f85cc156b90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00e870 tx=0x7f85bc00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:02.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.402+0000 7f85c8ff9700 1 -- 172.21.15.156:0/4129235824 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc009590 con 0x7f85cc0cd7c0 2023-11-28T13:10:02.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.402+0000 7f85c8ff9700 1 -- 172.21.15.156:0/4129235824 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc013680 con 0x7f85cc0cd7c0 2023-11-28T13:10:02.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.402+0000 7f85d29ff700 1 -- 172.21.15.156:0/4129235824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85cc0d1300 con 0x7f85cc0cd7c0 2023-11-28T13:10:02.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.402+0000 7f85c8ff9700 1 -- 172.21.15.156:0/4129235824 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc0139f0 con 0x7f85cc0cd7c0 2023-11-28T13:10:02.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.402+0000 7f85d29ff700 1 -- 172.21.15.156:0/4129235824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85cc0b5520 con 0x7f85cc0cd7c0 2023-11-28T13:10:02.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.404+0000 7f85c8ff9700 1 -- 172.21.15.156:0/4129235824 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f85bc019030 con 0x7f85cc0cd7c0 2023-11-28T13:10:02.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.405+0000 7f85c8ff9700 1 --2- 172.21.15.156:0/4129235824 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f85b80682f0 0x7f85b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.405+0000 7f85c8ff9700 1 -- 172.21.15.156:0/4129235824 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f85bc085a10 con 0x7f85cc0cd7c0 2023-11-28T13:10:02.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.406+0000 7f85cb7fe700 1 --2- 172.21.15.156:0/4129235824 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f85b80682f0 0x7f85b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.407+0000 7f85cb7fe700 1 --2- 172.21.15.156:0/4129235824 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f85b80682f0 0x7f85b806a7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f85c0005910 tx=0x7f85c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:02.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.408+0000 7f85e6295b80 1 Processor -- start 2023-11-28T13:10:02.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.408+0000 7f85e6295b80 1 -- start start 2023-11-28T13:10:02.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.409+0000 7f85e6295b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70d860 0x561a9e6e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.409+0000 7f85e6295b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70b180 0x561a9e70dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.409+0000 7f85e6295b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70e170 0x561a9e7297c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.409+0000 7f85e6295b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e5cdc10 con 0x561a9e70e170 2023-11-28T13:10:02.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.409+0000 7f85e6295b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e63c3a0 con 0x561a9e70b180 2023-11-28T13:10:02.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.409+0000 7f85e6295b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a9e6358d0 con 0x561a9e70d860 2023-11-28T13:10:02.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.409+0000 7f85d0f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70e170 0x561a9e7297c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.409+0000 7f85d0f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70e170 0x561a9e7297c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46758/0 (socket says 172.21.15.156:46758) 2023-11-28T13:10:02.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.409+0000 7f85d0f9c700 1 -- 172.21.15.156:0/1335461226 learned_addr learned my addr 172.21.15.156:0/1335461226 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:02.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.409+0000 7f85cb7fe700 1 --2- 172.21.15.156:0/1335461226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70b180 0x561a9e70dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.409+0000 7f85cbfff700 1 --2- 172.21.15.156:0/1335461226 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70d860 0x561a9e6e3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.410+0000 7f85d0f9c700 1 -- 172.21.15.156:0/1335461226 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70d860 msgr2=0x561a9e6e3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.410+0000 7f85d0f9c700 1 --2- 172.21.15.156:0/1335461226 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70d860 0x561a9e6e3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.410+0000 7f85d0f9c700 1 -- 172.21.15.156:0/1335461226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70b180 msgr2=0x561a9e70dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.410+0000 7f85d0f9c700 1 --2- 172.21.15.156:0/1335461226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70b180 0x561a9e70dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.410+0000 7f85d0f9c700 1 -- 172.21.15.156:0/1335461226 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a9e4b7630 con 0x561a9e70e170 2023-11-28T13:10:02.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.410+0000 7f85d0f9c700 1 --2- 172.21.15.156:0/1335461226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70e170 0x561a9e7297c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f85cc067550 tx=0x7f85c4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a005bec534c828 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:02.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.410+0000 7f85b1ffb700 1 -- 172.21.15.156:0/1335461226 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c4003de0 con 0x561a9e70e170 2023-11-28T13:10:02.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.410+0000 7f85b1ffb700 1 -- 172.21.15.156:0/1335461226 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c40125e0 con 0x561a9e70e170 2023-11-28T13:10:02.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.410+0000 7f85b1ffb700 1 -- 172.21.15.156:0/1335461226 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c4012990 con 0x561a9e70e170 2023-11-28T13:10:02.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.411+0000 7f85e6295b80 1 -- 172.21.15.156:0/1335461226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70e170 msgr2=0x561a9e7297c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.411+0000 7f85e6295b80 1 --2- 172.21.15.156:0/1335461226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70e170 0x561a9e7297c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f85cc067550 tx=0x7f85c4010000 comp rx=0 tx=0).stop 2023-11-28T13:10:02.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.411+0000 7f85e6295b80 1 -- 172.21.15.156:0/1335461226 shutdown_connections 2023-11-28T13:10:02.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.411+0000 7f85e6295b80 1 --2- 172.21.15.156:0/1335461226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70e170 0x561a9e7297c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.411+0000 7f85e6295b80 1 --2- 172.21.15.156:0/1335461226 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70d860 0x561a9e6e3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.411+0000 7f85e6295b80 1 --2- 172.21.15.156:0/1335461226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70b180 0x561a9e70dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.411+0000 7f85e6295b80 1 -- 172.21.15.156:0/1335461226 >> 172.21.15.156:0/1335461226 conn(0x561a9e6e0db0 msgr2=0x561a9e6e1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:02.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.411+0000 7f85e6295b80 1 -- 172.21.15.156:0/1335461226 shutdown_connections 2023-11-28T13:10:02.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.411+0000 7f85e6295b80 1 -- 172.21.15.156:0/1335461226 wait complete. 2023-11-28T13:10:02.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.412+0000 7f85e6295b80 1 -- start start 2023-11-28T13:10:02.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.412+0000 7f85e6295b80 1 Processor -- start 2023-11-28T13:10:02.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.413+0000 7f85e6295b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70b180 0x561a9e703500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.413+0000 7f85e6295b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70d860 0x561a9e71ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.413+0000 7f85e6295b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70e170 0x561a9e6ff050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.413+0000 7f85e6295b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e4b9920 con 0x561a9e70b180 2023-11-28T13:10:02.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.413+0000 7f85e6295b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e6ffde0 con 0x561a9e70e170 2023-11-28T13:10:02.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.413+0000 7f85e6295b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a9e7207c0 con 0x561a9e70d860 2023-11-28T13:10:02.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.413+0000 7f85e6295b80 10 client.0 _ll_register_callbacks cb 0x561a9e4db290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:10:02.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.413+0000 7f85cbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70b180 0x561a9e703500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.413+0000 7f85cbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70b180 0x561a9e703500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46772/0 (socket says 172.21.15.156:46772) 2023-11-28T13:10:02.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.413+0000 7f85cbfff700 1 -- 172.21.15.156:0/4255355032 learned_addr learned my addr 172.21.15.156:0/4255355032 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:02.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.413+0000 7f85d0f9c700 1 --2- 172.21.15.156:0/4255355032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70e170 0x561a9e6ff050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.414+0000 7f85cbfff700 1 -- 172.21.15.156:0/4255355032 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70d860 msgr2=0x561a9e71ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.414+0000 7f85cbfff700 1 --2- 172.21.15.156:0/4255355032 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70d860 0x561a9e71ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.414+0000 7f85cbfff700 1 -- 172.21.15.156:0/4255355032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70e170 msgr2=0x561a9e6ff050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.414+0000 7f85cbfff700 1 --2- 172.21.15.156:0/4255355032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70e170 0x561a9e6ff050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.414+0000 7f85cbfff700 1 -- 172.21.15.156:0/4255355032 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85cc0d24f0 con 0x561a9e70b180 2023-11-28T13:10:02.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.414+0000 7f85d0f9c700 1 --2- 172.21.15.156:0/4255355032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70e170 0x561a9e6ff050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:02.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.414+0000 7f85cbfff700 1 --2- 172.21.15.156:0/4255355032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70b180 0x561a9e703500 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f85bc055910 tx=0x7f85bc0556e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d723d5b986bd214a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:02.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.414+0000 7f85aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:10:02.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.414+0000 7f85aaffd700 1 -- 172.21.15.156:0/4255355032 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc051c30 con 0x561a9e70b180 2023-11-28T13:10:02.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.414+0000 7f85aaffd700 1 -- 172.21.15.156:0/4255355032 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f85bc051d90 con 0x561a9e70b180 2023-11-28T13:10:02.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.414+0000 7f85e6295b80 1 -- 172.21.15.156:0/4255355032 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a9e603190 con 0x561a9e70b180 2023-11-28T13:10:02.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.414+0000 7f85aaffd700 1 -- 172.21.15.156:0/4255355032 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85bc051f50 con 0x561a9e70b180 2023-11-28T13:10:02.422 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.414+0000 7f85e6295b80 10 client.5700 fetch_fsmap learned FSMap version 23 2023-11-28T13:10:02.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.414+0000 7f85e6295b80 1 -- 172.21.15.156:0/4255355032 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x561a9e463de0 con 0x561a9e70b180 2023-11-28T13:10:02.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.414+0000 7f85aaffd700 1 -- 172.21.15.156:0/4255355032 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561a9e603190 con 0x561a9e70b180 2023-11-28T13:10:02.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.414+0000 7f85aaffd700 1 -- 172.21.15.156:0/4255355032 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f85bc054da0 con 0x561a9e70b180 2023-11-28T13:10:02.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.415+0000 7f85e6295b80 10 client.5700 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T13:10:02.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.415+0000 7f85e6295b80 10 client.5700 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T13:10:02.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.415+0000 7f85e6295b80 20 client.5700 populate_metadata read hostname 'smithi156' 2023-11-28T13:10:02.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.415+0000 7f85e6295b80 1 --2- 172.21.15.156:0/4255355032 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x561a9e6fd830 0x561a9e745570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.415+0000 7f85e6295b80 4 client.5700 mds_command: new command op to 5481 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:10:02.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.415+0000 7f85e6295b80 1 -- 172.21.15.156:0/4255355032 --> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a9e4b9f60 con 0x561a9e6fd830 2023-11-28T13:10:02.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.415+0000 7f85cb7fe700 1 --2- 172.21.15.156:0/4255355032 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x561a9e6fd830 0x561a9e745570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.415+0000 7f85aaffd700 10 client.5700 ms_handle_connect on v2:172.21.15.156:6832/1930782798 2023-11-28T13:10:02.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.415+0000 7f85cb7fe700 1 --2- 172.21.15.156:0/4255355032 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x561a9e6fd830 0x561a9e745570 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=657e00c3d0d1f35c server_cookie=c0f3f2cb1bc56617 in_seq=0 out_seq=0 2023-11-28T13:10:02.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.416+0000 7f85aaffd700 1 -- 172.21.15.156:0/4255355032 <== mds.0 v2:172.21.15.156:6832/1930782798 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x561a9e4b9f60 con 0x561a9e6fd830 2023-11-28T13:10:02.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.416+0000 7f85aaffd700 10 client.5700 handle_command_reply: tid=0 2023-11-28T13:10:02.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.416+0000 7f85e6295b80 1 client.5700 shutdown 2023-11-28T13:10:02.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.416+0000 7f85e6295b80 1 -- 172.21.15.156:0/4255355032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70b180 msgr2=0x561a9e703500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.416+0000 7f85e6295b80 1 --2- 172.21.15.156:0/4255355032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70b180 0x561a9e703500 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f85bc055910 tx=0x7f85bc0556e0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.416+0000 7f85e6295b80 1 -- 172.21.15.156:0/4255355032 shutdown_connections 2023-11-28T13:10:02.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.416+0000 7f85e6295b80 1 --2- 172.21.15.156:0/4255355032 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x561a9e6fd830 0x561a9e745570 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.416+0000 7f85e6295b80 1 --2- 172.21.15.156:0/4255355032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70e170 0x561a9e6ff050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.416+0000 7f85e6295b80 1 --2- 172.21.15.156:0/4255355032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70b180 0x561a9e703500 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.416+0000 7f85aaffd700 0 client.5700 ms_handle_reset on v2:172.21.15.156:6832/1930782798 2023-11-28T13:10:02.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.416+0000 7f85e6295b80 1 --2- 172.21.15.156:0/4255355032 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70d860 0x561a9e71ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.416+0000 7f85e6295b80 1 -- 172.21.15.156:0/4255355032 >> 172.21.15.156:0/4255355032 conn(0x561a9e6e0db0 msgr2=0x561a9e6e39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:02.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.417+0000 7f85e6295b80 1 -- 172.21.15.156:0/4255355032 shutdown_connections 2023-11-28T13:10:02.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.417+0000 7f85e6295b80 1 -- 172.21.15.156:0/4255355032 wait complete. 2023-11-28T13:10:02.425 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.417+0000 7f85e6295b80 20 client.5700 trim_cache size 0 max 16384 2023-11-28T13:10:02.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.434+0000 7f85e6295b80 1 Processor -- start 2023-11-28T13:10:02.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.434+0000 7f85e6295b80 1 -- start start 2023-11-28T13:10:02.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85e6295b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70b180 0x561a9e6de0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85e6295b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70d860 0x561a9e6fb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85e6295b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70e170 0x561a9e6fbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85e6295b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e6e03e0 con 0x561a9e70b180 2023-11-28T13:10:02.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85e6295b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e70de00 con 0x561a9e70d860 2023-11-28T13:10:02.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85e6295b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a9e6e1860 con 0x561a9e70e170 2023-11-28T13:10:02.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85cbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70b180 0x561a9e6de0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85cbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70b180 0x561a9e6de0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46778/0 (socket says 172.21.15.156:46778) 2023-11-28T13:10:02.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85cbfff700 1 -- 172.21.15.156:0/3692337598 learned_addr learned my addr 172.21.15.156:0/3692337598 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:02.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85cb7fe700 1 --2- 172.21.15.156:0/3692337598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70d860 0x561a9e6fb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85cbfff700 1 -- 172.21.15.156:0/3692337598 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70e170 msgr2=0x561a9e6fbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85d0f9c700 1 --2- 172.21.15.156:0/3692337598 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70e170 0x561a9e6fbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85cbfff700 1 --2- 172.21.15.156:0/3692337598 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70e170 0x561a9e6fbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85cbfff700 1 -- 172.21.15.156:0/3692337598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70d860 msgr2=0x561a9e6fb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85cbfff700 1 --2- 172.21.15.156:0/3692337598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70d860 0x561a9e6fb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85cbfff700 1 -- 172.21.15.156:0/3692337598 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a9e5fc190 con 0x561a9e70b180 2023-11-28T13:10:02.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85cb7fe700 1 --2- 172.21.15.156:0/3692337598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70d860 0x561a9e6fb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:02.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.435+0000 7f85d0f9c700 1 --2- 172.21.15.156:0/3692337598 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70e170 0x561a9e6fbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:02.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.436+0000 7f85cbfff700 1 --2- 172.21.15.156:0/3692337598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70b180 0x561a9e6de0c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f85bc052340 tx=0x7f85bc06a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9591fa2af0ca8e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:02.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.436+0000 7f85b27fc700 1 -- 172.21.15.156:0/3692337598 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc052180 con 0x561a9e70b180 2023-11-28T13:10:02.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.436+0000 7f85b27fc700 1 -- 172.21.15.156:0/3692337598 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc069210 con 0x561a9e70b180 2023-11-28T13:10:02.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.436+0000 7f85b27fc700 1 -- 172.21.15.156:0/3692337598 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc054da0 con 0x561a9e70b180 2023-11-28T13:10:02.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.437+0000 7f85e6295b80 1 -- 172.21.15.156:0/3692337598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70b180 msgr2=0x561a9e6de0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.437+0000 7f85e6295b80 1 --2- 172.21.15.156:0/3692337598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70b180 0x561a9e6de0c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f85bc052340 tx=0x7f85bc06a000 comp rx=0 tx=0).stop 2023-11-28T13:10:02.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.437+0000 7f85e6295b80 1 -- 172.21.15.156:0/3692337598 shutdown_connections 2023-11-28T13:10:02.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.437+0000 7f85e6295b80 1 --2- 172.21.15.156:0/3692337598 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70e170 0x561a9e6fbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.437+0000 7f85e6295b80 1 --2- 172.21.15.156:0/3692337598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70b180 0x561a9e6de0c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.437+0000 7f85e6295b80 1 --2- 172.21.15.156:0/3692337598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70d860 0x561a9e6fb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.437+0000 7f85e6295b80 1 -- 172.21.15.156:0/3692337598 >> 172.21.15.156:0/3692337598 conn(0x561a9e6e0db0 msgr2=0x561a9e71fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:02.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.437+0000 7f85e6295b80 1 -- 172.21.15.156:0/3692337598 shutdown_connections 2023-11-28T13:10:02.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.437+0000 7f85e6295b80 1 -- 172.21.15.156:0/3692337598 wait complete. 2023-11-28T13:10:02.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.437+0000 7f85e6295b80 1 -- start start 2023-11-28T13:10:02.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.438+0000 7f85e6295b80 1 Processor -- start 2023-11-28T13:10:02.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.439+0000 7f85e6295b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70b180 0x561a9e701f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.439+0000 7f85e6295b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70d860 0x561a9e702470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.439+0000 7f85e6295b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70e170 0x561a9e71e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.439+0000 7f85e6295b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e741e80 con 0x561a9e70e170 2023-11-28T13:10:02.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.439+0000 7f85e6295b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e78e6a0 con 0x561a9e70b180 2023-11-28T13:10:02.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.439+0000 7f85e6295b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a9e78e490 con 0x561a9e70d860 2023-11-28T13:10:02.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.439+0000 7f85e6295b80 10 client.0 _ll_register_callbacks cb 0x561a9e4db290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:10:02.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.440+0000 7f85d0f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70e170 0x561a9e71e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.440+0000 7f85d0f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70e170 0x561a9e71e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46790/0 (socket says 172.21.15.156:46790) 2023-11-28T13:10:02.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.440+0000 7f85d0f9c700 1 -- 172.21.15.156:0/3889416223 learned_addr learned my addr 172.21.15.156:0/3889416223 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:02.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.440+0000 7f85cbfff700 1 --2- 172.21.15.156:0/3889416223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70b180 0x561a9e701f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.440+0000 7f85cb7fe700 1 --2- 172.21.15.156:0/3889416223 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70d860 0x561a9e702470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.440+0000 7f85d0f9c700 1 -- 172.21.15.156:0/3889416223 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70d860 msgr2=0x561a9e702470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.440+0000 7f85d0f9c700 1 --2- 172.21.15.156:0/3889416223 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70d860 0x561a9e702470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.440+0000 7f85d0f9c700 1 -- 172.21.15.156:0/3889416223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70b180 msgr2=0x561a9e701f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:02.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.440+0000 7f85d0f9c700 1 --2- 172.21.15.156:0/3889416223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70b180 0x561a9e701f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.440+0000 7f85d0f9c700 1 -- 172.21.15.156:0/3889416223 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a9e463de0 con 0x561a9e70e170 2023-11-28T13:10:02.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.440+0000 7f85cb7fe700 1 --2- 172.21.15.156:0/3889416223 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70d860 0x561a9e702470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:02.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.440+0000 7f85d0f9c700 1 --2- 172.21.15.156:0/3889416223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70e170 0x561a9e71e000 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009a40 tx=0x7f85c4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da95fc4def7a4940 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:02.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.440+0000 7f85aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:10:02.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.441+0000 7f85aaffd700 1 -- 172.21.15.156:0/3889416223 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c4012d70 con 0x561a9e70e170 2023-11-28T13:10:02.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.441+0000 7f85aaffd700 1 -- 172.21.15.156:0/3889416223 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f85c4038c20 con 0x561a9e70e170 2023-11-28T13:10:02.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.441+0000 7f85e6295b80 1 -- 172.21.15.156:0/3889416223 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a9e5f31b0 con 0x561a9e70e170 2023-11-28T13:10:02.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.441+0000 7f85aaffd700 1 -- 172.21.15.156:0/3889416223 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85c4038de0 con 0x561a9e70e170 2023-11-28T13:10:02.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.441+0000 7f85e6295b80 10 client.5706 fetch_fsmap learned FSMap version 23 2023-11-28T13:10:02.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.441+0000 7f85e6295b80 1 -- 172.21.15.156:0/3889416223 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x561a9e48a250 con 0x561a9e70e170 2023-11-28T13:10:02.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.441+0000 7f85aaffd700 1 -- 172.21.15.156:0/3889416223 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561a9e5f31b0 con 0x561a9e70e170 2023-11-28T13:10:02.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.441+0000 7f85aaffd700 1 -- 172.21.15.156:0/3889416223 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f85c4035030 con 0x561a9e70e170 2023-11-28T13:10:02.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.441+0000 7f85e6295b80 10 client.5706 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T13:10:02.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.441+0000 7f85e6295b80 10 client.5706 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T13:10:02.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.441+0000 7f85e6295b80 20 client.5706 populate_metadata read hostname 'smithi156' 2023-11-28T13:10:02.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.441+0000 7f85e6295b80 1 --2- 172.21.15.156:0/3889416223 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x561a9e7734e0 0x561a9e71e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:02.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.441+0000 7f85e6295b80 4 client.5706 mds_command: new command op to 5481 tid=0[{"prefix": "scrub status"}] 2023-11-28T13:10:02.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.441+0000 7f85e6295b80 1 -- 172.21.15.156:0/3889416223 --> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561a9e5dd1c0 con 0x561a9e7734e0 2023-11-28T13:10:02.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.441+0000 7f85cbfff700 1 --2- 172.21.15.156:0/3889416223 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x561a9e7734e0 0x561a9e71e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:02.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.442+0000 7f85cbfff700 1 --2- 172.21.15.156:0/3889416223 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x561a9e7734e0 0x561a9e71e540 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c34ac02433f14f37 server_cookie=45a80fbb22a28b1a in_seq=0 out_seq=0 2023-11-28T13:10:02.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.442+0000 7f85aaffd700 10 client.5706 ms_handle_connect on v2:172.21.15.156:6832/1930782798 2023-11-28T13:10:02.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.442+0000 7f85aaffd700 1 -- 172.21.15.156:0/3889416223 <== mds.0 v2:172.21.15.156:6832/1930782798 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x561a9e5dd1c0 con 0x561a9e7734e0 2023-11-28T13:10:02.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.442+0000 7f85aaffd700 10 client.5706 handle_command_reply: tid=0 2023-11-28T13:10:02.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.442+0000 7f85e6295b80 1 client.5706 shutdown 2023-11-28T13:10:02.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.442+0000 7f85e6295b80 1 -- 172.21.15.156:0/3889416223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70e170 msgr2=0x561a9e71e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.442+0000 7f85e6295b80 1 --2- 172.21.15.156:0/3889416223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70e170 0x561a9e71e000 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009a40 tx=0x7f85c4004b40 comp rx=0 tx=0).stop 2023-11-28T13:10:02.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.442+0000 7f85e6295b80 1 -- 172.21.15.156:0/3889416223 shutdown_connections 2023-11-28T13:10:02.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.442+0000 7f85e6295b80 1 --2- 172.21.15.156:0/3889416223 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x561a9e7734e0 0x561a9e71e540 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.442+0000 7f85e6295b80 1 --2- 172.21.15.156:0/3889416223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a9e70e170 0x561a9e71e000 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.442+0000 7f85e6295b80 1 --2- 172.21.15.156:0/3889416223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a9e70b180 0x561a9e701f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.442+0000 7f85aaffd700 0 client.5706 ms_handle_reset on v2:172.21.15.156:6832/1930782798 2023-11-28T13:10:02.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.442+0000 7f85e6295b80 1 --2- 172.21.15.156:0/3889416223 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a9e70d860 0x561a9e702470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.442+0000 7f85e6295b80 1 -- 172.21.15.156:0/3889416223 >> 172.21.15.156:0/3889416223 conn(0x561a9e6e0db0 msgr2=0x561a9e749700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:02.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.443+0000 7f85e6295b80 1 -- 172.21.15.156:0/3889416223 shutdown_connections 2023-11-28T13:10:02.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.443+0000 7f85e6295b80 1 -- 172.21.15.156:0/3889416223 wait complete. 2023-11-28T13:10:02.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.443+0000 7f85e6295b80 20 client.5706 trim_cache size 0 max 16384 2023-11-28T13:10:02.451 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:10:02.451 INFO:teuthology.orchestra.run.smithi156.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T13:10:02.451 INFO:teuthology.orchestra.run.smithi156.stdout: "scrubs": { 2023-11-28T13:10:02.451 INFO:teuthology.orchestra.run.smithi156.stdout: "bc42f3c3-a0e0-4bec-921d-b1eb75085cd1": { 2023-11-28T13:10:02.451 INFO:teuthology.orchestra.run.smithi156.stdout: "path": "/", 2023-11-28T13:10:02.451 INFO:teuthology.orchestra.run.smithi156.stdout: "tag": "bc42f3c3-a0e0-4bec-921d-b1eb75085cd1", 2023-11-28T13:10:02.452 INFO:teuthology.orchestra.run.smithi156.stdout: "options": "recursive,repair" 2023-11-28T13:10:02.452 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:02.452 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:02.452 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:10:02.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.443+0000 7f85d29ff700 1 -- 172.21.15.156:0/4129235824 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f85b80682f0 msgr2=0x7f85b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.443+0000 7f85d29ff700 1 --2- 172.21.15.156:0/4129235824 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f85b80682f0 0x7f85b806a7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f85c0005910 tx=0x7f85c0009040 comp rx=0 tx=0).stop 2023-11-28T13:10:02.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.443+0000 7f85d29ff700 1 -- 172.21.15.156:0/4129235824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85cc0cd7c0 msgr2=0x7f85cc156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:02.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.443+0000 7f85d29ff700 1 --2- 172.21.15.156:0/4129235824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85cc0cd7c0 0x7f85cc156b90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00e870 tx=0x7f85bc00eb80 comp rx=0 tx=0).stop 2023-11-28T13:10:02.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.443+0000 7f85d29ff700 1 -- 172.21.15.156:0/4129235824 shutdown_connections 2023-11-28T13:10:02.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.443+0000 7f85d29ff700 1 --2- 172.21.15.156:0/4129235824 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f85b80682f0 0x7f85b806a7a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.443+0000 7f85d29ff700 1 --2- 172.21.15.156:0/4129235824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85cc161340 0x7f85cc15e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.443+0000 7f85d29ff700 1 --2- 172.21.15.156:0/4129235824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85cc0cd7c0 0x7f85cc156b90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.443+0000 7f85d29ff700 1 --2- 172.21.15.156:0/4129235824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85cc15c9b0 0x7f85cc0c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:02.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.443+0000 7f85d29ff700 1 -- 172.21.15.156:0/4129235824 >> 172.21.15.156:0/4129235824 conn(0x7f85cc072600 msgr2=0x7f85cc06a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:02.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.443+0000 7f85d29ff700 1 -- 172.21.15.156:0/4129235824 shutdown_connections 2023-11-28T13:10:02.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.444+0000 7f85d29ff700 1 -- 172.21.15.156:0/4129235824 wait complete. 2023-11-28T13:10:02.462 INFO:tasks.cephfs.filesystem:scrub status for tag:bc42f3c3-a0e0-4bec-921d-b1eb75085cd1 - {'path': '/', 'tag': 'bc42f3c3-a0e0-4bec-921d-b1eb75085cd1', 'options': 'recursive,repair'} 2023-11-28T13:10:02.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:02.633+0000 7f91fa7fc700 1 -- 172.21.15.156:0/1882351353 <== mon.0 v2:172.21.15.156:3300/0 7 ==== log(6 entries from seq 1 at 2023-11-28T13:10:02.176452+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f91fc0862e0 con 0x7f920415c9f0 2023-11-28T13:10:02.636 INFO:teuthology.orchestra.run.smithi156.stdout: cluster: 2023-11-28T13:10:02.636 INFO:teuthology.orchestra.run.smithi156.stdout: id: 12d9e849-dc60-46f0-8e8e-9e291633f61b 2023-11-28T13:10:02.636 INFO:teuthology.orchestra.run.smithi156.stdout: health: HEALTH_OK 2023-11-28T13:10:02.637 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:02.637 INFO:teuthology.orchestra.run.smithi156.stdout: services: 2023-11-28T13:10:02.637 INFO:teuthology.orchestra.run.smithi156.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2023-11-28T13:10:02.637 INFO:teuthology.orchestra.run.smithi156.stdout: mgr: x(active, since 2m), standbys: y 2023-11-28T13:10:02.638 INFO:teuthology.orchestra.run.smithi156.stdout: mds: 1/1 daemons up, 3 standby 2023-11-28T13:10:02.638 INFO:teuthology.orchestra.run.smithi156.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2023-11-28T13:10:02.638 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:02.638 INFO:teuthology.orchestra.run.smithi156.stdout: data: 2023-11-28T13:10:02.638 INFO:teuthology.orchestra.run.smithi156.stdout: volumes: 1/1 healthy 2023-11-28T13:10:02.639 INFO:teuthology.orchestra.run.smithi156.stdout: pools: 4 pools, 105 pgs 2023-11-28T13:10:02.639 INFO:teuthology.orchestra.run.smithi156.stdout: objects: 23 objects, 579 KiB 2023-11-28T13:10:02.639 INFO:teuthology.orchestra.run.smithi156.stdout: usage: 88 MiB used, 720 GiB / 720 GiB avail 2023-11-28T13:10:02.639 INFO:teuthology.orchestra.run.smithi156.stdout: pgs: 105 active+clean 2023-11-28T13:10:02.639 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:02.639 INFO:teuthology.orchestra.run.smithi156.stdout: io: 2023-11-28T13:10:02.640 INFO:teuthology.orchestra.run.smithi156.stdout: client: 1.7 KiB/s wr, 0 op/s rd, 5 op/s wr 2023-11-28T13:10:02.640 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:02.640 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:02.640 INFO:teuthology.orchestra.run.smithi156.stdout:2023-11-28T13:10:02.176452+0000 mds.c [INF] scrub queued for path: / 2023-11-28T13:10:02.640 INFO:teuthology.orchestra.run.smithi156.stdout:2023-11-28T13:10:02.176456+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2023-11-28T13:10:02.641 INFO:teuthology.orchestra.run.smithi156.stdout:2023-11-28T13:10:02.176462+0000 mds.c [INF] scrub summary: active paths [/] 2023-11-28T13:10:02.641 INFO:teuthology.orchestra.run.smithi156.stdout:2023-11-28T13:10:02.177259+0000 mds.c [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2023-11-28T13:10:02.641 INFO:teuthology.orchestra.run.smithi156.stdout:2023-11-28T13:10:02.177267+0000 mds.c [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2023-11-28T13:10:02.641 INFO:teuthology.orchestra.run.smithi156.stdout:2023-11-28T13:10:02.178209+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2023-11-28T13:10:03.109 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:10:03.115 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:10:07.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:07.643+0000 7f91fa7fc700 1 -- 172.21.15.156:0/1882351353 <== mon.0 v2:172.21.15.156:3300/0 8 ==== log(1 entries from seq 7 at 2023-11-28T13:10:06.242588+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f91fc05e410 con 0x7f920415c9f0 2023-11-28T13:10:07.645 INFO:teuthology.orchestra.run.smithi156.stdout:2023-11-28T13:10:06.242588+0000 mds.c [INF] scrub summary: idle 2023-11-28T13:10:32.463 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T13:10:32.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.672+0000 7fd5879ff700 1 Processor -- start 2023-11-28T13:10:32.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.672+0000 7fd5879ff700 1 -- start start 2023-11-28T13:10:32.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.673+0000 7fd5879ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd58015c9b0 0x7fd5800b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.673+0000 7fd5879ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd580161340 0x7fd5800d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.673+0000 7fd5879ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5800d1950 con 0x7fd5800d2320 2023-11-28T13:10:32.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.673+0000 7fd5879ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5800b5800 con 0x7fd58015c9b0 2023-11-28T13:10:32.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.673+0000 7fd5879ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5800b5520 con 0x7fd580161340 2023-11-28T13:10:32.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.673+0000 7fd58579b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd58015c9b0 0x7fd5800b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:32.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.673+0000 7fd58579b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd58015c9b0 0x7fd5800b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48518/0 (socket says 172.21.15.156:48518) 2023-11-28T13:10:32.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.673+0000 7fd58579b700 1 -- 172.21.15.156:0/1333804560 learned_addr learned my addr 172.21.15.156:0/1333804560 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:32.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.673+0000 7fd58579b700 1 -- 172.21.15.156:0/1333804560 >> v2:172.21.15.186:3300/0 conn(0x7fd580161340 msgr2=0x7fd5800d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:32.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.673+0000 7fd58579b700 1 --2- 172.21.15.156:0/1333804560 >> v2:172.21.15.186:3300/0 conn(0x7fd580161340 0x7fd5800d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.673+0000 7fd58579b700 1 -- 172.21.15.156:0/1333804560 >> v1:172.21.15.186:6789/0 conn(0x7fd5800d2320 legacy=0x7fd5800cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:32.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.673+0000 7fd58579b700 1 -- 172.21.15.156:0/1333804560 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5800d1300 con 0x7fd58015c9b0 2023-11-28T13:10:32.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.674+0000 7fd58579b700 1 --2- 172.21.15.156:0/1333804560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd58015c9b0 0x7fd5800b4bf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd5800685c0 tx=0x7fd57c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4433affeaa01679f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:32.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.674+0000 7fd577fff700 1 -- 172.21.15.156:0/1333804560 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c004230 con 0x7fd58015c9b0 2023-11-28T13:10:32.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.674+0000 7fd577fff700 1 -- 172.21.15.156:0/1333804560 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c004390 con 0x7fd58015c9b0 2023-11-28T13:10:32.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.674+0000 7fd577fff700 1 -- 172.21.15.156:0/1333804560 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c0036a0 con 0x7fd58015c9b0 2023-11-28T13:10:32.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.674+0000 7fd5879ff700 1 -- 172.21.15.156:0/1333804560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd58015c9b0 msgr2=0x7fd5800b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:32.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.674+0000 7fd5879ff700 1 --2- 172.21.15.156:0/1333804560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd58015c9b0 0x7fd5800b4bf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd5800685c0 tx=0x7fd57c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.675+0000 7fd5879ff700 1 -- 172.21.15.156:0/1333804560 shutdown_connections 2023-11-28T13:10:32.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.675+0000 7fd5879ff700 1 --2- 172.21.15.156:0/1333804560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd58015c9b0 0x7fd5800b4bf0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.675+0000 7fd5879ff700 1 --2- 172.21.15.156:0/1333804560 >> v2:172.21.15.186:3300/0 conn(0x7fd580161340 0x7fd5800d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.675+0000 7fd5879ff700 1 -- 172.21.15.156:0/1333804560 >> 172.21.15.156:0/1333804560 conn(0x7fd580072600 msgr2=0x7fd5800cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:32.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.675+0000 7fd5879ff700 1 -- 172.21.15.156:0/1333804560 shutdown_connections 2023-11-28T13:10:32.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.675+0000 7fd5879ff700 1 -- 172.21.15.156:0/1333804560 wait complete. 2023-11-28T13:10:32.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.676+0000 7fd5879ff700 1 Processor -- start 2023-11-28T13:10:32.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.676+0000 7fd5879ff700 1 -- start start 2023-11-28T13:10:32.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.677+0000 7fd5879ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5800d2320 0x7fd5801463b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.677+0000 7fd5879ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd58015c9b0 0x7fd5801543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.677+0000 7fd5879ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd580161340 0x7fd5801f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.677+0000 7fd5879ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd580157250 con 0x7fd58015c9b0 2023-11-28T13:10:32.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.677+0000 7fd5879ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd580156610 con 0x7fd5800d2320 2023-11-28T13:10:32.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.677+0000 7fd5879ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5801568b0 con 0x7fd580161340 2023-11-28T13:10:32.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.678+0000 7fd584f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd58015c9b0 0x7fd5801543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:32.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.678+0000 7fd584f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd58015c9b0 0x7fd5801543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48534/0 (socket says 172.21.15.156:48534) 2023-11-28T13:10:32.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.678+0000 7fd584f9a700 1 -- 172.21.15.156:0/3801846200 learned_addr learned my addr 172.21.15.156:0/3801846200 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:32.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.678+0000 7fd58579b700 1 --2- 172.21.15.156:0/3801846200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5800d2320 0x7fd5801463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:32.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.678+0000 7fd585f9c700 1 --2- 172.21.15.156:0/3801846200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd580161340 0x7fd5801f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:32.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.678+0000 7fd584f9a700 1 -- 172.21.15.156:0/3801846200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd580161340 msgr2=0x7fd5801f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:32.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.678+0000 7fd584f9a700 1 --2- 172.21.15.156:0/3801846200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd580161340 0x7fd5801f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.678+0000 7fd584f9a700 1 -- 172.21.15.156:0/3801846200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5800d2320 msgr2=0x7fd5801463b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:32.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.678+0000 7fd584f9a700 1 --2- 172.21.15.156:0/3801846200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5800d2320 0x7fd5801463b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.678+0000 7fd584f9a700 1 -- 172.21.15.156:0/3801846200 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c009550 con 0x7fd58015c9b0 2023-11-28T13:10:32.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.678+0000 7fd584f9a700 1 --2- 172.21.15.156:0/3801846200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd58015c9b0 0x7fd5801543f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd57000b530 tx=0x7fd57000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:32.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.678+0000 7fd575ffb700 1 -- 172.21.15.156:0/3801846200 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570011830 con 0x7fd58015c9b0 2023-11-28T13:10:32.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.679+0000 7fd575ffb700 1 -- 172.21.15.156:0/3801846200 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd570011990 con 0x7fd58015c9b0 2023-11-28T13:10:32.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.679+0000 7fd5879ff700 1 -- 172.21.15.156:0/3801846200 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5800b5130 con 0x7fd58015c9b0 2023-11-28T13:10:32.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.679+0000 7fd575ffb700 1 -- 172.21.15.156:0/3801846200 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570011d20 con 0x7fd58015c9b0 2023-11-28T13:10:32.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.679+0000 7fd5879ff700 1 -- 172.21.15.156:0/3801846200 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5800d15e0 con 0x7fd58015c9b0 2023-11-28T13:10:32.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.681+0000 7fd575ffb700 1 -- 172.21.15.156:0/3801846200 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd570021070 con 0x7fd58015c9b0 2023-11-28T13:10:32.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.682+0000 7fd575ffb700 1 --2- 172.21.15.156:0/3801846200 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd5640682f0 0x7fd56406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.682+0000 7fd575ffb700 1 -- 172.21.15.156:0/3801846200 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd570085cf0 con 0x7fd58015c9b0 2023-11-28T13:10:32.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.683+0000 7fd58579b700 1 --2- 172.21.15.156:0/3801846200 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd5640682f0 0x7fd56406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:32.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.683+0000 7fd58579b700 1 --2- 172.21.15.156:0/3801846200 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd5640682f0 0x7fd56406a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009130 tx=0x7fd57c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:32.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.684+0000 7fd59b381b80 1 Processor -- start 2023-11-28T13:10:32.686 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.684+0000 7fd59b381b80 1 -- start start 2023-11-28T13:10:32.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.685+0000 7fd59b381b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983e860 0x55a349814a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.685+0000 7fd59b381b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983c180 0x55a34983ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.685+0000 7fd59b381b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983f170 0x55a34985a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.685+0000 7fd59b381b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3496fec10 con 0x55a34983f170 2023-11-28T13:10:32.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.685+0000 7fd59b381b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a34976d3a0 con 0x55a34983c180 2023-11-28T13:10:32.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.685+0000 7fd59b381b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3497668d0 con 0x55a34983e860 2023-11-28T13:10:32.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.685+0000 7fd585f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983f170 0x55a34985a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:32.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.685+0000 7fd585f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983f170 0x55a34985a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48542/0 (socket says 172.21.15.156:48542) 2023-11-28T13:10:32.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.685+0000 7fd58579b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983e860 0x55a349814a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:32.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.685+0000 7fd585f9c700 1 -- 172.21.15.156:0/2780906912 learned_addr learned my addr 172.21.15.156:0/2780906912 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:32.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.685+0000 7fd584f9a700 1 --2- 172.21.15.156:0/2780906912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983c180 0x55a34983ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:32.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.686+0000 7fd585f9c700 1 -- 172.21.15.156:0/2780906912 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983e860 msgr2=0x55a349814a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:32.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.686+0000 7fd585f9c700 1 --2- 172.21.15.156:0/2780906912 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983e860 0x55a349814a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.686+0000 7fd585f9c700 1 -- 172.21.15.156:0/2780906912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983c180 msgr2=0x55a34983ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:32.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.686+0000 7fd585f9c700 1 --2- 172.21.15.156:0/2780906912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983c180 0x55a34983ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.686+0000 7fd585f9c700 1 -- 172.21.15.156:0/2780906912 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3495e8630 con 0x55a34983f170 2023-11-28T13:10:32.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.686+0000 7fd585f9c700 1 --2- 172.21.15.156:0/2780906912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983f170 0x55a34985a7c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd578009f60 tx=0x7fd578009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68b40cea66dc356 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:32.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.686+0000 7fd56affd700 1 -- 172.21.15.156:0/2780906912 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578011070 con 0x55a34983f170 2023-11-28T13:10:32.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.686+0000 7fd56affd700 1 -- 172.21.15.156:0/2780906912 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57800ca30 con 0x55a34983f170 2023-11-28T13:10:32.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.686+0000 7fd56affd700 1 -- 172.21.15.156:0/2780906912 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57800ce20 con 0x55a34983f170 2023-11-28T13:10:32.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.687+0000 7fd59b381b80 1 -- 172.21.15.156:0/2780906912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983f170 msgr2=0x55a34985a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:32.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.687+0000 7fd59b381b80 1 --2- 172.21.15.156:0/2780906912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983f170 0x55a34985a7c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd578009f60 tx=0x7fd578009f90 comp rx=0 tx=0).stop 2023-11-28T13:10:32.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.687+0000 7fd59b381b80 1 -- 172.21.15.156:0/2780906912 shutdown_connections 2023-11-28T13:10:32.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.687+0000 7fd59b381b80 1 --2- 172.21.15.156:0/2780906912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983f170 0x55a34985a7c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.687+0000 7fd59b381b80 1 --2- 172.21.15.156:0/2780906912 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983e860 0x55a349814a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.687+0000 7fd59b381b80 1 --2- 172.21.15.156:0/2780906912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983c180 0x55a34983ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.687+0000 7fd59b381b80 1 -- 172.21.15.156:0/2780906912 >> 172.21.15.156:0/2780906912 conn(0x55a349811db0 msgr2=0x55a349812740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:32.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.687+0000 7fd59b381b80 1 -- 172.21.15.156:0/2780906912 shutdown_connections 2023-11-28T13:10:32.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.687+0000 7fd59b381b80 1 -- 172.21.15.156:0/2780906912 wait complete. 2023-11-28T13:10:32.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.688+0000 7fd59b381b80 1 -- start start 2023-11-28T13:10:32.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.688+0000 7fd59b381b80 1 Processor -- start 2023-11-28T13:10:32.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.689+0000 7fd59b381b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983c180 0x55a349840430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.689+0000 7fd59b381b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983e860 0x55a34983cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.689+0000 7fd59b381b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983f170 0x55a34983d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.689+0000 7fd59b381b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3495ea920 con 0x55a34983e860 2023-11-28T13:10:32.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.689+0000 7fd59b381b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3498420e0 con 0x55a34983f170 2023-11-28T13:10:32.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.689+0000 7fd59b381b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a34985e270 con 0x55a34983c180 2023-11-28T13:10:32.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.689+0000 7fd59b381b80 10 client.0 _ll_register_callbacks cb 0x55a34960c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:10:32.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.689+0000 7fd584f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983e860 0x55a34983cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:32.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.689+0000 7fd584f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983e860 0x55a34983cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48552/0 (socket says 172.21.15.156:48552) 2023-11-28T13:10:32.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.689+0000 7fd584f9a700 1 -- 172.21.15.156:0/1508656619 learned_addr learned my addr 172.21.15.156:0/1508656619 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:32.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd58579b700 1 --2- 172.21.15.156:0/1508656619 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983c180 0x55a349840430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:32.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd584f9a700 1 -- 172.21.15.156:0/1508656619 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983c180 msgr2=0x55a349840430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:32.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd584f9a700 1 --2- 172.21.15.156:0/1508656619 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983c180 0x55a349840430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd585f9c700 1 --2- 172.21.15.156:0/1508656619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983f170 0x55a34983d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:32.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd584f9a700 1 -- 172.21.15.156:0/1508656619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983f170 msgr2=0x55a34983d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:32.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd584f9a700 1 --2- 172.21.15.156:0/1508656619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983f170 0x55a34983d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd584f9a700 1 -- 172.21.15.156:0/1508656619 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd578009be0 con 0x55a34983e860 2023-11-28T13:10:32.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd58579b700 1 --2- 172.21.15.156:0/1508656619 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983c180 0x55a349840430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:32.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd585f9c700 1 --2- 172.21.15.156:0/1508656619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983f170 0x55a34983d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:32.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd584f9a700 1 --2- 172.21.15.156:0/1508656619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983e860 0x55a34983cb10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55a349816210 tx=0x7fd5700558f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc3491a28dd2859 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:32.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd55ffff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:10:32.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd55ffff700 1 -- 172.21.15.156:0/1508656619 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570051f10 con 0x55a34983e860 2023-11-28T13:10:32.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd55ffff700 1 -- 172.21.15.156:0/1508656619 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd570052070 con 0x55a34983e860 2023-11-28T13:10:32.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd59b381b80 1 -- 172.21.15.156:0/1508656619 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a349734190 con 0x55a34983e860 2023-11-28T13:10:32.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd55ffff700 1 -- 172.21.15.156:0/1508656619 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd570052230 con 0x55a34983e860 2023-11-28T13:10:32.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd59b381b80 10 client.5718 fetch_fsmap learned FSMap version 23 2023-11-28T13:10:32.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd59b381b80 1 -- 172.21.15.156:0/1508656619 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55a349594de0 con 0x55a34983e860 2023-11-28T13:10:32.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.690+0000 7fd55ffff700 1 -- 172.21.15.156:0/1508656619 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a349734190 con 0x55a34983e860 2023-11-28T13:10:32.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.691+0000 7fd55ffff700 1 -- 172.21.15.156:0/1508656619 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fd570055080 con 0x55a34983e860 2023-11-28T13:10:32.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.691+0000 7fd59b381b80 10 client.5718 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T13:10:32.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.691+0000 7fd59b381b80 10 client.5718 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T13:10:32.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.691+0000 7fd59b381b80 20 client.5718 populate_metadata read hostname 'smithi156' 2023-11-28T13:10:32.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.691+0000 7fd59b381b80 1 --2- 172.21.15.156:0/1508656619 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55a349834420 0x55a349841250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.691+0000 7fd59b381b80 4 client.5718 mds_command: new command op to 5481 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:10:32.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.691+0000 7fd59b381b80 1 -- 172.21.15.156:0/1508656619 --> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3495eaf60 con 0x55a349834420 2023-11-28T13:10:32.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.691+0000 7fd58579b700 1 --2- 172.21.15.156:0/1508656619 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55a349834420 0x55a349841250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:32.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.691+0000 7fd55ffff700 10 client.5718 ms_handle_connect on v2:172.21.15.156:6832/1930782798 2023-11-28T13:10:32.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.691+0000 7fd58579b700 1 --2- 172.21.15.156:0/1508656619 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55a349834420 0x55a349841250 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4f5309926f3d5 server_cookie=cac5535316b0ef79 in_seq=0 out_seq=0 2023-11-28T13:10:32.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.692+0000 7fd55ffff700 1 -- 172.21.15.156:0/1508656619 <== mds.0 v2:172.21.15.156:6832/1930782798 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55a3495eaf60 con 0x55a349834420 2023-11-28T13:10:32.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.692+0000 7fd55ffff700 10 client.5718 handle_command_reply: tid=0 2023-11-28T13:10:32.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.692+0000 7fd59b381b80 1 client.5718 shutdown 2023-11-28T13:10:32.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.692+0000 7fd59b381b80 1 -- 172.21.15.156:0/1508656619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983e860 msgr2=0x55a34983cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:32.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.692+0000 7fd59b381b80 1 --2- 172.21.15.156:0/1508656619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983e860 0x55a34983cb10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55a349816210 tx=0x7fd5700558f0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.692+0000 7fd59b381b80 1 -- 172.21.15.156:0/1508656619 shutdown_connections 2023-11-28T13:10:32.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.692+0000 7fd59b381b80 1 --2- 172.21.15.156:0/1508656619 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55a349834420 0x55a349841250 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.692+0000 7fd59b381b80 1 --2- 172.21.15.156:0/1508656619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983f170 0x55a34983d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.692+0000 7fd59b381b80 1 --2- 172.21.15.156:0/1508656619 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983c180 0x55a349840430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.692+0000 7fd55ffff700 0 client.5718 ms_handle_reset on v2:172.21.15.156:6832/1930782798 2023-11-28T13:10:32.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.692+0000 7fd59b381b80 1 --2- 172.21.15.156:0/1508656619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983e860 0x55a34983cb10 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.692+0000 7fd59b381b80 1 -- 172.21.15.156:0/1508656619 >> 172.21.15.156:0/1508656619 conn(0x55a349811db0 msgr2=0x55a349813280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:32.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.692+0000 7fd59b381b80 1 -- 172.21.15.156:0/1508656619 shutdown_connections 2023-11-28T13:10:32.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.693+0000 7fd59b381b80 1 -- 172.21.15.156:0/1508656619 wait complete. 2023-11-28T13:10:32.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.693+0000 7fd59b381b80 20 client.5718 trim_cache size 0 max 16384 2023-11-28T13:10:32.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.710+0000 7fd59b381b80 1 Processor -- start 2023-11-28T13:10:32.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.710+0000 7fd59b381b80 1 -- start start 2023-11-28T13:10:32.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.710+0000 7fd59b381b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983c180 0x55a349840430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.710+0000 7fd59b381b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983e860 0x55a349812180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.710+0000 7fd59b381b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983f170 0x55a349842410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.710+0000 7fd59b381b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a34980e8e0 con 0x55a34983e860 2023-11-28T13:10:32.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.710+0000 7fd59b381b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a34983bd60 con 0x55a34983f170 2023-11-28T13:10:32.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.710+0000 7fd59b381b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3498342a0 con 0x55a34983c180 2023-11-28T13:10:32.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.711+0000 7fd584f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983e860 0x55a349812180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:32.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.711+0000 7fd584f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983e860 0x55a349812180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48556/0 (socket says 172.21.15.156:48556) 2023-11-28T13:10:32.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.711+0000 7fd584f9a700 1 -- 172.21.15.156:0/841657327 learned_addr learned my addr 172.21.15.156:0/841657327 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:32.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.711+0000 7fd584f9a700 1 -- 172.21.15.156:0/841657327 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983c180 msgr2=0x55a349840430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:32.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.711+0000 7fd584f9a700 1 --2- 172.21.15.156:0/841657327 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983c180 0x55a349840430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.711+0000 7fd584f9a700 1 -- 172.21.15.156:0/841657327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983f170 msgr2=0x55a349842410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:32.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.711+0000 7fd584f9a700 1 --2- 172.21.15.156:0/841657327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983f170 0x55a349842410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.711+0000 7fd584f9a700 1 -- 172.21.15.156:0/841657327 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a34972d190 con 0x55a34983e860 2023-11-28T13:10:32.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.711+0000 7fd584f9a700 1 --2- 172.21.15.156:0/841657327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983e860 0x55a349812180 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd57005e370 tx=0x7fd57006a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b986fc0033fbe524 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:32.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.711+0000 7fd56b7fe700 1 -- 172.21.15.156:0/841657327 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570055c60 con 0x55a34983e860 2023-11-28T13:10:32.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.711+0000 7fd56b7fe700 1 -- 172.21.15.156:0/841657327 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd570055080 con 0x55a34983e860 2023-11-28T13:10:32.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.711+0000 7fd56b7fe700 1 -- 172.21.15.156:0/841657327 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570005c70 con 0x55a34983e860 2023-11-28T13:10:32.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.712+0000 7fd59b381b80 1 -- 172.21.15.156:0/841657327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983e860 msgr2=0x55a349812180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:32.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.712+0000 7fd59b381b80 1 --2- 172.21.15.156:0/841657327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983e860 0x55a349812180 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd57005e370 tx=0x7fd57006a000 comp rx=0 tx=0).stop 2023-11-28T13:10:32.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.712+0000 7fd59b381b80 1 -- 172.21.15.156:0/841657327 shutdown_connections 2023-11-28T13:10:32.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.712+0000 7fd59b381b80 1 --2- 172.21.15.156:0/841657327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983f170 0x55a349842410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.712+0000 7fd59b381b80 1 --2- 172.21.15.156:0/841657327 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983c180 0x55a349840430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.712+0000 7fd59b381b80 1 --2- 172.21.15.156:0/841657327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983e860 0x55a349812180 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.712+0000 7fd59b381b80 1 -- 172.21.15.156:0/841657327 >> 172.21.15.156:0/841657327 conn(0x55a349811db0 msgr2=0x55a349840e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:32.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.712+0000 7fd59b381b80 1 -- 172.21.15.156:0/841657327 shutdown_connections 2023-11-28T13:10:32.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.712+0000 7fd59b381b80 1 -- 172.21.15.156:0/841657327 wait complete. 2023-11-28T13:10:32.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.713+0000 7fd59b381b80 1 -- start start 2023-11-28T13:10:32.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.713+0000 7fd59b381b80 1 Processor -- start 2023-11-28T13:10:32.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.714+0000 7fd59b381b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983c180 0x55a3498bc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.714+0000 7fd59b381b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983e860 0x55a3498bf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.714+0000 7fd59b381b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983f170 0x55a3498bf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.714+0000 7fd59b381b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3498bf010 con 0x55a34983f170 2023-11-28T13:10:32.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.714+0000 7fd59b381b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a349814320 con 0x55a34983e860 2023-11-28T13:10:32.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.714+0000 7fd59b381b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a34985bdc0 con 0x55a34983c180 2023-11-28T13:10:32.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.714+0000 7fd59b381b80 10 client.0 _ll_register_callbacks cb 0x55a34960c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:10:32.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.714+0000 7fd585f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983f170 0x55a3498bf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:32.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.714+0000 7fd58579b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983c180 0x55a3498bc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:32.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.714+0000 7fd585f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983f170 0x55a3498bf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48564/0 (socket says 172.21.15.156:48564) 2023-11-28T13:10:32.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.714+0000 7fd585f9c700 1 -- 172.21.15.156:0/455295546 learned_addr learned my addr 172.21.15.156:0/455295546 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:32.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.714+0000 7fd58579b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983c180 0x55a3498bc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:49666/0 (socket says 172.21.15.156:49666) 2023-11-28T13:10:32.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.714+0000 7fd584f9a700 1 --2- 172.21.15.156:0/455295546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983e860 0x55a3498bf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:32.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.715+0000 7fd58579b700 1 -- 172.21.15.156:0/455295546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983e860 msgr2=0x55a3498bf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:32.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.715+0000 7fd58579b700 1 --2- 172.21.15.156:0/455295546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983e860 0x55a3498bf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.715+0000 7fd58579b700 1 -- 172.21.15.156:0/455295546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983f170 msgr2=0x55a3498bf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:32.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.715+0000 7fd58579b700 1 --2- 172.21.15.156:0/455295546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983f170 0x55a3498bf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.715+0000 7fd58579b700 1 -- 172.21.15.156:0/455295546 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a349594de0 con 0x55a34983c180 2023-11-28T13:10:32.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.715+0000 7fd585f9c700 1 --2- 172.21.15.156:0/455295546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983f170 0x55a3498bf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:10:32.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.715+0000 7fd58579b700 1 --2- 172.21.15.156:0/455295546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983c180 0x55a3498bc540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd57c015000 tx=0x7fd57c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4f566b9a284090d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:32.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.715+0000 7fd55ffff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3301/0 2023-11-28T13:10:32.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.715+0000 7fd55ffff700 1 -- 172.21.15.156:0/455295546 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c020070 con 0x55a34983c180 2023-11-28T13:10:32.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.715+0000 7fd55ffff700 1 -- 172.21.15.156:0/455295546 <== mon.2 v2:172.21.15.186:3301/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd57c034430 con 0x55a34983c180 2023-11-28T13:10:32.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.715+0000 7fd59b381b80 1 -- 172.21.15.156:0/455295546 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3497241b0 con 0x55a34983c180 2023-11-28T13:10:32.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.715+0000 7fd55ffff700 1 -- 172.21.15.156:0/455295546 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd57c034840 con 0x55a34983c180 2023-11-28T13:10:32.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.715+0000 7fd59b381b80 10 client.5144 fetch_fsmap learned FSMap version 23 2023-11-28T13:10:32.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.715+0000 7fd59b381b80 1 -- 172.21.15.156:0/455295546 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55a3495bb250 con 0x55a34983c180 2023-11-28T13:10:32.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.716+0000 7fd55ffff700 1 -- 172.21.15.156:0/455295546 <== mon.2 v2:172.21.15.186:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a3497241b0 con 0x55a34983c180 2023-11-28T13:10:32.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.716+0000 7fd55ffff700 1 -- 172.21.15.156:0/455295546 <== mon.2 v2:172.21.15.186:3301/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fd57c0349a0 con 0x55a34983c180 2023-11-28T13:10:32.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.716+0000 7fd59b381b80 10 client.5144 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T13:10:32.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.716+0000 7fd59b381b80 10 client.5144 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T13:10:32.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.716+0000 7fd59b381b80 20 client.5144 populate_metadata read hostname 'smithi156' 2023-11-28T13:10:32.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.716+0000 7fd59b381b80 1 --2- 172.21.15.156:0/455295546 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55a349833af0 0x55a34984f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:32.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.716+0000 7fd59b381b80 4 client.5144 mds_command: new command op to 5481 tid=0[{"prefix": "scrub status"}] 2023-11-28T13:10:32.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.716+0000 7fd59b381b80 1 -- 172.21.15.156:0/455295546 --> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a34970e1c0 con 0x55a349833af0 2023-11-28T13:10:32.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.716+0000 7fd584f9a700 1 --2- 172.21.15.156:0/455295546 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55a349833af0 0x55a34984f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:32.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.716+0000 7fd584f9a700 1 --2- 172.21.15.156:0/455295546 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55a349833af0 0x55a34984f2a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32cb2f13a7d34d89 server_cookie=a261bc04737918e8 in_seq=0 out_seq=0 2023-11-28T13:10:32.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.716+0000 7fd55ffff700 10 client.5144 ms_handle_connect on v2:172.21.15.156:6832/1930782798 2023-11-28T13:10:32.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.717+0000 7fd55ffff700 1 -- 172.21.15.156:0/455295546 <== mds.0 v2:172.21.15.156:6832/1930782798 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a34970e1c0 con 0x55a349833af0 2023-11-28T13:10:32.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.717+0000 7fd55ffff700 10 client.5144 handle_command_reply: tid=0 2023-11-28T13:10:32.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.717+0000 7fd59b381b80 1 client.5144 shutdown 2023-11-28T13:10:32.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.717+0000 7fd59b381b80 1 -- 172.21.15.156:0/455295546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983c180 msgr2=0x55a3498bc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:32.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.717+0000 7fd59b381b80 1 --2- 172.21.15.156:0/455295546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983c180 0x55a3498bc540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd57c015000 tx=0x7fd57c004750 comp rx=0 tx=0).stop 2023-11-28T13:10:32.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.717+0000 7fd59b381b80 1 -- 172.21.15.156:0/455295546 shutdown_connections 2023-11-28T13:10:32.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.717+0000 7fd59b381b80 1 --2- 172.21.15.156:0/455295546 >> [v2:172.21.15.156:6832/1930782798,v1:172.21.15.156:6834/1930782798] conn(0x55a349833af0 0x55a34984f2a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.717+0000 7fd59b381b80 1 --2- 172.21.15.156:0/455295546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a34983f170 0x55a3498bf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.717+0000 7fd59b381b80 1 --2- 172.21.15.156:0/455295546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55a34983c180 0x55a3498bc540 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.717+0000 7fd59b381b80 1 --2- 172.21.15.156:0/455295546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a34983e860 0x55a3498bf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.717+0000 7fd55ffff700 0 client.5144 ms_handle_reset on v2:172.21.15.156:6832/1930782798 2023-11-28T13:10:32.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.717+0000 7fd59b381b80 1 -- 172.21.15.156:0/455295546 >> 172.21.15.156:0/455295546 conn(0x55a349811db0 msgr2=0x55a349842410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:32.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.717+0000 7fd59b381b80 1 -- 172.21.15.156:0/455295546 shutdown_connections 2023-11-28T13:10:32.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.717+0000 7fd59b381b80 1 -- 172.21.15.156:0/455295546 wait complete. 2023-11-28T13:10:32.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.717+0000 7fd59b381b80 20 client.5144 trim_cache size 0 max 16384 2023-11-28T13:10:32.729 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:10:32.729 INFO:teuthology.orchestra.run.smithi156.stdout: "status": "no active scrubs running", 2023-11-28T13:10:32.729 INFO:teuthology.orchestra.run.smithi156.stdout: "scrubs": {} 2023-11-28T13:10:32.729 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:10:32.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.718+0000 7fd5879ff700 1 -- 172.21.15.156:0/3801846200 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd5640682f0 msgr2=0x7fd56406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:32.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.718+0000 7fd5879ff700 1 --2- 172.21.15.156:0/3801846200 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd5640682f0 0x7fd56406a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009130 tx=0x7fd57c008030 comp rx=0 tx=0).stop 2023-11-28T13:10:32.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.718+0000 7fd5879ff700 1 -- 172.21.15.156:0/3801846200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd58015c9b0 msgr2=0x7fd5801543f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:32.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.718+0000 7fd5879ff700 1 --2- 172.21.15.156:0/3801846200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd58015c9b0 0x7fd5801543f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd57000b530 tx=0x7fd57000b840 comp rx=0 tx=0).stop 2023-11-28T13:10:32.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.718+0000 7fd5879ff700 1 -- 172.21.15.156:0/3801846200 shutdown_connections 2023-11-28T13:10:32.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.718+0000 7fd5879ff700 1 --2- 172.21.15.156:0/3801846200 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd5640682f0 0x7fd56406a7a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.718+0000 7fd5879ff700 1 --2- 172.21.15.156:0/3801846200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd580161340 0x7fd5801f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.718+0000 7fd5879ff700 1 --2- 172.21.15.156:0/3801846200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5800d2320 0x7fd5801463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.718+0000 7fd5879ff700 1 --2- 172.21.15.156:0/3801846200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd58015c9b0 0x7fd5801543f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.718+0000 7fd5879ff700 1 -- 172.21.15.156:0/3801846200 >> 172.21.15.156:0/3801846200 conn(0x7fd580072600 msgr2=0x7fd58015a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.718+0000 7fd5879ff700 1 -- 172.21.15.156:0/3801846200 shutdown_connections 2023-11-28T13:10:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:32.718+0000 7fd5879ff700 1 -- 172.21.15.156:0/3801846200 wait complete. 2023-11-28T13:10:32.737 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T13:10:32.838 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:10:33.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.036+0000 7fcf8edff700 1 Processor -- start 2023-11-28T13:10:33.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.036+0000 7fcf8edff700 1 -- start start 2023-11-28T13:10:33.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.038+0000 7fcf8edff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcf88161340 0x7fcf880d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:33.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.038+0000 7fcf8edff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf880d2320 0x7fcf880cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:33.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.038+0000 7fcf8edff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf880d1950 con 0x7fcf8815c9b0 2023-11-28T13:10:33.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.038+0000 7fcf8edff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf880b5800 con 0x7fcf88161340 2023-11-28T13:10:33.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.038+0000 7fcf8edff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf880b5300 con 0x7fcf880d2320 2023-11-28T13:10:33.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.038+0000 7fcf87fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcf88161340 0x7fcf880d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:33.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.038+0000 7fcf87fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcf88161340 0x7fcf880d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48574/0 (socket says 172.21.15.156:48574) 2023-11-28T13:10:33.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.038+0000 7fcf87fff700 1 -- 172.21.15.156:0/3182819163 learned_addr learned my addr 172.21.15.156:0/3182819163 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:33.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.038+0000 7fcf8d39c700 1 --2- 172.21.15.156:0/3182819163 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf880d2320 0x7fcf880cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:33.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.038+0000 7fcf87fff700 1 -- 172.21.15.156:0/3182819163 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf880d2320 msgr2=0x7fcf880cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:33.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.038+0000 7fcf87fff700 1 --2- 172.21.15.156:0/3182819163 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf880d2320 0x7fcf880cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.038+0000 7fcf87fff700 1 -- 172.21.15.156:0/3182819163 >> v1:172.21.15.186:6789/0 conn(0x7fcf8815c9b0 legacy=0x7fcf880b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:33.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.038+0000 7fcf87fff700 1 -- 172.21.15.156:0/3182819163 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf880b5520 con 0x7fcf88161340 2023-11-28T13:10:33.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.038+0000 7fcf8d39c700 1 --2- 172.21.15.156:0/3182819163 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf880d2320 0x7fcf880cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:33.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.039+0000 7fcf87fff700 1 --2- 172.21.15.156:0/3182819163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf88161340 0x7fcf880d1de0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcf740098f0 tx=0x7fcf74009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffc51f65ce44aff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:33.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.039+0000 7fcf877fe700 1 -- 172.21.15.156:0/3182819163 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7400b8a0 con 0x7fcf88161340 2023-11-28T13:10:33.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.039+0000 7fcf877fe700 1 -- 172.21.15.156:0/3182819163 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7400ba00 con 0x7fcf88161340 2023-11-28T13:10:33.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.039+0000 7fcf877fe700 1 -- 172.21.15.156:0/3182819163 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7400bd90 con 0x7fcf88161340 2023-11-28T13:10:33.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.039+0000 7fcf8edff700 1 -- 172.21.15.156:0/3182819163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf88161340 msgr2=0x7fcf880d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:33.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.039+0000 7fcf8edff700 1 --2- 172.21.15.156:0/3182819163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf88161340 0x7fcf880d1de0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcf740098f0 tx=0x7fcf74009c00 comp rx=0 tx=0).stop 2023-11-28T13:10:33.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.039+0000 7fcf8edff700 1 -- 172.21.15.156:0/3182819163 shutdown_connections 2023-11-28T13:10:33.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.039+0000 7fcf8edff700 1 --2- 172.21.15.156:0/3182819163 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf880d2320 0x7fcf880cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.039+0000 7fcf8edff700 1 --2- 172.21.15.156:0/3182819163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf88161340 0x7fcf880d1de0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.039+0000 7fcf8edff700 1 -- 172.21.15.156:0/3182819163 >> 172.21.15.156:0/3182819163 conn(0x7fcf88072600 msgr2=0x7fcf880cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:33.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.040+0000 7fcf8edff700 1 -- 172.21.15.156:0/3182819163 shutdown_connections 2023-11-28T13:10:33.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.040+0000 7fcf8edff700 1 -- 172.21.15.156:0/3182819163 wait complete. 2023-11-28T13:10:33.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.041+0000 7fcf8edff700 1 Processor -- start 2023-11-28T13:10:33.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.041+0000 7fcf8edff700 1 -- start start 2023-11-28T13:10:33.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.042+0000 7fcf8edff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf880d2320 0x7fcf8814c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:33.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.042+0000 7fcf8edff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf8815c9b0 0x7fcf880b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:33.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.042+0000 7fcf8edff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf88161340 0x7fcf880b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:33.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.042+0000 7fcf8edff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf880cb030 con 0x7fcf88161340 2023-11-28T13:10:33.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.042+0000 7fcf8edff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf881f2b60 con 0x7fcf880d2320 2023-11-28T13:10:33.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.042+0000 7fcf8edff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf881f2e40 con 0x7fcf8815c9b0 2023-11-28T13:10:33.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.043+0000 7fcf8d39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf88161340 0x7fcf880b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:33.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.043+0000 7fcf8d39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf88161340 0x7fcf880b8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48580/0 (socket says 172.21.15.156:48580) 2023-11-28T13:10:33.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.043+0000 7fcf8d39c700 1 -- 172.21.15.156:0/2415317152 learned_addr learned my addr 172.21.15.156:0/2415317152 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:33.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.043+0000 7fcf87fff700 1 --2- 172.21.15.156:0/2415317152 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf8815c9b0 0x7fcf880b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:33.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.043+0000 7fcf8cb9b700 1 --2- 172.21.15.156:0/2415317152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf880d2320 0x7fcf8814c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:33.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.043+0000 7fcf8d39c700 1 -- 172.21.15.156:0/2415317152 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf8815c9b0 msgr2=0x7fcf880b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:33.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.043+0000 7fcf8d39c700 1 --2- 172.21.15.156:0/2415317152 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf8815c9b0 0x7fcf880b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.043+0000 7fcf8d39c700 1 -- 172.21.15.156:0/2415317152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf880d2320 msgr2=0x7fcf8814c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:33.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.043+0000 7fcf8d39c700 1 --2- 172.21.15.156:0/2415317152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf880d2320 0x7fcf8814c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.043+0000 7fcf8d39c700 1 -- 172.21.15.156:0/2415317152 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf74009550 con 0x7fcf88161340 2023-11-28T13:10:33.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.043+0000 7fcf8d39c700 1 --2- 172.21.15.156:0/2415317152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf88161340 0x7fcf880b8b00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcf80007e40 tx=0x7fcf8000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:33.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.044+0000 7fcf857fa700 1 -- 172.21.15.156:0/2415317152 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8000dd20 con 0x7fcf88161340 2023-11-28T13:10:33.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.044+0000 7fcf857fa700 1 -- 172.21.15.156:0/2415317152 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8000f040 con 0x7fcf88161340 2023-11-28T13:10:33.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.044+0000 7fcf857fa700 1 -- 172.21.15.156:0/2415317152 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8000dea0 con 0x7fcf88161340 2023-11-28T13:10:33.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.044+0000 7fcf8edff700 1 -- 172.21.15.156:0/2415317152 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf880d15e0 con 0x7fcf88161340 2023-11-28T13:10:33.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.044+0000 7fcf8edff700 1 -- 172.21.15.156:0/2415317152 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf880d1300 con 0x7fcf88161340 2023-11-28T13:10:33.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.045+0000 7fcf8edff700 1 -- 172.21.15.156:0/2415317152 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf8815cd80 con 0x7fcf88161340 2023-11-28T13:10:33.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.046+0000 7fcf857fa700 1 -- 172.21.15.156:0/2415317152 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcf8001c030 con 0x7fcf88161340 2023-11-28T13:10:33.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.047+0000 7fcf857fa700 1 --2- 172.21.15.156:0/2415317152 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcf6c0682f0 0x7fcf6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:33.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.047+0000 7fcf857fa700 1 -- 172.21.15.156:0/2415317152 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcf80085bf0 con 0x7fcf88161340 2023-11-28T13:10:33.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.047+0000 7fcf8cb9b700 1 --2- 172.21.15.156:0/2415317152 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcf6c0682f0 0x7fcf6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:33.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.048+0000 7fcf8cb9b700 1 --2- 172.21.15.156:0/2415317152 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcf6c0682f0 0x7fcf6c06a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c007fd0 tx=0x7fcf7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:33.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.051+0000 7fcf857fa700 1 -- 172.21.15.156:0/2415317152 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcf80052360 con 0x7fcf88161340 2023-11-28T13:10:33.233 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:10:33.238 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:10:33.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.285+0000 7fcf8edff700 1 -- 172.21.15.156:0/2415317152 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf881f3360 con 0x7fcf88161340 2023-11-28T13:10:33.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.286+0000 7fcf857fa700 1 -- 172.21.15.156:0/2415317152 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fcf80055980 con 0x7fcf88161340 2023-11-28T13:10:33.288 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:33.288 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3828141874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3828141874},{"type":"v1","addr":"172.21.15.156:6835","nonce":3828141874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1012489762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1012489762},{"type":"v1","addr":"172.21.15.186:6837","nonce":1012489762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2894670074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2894670074},{"type":"v1","addr":"172.21.15.186:6835","nonce":2894670074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:09:52.245032+0000","modified":"2023-11-28T13:09:55.251254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5481":{"gid":5481,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6834/1930782798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1930782798},{"type":"v1","addr":"172.21.15.156:6834","nonce":1930782798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:10:33.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.289+0000 7fcf8edff700 1 -- 172.21.15.156:0/2415317152 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcf6c0682f0 msgr2=0x7fcf6c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:33.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.289+0000 7fcf8edff700 1 --2- 172.21.15.156:0/2415317152 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcf6c0682f0 0x7fcf6c06a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c007fd0 tx=0x7fcf7c009040 comp rx=0 tx=0).stop 2023-11-28T13:10:33.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.289+0000 7fcf8edff700 1 -- 172.21.15.156:0/2415317152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf88161340 msgr2=0x7fcf880b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:33.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.289+0000 7fcf8edff700 1 --2- 172.21.15.156:0/2415317152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf88161340 0x7fcf880b8b00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcf80007e40 tx=0x7fcf8000d3b0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.289+0000 7fcf8edff700 1 -- 172.21.15.156:0/2415317152 shutdown_connections 2023-11-28T13:10:33.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.289+0000 7fcf8edff700 1 --2- 172.21.15.156:0/2415317152 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcf6c0682f0 0x7fcf6c06a7a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.289+0000 7fcf8edff700 1 --2- 172.21.15.156:0/2415317152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf88161340 0x7fcf880b8b00 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.289+0000 7fcf8edff700 1 --2- 172.21.15.156:0/2415317152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf880d2320 0x7fcf8814c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.289+0000 7fcf8edff700 1 --2- 172.21.15.156:0/2415317152 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf8815c9b0 0x7fcf880b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.289+0000 7fcf8edff700 1 -- 172.21.15.156:0/2415317152 >> 172.21.15.156:0/2415317152 conn(0x7fcf88072600 msgr2=0x7fcf8815fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:33.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.289+0000 7fcf8edff700 1 -- 172.21.15.156:0/2415317152 shutdown_connections 2023-11-28T13:10:33.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.289+0000 7fcf8edff700 1 -- 172.21.15.156:0/2415317152 wait complete. 2023-11-28T13:10:33.294 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 23 2023-11-28T13:10:33.310 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-11-28T13:10:33.533 INFO:teuthology.orchestra.run.smithi156.stdout:{"message":"","return_code":0} 2023-11-28T13:10:33.534 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T13:10:33.534 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T13:10:33.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.617+0000 7f762f03f080 1 Processor -- start 2023-11-28T13:10:33.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.617+0000 7f762f03f080 1 -- start start 2023-11-28T13:10:33.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f762f03f080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abc2a807b0 0x55abc2b29610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:33.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f762f03f080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abc2a960f0 0x55abc2a7d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:33.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f762f03f080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55abc2a964c0 0x55abc2a94470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:33.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f762f03f080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc298afe0 con 0x55abc2a960f0 2023-11-28T13:10:33.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f762f03f080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc2a7c3e0 con 0x55abc2a807b0 2023-11-28T13:10:33.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f762f03f080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abc2b28620 con 0x55abc2a964c0 2023-11-28T13:10:33.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f761ff9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abc2a960f0 0x55abc2a7d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:33.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f761ff9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abc2a960f0 0x55abc2a7d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48598/0 (socket says 172.21.15.156:48598) 2023-11-28T13:10:33.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f761ff9a700 1 -- 172.21.15.156:0/2981370700 learned_addr learned my addr 172.21.15.156:0/2981370700 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:33.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f7620f9c700 1 --2- 172.21.15.156:0/2981370700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55abc2a964c0 0x55abc2a94470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:33.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f762079b700 1 --2- 172.21.15.156:0/2981370700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abc2a807b0 0x55abc2b29610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:33.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f761ff9a700 1 -- 172.21.15.156:0/2981370700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55abc2a964c0 msgr2=0x55abc2a94470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:33.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f761ff9a700 1 --2- 172.21.15.156:0/2981370700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55abc2a964c0 0x55abc2a94470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f761ff9a700 1 -- 172.21.15.156:0/2981370700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abc2a807b0 msgr2=0x55abc2b29610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:33.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f761ff9a700 1 --2- 172.21.15.156:0/2981370700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abc2a807b0 0x55abc2b29610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f761ff9a700 1 -- 172.21.15.156:0/2981370700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7610005de0 con 0x55abc2a960f0 2023-11-28T13:10:33.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f7620f9c700 1 --2- 172.21.15.156:0/2981370700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55abc2a964c0 0x55abc2a94470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:33.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f762079b700 1 --2- 172.21.15.156:0/2981370700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abc2a807b0 0x55abc2b29610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:33.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.618+0000 7f761ff9a700 1 --2- 172.21.15.156:0/2981370700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abc2a960f0 0x55abc2a7d5f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f761400a410 tx=0x7f761400a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c488da86d77e386 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:33.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.619+0000 7f761ef98700 1 -- 172.21.15.156:0/2981370700 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761400c850 con 0x55abc2a960f0 2023-11-28T13:10:33.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.619+0000 7f761ef98700 1 -- 172.21.15.156:0/2981370700 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761400c9b0 con 0x55abc2a960f0 2023-11-28T13:10:33.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.619+0000 7f761ef98700 1 -- 172.21.15.156:0/2981370700 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761400cd40 con 0x55abc2a960f0 2023-11-28T13:10:33.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.619+0000 7f762f03f080 1 -- 172.21.15.156:0/2981370700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abc2a960f0 msgr2=0x55abc2a7d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:33.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.619+0000 7f762f03f080 1 --2- 172.21.15.156:0/2981370700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abc2a960f0 0x55abc2a7d5f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f761400a410 tx=0x7f761400a720 comp rx=0 tx=0).stop 2023-11-28T13:10:33.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.619+0000 7f762f03f080 1 -- 172.21.15.156:0/2981370700 shutdown_connections 2023-11-28T13:10:33.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.619+0000 7f762f03f080 1 --2- 172.21.15.156:0/2981370700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55abc2a964c0 0x55abc2a94470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.619+0000 7f762f03f080 1 --2- 172.21.15.156:0/2981370700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abc2a807b0 0x55abc2b29610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.619+0000 7f762f03f080 1 --2- 172.21.15.156:0/2981370700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abc2a960f0 0x55abc2a7d5f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.619+0000 7f762f03f080 1 -- 172.21.15.156:0/2981370700 >> 172.21.15.156:0/2981370700 conn(0x55abc2a9c9d0 msgr2=0x55abc2b29d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:33.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.619+0000 7f762f03f080 1 -- 172.21.15.156:0/2981370700 shutdown_connections 2023-11-28T13:10:33.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.619+0000 7f762f03f080 1 -- 172.21.15.156:0/2981370700 wait complete. 2023-11-28T13:10:33.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.620+0000 7f762f03f080 1 Processor -- start 2023-11-28T13:10:33.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.620+0000 7f762f03f080 1 -- start start 2023-11-28T13:10:33.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.621+0000 7f762f03f080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abc2a807b0 0x55abc2bb3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:33.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.621+0000 7f762f03f080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abc2a960f0 0x55abc2b1e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:33.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.621+0000 7f762f03f080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55abc2a964c0 0x55abc2b29d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:33.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.621+0000 7f762f03f080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc2a94730 con 0x55abc2a960f0 2023-11-28T13:10:33.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.621+0000 7f762f03f080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc2b27920 con 0x55abc2a807b0 2023-11-28T13:10:33.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.621+0000 7f762f03f080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abc2b1e970 con 0x55abc2a964c0 2023-11-28T13:10:33.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.621+0000 7f761ff9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abc2a960f0 0x55abc2b1e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:33.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.621+0000 7f761ff9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abc2a960f0 0x55abc2b1e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48606/0 (socket says 172.21.15.156:48606) 2023-11-28T13:10:33.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.621+0000 7f762079b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abc2a807b0 0x55abc2bb3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:33.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.621+0000 7f761ff9a700 1 -- 172.21.15.156:0/2604710340 learned_addr learned my addr 172.21.15.156:0/2604710340 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:33.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.622+0000 7f7620f9c700 1 --2- 172.21.15.156:0/2604710340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55abc2a964c0 0x55abc2b29d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:33.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.622+0000 7f761ff9a700 1 -- 172.21.15.156:0/2604710340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55abc2a964c0 msgr2=0x55abc2b29d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:33.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.622+0000 7f761ff9a700 1 --2- 172.21.15.156:0/2604710340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55abc2a964c0 0x55abc2b29d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.622+0000 7f761ff9a700 1 -- 172.21.15.156:0/2604710340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abc2a807b0 msgr2=0x55abc2bb3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:33.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.622+0000 7f761ff9a700 1 --2- 172.21.15.156:0/2604710340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55abc2a807b0 0x55abc2bb3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.622+0000 7f761ff9a700 1 -- 172.21.15.156:0/2604710340 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7614005c90 con 0x55abc2a960f0 2023-11-28T13:10:33.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.622+0000 7f761ff9a700 1 --2- 172.21.15.156:0/2604710340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abc2a960f0 0x55abc2b1e020 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f761400a760 tx=0x7f7614002f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:33.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.622+0000 7f761df96700 1 -- 172.21.15.156:0/2604710340 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614016970 con 0x55abc2a960f0 2023-11-28T13:10:33.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.622+0000 7f761df96700 1 -- 172.21.15.156:0/2604710340 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7614016ad0 con 0x55abc2a960f0 2023-11-28T13:10:33.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.622+0000 7f761df96700 1 -- 172.21.15.156:0/2604710340 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614016e60 con 0x55abc2a960f0 2023-11-28T13:10:33.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.622+0000 7f762f03f080 1 -- 172.21.15.156:0/2604710340 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55abc29a3120 con 0x55abc2a960f0 2023-11-28T13:10:33.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.622+0000 7f762f03f080 1 -- 172.21.15.156:0/2604710340 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abc2996830 con 0x55abc2a960f0 2023-11-28T13:10:33.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.624+0000 7f761df96700 1 -- 172.21.15.156:0/2604710340 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7614028030 con 0x55abc2a960f0 2023-11-28T13:10:33.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.625+0000 7f761df96700 1 --2- 172.21.15.156:0/2604710340 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f760c066c40 0x7f760c0690f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:33.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.625+0000 7f761df96700 1 -- 172.21.15.156:0/2604710340 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7614086240 con 0x55abc2a960f0 2023-11-28T13:10:33.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.625+0000 7f762079b700 1 --2- 172.21.15.156:0/2604710340 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f760c066c40 0x7f760c0690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:33.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.625+0000 7f762f03f080 1 --2- 172.21.15.156:0/2604710340 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x55abc2a7c670 0x55abc2b1f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:33.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.625+0000 7f762f03f080 1 -- 172.21.15.156:0/2604710340 --> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v8 -- 0x55abc2b1fc70 con 0x55abc2a7c670 2023-11-28T13:10:33.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.626+0000 7f7620f9c700 1 --2- 172.21.15.156:0/2604710340 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x55abc2a7c670 0x55abc2b1f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:33.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.626+0000 7f762079b700 1 --2- 172.21.15.156:0/2604710340 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f760c066c40 0x7f760c0690f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55abc2a98860 tx=0x7f761000f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:33.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.626+0000 7f7620f9c700 1 --2- 172.21.15.156:0/2604710340 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x55abc2a7c670 0x55abc2b1f730 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:33.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.627+0000 7f7620f9c700 1 -- 172.21.15.156:0/2604710340 <== osd.1 v2:172.21.15.156:6800/132479 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f761800c7e0 con 0x55abc2a7c670 2023-11-28T13:10:33.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.627+0000 7f762f03f080 1 -- 172.21.15.156:0/2604710340 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x55abc2a7c670 msgr2=0x55abc2b1f730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:33.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.627+0000 7f762f03f080 1 --2- 172.21.15.156:0/2604710340 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x55abc2a7c670 0x55abc2b1f730 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:33.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.627+0000 7f762f03f080 1 -- 172.21.15.156:0/2604710340 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f760c066c40 msgr2=0x7f760c0690f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:33.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.627+0000 7f762f03f080 1 --2- 172.21.15.156:0/2604710340 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f760c066c40 0x7f760c0690f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55abc2a98860 tx=0x7f761000f000 comp rx=0 tx=0).stop 2023-11-28T13:10:33.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.627+0000 7f762f03f080 1 -- 172.21.15.156:0/2604710340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abc2a960f0 msgr2=0x55abc2b1e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:33.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.627+0000 7f762f03f080 1 --2- 172.21.15.156:0/2604710340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abc2a960f0 0x55abc2b1e020 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f761400a760 tx=0x7f7614002f30 comp rx=0 tx=0).stop 2023-11-28T13:10:33.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.627+0000 7f7620f9c700 1 -- 172.21.15.156:0/2604710340 reap_dead start 2023-11-28T13:10:33.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.627+0000 7f762f03f080 1 -- 172.21.15.156:0/2604710340 shutdown_connections 2023-11-28T13:10:33.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.627+0000 7f762f03f080 1 -- 172.21.15.156:0/2604710340 >> 172.21.15.156:0/2604710340 conn(0x55abc2a9c9d0 msgr2=0x55abc2a7d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:33.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.627+0000 7f762f03f080 1 -- 172.21.15.156:0/2604710340 shutdown_connections 2023-11-28T13:10:33.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:33.627+0000 7f762f03f080 1 -- 172.21.15.156:0/2604710340 wait complete. 2023-11-28T13:10:33.635 INFO:teuthology.orchestra.run.smithi156.stdout:Yalphaparent_a 2023-11-28T13:10:33.635 DEBUG:teuthology.orchestra.run.smithi156:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T13:10:33.731 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:10:33.731 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 1099511627777, 2023-11-28T13:10:33.731 INFO:teuthology.orchestra.run.smithi156.stdout: "ancestors": [ 2023-11-28T13:10:33.731 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:33.732 INFO:teuthology.orchestra.run.smithi156.stdout: "dirino": 1099511627776, 2023-11-28T13:10:33.732 INFO:teuthology.orchestra.run.smithi156.stdout: "dname": "alpha", 2023-11-28T13:10:33.732 INFO:teuthology.orchestra.run.smithi156.stdout: "version": 4 2023-11-28T13:10:33.732 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:33.732 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:33.732 INFO:teuthology.orchestra.run.smithi156.stdout: "dirino": 1, 2023-11-28T13:10:33.733 INFO:teuthology.orchestra.run.smithi156.stdout: "dname": "parent_a", 2023-11-28T13:10:33.733 INFO:teuthology.orchestra.run.smithi156.stdout: "version": 7 2023-11-28T13:10:33.733 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:33.733 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2023-11-28T13:10:33.733 INFO:teuthology.orchestra.run.smithi156.stdout: "pool": 8, 2023-11-28T13:10:33.734 INFO:teuthology.orchestra.run.smithi156.stdout: "old_pools": [] 2023-11-28T13:10:33.734 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:10:33.734 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:33.738 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:10:33.739 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:10:33.845 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:10:33.937 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:10:33.937 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:10:33.937 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:10:33.980 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:10:33.981 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:10:34.065 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:10:34.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.300+0000 7fa0235ff700 1 Processor -- start 2023-11-28T13:10:34.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.300+0000 7fa0235ff700 1 -- start start 2023-11-28T13:10:34.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.301+0000 7fa0235ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa01c0cd7c0 0x7fa01c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:34.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.301+0000 7fa0235ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01c0d1950 con 0x7fa01c161340 2023-11-28T13:10:34.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.301+0000 7fa0235ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01c0ce140 con 0x7fa01c15c9b0 2023-11-28T13:10:34.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.301+0000 7fa0235ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c0b5800 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.301+0000 7fa020b9a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa01c161340 0x7fa01c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:59040/0 (socket says 172.21.15.156:59040) 2023-11-28T13:10:34.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.301+0000 7fa020b9a700 1 -- 172.21.15.156:0/3821126419 learned_addr learned my addr 172.21.15.156:0/3821126419 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:34.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.301+0000 7fa021b9c700 1 --2- 172.21.15.156:0/3821126419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa01c0cd7c0 0x7fa01c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:34.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.302+0000 7fa013fff700 1 -- 172.21.15.156:0/3821126419 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700438511 0 0) 0x7fa01c0d1950 con 0x7fa01c161340 2023-11-28T13:10:34.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.302+0000 7fa013fff700 1 -- 172.21.15.156:0/3821126419 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0080035b0 con 0x7fa01c161340 2023-11-28T13:10:34.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.302+0000 7fa013fff700 1 -- 172.21.15.156:0/3821126419 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058244065 0 0) 0x7fa01c0ce140 con 0x7fa01c15c9b0 2023-11-28T13:10:34.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.302+0000 7fa013fff700 1 -- 172.21.15.156:0/3821126419 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa01c0d1950 con 0x7fa01c15c9b0 2023-11-28T13:10:34.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.302+0000 7fa021b9c700 1 -- 172.21.15.156:0/3821126419 >> v1:172.21.15.186:6789/0 conn(0x7fa01c15c9b0 legacy=0x7fa01c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:34.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.302+0000 7fa021b9c700 1 -- 172.21.15.156:0/3821126419 >> v1:172.21.15.156:6789/0 conn(0x7fa01c161340 legacy=0x7fa01c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:34.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.302+0000 7fa013fff700 1 -- 172.21.15.156:0/3821126419 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4127222626 0 0) 0x7fa0080035b0 con 0x7fa01c161340 2023-11-28T13:10:34.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.302+0000 7fa021b9c700 1 -- 172.21.15.156:0/3821126419 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c0d24f0 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.302+0000 7fa021b9c700 1 --2- 172.21.15.156:0/3821126419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa01c0cd7c0 0x7fa01c0cdc00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa014009dc0 tx=0x7fa01400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab531c578d89b4b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:34.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.303+0000 7fa013fff700 1 -- 172.21.15.156:0/3821126419 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01400e070 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.303+0000 7fa013fff700 1 -- 172.21.15.156:0/3821126419 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa014003a20 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.303+0000 7fa013fff700 1 -- 172.21.15.156:0/3821126419 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c001670 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.303+0000 7fa0235ff700 1 -- 172.21.15.156:0/3821126419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa01c0cd7c0 msgr2=0x7fa01c0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:34.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.303+0000 7fa0235ff700 1 --2- 172.21.15.156:0/3821126419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa01c0cd7c0 0x7fa01c0cdc00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa014009dc0 tx=0x7fa01400b330 comp rx=0 tx=0).stop 2023-11-28T13:10:34.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.303+0000 7fa0235ff700 1 -- 172.21.15.156:0/3821126419 shutdown_connections 2023-11-28T13:10:34.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.303+0000 7fa0235ff700 1 --2- 172.21.15.156:0/3821126419 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa01c0cd7c0 0x7fa01c0cdc00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:34.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.303+0000 7fa0235ff700 1 -- 172.21.15.156:0/3821126419 >> 172.21.15.156:0/3821126419 conn(0x7fa01c072600 msgr2=0x7fa01c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:34.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.303+0000 7fa0235ff700 1 -- 172.21.15.156:0/3821126419 shutdown_connections 2023-11-28T13:10:34.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.303+0000 7fa0235ff700 1 -- 172.21.15.156:0/3821126419 wait complete. 2023-11-28T13:10:34.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.305+0000 7fa0235ff700 1 Processor -- start 2023-11-28T13:10:34.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.305+0000 7fa0235ff700 1 -- start start 2023-11-28T13:10:34.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.306+0000 7fa0235ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa01c0cd7c0 0x7fa01c0c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:34.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.306+0000 7fa0235ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa01c15c9b0 0x7fa01c0b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:34.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.306+0000 7fa0235ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa01c161340 0x7fa01c0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:34.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.306+0000 7fa0235ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c1e6480 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.306+0000 7fa0235ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c157270 con 0x7fa01c15c9b0 2023-11-28T13:10:34.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.306+0000 7fa0235ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c1e66d0 con 0x7fa01c161340 2023-11-28T13:10:34.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.306+0000 7fa02139b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa01c0cd7c0 0x7fa01c0c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:34.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.306+0000 7fa02139b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa01c0cd7c0 0x7fa01c0c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48612/0 (socket says 172.21.15.156:48612) 2023-11-28T13:10:34.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.306+0000 7fa02139b700 1 -- 172.21.15.156:0/1904309859 learned_addr learned my addr 172.21.15.156:0/1904309859 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:34.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.306+0000 7fa021b9c700 1 --2- 172.21.15.156:0/1904309859 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa01c161340 0x7fa01c0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:34.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.306+0000 7fa020b9a700 1 --2- 172.21.15.156:0/1904309859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa01c15c9b0 0x7fa01c0b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:34.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.307+0000 7fa02139b700 1 -- 172.21.15.156:0/1904309859 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa01c161340 msgr2=0x7fa01c0b8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:34.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.307+0000 7fa02139b700 1 --2- 172.21.15.156:0/1904309859 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa01c161340 0x7fa01c0b8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:34.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.307+0000 7fa02139b700 1 -- 172.21.15.156:0/1904309859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa01c15c9b0 msgr2=0x7fa01c0b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:34.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.307+0000 7fa02139b700 1 --2- 172.21.15.156:0/1904309859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa01c15c9b0 0x7fa01c0b84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:34.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.307+0000 7fa02139b700 1 -- 172.21.15.156:0/1904309859 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa014009a70 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.307+0000 7fa02139b700 1 --2- 172.21.15.156:0/1904309859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa01c0cd7c0 0x7fa01c0c87b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa01800c450 tx=0x7fa01800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:34.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.307+0000 7fa011ffb700 1 -- 172.21.15.156:0/1904309859 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0180052d0 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.307+0000 7fa011ffb700 1 -- 172.21.15.156:0/1904309859 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa018005430 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.307+0000 7fa0235ff700 1 -- 172.21.15.156:0/1904309859 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01c0d1300 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.307+0000 7fa011ffb700 1 -- 172.21.15.156:0/1904309859 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0180038c0 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.307+0000 7fa0235ff700 1 -- 172.21.15.156:0/1904309859 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01c0b5520 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.309+0000 7fa0235ff700 1 -- 172.21.15.156:0/1904309859 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01c0d15e0 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.310+0000 7fa011ffb700 1 -- 172.21.15.156:0/1904309859 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa018019030 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.311+0000 7fa011ffb700 1 --2- 172.21.15.156:0/1904309859 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa0000682f0 0x7fa00006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:34.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.311+0000 7fa011ffb700 1 -- 172.21.15.156:0/1904309859 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa018084f50 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.311+0000 7fa020b9a700 1 --2- 172.21.15.156:0/1904309859 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa0000682f0 0x7fa00006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:34.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.312+0000 7fa020b9a700 1 --2- 172.21.15.156:0/1904309859 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa0000682f0 0x7fa00006a7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa00c001380 tx=0x7fa00c001770 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:34.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.315+0000 7fa011ffb700 1 -- 172.21.15.156:0/1904309859 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa0180516c0 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.508+0000 7fa0235ff700 1 -- 172.21.15.156:0/1904309859 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa01c0745c0 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.510+0000 7fa011ffb700 1 -- 172.21.15.156:0/1904309859 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+17969 (secure 0 0 0) 0x7fa018017090 con 0x7fa01c0cd7c0 2023-11-28T13:10:34.512 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:34.512 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":47,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:09:53.247402+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:09:49.958043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T13:09:51.219302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:10:34.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.512+0000 7fa0235ff700 1 -- 172.21.15.156:0/1904309859 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa0000682f0 msgr2=0x7fa00006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:34.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.512+0000 7fa0235ff700 1 --2- 172.21.15.156:0/1904309859 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa0000682f0 0x7fa00006a7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa00c001380 tx=0x7fa00c001770 comp rx=0 tx=0).stop 2023-11-28T13:10:34.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.512+0000 7fa0235ff700 1 -- 172.21.15.156:0/1904309859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa01c0cd7c0 msgr2=0x7fa01c0c87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:34.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.512+0000 7fa0235ff700 1 --2- 172.21.15.156:0/1904309859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa01c0cd7c0 0x7fa01c0c87b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa01800c450 tx=0x7fa01800c760 comp rx=0 tx=0).stop 2023-11-28T13:10:34.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.512+0000 7fa0235ff700 1 -- 172.21.15.156:0/1904309859 shutdown_connections 2023-11-28T13:10:34.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.512+0000 7fa0235ff700 1 --2- 172.21.15.156:0/1904309859 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa0000682f0 0x7fa00006a7a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:34.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.512+0000 7fa0235ff700 1 --2- 172.21.15.156:0/1904309859 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa01c161340 0x7fa01c0b8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:34.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.512+0000 7fa0235ff700 1 --2- 172.21.15.156:0/1904309859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa01c0cd7c0 0x7fa01c0c87b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:34.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.512+0000 7fa0235ff700 1 --2- 172.21.15.156:0/1904309859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa01c15c9b0 0x7fa01c0b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:34.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.512+0000 7fa0235ff700 1 -- 172.21.15.156:0/1904309859 >> 172.21.15.156:0/1904309859 conn(0x7fa01c072600 msgr2=0x7fa01c06a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:34.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.512+0000 7fa0235ff700 1 -- 172.21.15.156:0/1904309859 shutdown_connections 2023-11-28T13:10:34.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.512+0000 7fa0235ff700 1 -- 172.21.15.156:0/1904309859 wait complete. 2023-11-28T13:10:34.532 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:09:49.958043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:09:51.219302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:10:34.533 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:10:34.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.732+0000 7f3d4bfff700 1 Processor -- start 2023-11-28T13:10:34.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.732+0000 7f3d4bfff700 1 -- start start 2023-11-28T13:10:34.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.734+0000 7f3d4bfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3d4415c9b0 0x7f3d440b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:34.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.734+0000 7f3d4bfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d44161340 0x7f3d440d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:34.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.734+0000 7f3d4bfff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d440d1950 con 0x7f3d440d2320 2023-11-28T13:10:34.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.734+0000 7f3d4bfff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d440b5800 con 0x7f3d4415c9b0 2023-11-28T13:10:34.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.734+0000 7f3d4bfff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d440b5520 con 0x7f3d44161340 2023-11-28T13:10:34.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.734+0000 7f3d49d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3d4415c9b0 0x7f3d440b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:34.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.734+0000 7f3d49d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3d4415c9b0 0x7f3d440b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48618/0 (socket says 172.21.15.156:48618) 2023-11-28T13:10:34.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.734+0000 7f3d49d9b700 1 -- 172.21.15.156:0/3495722520 learned_addr learned my addr 172.21.15.156:0/3495722520 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:34.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.734+0000 7f3d4959a700 1 --2- 172.21.15.156:0/3495722520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d44161340 0x7f3d440d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:34.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.734+0000 7f3d49d9b700 1 -- 172.21.15.156:0/3495722520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d44161340 msgr2=0x7f3d440d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:34.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.734+0000 7f3d49d9b700 1 --2- 172.21.15.156:0/3495722520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d44161340 0x7f3d440d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:34.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.734+0000 7f3d49d9b700 1 -- 172.21.15.156:0/3495722520 >> v1:172.21.15.186:6789/0 conn(0x7f3d440d2320 legacy=0x7f3d440cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:34.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.734+0000 7f3d49d9b700 1 -- 172.21.15.156:0/3495722520 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d440d1300 con 0x7f3d4415c9b0 2023-11-28T13:10:34.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.734+0000 7f3d4959a700 1 --2- 172.21.15.156:0/3495722520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d44161340 0x7f3d440d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:34.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.735+0000 7f3d49d9b700 1 --2- 172.21.15.156:0/3495722520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d4415c9b0 0x7f3d440b4bf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3d400098f0 tx=0x7f3d40009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f46d2e38864661d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:34.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.735+0000 7f3d48d99700 1 -- 172.21.15.156:0/3495722520 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d40004230 con 0x7f3d4415c9b0 2023-11-28T13:10:34.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.735+0000 7f3d48d99700 1 -- 172.21.15.156:0/3495722520 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d40004390 con 0x7f3d4415c9b0 2023-11-28T13:10:34.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.735+0000 7f3d48d99700 1 -- 172.21.15.156:0/3495722520 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d400036a0 con 0x7f3d4415c9b0 2023-11-28T13:10:34.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.735+0000 7f3d4bfff700 1 -- 172.21.15.156:0/3495722520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d4415c9b0 msgr2=0x7f3d440b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:34.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.735+0000 7f3d4bfff700 1 --2- 172.21.15.156:0/3495722520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d4415c9b0 0x7f3d440b4bf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3d400098f0 tx=0x7f3d40009c00 comp rx=0 tx=0).stop 2023-11-28T13:10:34.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.735+0000 7f3d4bfff700 1 -- 172.21.15.156:0/3495722520 shutdown_connections 2023-11-28T13:10:34.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.735+0000 7f3d4bfff700 1 --2- 172.21.15.156:0/3495722520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d4415c9b0 0x7f3d440b4bf0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:34.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.735+0000 7f3d4bfff700 1 --2- 172.21.15.156:0/3495722520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d44161340 0x7f3d440d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:34.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.735+0000 7f3d4bfff700 1 -- 172.21.15.156:0/3495722520 >> 172.21.15.156:0/3495722520 conn(0x7f3d44072600 msgr2=0x7f3d440cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:34.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.736+0000 7f3d4bfff700 1 -- 172.21.15.156:0/3495722520 shutdown_connections 2023-11-28T13:10:34.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.736+0000 7f3d4bfff700 1 -- 172.21.15.156:0/3495722520 wait complete. 2023-11-28T13:10:34.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.737+0000 7f3d4bfff700 1 Processor -- start 2023-11-28T13:10:34.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.737+0000 7f3d4bfff700 1 -- start start 2023-11-28T13:10:34.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.738+0000 7f3d4bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d440d2320 0x7f3d44146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:34.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.738+0000 7f3d4bfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d4415c9b0 0x7f3d441562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:34.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.738+0000 7f3d4bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d44161340 0x7f3d4415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:34.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.738+0000 7f3d4bfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d44155e10 con 0x7f3d440d2320 2023-11-28T13:10:34.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.738+0000 7f3d4bfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d44155200 con 0x7f3d44161340 2023-11-28T13:10:34.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.738+0000 7f3d4bfff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d441554e0 con 0x7f3d4415c9b0 2023-11-28T13:10:34.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.738+0000 7f3d49d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d440d2320 0x7f3d44146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:34.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.739+0000 7f3d49d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d440d2320 0x7f3d44146510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48624/0 (socket says 172.21.15.156:48624) 2023-11-28T13:10:34.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.739+0000 7f3d49d9b700 1 -- 172.21.15.156:0/4212868181 learned_addr learned my addr 172.21.15.156:0/4212868181 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:34.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.739+0000 7f3d4959a700 1 --2- 172.21.15.156:0/4212868181 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d4415c9b0 0x7f3d441562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:34.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.739+0000 7f3d4a59c700 1 --2- 172.21.15.156:0/4212868181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d44161340 0x7f3d4415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:34.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.739+0000 7f3d49d9b700 1 -- 172.21.15.156:0/4212868181 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d4415c9b0 msgr2=0x7f3d441562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:34.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.739+0000 7f3d49d9b700 1 --2- 172.21.15.156:0/4212868181 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d4415c9b0 0x7f3d441562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:34.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.739+0000 7f3d49d9b700 1 -- 172.21.15.156:0/4212868181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d44161340 msgr2=0x7f3d4415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:34.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.739+0000 7f3d49d9b700 1 --2- 172.21.15.156:0/4212868181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d44161340 0x7f3d4415ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:34.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.739+0000 7f3d49d9b700 1 -- 172.21.15.156:0/4212868181 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d40009550 con 0x7f3d440d2320 2023-11-28T13:10:34.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.739+0000 7f3d4a59c700 1 --2- 172.21.15.156:0/4212868181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d44161340 0x7f3d4415ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:34.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.739+0000 7f3d49d9b700 1 --2- 172.21.15.156:0/4212868181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d440d2320 0x7f3d44146510 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009130 tx=0x7f3d40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:34.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.739+0000 7f3d3a7fc700 1 -- 172.21.15.156:0/4212868181 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d40003e30 con 0x7f3d440d2320 2023-11-28T13:10:34.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.739+0000 7f3d3a7fc700 1 -- 172.21.15.156:0/4212868181 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4001f070 con 0x7f3d440d2320 2023-11-28T13:10:34.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.739+0000 7f3d4bfff700 1 -- 172.21.15.156:0/4212868181 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d440b5130 con 0x7f3d440d2320 2023-11-28T13:10:34.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.739+0000 7f3d3a7fc700 1 -- 172.21.15.156:0/4212868181 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4000b8a0 con 0x7f3d440d2320 2023-11-28T13:10:34.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.740+0000 7f3d4bfff700 1 -- 172.21.15.156:0/4212868181 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d440d15e0 con 0x7f3d440d2320 2023-11-28T13:10:34.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.741+0000 7f3d4bfff700 1 -- 172.21.15.156:0/4212868181 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d440b52c0 con 0x7f3d440d2320 2023-11-28T13:10:34.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.741+0000 7f3d3a7fc700 1 -- 172.21.15.156:0/4212868181 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3d4002a020 con 0x7f3d440d2320 2023-11-28T13:10:34.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.742+0000 7f3d3a7fc700 1 --2- 172.21.15.156:0/4212868181 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d280682f0 0x7f3d2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:34.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.742+0000 7f3d3a7fc700 1 -- 172.21.15.156:0/4212868181 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3d400875f0 con 0x7f3d440d2320 2023-11-28T13:10:34.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.743+0000 7f3d4959a700 1 --2- 172.21.15.156:0/4212868181 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d280682f0 0x7f3d2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:34.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.744+0000 7f3d4959a700 1 --2- 172.21.15.156:0/4212868181 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d280682f0 0x7f3d2806a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3d44154680 tx=0x7f3d34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:34.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.746+0000 7f3d3a7fc700 1 -- 172.21.15.156:0/4212868181 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3d40053d60 con 0x7f3d440d2320 2023-11-28T13:10:34.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.935+0000 7f3d4bfff700 1 -- 172.21.15.156:0/4212868181 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3d440b52c0 con 0x7f3d440d2320 2023-11-28T13:10:34.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.936+0000 7f3d3a7fc700 1 -- 172.21.15.156:0/4212868181 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v47) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3d40057380 con 0x7f3d440d2320 2023-11-28T13:10:34.938 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:10:34.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.938+0000 7f3d4bfff700 1 -- 172.21.15.156:0/4212868181 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d280682f0 msgr2=0x7f3d2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:34.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.938+0000 7f3d4bfff700 1 --2- 172.21.15.156:0/4212868181 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d280682f0 0x7f3d2806a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3d44154680 tx=0x7f3d34008040 comp rx=0 tx=0).stop 2023-11-28T13:10:34.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.938+0000 7f3d4bfff700 1 -- 172.21.15.156:0/4212868181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d440d2320 msgr2=0x7f3d44146510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:34.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.938+0000 7f3d4bfff700 1 --2- 172.21.15.156:0/4212868181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d440d2320 0x7f3d44146510 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009130 tx=0x7f3d40004750 comp rx=0 tx=0).stop 2023-11-28T13:10:34.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.938+0000 7f3d4bfff700 1 -- 172.21.15.156:0/4212868181 shutdown_connections 2023-11-28T13:10:34.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.938+0000 7f3d4bfff700 1 --2- 172.21.15.156:0/4212868181 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d280682f0 0x7f3d2806a7a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:34.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.938+0000 7f3d4bfff700 1 --2- 172.21.15.156:0/4212868181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d44161340 0x7f3d4415ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:34.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.938+0000 7f3d4bfff700 1 --2- 172.21.15.156:0/4212868181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d440d2320 0x7f3d44146510 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:34.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.938+0000 7f3d4bfff700 1 --2- 172.21.15.156:0/4212868181 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d4415c9b0 0x7f3d441562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:34.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.939+0000 7f3d4bfff700 1 -- 172.21.15.156:0/4212868181 >> 172.21.15.156:0/4212868181 conn(0x7f3d44072600 msgr2=0x7f3d4415ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:34.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.939+0000 7f3d4bfff700 1 -- 172.21.15.156:0/4212868181 shutdown_connections 2023-11-28T13:10:34.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:34.939+0000 7f3d4bfff700 1 -- 172.21.15.156:0/4212868181 wait complete. 2023-11-28T13:10:34.959 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:10:35.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.157+0000 7f8b80dff700 1 Processor -- start 2023-11-28T13:10:35.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.157+0000 7f8b80dff700 1 -- start start 2023-11-28T13:10:35.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.159+0000 7f8b80dff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8b7c15c9b0 0x7f8b7c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:35.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.159+0000 7f8b80dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b7c161340 0x7f8b7c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:35.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.159+0000 7f8b80dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7c0d1950 con 0x7f8b7c0d2320 2023-11-28T13:10:35.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.159+0000 7f8b80dff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b7c0b5800 con 0x7f8b7c15c9b0 2023-11-28T13:10:35.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.159+0000 7f8b80dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0b5520 con 0x7f8b7c161340 2023-11-28T13:10:35.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.159+0000 7f8b7a59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8b7c15c9b0 0x7f8b7c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:35.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.159+0000 7f8b7a59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8b7c15c9b0 0x7f8b7c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48640/0 (socket says 172.21.15.156:48640) 2023-11-28T13:10:35.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.159+0000 7f8b7a59c700 1 -- 172.21.15.156:0/279395302 learned_addr learned my addr 172.21.15.156:0/279395302 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:35.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.159+0000 7f8b73fff700 1 --2- 172.21.15.156:0/279395302 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b7c161340 0x7f8b7c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:35.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.159+0000 7f8b7a59c700 1 -- 172.21.15.156:0/279395302 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b7c161340 msgr2=0x7f8b7c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:35.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.159+0000 7f8b7a59c700 1 --2- 172.21.15.156:0/279395302 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b7c161340 0x7f8b7c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.159+0000 7f8b7a59c700 1 -- 172.21.15.156:0/279395302 >> v1:172.21.15.186:6789/0 conn(0x7f8b7c0d2320 legacy=0x7f8b7c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:35.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.159+0000 7f8b7a59c700 1 -- 172.21.15.156:0/279395302 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c0d1300 con 0x7f8b7c15c9b0 2023-11-28T13:10:35.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.160+0000 7f8b7a59c700 1 --2- 172.21.15.156:0/279395302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b7c15c9b0 0x7f8b7c0b4bf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0098f0 tx=0x7f8b6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1b884ee2b79955 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:35.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.160+0000 7f8b79d9b700 1 -- 172.21.15.156:0/279395302 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c00b8a0 con 0x7f8b7c15c9b0 2023-11-28T13:10:35.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.160+0000 7f8b79d9b700 1 -- 172.21.15.156:0/279395302 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c00ba00 con 0x7f8b7c15c9b0 2023-11-28T13:10:35.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.160+0000 7f8b79d9b700 1 -- 172.21.15.156:0/279395302 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c00bd90 con 0x7f8b7c15c9b0 2023-11-28T13:10:35.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.160+0000 7f8b80dff700 1 -- 172.21.15.156:0/279395302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b7c15c9b0 msgr2=0x7f8b7c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:35.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.160+0000 7f8b80dff700 1 --2- 172.21.15.156:0/279395302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b7c15c9b0 0x7f8b7c0b4bf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0098f0 tx=0x7f8b6c009c00 comp rx=0 tx=0).stop 2023-11-28T13:10:35.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.161+0000 7f8b80dff700 1 -- 172.21.15.156:0/279395302 shutdown_connections 2023-11-28T13:10:35.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.161+0000 7f8b80dff700 1 --2- 172.21.15.156:0/279395302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b7c15c9b0 0x7f8b7c0b4bf0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.161+0000 7f8b80dff700 1 --2- 172.21.15.156:0/279395302 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b7c161340 0x7f8b7c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.161+0000 7f8b80dff700 1 -- 172.21.15.156:0/279395302 >> 172.21.15.156:0/279395302 conn(0x7f8b7c072600 msgr2=0x7f8b7c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:35.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.161+0000 7f8b80dff700 1 -- 172.21.15.156:0/279395302 shutdown_connections 2023-11-28T13:10:35.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.161+0000 7f8b80dff700 1 -- 172.21.15.156:0/279395302 wait complete. 2023-11-28T13:10:35.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.162+0000 7f8b80dff700 1 Processor -- start 2023-11-28T13:10:35.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.163+0000 7f8b80dff700 1 -- start start 2023-11-28T13:10:35.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.163+0000 7f8b80dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b7c0d2320 0x7f8b7c15eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:35.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.163+0000 7f8b80dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b7c15c9b0 0x7f8b7c1e64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:35.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.164+0000 7f8b80dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b7c161340 0x7f8b7c1504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:35.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.164+0000 7f8b80dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c1e74d0 con 0x7f8b7c0d2320 2023-11-28T13:10:35.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.164+0000 7f8b80dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c1e6f10 con 0x7f8b7c161340 2023-11-28T13:10:35.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.164+0000 7f8b80dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c1e71f0 con 0x7f8b7c15c9b0 2023-11-28T13:10:35.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.164+0000 7f8b7a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b7c0d2320 0x7f8b7c15eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:35.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.164+0000 7f8b7a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b7c0d2320 0x7f8b7c15eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48652/0 (socket says 172.21.15.156:48652) 2023-11-28T13:10:35.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.164+0000 7f8b7a59c700 1 -- 172.21.15.156:0/4286109570 learned_addr learned my addr 172.21.15.156:0/4286109570 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:35.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.164+0000 7f8b73fff700 1 --2- 172.21.15.156:0/4286109570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b7c15c9b0 0x7f8b7c1e64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:35.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.164+0000 7f8b7ad9d700 1 --2- 172.21.15.156:0/4286109570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b7c161340 0x7f8b7c1504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:35.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.164+0000 7f8b7a59c700 1 -- 172.21.15.156:0/4286109570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b7c15c9b0 msgr2=0x7f8b7c1e64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:35.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.164+0000 7f8b7a59c700 1 --2- 172.21.15.156:0/4286109570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b7c15c9b0 0x7f8b7c1e64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.164+0000 7f8b7a59c700 1 -- 172.21.15.156:0/4286109570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b7c161340 msgr2=0x7f8b7c1504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:35.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.164+0000 7f8b7a59c700 1 --2- 172.21.15.156:0/4286109570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b7c161340 0x7f8b7c1504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.164+0000 7f8b7a59c700 1 -- 172.21.15.156:0/4286109570 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c009550 con 0x7f8b7c0d2320 2023-11-28T13:10:35.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.164+0000 7f8b73fff700 1 --2- 172.21.15.156:0/4286109570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b7c15c9b0 0x7f8b7c1e64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:35.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.164+0000 7f8b7ad9d700 1 --2- 172.21.15.156:0/4286109570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b7c161340 0x7f8b7c1504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:35.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.164+0000 7f8b7a59c700 1 --2- 172.21.15.156:0/4286109570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b7c0d2320 0x7f8b7c15eeb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c000c00 tx=0x7f8b6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:35.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.165+0000 7f8b72ffd700 1 -- 172.21.15.156:0/4286109570 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c004480 con 0x7f8b7c0d2320 2023-11-28T13:10:35.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.165+0000 7f8b72ffd700 1 -- 172.21.15.156:0/4286109570 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c01f070 con 0x7f8b7c0d2320 2023-11-28T13:10:35.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.165+0000 7f8b80dff700 1 -- 172.21.15.156:0/4286109570 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c0b5130 con 0x7f8b7c0d2320 2023-11-28T13:10:35.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.165+0000 7f8b72ffd700 1 -- 172.21.15.156:0/4286109570 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c0038c0 con 0x7f8b7c0d2320 2023-11-28T13:10:35.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.165+0000 7f8b80dff700 1 -- 172.21.15.156:0/4286109570 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7c0d15e0 con 0x7f8b7c0d2320 2023-11-28T13:10:35.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.166+0000 7f8b80dff700 1 -- 172.21.15.156:0/4286109570 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7c0b52c0 con 0x7f8b7c0d2320 2023-11-28T13:10:35.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.167+0000 7f8b72ffd700 1 -- 172.21.15.156:0/4286109570 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8b6c02a020 con 0x7f8b7c0d2320 2023-11-28T13:10:35.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.167+0000 7f8b72ffd700 1 --2- 172.21.15.156:0/4286109570 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8b600682f0 0x7f8b6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:35.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.168+0000 7f8b72ffd700 1 -- 172.21.15.156:0/4286109570 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8b6c0874c0 con 0x7f8b7c0d2320 2023-11-28T13:10:35.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.168+0000 7f8b73fff700 1 --2- 172.21.15.156:0/4286109570 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8b600682f0 0x7f8b6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:35.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.169+0000 7f8b73fff700 1 --2- 172.21.15.156:0/4286109570 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8b600682f0 0x7f8b6006a7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c154c40 tx=0x7f8b64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:35.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.171+0000 7f8b72ffd700 1 -- 172.21.15.156:0/4286109570 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8b6c053c30 con 0x7f8b7c0d2320 2023-11-28T13:10:35.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.361+0000 7f8b80dff700 1 -- 172.21.15.156:0/4286109570 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b7c0745c0 con 0x7f8b7c0d2320 2023-11-28T13:10:35.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.362+0000 7f8b72ffd700 1 -- 172.21.15.156:0/4286109570 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b6c057250 con 0x7f8b7c0d2320 2023-11-28T13:10:35.363 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:10:35.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.364+0000 7f8b80dff700 1 -- 172.21.15.156:0/4286109570 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8b600682f0 msgr2=0x7f8b6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:35.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.364+0000 7f8b80dff700 1 --2- 172.21.15.156:0/4286109570 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8b600682f0 0x7f8b6006a7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c154c40 tx=0x7f8b64008040 comp rx=0 tx=0).stop 2023-11-28T13:10:35.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.364+0000 7f8b80dff700 1 -- 172.21.15.156:0/4286109570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b7c0d2320 msgr2=0x7f8b7c15eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:35.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.364+0000 7f8b80dff700 1 --2- 172.21.15.156:0/4286109570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b7c0d2320 0x7f8b7c15eeb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c000c00 tx=0x7f8b6c004750 comp rx=0 tx=0).stop 2023-11-28T13:10:35.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.364+0000 7f8b80dff700 1 -- 172.21.15.156:0/4286109570 shutdown_connections 2023-11-28T13:10:35.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.364+0000 7f8b80dff700 1 --2- 172.21.15.156:0/4286109570 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8b600682f0 0x7f8b6006a7a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.364+0000 7f8b80dff700 1 --2- 172.21.15.156:0/4286109570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b7c161340 0x7f8b7c1504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.364+0000 7f8b80dff700 1 --2- 172.21.15.156:0/4286109570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b7c0d2320 0x7f8b7c15eeb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.364+0000 7f8b80dff700 1 --2- 172.21.15.156:0/4286109570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b7c15c9b0 0x7f8b7c1e64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.364+0000 7f8b80dff700 1 -- 172.21.15.156:0/4286109570 >> 172.21.15.156:0/4286109570 conn(0x7f8b7c072600 msgr2=0x7f8b7c15d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:35.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.364+0000 7f8b80dff700 1 -- 172.21.15.156:0/4286109570 shutdown_connections 2023-11-28T13:10:35.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.364+0000 7f8b80dff700 1 -- 172.21.15.156:0/4286109570 wait complete. 2023-11-28T13:10:35.384 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:10:35.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.583+0000 7f15c39ff700 1 Processor -- start 2023-11-28T13:10:35.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.584+0000 7f15c39ff700 1 -- start start 2023-11-28T13:10:35.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.585+0000 7f15c39ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f15bc161340 0x7f15bc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:35.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.585+0000 7f15c39ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f15bc0d2320 0x7f15bc0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:35.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.585+0000 7f15c39ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15bc0d1950 con 0x7f15bc15c9b0 2023-11-28T13:10:35.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.585+0000 7f15c39ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15bc0b5800 con 0x7f15bc161340 2023-11-28T13:10:35.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.585+0000 7f15c39ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15bc0b5300 con 0x7f15bc0d2320 2023-11-28T13:10:35.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.585+0000 7f15c0f9a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f15bc161340 0x7f15bc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:35.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.585+0000 7f15c0f9a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f15bc161340 0x7f15bc0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48656/0 (socket says 172.21.15.156:48656) 2023-11-28T13:10:35.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.585+0000 7f15c0f9a700 1 -- 172.21.15.156:0/324964979 learned_addr learned my addr 172.21.15.156:0/324964979 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:35.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.585+0000 7f15c1f9c700 1 --2- 172.21.15.156:0/324964979 >> v2:172.21.15.186:3300/0 conn(0x7f15bc0d2320 0x7f15bc0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:35.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.585+0000 7f15c0f9a700 1 -- 172.21.15.156:0/324964979 >> v2:172.21.15.186:3300/0 conn(0x7f15bc0d2320 msgr2=0x7f15bc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:35.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.585+0000 7f15c0f9a700 1 --2- 172.21.15.156:0/324964979 >> v2:172.21.15.186:3300/0 conn(0x7f15bc0d2320 0x7f15bc0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.585+0000 7f15c0f9a700 1 -- 172.21.15.156:0/324964979 >> v1:172.21.15.186:6789/0 conn(0x7f15bc15c9b0 legacy=0x7f15bc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:35.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.586+0000 7f15c0f9a700 1 -- 172.21.15.156:0/324964979 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc0b5520 con 0x7f15bc161340 2023-11-28T13:10:35.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.586+0000 7f15c1f9c700 1 --2- 172.21.15.156:0/324964979 >> v2:172.21.15.186:3300/0 conn(0x7f15bc0d2320 0x7f15bc0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:35.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.586+0000 7f15c0f9a700 1 --2- 172.21.15.156:0/324964979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15bc161340 0x7f15bc0d1de0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f15bc06c170 tx=0x7f15ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9a1c62ab14a6b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:35.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.586+0000 7f15b3fff700 1 -- 172.21.15.156:0/324964979 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac004230 con 0x7f15bc161340 2023-11-28T13:10:35.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.586+0000 7f15b3fff700 1 -- 172.21.15.156:0/324964979 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac004390 con 0x7f15bc161340 2023-11-28T13:10:35.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.586+0000 7f15b3fff700 1 -- 172.21.15.156:0/324964979 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac0036a0 con 0x7f15bc161340 2023-11-28T13:10:35.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.587+0000 7f15c39ff700 1 -- 172.21.15.156:0/324964979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15bc161340 msgr2=0x7f15bc0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:35.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.587+0000 7f15c39ff700 1 --2- 172.21.15.156:0/324964979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15bc161340 0x7f15bc0d1de0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f15bc06c170 tx=0x7f15ac009c00 comp rx=0 tx=0).stop 2023-11-28T13:10:35.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.587+0000 7f15c39ff700 1 -- 172.21.15.156:0/324964979 shutdown_connections 2023-11-28T13:10:35.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.587+0000 7f15c39ff700 1 --2- 172.21.15.156:0/324964979 >> v2:172.21.15.186:3300/0 conn(0x7f15bc0d2320 0x7f15bc0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.587+0000 7f15c39ff700 1 --2- 172.21.15.156:0/324964979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15bc161340 0x7f15bc0d1de0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.587+0000 7f15c39ff700 1 -- 172.21.15.156:0/324964979 >> 172.21.15.156:0/324964979 conn(0x7f15bc072600 msgr2=0x7f15bc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:35.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.587+0000 7f15c39ff700 1 -- 172.21.15.156:0/324964979 shutdown_connections 2023-11-28T13:10:35.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.587+0000 7f15c39ff700 1 -- 172.21.15.156:0/324964979 wait complete. 2023-11-28T13:10:35.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.588+0000 7f15c39ff700 1 Processor -- start 2023-11-28T13:10:35.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.589+0000 7f15c39ff700 1 -- start start 2023-11-28T13:10:35.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.589+0000 7f15c39ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15bc0d2320 0x7f15bc1559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:35.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.590+0000 7f15c39ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15bc15c9b0 0x7f15bc1f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:35.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.590+0000 7f15c39ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15bc161340 0x7f15bc150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:35.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.590+0000 7f15c39ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc155f10 con 0x7f15bc0d2320 2023-11-28T13:10:35.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.590+0000 7f15c39ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc1f6c90 con 0x7f15bc15c9b0 2023-11-28T13:10:35.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.590+0000 7f15c39ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc156100 con 0x7f15bc161340 2023-11-28T13:10:35.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.590+0000 7f15c179b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15bc0d2320 0x7f15bc1559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:35.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.590+0000 7f15c179b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15bc0d2320 0x7f15bc1559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48666/0 (socket says 172.21.15.156:48666) 2023-11-28T13:10:35.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.590+0000 7f15c179b700 1 -- 172.21.15.156:0/221929526 learned_addr learned my addr 172.21.15.156:0/221929526 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:35.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.590+0000 7f15c179b700 1 -- 172.21.15.156:0/221929526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15bc161340 msgr2=0x7f15bc150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:35.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.590+0000 7f15c179b700 1 --2- 172.21.15.156:0/221929526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15bc161340 0x7f15bc150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.590+0000 7f15c179b700 1 -- 172.21.15.156:0/221929526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15bc15c9b0 msgr2=0x7f15bc1f6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:35.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.590+0000 7f15c179b700 1 --2- 172.21.15.156:0/221929526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15bc15c9b0 0x7f15bc1f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.590+0000 7f15c179b700 1 -- 172.21.15.156:0/221929526 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac009550 con 0x7f15bc0d2320 2023-11-28T13:10:35.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.590+0000 7f15c179b700 1 --2- 172.21.15.156:0/221929526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15bc0d2320 0x7f15bc1559d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f15b800b4a0 tx=0x7f15b800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:35.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.591+0000 7f15b1ffb700 1 -- 172.21.15.156:0/221929526 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b8011830 con 0x7f15bc0d2320 2023-11-28T13:10:35.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.591+0000 7f15b1ffb700 1 -- 172.21.15.156:0/221929526 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b8011990 con 0x7f15bc0d2320 2023-11-28T13:10:35.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.591+0000 7f15b1ffb700 1 -- 172.21.15.156:0/221929526 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b8011d40 con 0x7f15bc0d2320 2023-11-28T13:10:35.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.591+0000 7f15c39ff700 1 -- 172.21.15.156:0/221929526 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15bc0d15e0 con 0x7f15bc0d2320 2023-11-28T13:10:35.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.591+0000 7f15c39ff700 1 -- 172.21.15.156:0/221929526 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15bc0d1300 con 0x7f15bc0d2320 2023-11-28T13:10:35.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.592+0000 7f15c39ff700 1 -- 172.21.15.156:0/221929526 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15bc15cd80 con 0x7f15bc0d2320 2023-11-28T13:10:35.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.593+0000 7f15b1ffb700 1 -- 172.21.15.156:0/221929526 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f15b800f050 con 0x7f15bc0d2320 2023-11-28T13:10:35.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.594+0000 7f15b1ffb700 1 --2- 172.21.15.156:0/221929526 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f15a00682f0 0x7f15a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:35.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.594+0000 7f15b1ffb700 1 -- 172.21.15.156:0/221929526 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f15b8085800 con 0x7f15bc0d2320 2023-11-28T13:10:35.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.594+0000 7f15c0f9a700 1 --2- 172.21.15.156:0/221929526 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f15a00682f0 0x7f15a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:35.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.595+0000 7f15c0f9a700 1 --2- 172.21.15.156:0/221929526 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f15a00682f0 0x7f15a006a7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0d0c90 tx=0x7f15ac010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:35.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.598+0000 7f15b1ffb700 1 -- 172.21.15.156:0/221929526 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f15b8052060 con 0x7f15bc0d2320 2023-11-28T13:10:35.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.788+0000 7f15c39ff700 1 -- 172.21.15.156:0/221929526 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f15bc156620 con 0x7f15bc0d2320 2023-11-28T13:10:35.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.789+0000 7f15b1ffb700 1 -- 172.21.15.156:0/221929526 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7f15bc156620 con 0x7f15bc0d2320 2023-11-28T13:10:35.790 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T13:10:35.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.791+0000 7f15c39ff700 1 -- 172.21.15.156:0/221929526 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f15a00682f0 msgr2=0x7f15a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:35.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.791+0000 7f15c39ff700 1 --2- 172.21.15.156:0/221929526 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f15a00682f0 0x7f15a006a7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0d0c90 tx=0x7f15ac010000 comp rx=0 tx=0).stop 2023-11-28T13:10:35.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.791+0000 7f15c39ff700 1 -- 172.21.15.156:0/221929526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15bc0d2320 msgr2=0x7f15bc1559d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:35.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.791+0000 7f15c39ff700 1 --2- 172.21.15.156:0/221929526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15bc0d2320 0x7f15bc1559d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f15b800b4a0 tx=0x7f15b800b860 comp rx=0 tx=0).stop 2023-11-28T13:10:35.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.791+0000 7f15c39ff700 1 -- 172.21.15.156:0/221929526 shutdown_connections 2023-11-28T13:10:35.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.791+0000 7f15c39ff700 1 --2- 172.21.15.156:0/221929526 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f15a00682f0 0x7f15a006a7a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.791+0000 7f15c39ff700 1 --2- 172.21.15.156:0/221929526 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15bc161340 0x7f15bc150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.791+0000 7f15c39ff700 1 --2- 172.21.15.156:0/221929526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15bc0d2320 0x7f15bc1559d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.791+0000 7f15c39ff700 1 --2- 172.21.15.156:0/221929526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15bc15c9b0 0x7f15bc1f6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:35.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.791+0000 7f15c39ff700 1 -- 172.21.15.156:0/221929526 >> 172.21.15.156:0/221929526 conn(0x7f15bc072600 msgr2=0x7f15bc0b6280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:35.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.791+0000 7f15c39ff700 1 -- 172.21.15.156:0/221929526 shutdown_connections 2023-11-28T13:10:35.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:35.791+0000 7f15c39ff700 1 -- 172.21.15.156:0/221929526 wait complete. 2023-11-28T13:10:35.811 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:10:36.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.009+0000 7f8377bff700 1 Processor -- start 2023-11-28T13:10:36.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.009+0000 7f8377bff700 1 -- start start 2023-11-28T13:10:36.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.010+0000 7f8377bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f837015c9b0 0x7f83700b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:36.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.010+0000 7f8377bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8370161340 0x7f83700d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:36.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.010+0000 7f8377bff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f83700d2320 0x7f83700cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:36.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.010+0000 7f8377bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83700b5800 con 0x7f8370161340 2023-11-28T13:10:36.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.010+0000 7f8377bff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83700d1950 con 0x7f83700d2320 2023-11-28T13:10:36.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.010+0000 7f8377bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83700b5520 con 0x7f837015c9b0 2023-11-28T13:10:36.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.010+0000 7f837519a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8370161340 0x7f83700d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:36.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.011+0000 7f837519a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8370161340 0x7f83700d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41906/0 (socket says 172.21.15.156:41906) 2023-11-28T13:10:36.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.011+0000 7f837519a700 1 -- 172.21.15.156:0/3631529777 learned_addr learned my addr 172.21.15.156:0/3631529777 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:36.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.011+0000 7f837599b700 1 --2- 172.21.15.156:0/3631529777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f837015c9b0 0x7f83700b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:36.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.011+0000 7f837619c700 1 --2- 172.21.15.156:0/3631529777 >> v2:172.21.15.186:3300/0 conn(0x7f83700d2320 0x7f83700cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:36.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.011+0000 7f837519a700 1 -- 172.21.15.156:0/3631529777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f837015c9b0 msgr2=0x7f83700b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:36.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.011+0000 7f837519a700 1 --2- 172.21.15.156:0/3631529777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f837015c9b0 0x7f83700b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.011+0000 7f837519a700 1 -- 172.21.15.156:0/3631529777 >> v2:172.21.15.186:3300/0 conn(0x7f83700d2320 msgr2=0x7f83700cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:36.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.011+0000 7f837519a700 1 --2- 172.21.15.156:0/3631529777 >> v2:172.21.15.186:3300/0 conn(0x7f83700d2320 0x7f83700cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.011+0000 7f837519a700 1 -- 172.21.15.156:0/3631529777 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83700d1300 con 0x7f8370161340 2023-11-28T13:10:36.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.011+0000 7f837619c700 1 --2- 172.21.15.156:0/3631529777 >> v2:172.21.15.186:3300/0 conn(0x7f83700d2320 0x7f83700cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:36.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.011+0000 7f837519a700 1 --2- 172.21.15.156:0/3631529777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8370161340 0x7f83700d1de0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f837008c4d0 tx=0x7f8360009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e49c48560e475fe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:36.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.011+0000 7f8374999700 1 -- 172.21.15.156:0/3631529777 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360011830 con 0x7f8370161340 2023-11-28T13:10:36.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.011+0000 7f8374999700 1 -- 172.21.15.156:0/3631529777 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8360011990 con 0x7f8370161340 2023-11-28T13:10:36.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.012+0000 7f8374999700 1 -- 172.21.15.156:0/3631529777 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360011d20 con 0x7f8370161340 2023-11-28T13:10:36.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.012+0000 7f8377bff700 1 -- 172.21.15.156:0/3631529777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8370161340 msgr2=0x7f83700d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.012+0000 7f8377bff700 1 --2- 172.21.15.156:0/3631529777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8370161340 0x7f83700d1de0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f837008c4d0 tx=0x7f8360009bd0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.012+0000 7f8377bff700 1 -- 172.21.15.156:0/3631529777 shutdown_connections 2023-11-28T13:10:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.012+0000 7f8377bff700 1 --2- 172.21.15.156:0/3631529777 >> v2:172.21.15.186:3300/0 conn(0x7f83700d2320 0x7f83700cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.012+0000 7f8377bff700 1 --2- 172.21.15.156:0/3631529777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f837015c9b0 0x7f83700b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.012+0000 7f8377bff700 1 --2- 172.21.15.156:0/3631529777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8370161340 0x7f83700d1de0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.012+0000 7f8377bff700 1 -- 172.21.15.156:0/3631529777 >> 172.21.15.156:0/3631529777 conn(0x7f8370072600 msgr2=0x7f83700cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.012+0000 7f8377bff700 1 -- 172.21.15.156:0/3631529777 shutdown_connections 2023-11-28T13:10:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.012+0000 7f8377bff700 1 -- 172.21.15.156:0/3631529777 wait complete. 2023-11-28T13:10:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.014+0000 7f8377bff700 1 Processor -- start 2023-11-28T13:10:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.014+0000 7f8377bff700 1 -- start start 2023-11-28T13:10:36.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.015+0000 7f8377bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83700d2320 0x7f8370150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:36.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.015+0000 7f8377bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f837015c9b0 0x7f83700c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:36.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.015+0000 7f8377bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8370161340 0x7f83701536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:36.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.015+0000 7f8377bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8370150940 con 0x7f8370161340 2023-11-28T13:10:36.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.015+0000 7f8377bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8370154bb0 con 0x7f837015c9b0 2023-11-28T13:10:36.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.015+0000 7f8377bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8370154e50 con 0x7f83700d2320 2023-11-28T13:10:36.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.015+0000 7f837619c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8370161340 0x7f83701536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:36.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.015+0000 7f837619c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8370161340 0x7f83701536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41914/0 (socket says 172.21.15.156:41914) 2023-11-28T13:10:36.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.015+0000 7f837619c700 1 -- 172.21.15.156:0/1979346229 learned_addr learned my addr 172.21.15.156:0/1979346229 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:36.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.016+0000 7f837599b700 1 --2- 172.21.15.156:0/1979346229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83700d2320 0x7f8370150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:36.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.016+0000 7f837519a700 1 --2- 172.21.15.156:0/1979346229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f837015c9b0 0x7f83700c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:36.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.016+0000 7f837619c700 1 -- 172.21.15.156:0/1979346229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83700d2320 msgr2=0x7f8370150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:36.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.016+0000 7f837619c700 1 --2- 172.21.15.156:0/1979346229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83700d2320 0x7f8370150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.016+0000 7f837619c700 1 -- 172.21.15.156:0/1979346229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f837015c9b0 msgr2=0x7f83700c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:36.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.016+0000 7f837619c700 1 --2- 172.21.15.156:0/1979346229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f837015c9b0 0x7f83700c98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.016+0000 7f837619c700 1 -- 172.21.15.156:0/1979346229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8360009550 con 0x7f8370161340 2023-11-28T13:10:36.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.016+0000 7f837619c700 1 --2- 172.21.15.156:0/1979346229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8370161340 0x7f83701536d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8368007e70 tx=0x7f836800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:36.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.016+0000 7f83667fc700 1 -- 172.21.15.156:0/1979346229 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836800ddd0 con 0x7f8370161340 2023-11-28T13:10:36.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.016+0000 7f83667fc700 1 -- 172.21.15.156:0/1979346229 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836800f040 con 0x7f8370161340 2023-11-28T13:10:36.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.016+0000 7f8377bff700 1 -- 172.21.15.156:0/1979346229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83700b5130 con 0x7f8370161340 2023-11-28T13:10:36.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.016+0000 7f83667fc700 1 -- 172.21.15.156:0/1979346229 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368003c60 con 0x7f8370161340 2023-11-28T13:10:36.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.016+0000 7f8377bff700 1 -- 172.21.15.156:0/1979346229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83700d15e0 con 0x7f8370161340 2023-11-28T13:10:36.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.018+0000 7f8377bff700 1 -- 172.21.15.156:0/1979346229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83700b52c0 con 0x7f8370161340 2023-11-28T13:10:36.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.018+0000 7f83667fc700 1 -- 172.21.15.156:0/1979346229 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f836801c030 con 0x7f8370161340 2023-11-28T13:10:36.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.019+0000 7f83667fc700 1 --2- 172.21.15.156:0/1979346229 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f83540682f0 0x7f835406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:36.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.019+0000 7f83667fc700 1 -- 172.21.15.156:0/1979346229 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8368085cc0 con 0x7f8370161340 2023-11-28T13:10:36.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.020+0000 7f837599b700 1 --2- 172.21.15.156:0/1979346229 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f83540682f0 0x7f835406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:36.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.020+0000 7f837599b700 1 --2- 172.21.15.156:0/1979346229 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f83540682f0 0x7f835406a7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f836c009ac0 tx=0x7f836c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:36.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.023+0000 7f83667fc700 1 -- 172.21.15.156:0/1979346229 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f836808c080 con 0x7f8370161340 2023-11-28T13:10:36.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.211+0000 7f8377bff700 1 -- 172.21.15.156:0/1979346229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f83700745c0 con 0x7f8370161340 2023-11-28T13:10:36.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.212+0000 7f83667fc700 1 -- 172.21.15.156:0/1979346229 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f836801a0a0 con 0x7f8370161340 2023-11-28T13:10:36.214 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T13:10:36.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.214+0000 7f8377bff700 1 -- 172.21.15.156:0/1979346229 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f83540682f0 msgr2=0x7f835406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:36.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.214+0000 7f8377bff700 1 --2- 172.21.15.156:0/1979346229 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f83540682f0 0x7f835406a7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f836c009ac0 tx=0x7f836c009210 comp rx=0 tx=0).stop 2023-11-28T13:10:36.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.214+0000 7f8377bff700 1 -- 172.21.15.156:0/1979346229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8370161340 msgr2=0x7f83701536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:36.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.214+0000 7f8377bff700 1 --2- 172.21.15.156:0/1979346229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8370161340 0x7f83701536d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8368007e70 tx=0x7f836800d3b0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.214+0000 7f8377bff700 1 -- 172.21.15.156:0/1979346229 shutdown_connections 2023-11-28T13:10:36.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.214+0000 7f8377bff700 1 --2- 172.21.15.156:0/1979346229 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f83540682f0 0x7f835406a7a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.214+0000 7f8377bff700 1 --2- 172.21.15.156:0/1979346229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8370161340 0x7f83701536d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.214+0000 7f8377bff700 1 --2- 172.21.15.156:0/1979346229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83700d2320 0x7f8370150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.214+0000 7f8377bff700 1 --2- 172.21.15.156:0/1979346229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f837015c9b0 0x7f83700c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.214+0000 7f8377bff700 1 -- 172.21.15.156:0/1979346229 >> 172.21.15.156:0/1979346229 conn(0x7f8370072600 msgr2=0x7f8370160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:36.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.214+0000 7f8377bff700 1 -- 172.21.15.156:0/1979346229 shutdown_connections 2023-11-28T13:10:36.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.214+0000 7f8377bff700 1 -- 172.21.15.156:0/1979346229 wait complete. 2023-11-28T13:10:36.235 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:10:36.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.432+0000 7f3f381ff700 1 Processor -- start 2023-11-28T13:10:36.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.432+0000 7f3f381ff700 1 -- start start 2023-11-28T13:10:36.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.433+0000 7f3f381ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3f30161340 0x7f3f300d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:36.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.433+0000 7f3f381ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f300d2320 0x7f3f300cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:36.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.433+0000 7f3f381ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f300d1950 con 0x7f3f3015c9b0 2023-11-28T13:10:36.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.433+0000 7f3f381ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f300b5800 con 0x7f3f30161340 2023-11-28T13:10:36.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.433+0000 7f3f381ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300b5300 con 0x7f3f300d2320 2023-11-28T13:10:36.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.434+0000 7f3f3679c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f300d2320 0x7f3f300cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:36.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.434+0000 7f3f3679c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f300d2320 0x7f3f300cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:37504/0 (socket says 172.21.15.156:37504) 2023-11-28T13:10:36.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.434+0000 7f3f3679c700 1 -- 172.21.15.156:0/2290340401 learned_addr learned my addr 172.21.15.156:0/2290340401 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:36.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.434+0000 7f3f3579a700 1 --2- 172.21.15.156:0/2290340401 >> v2:172.21.15.186:3300/0 conn(0x7f3f30161340 0x7f3f300d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:36.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.434+0000 7f3f3679c700 1 -- 172.21.15.156:0/2290340401 >> v2:172.21.15.186:3300/0 conn(0x7f3f30161340 msgr2=0x7f3f300d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:36.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.434+0000 7f3f3679c700 1 --2- 172.21.15.156:0/2290340401 >> v2:172.21.15.186:3300/0 conn(0x7f3f30161340 0x7f3f300d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.434+0000 7f3f3679c700 1 -- 172.21.15.156:0/2290340401 >> v1:172.21.15.186:6789/0 conn(0x7f3f3015c9b0 legacy=0x7f3f300b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:36.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.434+0000 7f3f3679c700 1 -- 172.21.15.156:0/2290340401 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f300b5520 con 0x7f3f300d2320 2023-11-28T13:10:36.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.435+0000 7f3f3679c700 1 --2- 172.21.15.156:0/2290340401 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f300d2320 0x7f3f300cd7c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009e50 tx=0x7f3f2800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae30a28cbd3fa1fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:36.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.435+0000 7f3f34f99700 1 -- 172.21.15.156:0/2290340401 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2800e070 con 0x7f3f300d2320 2023-11-28T13:10:36.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.435+0000 7f3f34f99700 1 -- 172.21.15.156:0/2290340401 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2800b910 con 0x7f3f300d2320 2023-11-28T13:10:36.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.435+0000 7f3f34f99700 1 -- 172.21.15.156:0/2290340401 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2800bcc0 con 0x7f3f300d2320 2023-11-28T13:10:36.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.436+0000 7f3f381ff700 1 -- 172.21.15.156:0/2290340401 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f300d2320 msgr2=0x7f3f300cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:36.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.436+0000 7f3f381ff700 1 --2- 172.21.15.156:0/2290340401 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f300d2320 0x7f3f300cd7c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009e50 tx=0x7f3f2800b330 comp rx=0 tx=0).stop 2023-11-28T13:10:36.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.436+0000 7f3f381ff700 1 -- 172.21.15.156:0/2290340401 shutdown_connections 2023-11-28T13:10:36.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.436+0000 7f3f381ff700 1 --2- 172.21.15.156:0/2290340401 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f300d2320 0x7f3f300cd7c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.436+0000 7f3f381ff700 1 --2- 172.21.15.156:0/2290340401 >> v2:172.21.15.186:3300/0 conn(0x7f3f30161340 0x7f3f300d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.436+0000 7f3f381ff700 1 -- 172.21.15.156:0/2290340401 >> 172.21.15.156:0/2290340401 conn(0x7f3f30072600 msgr2=0x7f3f300cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:36.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.436+0000 7f3f381ff700 1 -- 172.21.15.156:0/2290340401 shutdown_connections 2023-11-28T13:10:36.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.436+0000 7f3f381ff700 1 -- 172.21.15.156:0/2290340401 wait complete. 2023-11-28T13:10:36.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.437+0000 7f3f381ff700 1 Processor -- start 2023-11-28T13:10:36.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.438+0000 7f3f381ff700 1 -- start start 2023-11-28T13:10:36.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.438+0000 7f3f381ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f300d2320 0x7f3f30157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:36.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.439+0000 7f3f381ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f3015c9b0 0x7f3f300c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:36.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.439+0000 7f3f381ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f30161340 0x7f3f300c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:36.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.439+0000 7f3f381ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300c6ca0 con 0x7f3f300d2320 2023-11-28T13:10:36.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.439+0000 7f3f381ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300c7b00 con 0x7f3f30161340 2023-11-28T13:10:36.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.439+0000 7f3f381ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300c6ef0 con 0x7f3f3015c9b0 2023-11-28T13:10:36.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.439+0000 7f3f35f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f300d2320 0x7f3f30157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:36.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.439+0000 7f3f35f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f300d2320 0x7f3f30157950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41922/0 (socket says 172.21.15.156:41922) 2023-11-28T13:10:36.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.439+0000 7f3f35f9b700 1 -- 172.21.15.156:0/2314672522 learned_addr learned my addr 172.21.15.156:0/2314672522 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:36.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.439+0000 7f3f3679c700 1 --2- 172.21.15.156:0/2314672522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f30161340 0x7f3f300c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:36.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.439+0000 7f3f3579a700 1 --2- 172.21.15.156:0/2314672522 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f3015c9b0 0x7f3f300c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:36.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.439+0000 7f3f35f9b700 1 -- 172.21.15.156:0/2314672522 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f3015c9b0 msgr2=0x7f3f300c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:36.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.439+0000 7f3f35f9b700 1 --2- 172.21.15.156:0/2314672522 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f3015c9b0 0x7f3f300c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.439+0000 7f3f35f9b700 1 -- 172.21.15.156:0/2314672522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f30161340 msgr2=0x7f3f300c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:36.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.439+0000 7f3f35f9b700 1 --2- 172.21.15.156:0/2314672522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f30161340 0x7f3f300c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.439+0000 7f3f35f9b700 1 -- 172.21.15.156:0/2314672522 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f28009b30 con 0x7f3f300d2320 2023-11-28T13:10:36.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.439+0000 7f3f35f9b700 1 --2- 172.21.15.156:0/2314672522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f300d2320 0x7f3f30157950 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00c480 tx=0x7f3f2c00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:36.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.440+0000 7f3f267fc700 1 -- 172.21.15.156:0/2314672522 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c0052d0 con 0x7f3f300d2320 2023-11-28T13:10:36.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.440+0000 7f3f267fc700 1 -- 172.21.15.156:0/2314672522 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2c005430 con 0x7f3f300d2320 2023-11-28T13:10:36.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.440+0000 7f3f267fc700 1 -- 172.21.15.156:0/2314672522 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c0038c0 con 0x7f3f300d2320 2023-11-28T13:10:36.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.440+0000 7f3f381ff700 1 -- 172.21.15.156:0/2314672522 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f300d15e0 con 0x7f3f300d2320 2023-11-28T13:10:36.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.440+0000 7f3f381ff700 1 -- 172.21.15.156:0/2314672522 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f300d1300 con 0x7f3f300d2320 2023-11-28T13:10:36.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.441+0000 7f3f381ff700 1 -- 172.21.15.156:0/2314672522 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3015cd80 con 0x7f3f300d2320 2023-11-28T13:10:36.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.442+0000 7f3f267fc700 1 -- 172.21.15.156:0/2314672522 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3f2c019030 con 0x7f3f300d2320 2023-11-28T13:10:36.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.443+0000 7f3f267fc700 1 --2- 172.21.15.156:0/2314672522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3f140682f0 0x7f3f1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:36.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.443+0000 7f3f267fc700 1 -- 172.21.15.156:0/2314672522 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f2c084fd0 con 0x7f3f300d2320 2023-11-28T13:10:36.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.443+0000 7f3f3579a700 1 --2- 172.21.15.156:0/2314672522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3f140682f0 0x7f3f1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:36.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.444+0000 7f3f3579a700 1 --2- 172.21.15.156:0/2314672522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3f140682f0 0x7f3f1406a7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3f300cdca0 tx=0x7f3f20009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:36.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.447+0000 7f3f267fc700 1 -- 172.21.15.156:0/2314672522 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3f2c051650 con 0x7f3f300d2320 2023-11-28T13:10:36.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.654+0000 7f3f381ff700 1 -- 172.21.15.156:0/2314672522 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3f30157ef0 con 0x7f3f300d2320 2023-11-28T13:10:36.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.654+0000 7f3f267fc700 1 -- 172.21.15.156:0/2314672522 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3f2c017020 con 0x7f3f300d2320 2023-11-28T13:10:36.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.656+0000 7f3f381ff700 1 -- 172.21.15.156:0/2314672522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3f140682f0 msgr2=0x7f3f1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:36.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.656+0000 7f3f381ff700 1 --2- 172.21.15.156:0/2314672522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3f140682f0 0x7f3f1406a7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3f300cdca0 tx=0x7f3f20009210 comp rx=0 tx=0).stop 2023-11-28T13:10:36.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.656+0000 7f3f381ff700 1 -- 172.21.15.156:0/2314672522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f300d2320 msgr2=0x7f3f30157950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:36.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.656+0000 7f3f381ff700 1 --2- 172.21.15.156:0/2314672522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f300d2320 0x7f3f30157950 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00c480 tx=0x7f3f2c00c790 comp rx=0 tx=0).stop 2023-11-28T13:10:36.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.656+0000 7f3f381ff700 1 -- 172.21.15.156:0/2314672522 shutdown_connections 2023-11-28T13:10:36.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.656+0000 7f3f381ff700 1 --2- 172.21.15.156:0/2314672522 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3f140682f0 0x7f3f1406a7a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.656+0000 7f3f381ff700 1 --2- 172.21.15.156:0/2314672522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f30161340 0x7f3f300c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.656+0000 7f3f381ff700 1 --2- 172.21.15.156:0/2314672522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f300d2320 0x7f3f30157950 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.656+0000 7f3f381ff700 1 --2- 172.21.15.156:0/2314672522 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f3015c9b0 0x7f3f300c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:36.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.656+0000 7f3f381ff700 1 -- 172.21.15.156:0/2314672522 >> 172.21.15.156:0/2314672522 conn(0x7f3f30072600 msgr2=0x7f3f300b62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:36.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.657+0000 7f3f381ff700 1 -- 172.21.15.156:0/2314672522 shutdown_connections 2023-11-28T13:10:36.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:36.657+0000 7f3f381ff700 1 -- 172.21.15.156:0/2314672522 wait complete. 2023-11-28T13:10:36.660 INFO:teuthology.orchestra.run.smithi156.stderr:listed 0 entries 2023-11-28T13:10:36.676 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T13:10:36.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:10:36.677 DEBUG:teuthology.orchestra.run.smithi156:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T13:10:36.788 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T13:10:36.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:10:36.788 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T13:10:36.841 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:10:36.841 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:10:36.886 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:10:36.887 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:10:36.887 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:10:36.930 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:10:36.931 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:10:36.931 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:10:36.976 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:10:36.977 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:10:37.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.172+0000 7f7b11bff700 1 Processor -- start 2023-11-28T13:10:37.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.172+0000 7f7b11bff700 1 -- start start 2023-11-28T13:10:37.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.173+0000 7f7b11bff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7b0c15c9b0 0x7f7b0c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:37.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.174+0000 7f7b11bff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0c0d1950 con 0x7f7b0c161340 2023-11-28T13:10:37.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.174+0000 7f7b11bff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0c0d24f0 con 0x7f7b0c0cd7c0 2023-11-28T13:10:37.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.174+0000 7f7b11bff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0c0b5800 con 0x7f7b0c15c9b0 2023-11-28T13:10:37.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.174+0000 7f7b0affd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f7b0c161340 0x7f7b0c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37818/0 (socket says 172.21.15.156:37818) 2023-11-28T13:10:37.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.174+0000 7f7b0affd700 1 -- 172.21.15.156:0/2806597488 learned_addr learned my addr 172.21.15.156:0/2806597488 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:37.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.174+0000 7f7b0b7fe700 1 --2- 172.21.15.156:0/2806597488 >> v2:172.21.15.186:3300/0 conn(0x7f7b0c15c9b0 0x7f7b0c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:37.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.174+0000 7f7b0a7fc700 1 -- 172.21.15.156:0/2806597488 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941298474 0 0) 0x7f7b0c0d1950 con 0x7f7b0c161340 2023-11-28T13:10:37.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.174+0000 7f7b0a7fc700 1 -- 172.21.15.156:0/2806597488 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7af40035b0 con 0x7f7b0c161340 2023-11-28T13:10:37.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.174+0000 7f7b0a7fc700 1 -- 172.21.15.156:0/2806597488 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2758057096 0 0) 0x7f7af40035b0 con 0x7f7b0c161340 2023-11-28T13:10:37.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.174+0000 7f7b0a7fc700 1 -- 172.21.15.156:0/2806597488 >> v2:172.21.15.186:3300/0 conn(0x7f7b0c15c9b0 msgr2=0x7f7b0c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:37.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.174+0000 7f7b0a7fc700 1 --2- 172.21.15.156:0/2806597488 >> v2:172.21.15.186:3300/0 conn(0x7f7b0c15c9b0 0x7f7b0c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.174+0000 7f7b0a7fc700 1 -- 172.21.15.156:0/2806597488 >> v1:172.21.15.186:6789/0 conn(0x7f7b0c0cd7c0 legacy=0x7f7b0c0b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:37.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.174+0000 7f7b0a7fc700 1 -- 172.21.15.156:0/2806597488 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c0b5520 con 0x7f7b0c161340 2023-11-28T13:10:37.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.174+0000 7f7b0a7fc700 1 -- 172.21.15.156:0/2806597488 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f7b00001670 con 0x7f7b0c161340 2023-11-28T13:10:37.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.175+0000 7f7b0a7fc700 1 -- 172.21.15.156:0/2806597488 >> v1:172.21.15.156:6789/0 conn(0x7f7b0c161340 legacy=0x7f7b0c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:37.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.175+0000 7f7b0a7fc700 1 --2- 172.21.15.156:0/2806597488 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af4003a00 0x7f7af4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:37.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.175+0000 7f7b0a7fc700 1 --2- 172.21.15.156:0/2806597488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af40043f0 0x7f7af40048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:37.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.175+0000 7f7b0a7fc700 1 --2- 172.21.15.156:0/2806597488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7af4005150 0x7f7af400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:37.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.175+0000 7f7b0a7fc700 1 -- 172.21.15.156:0/2806597488 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c0b5800 con 0x7f7af40043f0 2023-11-28T13:10:37.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.175+0000 7f7b0a7fc700 1 -- 172.21.15.156:0/2806597488 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af400c080 con 0x7f7af4005150 2023-11-28T13:10:37.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.175+0000 7f7b0a7fc700 1 -- 172.21.15.156:0/2806597488 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af400c240 con 0x7f7af4003a00 2023-11-28T13:10:37.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.175+0000 7f7b0affd700 1 --2- 172.21.15.156:0/2806597488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af40043f0 0x7f7af40048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:37.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.175+0000 7f7b0b7fe700 1 --2- 172.21.15.156:0/2806597488 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af4003a00 0x7f7af4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:37.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.175+0000 7f7b0bfff700 1 --2- 172.21.15.156:0/2806597488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7af4005150 0x7f7af400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:37.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.175+0000 7f7b0affd700 1 -- 172.21.15.156:0/2806597488 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af4003a00 msgr2=0x7f7af4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:37.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.175+0000 7f7b0affd700 1 --2- 172.21.15.156:0/2806597488 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af4003a00 0x7f7af4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.175+0000 7f7b0affd700 1 -- 172.21.15.156:0/2806597488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7af4005150 msgr2=0x7f7af400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:37.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.175+0000 7f7b0affd700 1 --2- 172.21.15.156:0/2806597488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7af4005150 0x7f7af400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.175+0000 7f7b0affd700 1 -- 172.21.15.156:0/2806597488 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af40035b0 con 0x7f7af40043f0 2023-11-28T13:10:37.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.175+0000 7f7b0bfff700 1 -- 172.21.15.156:0/2806597488 reap_dead start 2023-11-28T13:10:37.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.176+0000 7f7b0affd700 1 --2- 172.21.15.156:0/2806597488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af40043f0 0x7f7af40048b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000c2f0 tx=0x7f7af4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1affbc1eab309b4f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:37.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.176+0000 7f7b0a7fc700 1 -- 172.21.15.156:0/2806597488 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0000ce30 con 0x7f7af40043f0 2023-11-28T13:10:37.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.176+0000 7f7b0a7fc700 1 -- 172.21.15.156:0/2806597488 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b00005960 con 0x7f7af40043f0 2023-11-28T13:10:37.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.176+0000 7f7b0a7fc700 1 -- 172.21.15.156:0/2806597488 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00005d10 con 0x7f7af40043f0 2023-11-28T13:10:37.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.176+0000 7f7b11bff700 1 -- 172.21.15.156:0/2806597488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af40043f0 msgr2=0x7f7af40048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:37.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.176+0000 7f7b11bff700 1 --2- 172.21.15.156:0/2806597488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af40043f0 0x7f7af40048b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000c2f0 tx=0x7f7af4005880 comp rx=0 tx=0).stop 2023-11-28T13:10:37.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.176+0000 7f7b11bff700 1 -- 172.21.15.156:0/2806597488 shutdown_connections 2023-11-28T13:10:37.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.176+0000 7f7b11bff700 1 --2- 172.21.15.156:0/2806597488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af40043f0 0x7f7af40048b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.176+0000 7f7b11bff700 1 -- 172.21.15.156:0/2806597488 >> 172.21.15.156:0/2806597488 conn(0x7f7b0c072600 msgr2=0x7f7b0c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:37.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.176+0000 7f7b11bff700 1 -- 172.21.15.156:0/2806597488 shutdown_connections 2023-11-28T13:10:37.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.176+0000 7f7b11bff700 1 -- 172.21.15.156:0/2806597488 wait complete. 2023-11-28T13:10:37.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.178+0000 7f7b11bff700 1 Processor -- start 2023-11-28T13:10:37.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.178+0000 7f7b11bff700 1 -- start start 2023-11-28T13:10:37.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.179+0000 7f7b11bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af40043f0 0x7f7b0c1f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:37.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.179+0000 7f7b11bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b0c0c8f00 0x7f7b0c0ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:37.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.179+0000 7f7b11bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b0c156a00 0x7f7b0c0ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:37.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.179+0000 7f7b11bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c15db00 con 0x7f7b0c156a00 2023-11-28T13:10:37.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.179+0000 7f7b11bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c1f6c60 con 0x7f7b0c0c8f00 2023-11-28T13:10:37.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.179+0000 7f7b11bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c1f63c0 con 0x7f7af40043f0 2023-11-28T13:10:37.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.180+0000 7f7b0bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b0c156a00 0x7f7b0c0ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:37.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.180+0000 7f7b0bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b0c156a00 0x7f7b0c0ca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41928/0 (socket says 172.21.15.156:41928) 2023-11-28T13:10:37.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.180+0000 7f7b0bfff700 1 -- 172.21.15.156:0/1842392941 learned_addr learned my addr 172.21.15.156:0/1842392941 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:37.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.180+0000 7f7b0b7fe700 1 --2- 172.21.15.156:0/1842392941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af40043f0 0x7f7b0c1f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:37.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.180+0000 7f7b0affd700 1 --2- 172.21.15.156:0/1842392941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b0c0c8f00 0x7f7b0c0ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:37.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.180+0000 7f7b0bfff700 1 -- 172.21.15.156:0/1842392941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af40043f0 msgr2=0x7f7b0c1f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:37.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.180+0000 7f7b0bfff700 1 --2- 172.21.15.156:0/1842392941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af40043f0 0x7f7b0c1f5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.180+0000 7f7b0bfff700 1 -- 172.21.15.156:0/1842392941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b0c0c8f00 msgr2=0x7f7b0c0ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:37.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.180+0000 7f7b0bfff700 1 --2- 172.21.15.156:0/1842392941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b0c0c8f00 0x7f7b0c0ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.180+0000 7f7b0bfff700 1 -- 172.21.15.156:0/1842392941 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c0d15e0 con 0x7f7b0c156a00 2023-11-28T13:10:37.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.180+0000 7f7b0bfff700 1 --2- 172.21.15.156:0/1842392941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b0c156a00 0x7f7b0c0ca8a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c08bfd0 tx=0x7f7b0400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:37.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.181+0000 7f7af3fff700 1 -- 172.21.15.156:0/1842392941 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0400fed0 con 0x7f7b0c156a00 2023-11-28T13:10:37.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.181+0000 7f7af3fff700 1 -- 172.21.15.156:0/1842392941 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b04003c10 con 0x7f7b0c156a00 2023-11-28T13:10:37.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.181+0000 7f7af3fff700 1 -- 172.21.15.156:0/1842392941 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0400d990 con 0x7f7b0c156a00 2023-11-28T13:10:37.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.181+0000 7f7b11bff700 1 -- 172.21.15.156:0/1842392941 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0c0d1300 con 0x7f7b0c156a00 2023-11-28T13:10:37.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.181+0000 7f7b11bff700 1 -- 172.21.15.156:0/1842392941 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0c15cd80 con 0x7f7b0c156a00 2023-11-28T13:10:37.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.182+0000 7f7b11bff700 1 -- 172.21.15.156:0/1842392941 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0c0745c0 con 0x7f7b0c156a00 2023-11-28T13:10:37.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.183+0000 7f7af3fff700 1 -- 172.21.15.156:0/1842392941 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7b0400daf0 con 0x7f7b0c156a00 2023-11-28T13:10:37.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.184+0000 7f7af3fff700 1 --2- 172.21.15.156:0/1842392941 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7af80682f0 0x7f7af806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:37.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.184+0000 7f7af3fff700 1 -- 172.21.15.156:0/1842392941 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7b040860d0 con 0x7f7b0c156a00 2023-11-28T13:10:37.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.184+0000 7f7b0b7fe700 1 --2- 172.21.15.156:0/1842392941 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7af80682f0 0x7f7af806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:37.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.185+0000 7f7b0b7fe700 1 --2- 172.21.15.156:0/1842392941 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7af80682f0 0x7f7af806a7a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0cc170 tx=0x7f7afc009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:37.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.188+0000 7f7af3fff700 1 -- 172.21.15.156:0/1842392941 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7b04052840 con 0x7f7b0c156a00 2023-11-28T13:10:37.428 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.427+0000 7f7b11bff700 1 -- 172.21.15.156:0/1842392941 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b0c0d08a0 con 0x7f7b0c156a00 2023-11-28T13:10:37.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.427+0000 7f7af3fff700 1 -- 172.21.15.156:0/1842392941 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f7b04055e60 con 0x7f7b0c156a00 2023-11-28T13:10:37.430 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:37.430 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3828141874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3828141874},{"type":"v1","addr":"172.21.15.156:6835","nonce":3828141874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1012489762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1012489762},{"type":"v1","addr":"172.21.15.186:6837","nonce":1012489762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2894670074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2894670074},{"type":"v1","addr":"172.21.15.186:6835","nonce":2894670074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:09:52.245032+0000","modified":"2023-11-28T13:09:55.251254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5481":{"gid":5481,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6834/1930782798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1930782798},{"type":"v1","addr":"172.21.15.156:6834","nonce":1930782798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:10:37.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.430+0000 7f7b11bff700 1 -- 172.21.15.156:0/1842392941 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7af80682f0 msgr2=0x7f7af806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:37.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.430+0000 7f7b11bff700 1 --2- 172.21.15.156:0/1842392941 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7af80682f0 0x7f7af806a7a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0cc170 tx=0x7f7afc009210 comp rx=0 tx=0).stop 2023-11-28T13:10:37.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.430+0000 7f7b11bff700 1 -- 172.21.15.156:0/1842392941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b0c156a00 msgr2=0x7f7b0c0ca8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:37.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.430+0000 7f7b11bff700 1 --2- 172.21.15.156:0/1842392941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b0c156a00 0x7f7b0c0ca8a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c08bfd0 tx=0x7f7b0400d430 comp rx=0 tx=0).stop 2023-11-28T13:10:37.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.430+0000 7f7b11bff700 1 -- 172.21.15.156:0/1842392941 shutdown_connections 2023-11-28T13:10:37.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.430+0000 7f7b11bff700 1 --2- 172.21.15.156:0/1842392941 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7af80682f0 0x7f7af806a7a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.430+0000 7f7b11bff700 1 --2- 172.21.15.156:0/1842392941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b0c156a00 0x7f7b0c0ca8a0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.430+0000 7f7b11bff700 1 --2- 172.21.15.156:0/1842392941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af40043f0 0x7f7b0c1f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.430+0000 7f7b11bff700 1 --2- 172.21.15.156:0/1842392941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b0c0c8f00 0x7f7b0c0ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.430+0000 7f7b11bff700 1 -- 172.21.15.156:0/1842392941 >> 172.21.15.156:0/1842392941 conn(0x7f7b0c072600 msgr2=0x7f7b0c15a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:37.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.430+0000 7f7b11bff700 1 -- 172.21.15.156:0/1842392941 shutdown_connections 2023-11-28T13:10:37.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.430+0000 7f7b11bff700 1 -- 172.21.15.156:0/1842392941 wait complete. 2023-11-28T13:10:37.434 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 23 2023-11-28T13:10:37.450 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:10:37.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.645+0000 7f6fb21ff700 1 Processor -- start 2023-11-28T13:10:37.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.646+0000 7f6fb21ff700 1 -- start start 2023-11-28T13:10:37.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.647+0000 7f6fb21ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6fac15c9b0 0x7f6fac0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:37.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.647+0000 7f6fb21ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fac161340 0x7f6fac0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:37.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.647+0000 7f6fb21ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac0d1950 con 0x7f6fac0d2320 2023-11-28T13:10:37.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.647+0000 7f6fb21ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fac0b5800 con 0x7f6fac15c9b0 2023-11-28T13:10:37.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.647+0000 7f6fb21ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac0b5520 con 0x7f6fac161340 2023-11-28T13:10:37.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.648+0000 7f6fabfff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f6fac0d2320 0x7f6fac0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37824/0 (socket says 172.21.15.156:37824) 2023-11-28T13:10:37.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.648+0000 7f6fabfff700 1 -- 172.21.15.156:0/3022872154 learned_addr learned my addr 172.21.15.156:0/3022872154 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:37.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.648+0000 7f6fa3fff700 1 --2- 172.21.15.156:0/3022872154 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fac161340 0x7f6fac0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:37.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.648+0000 7f6fab7fe700 1 --2- 172.21.15.156:0/3022872154 >> v2:172.21.15.186:3300/0 conn(0x7f6fac15c9b0 0x7f6fac0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:37.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.648+0000 7f6faaffd700 1 -- 172.21.15.156:0/3022872154 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 85554338 0 0) 0x7f6fac0d1950 con 0x7f6fac0d2320 2023-11-28T13:10:37.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.648+0000 7f6faaffd700 1 -- 172.21.15.156:0/3022872154 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f980035b0 con 0x7f6fac0d2320 2023-11-28T13:10:37.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.648+0000 7f6fa3fff700 1 -- 172.21.15.156:0/3022872154 >> v2:172.21.15.186:3300/0 conn(0x7f6fac15c9b0 msgr2=0x7f6fac0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:37.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.648+0000 7f6fa3fff700 1 --2- 172.21.15.156:0/3022872154 >> v2:172.21.15.186:3300/0 conn(0x7f6fac15c9b0 0x7f6fac0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.648+0000 7f6fa3fff700 1 -- 172.21.15.156:0/3022872154 >> v1:172.21.15.156:6789/0 conn(0x7f6fac0d2320 legacy=0x7f6fac0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:37.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.648+0000 7f6fa3fff700 1 -- 172.21.15.156:0/3022872154 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac0d1300 con 0x7f6fac161340 2023-11-28T13:10:37.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.648+0000 7f6fab7fe700 1 --2- 172.21.15.156:0/3022872154 >> v2:172.21.15.186:3300/0 conn(0x7f6fac15c9b0 0x7f6fac0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:10:37.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.649+0000 7f6fa3fff700 1 --2- 172.21.15.156:0/3022872154 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fac161340 0x7f6fac0d1de0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009840 tx=0x7f6f94009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca487f6b922c378e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:37.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.649+0000 7f6faaffd700 1 -- 172.21.15.156:0/3022872154 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f94011830 con 0x7f6fac161340 2023-11-28T13:10:37.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.649+0000 7f6faaffd700 1 -- 172.21.15.156:0/3022872154 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f94011990 con 0x7f6fac161340 2023-11-28T13:10:37.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.649+0000 7f6faaffd700 1 -- 172.21.15.156:0/3022872154 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f94011d20 con 0x7f6fac161340 2023-11-28T13:10:37.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.649+0000 7f6fb21ff700 1 -- 172.21.15.156:0/3022872154 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fac161340 msgr2=0x7f6fac0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:37.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.649+0000 7f6fb21ff700 1 --2- 172.21.15.156:0/3022872154 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fac161340 0x7f6fac0d1de0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009840 tx=0x7f6f94009b50 comp rx=0 tx=0).stop 2023-11-28T13:10:37.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.650+0000 7f6fb21ff700 1 -- 172.21.15.156:0/3022872154 shutdown_connections 2023-11-28T13:10:37.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.650+0000 7f6fb21ff700 1 --2- 172.21.15.156:0/3022872154 >> v2:172.21.15.186:3300/0 conn(0x7f6fac15c9b0 0x7f6fac0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.650+0000 7f6fb21ff700 1 --2- 172.21.15.156:0/3022872154 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fac161340 0x7f6fac0d1de0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.650+0000 7f6fb21ff700 1 -- 172.21.15.156:0/3022872154 >> 172.21.15.156:0/3022872154 conn(0x7f6fac072600 msgr2=0x7f6fac0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:37.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.650+0000 7f6fb21ff700 1 -- 172.21.15.156:0/3022872154 shutdown_connections 2023-11-28T13:10:37.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.650+0000 7f6fb21ff700 1 -- 172.21.15.156:0/3022872154 wait complete. 2023-11-28T13:10:37.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.652+0000 7f6fb21ff700 1 Processor -- start 2023-11-28T13:10:37.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.652+0000 7f6fb21ff700 1 -- start start 2023-11-28T13:10:37.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.653+0000 7f6fb21ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fac0d2320 0x7f6fac14da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:37.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.653+0000 7f6fb21ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fac15c9b0 0x7f6fac0c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:37.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.653+0000 7f6fb21ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fac161340 0x7f6fac151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:37.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.653+0000 7f6fb21ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac0c9c30 con 0x7f6fac0d2320 2023-11-28T13:10:37.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.653+0000 7f6fb21ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac0c7ea0 con 0x7f6fac161340 2023-11-28T13:10:37.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.653+0000 7f6fb21ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac14dfd0 con 0x7f6fac15c9b0 2023-11-28T13:10:37.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.654+0000 7f6fab7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fac0d2320 0x7f6fac14da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:37.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.654+0000 7f6fab7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fac0d2320 0x7f6fac14da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41944/0 (socket says 172.21.15.156:41944) 2023-11-28T13:10:37.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.654+0000 7f6fab7fe700 1 -- 172.21.15.156:0/45257561 learned_addr learned my addr 172.21.15.156:0/45257561 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:37.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.654+0000 7f6fabfff700 1 --2- 172.21.15.156:0/45257561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fac161340 0x7f6fac151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:37.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.654+0000 7f6fa3fff700 1 --2- 172.21.15.156:0/45257561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fac15c9b0 0x7f6fac0c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:37.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.654+0000 7f6fab7fe700 1 -- 172.21.15.156:0/45257561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fac15c9b0 msgr2=0x7f6fac0c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:37.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.654+0000 7f6fab7fe700 1 --2- 172.21.15.156:0/45257561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fac15c9b0 0x7f6fac0c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.654+0000 7f6fab7fe700 1 -- 172.21.15.156:0/45257561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fac161340 msgr2=0x7f6fac151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:37.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.654+0000 7f6fab7fe700 1 --2- 172.21.15.156:0/45257561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fac161340 0x7f6fac151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.654+0000 7f6fab7fe700 1 -- 172.21.15.156:0/45257561 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c0094a0 con 0x7f6fac0d2320 2023-11-28T13:10:37.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.654+0000 7f6fa3fff700 1 --2- 172.21.15.156:0/45257561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fac15c9b0 0x7f6fac0c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:37.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.654+0000 7f6fab7fe700 1 --2- 172.21.15.156:0/45257561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fac0d2320 0x7f6fac14da90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00ebe0 tx=0x7f6f9c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:37.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.655+0000 7f6fa8ff9700 1 -- 172.21.15.156:0/45257561 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c009800 con 0x7f6fac0d2320 2023-11-28T13:10:37.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.655+0000 7f6fa8ff9700 1 -- 172.21.15.156:0/45257561 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c00c910 con 0x7f6fac0d2320 2023-11-28T13:10:37.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.655+0000 7f6fb21ff700 1 -- 172.21.15.156:0/45257561 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f940094a0 con 0x7f6fac0d2320 2023-11-28T13:10:37.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.655+0000 7f6fa8ff9700 1 -- 172.21.15.156:0/45257561 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c00cc80 con 0x7f6fac0d2320 2023-11-28T13:10:37.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.655+0000 7f6fb21ff700 1 -- 172.21.15.156:0/45257561 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fac0b5130 con 0x7f6fac0d2320 2023-11-28T13:10:37.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.656+0000 7f6fb21ff700 1 -- 172.21.15.156:0/45257561 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fac0d15e0 con 0x7f6fac0d2320 2023-11-28T13:10:37.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.657+0000 7f6fa8ff9700 1 -- 172.21.15.156:0/45257561 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6f9c023070 con 0x7f6fac0d2320 2023-11-28T13:10:37.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.657+0000 7f6fa8ff9700 1 --2- 172.21.15.156:0/45257561 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6f8c0682f0 0x7f6f8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:37.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.657+0000 7f6fa8ff9700 1 -- 172.21.15.156:0/45257561 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6f9c086010 con 0x7f6fac0d2320 2023-11-28T13:10:37.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.658+0000 7f6fa3fff700 1 --2- 172.21.15.156:0/45257561 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6f8c0682f0 0x7f6f8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:37.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.658+0000 7f6fa3fff700 1 --2- 172.21.15.156:0/45257561 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6f8c0682f0 0x7f6f8c06a7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6f94000c00 tx=0x7f6f94008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:37.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.662+0000 7f6fa8ff9700 1 -- 172.21.15.156:0/45257561 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6f9c052810 con 0x7f6fac0d2320 2023-11-28T13:10:37.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.855+0000 7f6fb21ff700 1 -- 172.21.15.156:0/45257561 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fac0b52c0 con 0x7f6fac0d2320 2023-11-28T13:10:37.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.856+0000 7f6fa8ff9700 1 -- 172.21.15.156:0/45257561 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+17969 (secure 0 0 0) 0x7f6f9c055e30 con 0x7f6fac0d2320 2023-11-28T13:10:37.858 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:37.858 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":47,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:09:53.247402+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:09:49.958043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T13:09:51.219302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:10:37.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.858+0000 7f6fb21ff700 1 -- 172.21.15.156:0/45257561 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6f8c0682f0 msgr2=0x7f6f8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:37.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.858+0000 7f6fb21ff700 1 --2- 172.21.15.156:0/45257561 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6f8c0682f0 0x7f6f8c06a7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6f94000c00 tx=0x7f6f94008030 comp rx=0 tx=0).stop 2023-11-28T13:10:37.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.858+0000 7f6fb21ff700 1 -- 172.21.15.156:0/45257561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fac0d2320 msgr2=0x7f6fac14da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:37.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.858+0000 7f6fb21ff700 1 --2- 172.21.15.156:0/45257561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fac0d2320 0x7f6fac14da90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00ebe0 tx=0x7f6f9c00eef0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.859+0000 7f6fb21ff700 1 -- 172.21.15.156:0/45257561 shutdown_connections 2023-11-28T13:10:37.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.859+0000 7f6fb21ff700 1 --2- 172.21.15.156:0/45257561 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6f8c0682f0 0x7f6f8c06a7a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.859+0000 7f6fb21ff700 1 --2- 172.21.15.156:0/45257561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fac161340 0x7f6fac151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.859+0000 7f6fb21ff700 1 --2- 172.21.15.156:0/45257561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fac0d2320 0x7f6fac14da90 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.859+0000 7f6fb21ff700 1 --2- 172.21.15.156:0/45257561 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fac15c9b0 0x7f6fac0c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:37.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.859+0000 7f6fb21ff700 1 -- 172.21.15.156:0/45257561 >> 172.21.15.156:0/45257561 conn(0x7f6fac072600 msgr2=0x7f6fac15a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:37.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.859+0000 7f6fb21ff700 1 -- 172.21.15.156:0/45257561 shutdown_connections 2023-11-28T13:10:37.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:37.859+0000 7f6fb21ff700 1 -- 172.21.15.156:0/45257561 wait complete. 2023-11-28T13:10:37.879 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:09:49.958043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:09:51.219302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:10:37.880 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:10:38.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.078+0000 7f0d903ff700 1 Processor -- start 2023-11-28T13:10:38.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.078+0000 7f0d903ff700 1 -- start start 2023-11-28T13:10:38.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.079+0000 7f0d903ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d8815c9b0 0x7f0d880b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.079+0000 7f0d903ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0d880cd7c0 0x7f0d880cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.079+0000 7f0d903ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d880ce160 con 0x7f0d88161340 2023-11-28T13:10:38.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.079+0000 7f0d903ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d880b5800 con 0x7f0d880cd7c0 2023-11-28T13:10:38.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.079+0000 7f0d903ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880d1950 con 0x7f0d8815c9b0 2023-11-28T13:10:38.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.079+0000 7f0d8d99a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f0d88161340 0x7f0d880d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37834/0 (socket says 172.21.15.156:37834) 2023-11-28T13:10:38.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.079+0000 7f0d8d99a700 1 -- 172.21.15.156:0/2723436376 learned_addr learned my addr 172.21.15.156:0/2723436376 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:38.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8e99c700 1 --2- 172.21.15.156:0/2723436376 >> v2:172.21.15.186:3300/0 conn(0x7f0d880cd7c0 0x7f0d880cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8e19b700 1 --2- 172.21.15.156:0/2723436376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d8815c9b0 0x7f0d880b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8d199700 1 -- 172.21.15.156:0/2723436376 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825829555 0 0) 0x7f0d880ce160 con 0x7f0d88161340 2023-11-28T13:10:38.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8d199700 1 -- 172.21.15.156:0/2723436376 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d740035b0 con 0x7f0d88161340 2023-11-28T13:10:38.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8d199700 1 -- 172.21.15.156:0/2723436376 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2024377007 0 0) 0x7f0d740035b0 con 0x7f0d88161340 2023-11-28T13:10:38.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8d199700 1 -- 172.21.15.156:0/2723436376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d8815c9b0 msgr2=0x7f0d880b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:38.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8d199700 1 --2- 172.21.15.156:0/2723436376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d8815c9b0 0x7f0d880b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8d199700 1 -- 172.21.15.156:0/2723436376 >> v2:172.21.15.186:3300/0 conn(0x7f0d880cd7c0 msgr2=0x7f0d880cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:38.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8d199700 1 --2- 172.21.15.156:0/2723436376 >> v2:172.21.15.186:3300/0 conn(0x7f0d880cd7c0 0x7f0d880cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8d199700 1 -- 172.21.15.156:0/2723436376 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d880d24f0 con 0x7f0d88161340 2023-11-28T13:10:38.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8d199700 1 -- 172.21.15.156:0/2723436376 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f0d78001670 con 0x7f0d88161340 2023-11-28T13:10:38.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8e19b700 1 --2- 172.21.15.156:0/2723436376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d8815c9b0 0x7f0d880b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:10:38.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8d199700 1 -- 172.21.15.156:0/2723436376 >> v1:172.21.15.156:6789/0 conn(0x7f0d88161340 legacy=0x7f0d880d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:38.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8d199700 1 --2- 172.21.15.156:0/2723436376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d74003a00 0x7f0d74003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8d199700 1 --2- 172.21.15.156:0/2723436376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d8815c9b0 0x7f0d74004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8d199700 1 --2- 172.21.15.156:0/2723436376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d74004d90 0x7f0d74005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8d199700 1 -- 172.21.15.156:0/2723436376 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880b5800 con 0x7f0d74004d90 2023-11-28T13:10:38.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8d199700 1 -- 172.21.15.156:0/2723436376 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880d1950 con 0x7f0d74003a00 2023-11-28T13:10:38.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.080+0000 7f0d8d199700 1 -- 172.21.15.156:0/2723436376 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7400ba10 con 0x7f0d8815c9b0 2023-11-28T13:10:38.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.081+0000 7f0d8e99c700 1 --2- 172.21.15.156:0/2723436376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d74004d90 0x7f0d74005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.081+0000 7f0d8e19b700 1 --2- 172.21.15.156:0/2723436376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d74003a00 0x7f0d74003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.081+0000 7f0d8d99a700 1 --2- 172.21.15.156:0/2723436376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d8815c9b0 0x7f0d74004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.081+0000 7f0d8e99c700 1 -- 172.21.15.156:0/2723436376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d8815c9b0 msgr2=0x7f0d74004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:38.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.081+0000 7f0d8e99c700 1 --2- 172.21.15.156:0/2723436376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d8815c9b0 0x7f0d74004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.081+0000 7f0d8e99c700 1 -- 172.21.15.156:0/2723436376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d74003a00 msgr2=0x7f0d74003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:38.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.081+0000 7f0d8e99c700 1 --2- 172.21.15.156:0/2723436376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d74003a00 0x7f0d74003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.081+0000 7f0d8e99c700 1 -- 172.21.15.156:0/2723436376 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d840094a0 con 0x7f0d74004d90 2023-11-28T13:10:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.081+0000 7f0d8e19b700 1 --2- 172.21.15.156:0/2723436376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d74003a00 0x7f0d74003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.081+0000 7f0d8e99c700 1 --2- 172.21.15.156:0/2723436376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d74004d90 0x7f0d74005200 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009fc0 tx=0x7f0d74005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b347e4979e161b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.081+0000 7f0d8d199700 1 -- 172.21.15.156:0/2723436376 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8000ec30 con 0x7f0d74004d90 2023-11-28T13:10:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.081+0000 7f0d8d199700 1 -- 172.21.15.156:0/2723436376 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8000f040 con 0x7f0d74004d90 2023-11-28T13:10:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.081+0000 7f0d8d199700 1 -- 172.21.15.156:0/2723436376 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80014480 con 0x7f0d74004d90 2023-11-28T13:10:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.082+0000 7f0d903ff700 1 -- 172.21.15.156:0/2723436376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d74004d90 msgr2=0x7f0d74005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.082+0000 7f0d903ff700 1 --2- 172.21.15.156:0/2723436376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d74004d90 0x7f0d74005200 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009fc0 tx=0x7f0d74005880 comp rx=0 tx=0).stop 2023-11-28T13:10:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.082+0000 7f0d8e99c700 1 -- 172.21.15.156:0/2723436376 reap_dead start 2023-11-28T13:10:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.082+0000 7f0d903ff700 1 -- 172.21.15.156:0/2723436376 shutdown_connections 2023-11-28T13:10:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.082+0000 7f0d903ff700 1 -- 172.21.15.156:0/2723436376 >> 172.21.15.156:0/2723436376 conn(0x7f0d88072600 msgr2=0x7f0d880cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.082+0000 7f0d903ff700 1 -- 172.21.15.156:0/2723436376 shutdown_connections 2023-11-28T13:10:38.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.082+0000 7f0d903ff700 1 -- 172.21.15.156:0/2723436376 wait complete. 2023-11-28T13:10:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.084+0000 7f0d903ff700 1 Processor -- start 2023-11-28T13:10:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.084+0000 7f0d903ff700 1 -- start start 2023-11-28T13:10:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.085+0000 7f0d903ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d74004d90 0x7f0d88157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.085+0000 7f0d903ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d8814d7d0 0x7f0d880c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.085+0000 7f0d903ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d881f2a80 0x7f0d8815e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.085+0000 7f0d903ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880cd680 con 0x7f0d74004d90 2023-11-28T13:10:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.085+0000 7f0d903ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880c8fd0 con 0x7f0d881f2a80 2023-11-28T13:10:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.085+0000 7f0d903ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880c88e0 con 0x7f0d8814d7d0 2023-11-28T13:10:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.085+0000 7f0d8e19b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d74004d90 0x7f0d88157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.085+0000 7f0d8e19b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d74004d90 0x7f0d88157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41968/0 (socket says 172.21.15.156:41968) 2023-11-28T13:10:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.085+0000 7f0d8e19b700 1 -- 172.21.15.156:0/1689380617 learned_addr learned my addr 172.21.15.156:0/1689380617 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.085+0000 7f0d8e19b700 1 -- 172.21.15.156:0/1689380617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d8814d7d0 msgr2=0x7f0d880c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.085+0000 7f0d8e19b700 1 --2- 172.21.15.156:0/1689380617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d8814d7d0 0x7f0d880c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.085+0000 7f0d8e19b700 1 -- 172.21.15.156:0/1689380617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d881f2a80 msgr2=0x7f0d8815e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:38.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.085+0000 7f0d8e19b700 1 --2- 172.21.15.156:0/1689380617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d881f2a80 0x7f0d8815e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.085+0000 7f0d8e19b700 1 -- 172.21.15.156:0/1689380617 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d880d1300 con 0x7f0d74004d90 2023-11-28T13:10:38.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.086+0000 7f0d8e19b700 1 --2- 172.21.15.156:0/1689380617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d74004d90 0x7f0d88157780 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0d84010c30 tx=0x7f0d84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:38.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.086+0000 7f0d7effd700 1 -- 172.21.15.156:0/1689380617 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8400e910 con 0x7f0d74004d90 2023-11-28T13:10:38.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.086+0000 7f0d7effd700 1 -- 172.21.15.156:0/1689380617 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8400ea70 con 0x7f0d74004d90 2023-11-28T13:10:38.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.086+0000 7f0d903ff700 1 -- 172.21.15.156:0/1689380617 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d880b5520 con 0x7f0d74004d90 2023-11-28T13:10:38.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.086+0000 7f0d7effd700 1 -- 172.21.15.156:0/1689380617 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8400ede0 con 0x7f0d74004d90 2023-11-28T13:10:38.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.086+0000 7f0d903ff700 1 -- 172.21.15.156:0/1689380617 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d880d15e0 con 0x7f0d74004d90 2023-11-28T13:10:38.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.087+0000 7f0d903ff700 1 -- 172.21.15.156:0/1689380617 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d880ca570 con 0x7f0d74004d90 2023-11-28T13:10:38.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.088+0000 7f0d7effd700 1 -- 172.21.15.156:0/1689380617 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0d8401c030 con 0x7f0d74004d90 2023-11-28T13:10:38.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.089+0000 7f0d7effd700 1 --2- 172.21.15.156:0/1689380617 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0d6c0682f0 0x7f0d6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.089+0000 7f0d7effd700 1 -- 172.21.15.156:0/1689380617 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d84086100 con 0x7f0d74004d90 2023-11-28T13:10:38.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.089+0000 7f0d8d99a700 1 --2- 172.21.15.156:0/1689380617 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0d6c0682f0 0x7f0d6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.090+0000 7f0d8d99a700 1 --2- 172.21.15.156:0/1689380617 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0d6c0682f0 0x7f0d6c06a7a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d78001eb0 tx=0x7f0d78001dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:38.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.093+0000 7f0d7effd700 1 -- 172.21.15.156:0/1689380617 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0d84052870 con 0x7f0d74004d90 2023-11-28T13:10:38.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.283+0000 7f0d903ff700 1 -- 172.21.15.156:0/1689380617 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0d880745c0 con 0x7f0d74004d90 2023-11-28T13:10:38.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.284+0000 7f0d7effd700 1 -- 172.21.15.156:0/1689380617 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v47) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0d84055e90 con 0x7f0d74004d90 2023-11-28T13:10:38.285 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:10:38.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.286+0000 7f0d903ff700 1 -- 172.21.15.156:0/1689380617 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0d6c0682f0 msgr2=0x7f0d6c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:38.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.286+0000 7f0d903ff700 1 --2- 172.21.15.156:0/1689380617 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0d6c0682f0 0x7f0d6c06a7a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d78001eb0 tx=0x7f0d78001dc0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.286+0000 7f0d903ff700 1 -- 172.21.15.156:0/1689380617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d74004d90 msgr2=0x7f0d88157780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:38.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.286+0000 7f0d903ff700 1 --2- 172.21.15.156:0/1689380617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d74004d90 0x7f0d88157780 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0d84010c30 tx=0x7f0d84009e30 comp rx=0 tx=0).stop 2023-11-28T13:10:38.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.286+0000 7f0d903ff700 1 -- 172.21.15.156:0/1689380617 shutdown_connections 2023-11-28T13:10:38.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.286+0000 7f0d903ff700 1 --2- 172.21.15.156:0/1689380617 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0d6c0682f0 0x7f0d6c06a7a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.286+0000 7f0d903ff700 1 --2- 172.21.15.156:0/1689380617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d881f2a80 0x7f0d8815e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.286+0000 7f0d903ff700 1 --2- 172.21.15.156:0/1689380617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d74004d90 0x7f0d88157780 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.286+0000 7f0d903ff700 1 --2- 172.21.15.156:0/1689380617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d8814d7d0 0x7f0d880c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.286+0000 7f0d903ff700 1 -- 172.21.15.156:0/1689380617 >> 172.21.15.156:0/1689380617 conn(0x7f0d88072600 msgr2=0x7f0d880b7380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:38.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.286+0000 7f0d903ff700 1 -- 172.21.15.156:0/1689380617 shutdown_connections 2023-11-28T13:10:38.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.286+0000 7f0d903ff700 1 -- 172.21.15.156:0/1689380617 wait complete. 2023-11-28T13:10:38.307 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:10:38.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.508+0000 7f8eb93ff700 1 Processor -- start 2023-11-28T13:10:38.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.508+0000 7f8eb93ff700 1 -- start start 2023-11-28T13:10:38.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.509+0000 7f8eb93ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8eb415c9b0 0x7f8eb40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.509+0000 7f8eb93ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8eb40cd7c0 0x7f8eb40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.509+0000 7f8eb93ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb40d1950 con 0x7f8eb4161340 2023-11-28T13:10:38.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.509+0000 7f8eb93ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb40b5800 con 0x7f8eb415c9b0 2023-11-28T13:10:38.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.509+0000 7f8eb93ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb40ce140 con 0x7f8eb40cd7c0 2023-11-28T13:10:38.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.509+0000 7f8eb2ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8eb415c9b0 0x7f8eb40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.509+0000 7f8eb27fc700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f8eb4161340 0x7f8eb40d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37840/0 (socket says 172.21.15.156:37840) 2023-11-28T13:10:38.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.509+0000 7f8eb27fc700 1 -- 172.21.15.156:0/2383164935 learned_addr learned my addr 172.21.15.156:0/2383164935 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:38.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.510+0000 7f8eb37fe700 1 --2- 172.21.15.156:0/2383164935 >> v2:172.21.15.186:3300/0 conn(0x7f8eb40cd7c0 0x7f8eb40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.510+0000 7f8eb2ffd700 1 -- 172.21.15.156:0/2383164935 >> v2:172.21.15.186:3300/0 conn(0x7f8eb40cd7c0 msgr2=0x7f8eb40cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:38.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.510+0000 7f8eb2ffd700 1 --2- 172.21.15.156:0/2383164935 >> v2:172.21.15.186:3300/0 conn(0x7f8eb40cd7c0 0x7f8eb40cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.510+0000 7f8eb2ffd700 1 -- 172.21.15.156:0/2383164935 >> v1:172.21.15.156:6789/0 conn(0x7f8eb4161340 legacy=0x7f8eb40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:38.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.510+0000 7f8eb2ffd700 1 -- 172.21.15.156:0/2383164935 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb40d24f0 con 0x7f8eb415c9b0 2023-11-28T13:10:38.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.510+0000 7f8eb2ffd700 1 --2- 172.21.15.156:0/2383164935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8eb415c9b0 0x7f8eb40b4bf0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8eb406c170 tx=0x7f8ea4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5882f75c9a71fca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:38.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.510+0000 7f8eb1ffb700 1 -- 172.21.15.156:0/2383164935 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4011830 con 0x7f8eb415c9b0 2023-11-28T13:10:38.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.510+0000 7f8eb1ffb700 1 -- 172.21.15.156:0/2383164935 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea4011990 con 0x7f8eb415c9b0 2023-11-28T13:10:38.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.510+0000 7f8eb1ffb700 1 -- 172.21.15.156:0/2383164935 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4011d20 con 0x7f8eb415c9b0 2023-11-28T13:10:38.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.511+0000 7f8eb93ff700 1 -- 172.21.15.156:0/2383164935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8eb415c9b0 msgr2=0x7f8eb40b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:38.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.511+0000 7f8eb93ff700 1 --2- 172.21.15.156:0/2383164935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8eb415c9b0 0x7f8eb40b4bf0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8eb406c170 tx=0x7f8ea4009b20 comp rx=0 tx=0).stop 2023-11-28T13:10:38.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.511+0000 7f8eb93ff700 1 -- 172.21.15.156:0/2383164935 shutdown_connections 2023-11-28T13:10:38.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.511+0000 7f8eb93ff700 1 --2- 172.21.15.156:0/2383164935 >> v2:172.21.15.186:3300/0 conn(0x7f8eb40cd7c0 0x7f8eb40cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.511+0000 7f8eb93ff700 1 --2- 172.21.15.156:0/2383164935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8eb415c9b0 0x7f8eb40b4bf0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.511+0000 7f8eb93ff700 1 -- 172.21.15.156:0/2383164935 >> 172.21.15.156:0/2383164935 conn(0x7f8eb4072600 msgr2=0x7f8eb40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:38.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.511+0000 7f8eb93ff700 1 -- 172.21.15.156:0/2383164935 shutdown_connections 2023-11-28T13:10:38.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.511+0000 7f8eb93ff700 1 -- 172.21.15.156:0/2383164935 wait complete. 2023-11-28T13:10:38.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.513+0000 7f8eb93ff700 1 Processor -- start 2023-11-28T13:10:38.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.513+0000 7f8eb93ff700 1 -- start start 2023-11-28T13:10:38.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.514+0000 7f8eb93ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8eb40cd7c0 0x7f8eb40ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.514+0000 7f8eb93ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8eb415c9b0 0x7f8eb4157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.514+0000 7f8eb93ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8eb4161340 0x7f8eb414a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.514+0000 7f8eb93ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb40ca780 con 0x7f8eb415c9b0 2023-11-28T13:10:38.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.514+0000 7f8eb93ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb4156280 con 0x7f8eb4161340 2023-11-28T13:10:38.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.514+0000 7f8eb93ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb4156560 con 0x7f8eb40cd7c0 2023-11-28T13:10:38.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.514+0000 7f8eb27fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8eb415c9b0 0x7f8eb4157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.514+0000 7f8eb27fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8eb415c9b0 0x7f8eb4157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41984/0 (socket says 172.21.15.156:41984) 2023-11-28T13:10:38.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.514+0000 7f8eb27fc700 1 -- 172.21.15.156:0/4291804480 learned_addr learned my addr 172.21.15.156:0/4291804480 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:38.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.514+0000 7f8eb2ffd700 1 --2- 172.21.15.156:0/4291804480 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8eb40cd7c0 0x7f8eb40ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.514+0000 7f8eb37fe700 1 --2- 172.21.15.156:0/4291804480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8eb4161340 0x7f8eb414a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.515+0000 7f8eb27fc700 1 -- 172.21.15.156:0/4291804480 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8eb40cd7c0 msgr2=0x7f8eb40ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:38.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.515+0000 7f8eb27fc700 1 --2- 172.21.15.156:0/4291804480 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8eb40cd7c0 0x7f8eb40ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.515+0000 7f8eb27fc700 1 -- 172.21.15.156:0/4291804480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8eb4161340 msgr2=0x7f8eb414a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:38.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.515+0000 7f8eb27fc700 1 --2- 172.21.15.156:0/4291804480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8eb4161340 0x7f8eb414a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.515+0000 7f8eb27fc700 1 -- 172.21.15.156:0/4291804480 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea40094a0 con 0x7f8eb415c9b0 2023-11-28T13:10:38.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.515+0000 7f8eb27fc700 1 --2- 172.21.15.156:0/4291804480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8eb415c9b0 0x7f8eb4157600 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c480 tx=0x7f8ea800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:38.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.515+0000 7f8e9b7fe700 1 -- 172.21.15.156:0/4291804480 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8005d70 con 0x7f8eb415c9b0 2023-11-28T13:10:38.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.515+0000 7f8e9b7fe700 1 -- 172.21.15.156:0/4291804480 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea80070d0 con 0x7f8eb415c9b0 2023-11-28T13:10:38.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.515+0000 7f8eb93ff700 1 -- 172.21.15.156:0/4291804480 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb40d1300 con 0x7f8eb415c9b0 2023-11-28T13:10:38.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.515+0000 7f8e9b7fe700 1 -- 172.21.15.156:0/4291804480 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8007440 con 0x7f8eb415c9b0 2023-11-28T13:10:38.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.515+0000 7f8eb93ff700 1 -- 172.21.15.156:0/4291804480 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb40b5520 con 0x7f8eb415c9b0 2023-11-28T13:10:38.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.516+0000 7f8eb93ff700 1 -- 172.21.15.156:0/4291804480 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb40d15e0 con 0x7f8eb415c9b0 2023-11-28T13:10:38.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.517+0000 7f8e9b7fe700 1 -- 172.21.15.156:0/4291804480 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8ea8019030 con 0x7f8eb415c9b0 2023-11-28T13:10:38.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.518+0000 7f8e9b7fe700 1 --2- 172.21.15.156:0/4291804480 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8ea00682f0 0x7f8ea006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.518+0000 7f8e9b7fe700 1 -- 172.21.15.156:0/4291804480 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8ea8084ed0 con 0x7f8eb415c9b0 2023-11-28T13:10:38.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.518+0000 7f8eb2ffd700 1 --2- 172.21.15.156:0/4291804480 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8ea00682f0 0x7f8ea006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.519+0000 7f8eb2ffd700 1 --2- 172.21.15.156:0/4291804480 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8ea00682f0 0x7f8ea006a7a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8ea40090b0 tx=0x7f8ea401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:38.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.522+0000 7f8e9b7fe700 1 -- 172.21.15.156:0/4291804480 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8ea804b5e0 con 0x7f8eb415c9b0 2023-11-28T13:10:38.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.712+0000 7f8eb93ff700 1 -- 172.21.15.156:0/4291804480 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8eb4159a70 con 0x7f8eb415c9b0 2023-11-28T13:10:38.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.712+0000 7f8e9b7fe700 1 -- 172.21.15.156:0/4291804480 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ea80170a0 con 0x7f8eb415c9b0 2023-11-28T13:10:38.776 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:10:38.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.715+0000 7f8eb93ff700 1 -- 172.21.15.156:0/4291804480 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8ea00682f0 msgr2=0x7f8ea006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.715+0000 7f8eb93ff700 1 --2- 172.21.15.156:0/4291804480 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8ea00682f0 0x7f8ea006a7a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8ea40090b0 tx=0x7f8ea401c040 comp rx=0 tx=0).stop 2023-11-28T13:10:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.715+0000 7f8eb93ff700 1 -- 172.21.15.156:0/4291804480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8eb415c9b0 msgr2=0x7f8eb4157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.715+0000 7f8eb93ff700 1 --2- 172.21.15.156:0/4291804480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8eb415c9b0 0x7f8eb4157600 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c480 tx=0x7f8ea800c790 comp rx=0 tx=0).stop 2023-11-28T13:10:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.715+0000 7f8eb93ff700 1 -- 172.21.15.156:0/4291804480 shutdown_connections 2023-11-28T13:10:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.715+0000 7f8eb93ff700 1 --2- 172.21.15.156:0/4291804480 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8ea00682f0 0x7f8ea006a7a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.715+0000 7f8eb93ff700 1 --2- 172.21.15.156:0/4291804480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8eb4161340 0x7f8eb414a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.715+0000 7f8eb93ff700 1 --2- 172.21.15.156:0/4291804480 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8eb40cd7c0 0x7f8eb40ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.715+0000 7f8eb93ff700 1 --2- 172.21.15.156:0/4291804480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8eb415c9b0 0x7f8eb4157600 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.715+0000 7f8eb93ff700 1 -- 172.21.15.156:0/4291804480 >> 172.21.15.156:0/4291804480 conn(0x7f8eb4072600 msgr2=0x7f8eb4159de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.715+0000 7f8eb93ff700 1 -- 172.21.15.156:0/4291804480 shutdown_connections 2023-11-28T13:10:38.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.715+0000 7f8eb93ff700 1 -- 172.21.15.156:0/4291804480 wait complete. 2023-11-28T13:10:38.779 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:10:38.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.977+0000 7ff17ebff700 1 Processor -- start 2023-11-28T13:10:38.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.977+0000 7ff17ebff700 1 -- start start 2023-11-28T13:10:38.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.978+0000 7ff17ebff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff178161340 0x7ff1780d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.978+0000 7ff17ebff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1780d2320 0x7ff1780cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.979+0000 7ff17ebff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1780d1950 con 0x7ff17815c9b0 2023-11-28T13:10:38.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.979+0000 7ff17ebff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1780b5800 con 0x7ff178161340 2023-11-28T13:10:38.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.979+0000 7ff17ebff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1780b5300 con 0x7ff1780d2320 2023-11-28T13:10:38.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.979+0000 7ff177fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff178161340 0x7ff1780d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.979+0000 7ff17d19c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1780d2320 0x7ff1780cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.979+0000 7ff17c99b700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7ff17815c9b0 0x7ff1780b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:36866/0 (socket says 172.21.15.156:36866) 2023-11-28T13:10:38.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.979+0000 7ff17c99b700 1 -- 172.21.15.156:0/528759220 learned_addr learned my addr 172.21.15.156:0/528759220 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:38.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.979+0000 7ff1777fe700 1 -- 172.21.15.156:0/528759220 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341594149 0 0) 0x7ff1780d1950 con 0x7ff17815c9b0 2023-11-28T13:10:38.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.979+0000 7ff17d19c700 1 -- 172.21.15.156:0/528759220 >> v2:172.21.15.186:3300/0 conn(0x7ff178161340 msgr2=0x7ff1780d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:38.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.979+0000 7ff17d19c700 1 --2- 172.21.15.156:0/528759220 >> v2:172.21.15.186:3300/0 conn(0x7ff178161340 0x7ff1780d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.979+0000 7ff17d19c700 1 -- 172.21.15.156:0/528759220 >> v1:172.21.15.186:6789/0 conn(0x7ff17815c9b0 legacy=0x7ff1780b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:38.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.979+0000 7ff17d19c700 1 -- 172.21.15.156:0/528759220 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1780b5520 con 0x7ff1780d2320 2023-11-28T13:10:38.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.979+0000 7ff177fff700 1 --2- 172.21.15.156:0/528759220 >> v2:172.21.15.186:3300/0 conn(0x7ff178161340 0x7ff1780d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:10:38.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.980+0000 7ff17d19c700 1 --2- 172.21.15.156:0/528759220 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1780d2320 0x7ff1780cd7c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff170009d90 tx=0x7ff17000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2904cc09cfcc849e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:38.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.980+0000 7ff1777fe700 1 -- 172.21.15.156:0/528759220 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17000e070 con 0x7ff1780d2320 2023-11-28T13:10:38.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.980+0000 7ff1777fe700 1 -- 172.21.15.156:0/528759220 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17000bb20 con 0x7ff1780d2320 2023-11-28T13:10:38.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.980+0000 7ff1777fe700 1 -- 172.21.15.156:0/528759220 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170012430 con 0x7ff1780d2320 2023-11-28T13:10:38.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.981+0000 7ff17ebff700 1 -- 172.21.15.156:0/528759220 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1780d2320 msgr2=0x7ff1780cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:38.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.981+0000 7ff17ebff700 1 --2- 172.21.15.156:0/528759220 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1780d2320 0x7ff1780cd7c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff170009d90 tx=0x7ff17000b330 comp rx=0 tx=0).stop 2023-11-28T13:10:38.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.981+0000 7ff17ebff700 1 -- 172.21.15.156:0/528759220 shutdown_connections 2023-11-28T13:10:38.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.981+0000 7ff17ebff700 1 --2- 172.21.15.156:0/528759220 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1780d2320 0x7ff1780cd7c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.981+0000 7ff17ebff700 1 --2- 172.21.15.156:0/528759220 >> v2:172.21.15.186:3300/0 conn(0x7ff178161340 0x7ff1780d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.981+0000 7ff17ebff700 1 -- 172.21.15.156:0/528759220 >> 172.21.15.156:0/528759220 conn(0x7ff178072600 msgr2=0x7ff1780cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:38.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.981+0000 7ff17ebff700 1 -- 172.21.15.156:0/528759220 shutdown_connections 2023-11-28T13:10:38.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.981+0000 7ff17ebff700 1 -- 172.21.15.156:0/528759220 wait complete. 2023-11-28T13:10:38.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.983+0000 7ff17ebff700 1 Processor -- start 2023-11-28T13:10:38.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.983+0000 7ff17ebff700 1 -- start start 2023-11-28T13:10:38.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.983+0000 7ff17ebff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1780d2320 0x7ff178150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.984+0000 7ff17ebff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff17815c9b0 0x7ff1780c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.984+0000 7ff17ebff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff178161340 0x7ff17815e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.984+0000 7ff17ebff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1781f3b90 con 0x7ff17815c9b0 2023-11-28T13:10:38.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.984+0000 7ff17ebff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1781f2f10 con 0x7ff1780d2320 2023-11-28T13:10:38.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.984+0000 7ff17ebff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1781f31f0 con 0x7ff178161340 2023-11-28T13:10:38.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.984+0000 7ff177fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff17815c9b0 0x7ff1780c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.984+0000 7ff177fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff17815c9b0 0x7ff1780c60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41986/0 (socket says 172.21.15.156:41986) 2023-11-28T13:10:38.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.984+0000 7ff177fff700 1 -- 172.21.15.156:0/506035700 learned_addr learned my addr 172.21.15.156:0/506035700 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:38.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.984+0000 7ff17d19c700 1 --2- 172.21.15.156:0/506035700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff178161340 0x7ff17815e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.984+0000 7ff17c99b700 1 --2- 172.21.15.156:0/506035700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1780d2320 0x7ff178150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.984+0000 7ff177fff700 1 -- 172.21.15.156:0/506035700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff178161340 msgr2=0x7ff17815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:38.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.984+0000 7ff177fff700 1 --2- 172.21.15.156:0/506035700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff178161340 0x7ff17815e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.984+0000 7ff177fff700 1 -- 172.21.15.156:0/506035700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1780d2320 msgr2=0x7ff178150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:38.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.984+0000 7ff177fff700 1 --2- 172.21.15.156:0/506035700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1780d2320 0x7ff178150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:38.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.984+0000 7ff177fff700 1 -- 172.21.15.156:0/506035700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1640094a0 con 0x7ff17815c9b0 2023-11-28T13:10:38.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.985+0000 7ff177fff700 1 --2- 172.21.15.156:0/506035700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff17815c9b0 0x7ff1780c60f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff16400ebe0 tx=0x7ff16400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:38.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.985+0000 7ff1757fa700 1 -- 172.21.15.156:0/506035700 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16400c850 con 0x7ff17815c9b0 2023-11-28T13:10:38.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.985+0000 7ff1757fa700 1 -- 172.21.15.156:0/506035700 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16400c9b0 con 0x7ff17815c9b0 2023-11-28T13:10:38.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.985+0000 7ff1757fa700 1 -- 172.21.15.156:0/506035700 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16400cd40 con 0x7ff17815c9b0 2023-11-28T13:10:38.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.985+0000 7ff17ebff700 1 -- 172.21.15.156:0/506035700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff170009a70 con 0x7ff17815c9b0 2023-11-28T13:10:38.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.985+0000 7ff17ebff700 1 -- 172.21.15.156:0/506035700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1780d15e0 con 0x7ff17815c9b0 2023-11-28T13:10:38.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.986+0000 7ff17ebff700 1 -- 172.21.15.156:0/506035700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1780d1300 con 0x7ff17815c9b0 2023-11-28T13:10:38.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.987+0000 7ff1757fa700 1 -- 172.21.15.156:0/506035700 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff164019030 con 0x7ff17815c9b0 2023-11-28T13:10:38.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.988+0000 7ff1757fa700 1 --2- 172.21.15.156:0/506035700 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff15c068320 0x7ff15c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:38.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.988+0000 7ff1757fa700 1 -- 172.21.15.156:0/506035700 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff164085ce0 con 0x7ff17815c9b0 2023-11-28T13:10:38.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.988+0000 7ff17c99b700 1 --2- 172.21.15.156:0/506035700 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff15c068320 0x7ff15c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:38.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.989+0000 7ff17c99b700 1 --2- 172.21.15.156:0/506035700 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff15c068320 0x7ff15c06a7d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff16c000f60 tx=0x7ff16c007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:38.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:38.992+0000 7ff1757fa700 1 -- 172.21.15.156:0/506035700 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff164052450 con 0x7ff17815c9b0 2023-11-28T13:10:39.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.182+0000 7ff17ebff700 1 -- 172.21.15.156:0/506035700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff17815cd80 con 0x7ff17815c9b0 2023-11-28T13:10:39.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.183+0000 7ff1757fa700 1 -- 172.21.15.156:0/506035700 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff16400cea0 con 0x7ff17815c9b0 2023-11-28T13:10:39.185 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T13:10:39.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.185+0000 7ff17ebff700 1 -- 172.21.15.156:0/506035700 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff15c068320 msgr2=0x7ff15c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:39.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.185+0000 7ff17ebff700 1 --2- 172.21.15.156:0/506035700 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff15c068320 0x7ff15c06a7d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff16c000f60 tx=0x7ff16c007e70 comp rx=0 tx=0).stop 2023-11-28T13:10:39.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.185+0000 7ff17ebff700 1 -- 172.21.15.156:0/506035700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff17815c9b0 msgr2=0x7ff1780c60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:39.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.185+0000 7ff17ebff700 1 --2- 172.21.15.156:0/506035700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff17815c9b0 0x7ff1780c60f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff16400ebe0 tx=0x7ff16400eef0 comp rx=0 tx=0).stop 2023-11-28T13:10:39.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.185+0000 7ff17ebff700 1 -- 172.21.15.156:0/506035700 shutdown_connections 2023-11-28T13:10:39.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.185+0000 7ff17ebff700 1 --2- 172.21.15.156:0/506035700 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff15c068320 0x7ff15c06a7d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:39.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.185+0000 7ff17ebff700 1 --2- 172.21.15.156:0/506035700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff178161340 0x7ff17815e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:39.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.185+0000 7ff17ebff700 1 --2- 172.21.15.156:0/506035700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1780d2320 0x7ff178150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:39.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.185+0000 7ff17ebff700 1 --2- 172.21.15.156:0/506035700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff17815c9b0 0x7ff1780c60f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:39.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.185+0000 7ff17ebff700 1 -- 172.21.15.156:0/506035700 >> 172.21.15.156:0/506035700 conn(0x7ff178072600 msgr2=0x7ff17815f990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:39.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.185+0000 7ff17ebff700 1 -- 172.21.15.156:0/506035700 shutdown_connections 2023-11-28T13:10:39.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.185+0000 7ff17ebff700 1 -- 172.21.15.156:0/506035700 wait complete. 2023-11-28T13:10:39.205 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:10:39.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.404+0000 7f04d89ff700 1 Processor -- start 2023-11-28T13:10:39.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.404+0000 7f04d89ff700 1 -- start start 2023-11-28T13:10:39.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.406+0000 7f04d89ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f04d415c9b0 0x7f04d40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:39.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.406+0000 7f04d89ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d40d1950 con 0x7f04d40cd7c0 2023-11-28T13:10:39.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.406+0000 7f04d89ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d40d24f0 con 0x7f04d4161340 2023-11-28T13:10:39.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.406+0000 7f04d89ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d40b5800 con 0x7f04d415c9b0 2023-11-28T13:10:39.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.406+0000 7f04d259c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f04d415c9b0 0x7f04d40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:39.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.406+0000 7f04d2d9d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f04d40cd7c0 0x7f04d40b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37852/0 (socket says 172.21.15.156:37852) 2023-11-28T13:10:39.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.406+0000 7f04d2d9d700 1 -- 172.21.15.156:0/738692113 learned_addr learned my addr 172.21.15.156:0/738692113 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:39.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.406+0000 7f04d259c700 1 -- 172.21.15.156:0/738692113 >> v1:172.21.15.186:6789/0 conn(0x7f04d4161340 legacy=0x7f04d40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:39.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.406+0000 7f04d259c700 1 -- 172.21.15.156:0/738692113 >> v1:172.21.15.156:6789/0 conn(0x7f04d40cd7c0 legacy=0x7f04d40b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:39.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.406+0000 7f04d259c700 1 -- 172.21.15.156:0/738692113 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d40b5520 con 0x7f04d415c9b0 2023-11-28T13:10:39.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.406+0000 7f04d1d9b700 1 -- 172.21.15.156:0/738692113 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419209084 0 0) 0x7f04d40d1950 con 0x7f04d40cd7c0 2023-11-28T13:10:39.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.407+0000 7f04d259c700 1 --2- 172.21.15.156:0/738692113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04d415c9b0 0x7f04d40b4bf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009840 tx=0x7f04c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef17482d86aded09 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:39.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.407+0000 7f04d1d9b700 1 -- 172.21.15.156:0/738692113 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c4004230 con 0x7f04d415c9b0 2023-11-28T13:10:39.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.407+0000 7f04d1d9b700 1 -- 172.21.15.156:0/738692113 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c4004390 con 0x7f04d415c9b0 2023-11-28T13:10:39.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.407+0000 7f04d1d9b700 1 -- 172.21.15.156:0/738692113 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c40036a0 con 0x7f04d415c9b0 2023-11-28T13:10:39.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.407+0000 7f04d89ff700 1 -- 172.21.15.156:0/738692113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04d415c9b0 msgr2=0x7f04d40b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:39.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.407+0000 7f04d89ff700 1 --2- 172.21.15.156:0/738692113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04d415c9b0 0x7f04d40b4bf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009840 tx=0x7f04c4009b50 comp rx=0 tx=0).stop 2023-11-28T13:10:39.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.407+0000 7f04d89ff700 1 -- 172.21.15.156:0/738692113 shutdown_connections 2023-11-28T13:10:39.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.407+0000 7f04d89ff700 1 --2- 172.21.15.156:0/738692113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04d415c9b0 0x7f04d40b4bf0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:39.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.407+0000 7f04d89ff700 1 -- 172.21.15.156:0/738692113 >> 172.21.15.156:0/738692113 conn(0x7f04d4072600 msgr2=0x7f04d40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:39.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.408+0000 7f04d89ff700 1 -- 172.21.15.156:0/738692113 shutdown_connections 2023-11-28T13:10:39.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.408+0000 7f04d89ff700 1 -- 172.21.15.156:0/738692113 wait complete. 2023-11-28T13:10:39.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.410+0000 7f04d89ff700 1 Processor -- start 2023-11-28T13:10:39.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.410+0000 7f04d89ff700 1 -- start start 2023-11-28T13:10:39.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.411+0000 7f04d89ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04d40cd7c0 0x7f04d4156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:39.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.411+0000 7f04d89ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04d415c9b0 0x7f04d4156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:39.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.411+0000 7f04d89ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04d4161340 0x7f04d415ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:39.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.411+0000 7f04d89ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d40c9ff0 con 0x7f04d4161340 2023-11-28T13:10:39.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.411+0000 7f04d89ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d40c7080 con 0x7f04d415c9b0 2023-11-28T13:10:39.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.411+0000 7f04d89ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d40c7360 con 0x7f04d40cd7c0 2023-11-28T13:10:39.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.412+0000 7f04d2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04d4161340 0x7f04d415ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:39.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.412+0000 7f04d2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04d4161340 0x7f04d415ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42004/0 (socket says 172.21.15.156:42004) 2023-11-28T13:10:39.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.412+0000 7f04d2d9d700 1 -- 172.21.15.156:0/3821401909 learned_addr learned my addr 172.21.15.156:0/3821401909 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:39.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.412+0000 7f04cbfff700 1 --2- 172.21.15.156:0/3821401909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04d415c9b0 0x7f04d4156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:39.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.412+0000 7f04d259c700 1 --2- 172.21.15.156:0/3821401909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04d40cd7c0 0x7f04d4156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:39.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.412+0000 7f04d2d9d700 1 -- 172.21.15.156:0/3821401909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04d40cd7c0 msgr2=0x7f04d4156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:39.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.412+0000 7f04d2d9d700 1 --2- 172.21.15.156:0/3821401909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04d40cd7c0 0x7f04d4156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:39.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.412+0000 7f04d2d9d700 1 -- 172.21.15.156:0/3821401909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04d415c9b0 msgr2=0x7f04d4156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:39.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.412+0000 7f04d2d9d700 1 --2- 172.21.15.156:0/3821401909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04d415c9b0 0x7f04d4156640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:39.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.412+0000 7f04d2d9d700 1 -- 172.21.15.156:0/3821401909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c40094a0 con 0x7f04d4161340 2023-11-28T13:10:39.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.412+0000 7f04d2d9d700 1 --2- 172.21.15.156:0/3821401909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04d4161340 0x7f04d415ddd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00cb60 tx=0x7f04cc00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:39.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.413+0000 7f04caffd700 1 -- 172.21.15.156:0/3821401909 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc00ab50 con 0x7f04d4161340 2023-11-28T13:10:39.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.413+0000 7f04caffd700 1 -- 172.21.15.156:0/3821401909 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04cc00acb0 con 0x7f04d4161340 2023-11-28T13:10:39.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.413+0000 7f04d89ff700 1 -- 172.21.15.156:0/3821401909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d40d15e0 con 0x7f04d4161340 2023-11-28T13:10:39.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.413+0000 7f04caffd700 1 -- 172.21.15.156:0/3821401909 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc007310 con 0x7f04d4161340 2023-11-28T13:10:39.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.413+0000 7f04d89ff700 1 -- 172.21.15.156:0/3821401909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d40d1300 con 0x7f04d4161340 2023-11-28T13:10:39.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.414+0000 7f04d89ff700 1 -- 172.21.15.156:0/3821401909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d40c9e60 con 0x7f04d4161340 2023-11-28T13:10:39.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.415+0000 7f04caffd700 1 -- 172.21.15.156:0/3821401909 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f04cc010070 con 0x7f04d4161340 2023-11-28T13:10:39.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.416+0000 7f04caffd700 1 --2- 172.21.15.156:0/3821401909 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f04b40682f0 0x7f04b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:39.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.416+0000 7f04caffd700 1 -- 172.21.15.156:0/3821401909 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f04cc0857d0 con 0x7f04d4161340 2023-11-28T13:10:39.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.416+0000 7f04d259c700 1 --2- 172.21.15.156:0/3821401909 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f04b40682f0 0x7f04b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:39.418 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.417+0000 7f04d259c700 1 --2- 172.21.15.156:0/3821401909 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f04b40682f0 0x7f04b406a7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f04c40090b0 tx=0x7f04c4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:39.421 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.419+0000 7f04caffd700 1 -- 172.21.15.156:0/3821401909 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f04cc051f40 con 0x7f04d4161340 2023-11-28T13:10:39.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.613+0000 7f04d89ff700 1 -- 172.21.15.156:0/3821401909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f04d40745c0 con 0x7f04d4161340 2023-11-28T13:10:39.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.614+0000 7f04caffd700 1 -- 172.21.15.156:0/3821401909 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f04cc0180a0 con 0x7f04d4161340 2023-11-28T13:10:39.615 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T13:10:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.616+0000 7f04d89ff700 1 -- 172.21.15.156:0/3821401909 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f04b40682f0 msgr2=0x7f04b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.616+0000 7f04d89ff700 1 --2- 172.21.15.156:0/3821401909 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f04b40682f0 0x7f04b406a7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f04c40090b0 tx=0x7f04c4008030 comp rx=0 tx=0).stop 2023-11-28T13:10:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.616+0000 7f04d89ff700 1 -- 172.21.15.156:0/3821401909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04d4161340 msgr2=0x7f04d415ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.616+0000 7f04d89ff700 1 --2- 172.21.15.156:0/3821401909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04d4161340 0x7f04d415ddd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00cb60 tx=0x7f04cc00cf20 comp rx=0 tx=0).stop 2023-11-28T13:10:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.616+0000 7f04d89ff700 1 -- 172.21.15.156:0/3821401909 shutdown_connections 2023-11-28T13:10:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.616+0000 7f04d89ff700 1 --2- 172.21.15.156:0/3821401909 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f04b40682f0 0x7f04b406a7a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.616+0000 7f04d89ff700 1 --2- 172.21.15.156:0/3821401909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04d4161340 0x7f04d415ddd0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.616+0000 7f04d89ff700 1 --2- 172.21.15.156:0/3821401909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04d40cd7c0 0x7f04d4156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:39.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.616+0000 7f04d89ff700 1 --2- 172.21.15.156:0/3821401909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04d415c9b0 0x7f04d4156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:39.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.616+0000 7f04d89ff700 1 -- 172.21.15.156:0/3821401909 >> 172.21.15.156:0/3821401909 conn(0x7f04d4072600 msgr2=0x7f04d40b8550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:39.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.616+0000 7f04d89ff700 1 -- 172.21.15.156:0/3821401909 shutdown_connections 2023-11-28T13:10:39.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.616+0000 7f04d89ff700 1 -- 172.21.15.156:0/3821401909 wait complete. 2023-11-28T13:10:39.636 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:10:39.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.835+0000 7f3f9efff700 1 Processor -- start 2023-11-28T13:10:39.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.835+0000 7f3f9efff700 1 -- start start 2023-11-28T13:10:39.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.836+0000 7f3f9efff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f98161340 0x7f3f980d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:39.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.836+0000 7f3f9efff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3f980d2320 0x7f3f980cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:39.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.836+0000 7f3f9efff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f980d1950 con 0x7f3f9815c9b0 2023-11-28T13:10:39.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.836+0000 7f3f9efff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f980b5800 con 0x7f3f980d2320 2023-11-28T13:10:39.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.836+0000 7f3f9efff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f980b5300 con 0x7f3f98161340 2023-11-28T13:10:39.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.837+0000 7f3f9cd9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f3f9815c9b0 0x7f3f980b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37868/0 (socket says 172.21.15.156:37868) 2023-11-28T13:10:39.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.837+0000 7f3f9cd9b700 1 -- 172.21.15.156:0/3741523963 learned_addr learned my addr 172.21.15.156:0/3741523963 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:39.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.837+0000 7f3f97fff700 1 --2- 172.21.15.156:0/3741523963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f98161340 0x7f3f980d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:39.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.837+0000 7f3f9d59c700 1 --2- 172.21.15.156:0/3741523963 >> v2:172.21.15.186:3300/0 conn(0x7f3f980d2320 0x7f3f980cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:39.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.837+0000 7f3f977fe700 1 -- 172.21.15.156:0/3741523963 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977953938 0 0) 0x7f3f980d1950 con 0x7f3f9815c9b0 2023-11-28T13:10:39.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.837+0000 7f3f977fe700 1 -- 172.21.15.156:0/3741523963 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f880035b0 con 0x7f3f9815c9b0 2023-11-28T13:10:39.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.837+0000 7f3f97fff700 1 -- 172.21.15.156:0/3741523963 >> v2:172.21.15.186:3300/0 conn(0x7f3f980d2320 msgr2=0x7f3f980cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:39.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.837+0000 7f3f97fff700 1 --2- 172.21.15.156:0/3741523963 >> v2:172.21.15.186:3300/0 conn(0x7f3f980d2320 0x7f3f980cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:39.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.837+0000 7f3f97fff700 1 -- 172.21.15.156:0/3741523963 >> v1:172.21.15.156:6789/0 conn(0x7f3f9815c9b0 legacy=0x7f3f980b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:39.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.837+0000 7f3f97fff700 1 -- 172.21.15.156:0/3741523963 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f980b5520 con 0x7f3f98161340 2023-11-28T13:10:39.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.838+0000 7f3f97fff700 1 --2- 172.21.15.156:0/3741523963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f98161340 0x7f3f980d1de0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009840 tx=0x7f3f84009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1443e73d9af5a6b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:39.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.838+0000 7f3f977fe700 1 -- 172.21.15.156:0/3741523963 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f84011830 con 0x7f3f98161340 2023-11-28T13:10:39.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.838+0000 7f3f977fe700 1 -- 172.21.15.156:0/3741523963 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f84011990 con 0x7f3f98161340 2023-11-28T13:10:39.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.838+0000 7f3f977fe700 1 -- 172.21.15.156:0/3741523963 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f84011d00 con 0x7f3f98161340 2023-11-28T13:10:39.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.839+0000 7f3f9efff700 1 -- 172.21.15.156:0/3741523963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f98161340 msgr2=0x7f3f980d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:39.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.839+0000 7f3f9efff700 1 --2- 172.21.15.156:0/3741523963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f98161340 0x7f3f980d1de0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009840 tx=0x7f3f84009b50 comp rx=0 tx=0).stop 2023-11-28T13:10:39.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.839+0000 7f3f9efff700 1 -- 172.21.15.156:0/3741523963 shutdown_connections 2023-11-28T13:10:39.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.839+0000 7f3f9efff700 1 --2- 172.21.15.156:0/3741523963 >> v2:172.21.15.186:3300/0 conn(0x7f3f980d2320 0x7f3f980cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:39.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.839+0000 7f3f9efff700 1 --2- 172.21.15.156:0/3741523963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f98161340 0x7f3f980d1de0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:39.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.839+0000 7f3f9efff700 1 -- 172.21.15.156:0/3741523963 >> 172.21.15.156:0/3741523963 conn(0x7f3f98072600 msgr2=0x7f3f980cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:39.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.839+0000 7f3f9efff700 1 -- 172.21.15.156:0/3741523963 shutdown_connections 2023-11-28T13:10:39.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.839+0000 7f3f9efff700 1 -- 172.21.15.156:0/3741523963 wait complete. 2023-11-28T13:10:39.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.840+0000 7f3f9efff700 1 Processor -- start 2023-11-28T13:10:39.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.841+0000 7f3f9efff700 1 -- start start 2023-11-28T13:10:39.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.841+0000 7f3f9efff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f980d2320 0x7f3f98156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:39.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.841+0000 7f3f9efff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f9815c9b0 0x7f3f981f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:39.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.841+0000 7f3f9efff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f98161340 0x7f3f980cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:39.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.841+0000 7f3f9efff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f980cacf0 con 0x7f3f98161340 2023-11-28T13:10:39.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.841+0000 7f3f9efff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f980c9290 con 0x7f3f980d2320 2023-11-28T13:10:39.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.841+0000 7f3f9efff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f980caf90 con 0x7f3f9815c9b0 2023-11-28T13:10:39.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.842+0000 7f3f9d59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f98161340 0x7f3f980cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:39.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.842+0000 7f3f9d59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f98161340 0x7f3f980cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42016/0 (socket says 172.21.15.156:42016) 2023-11-28T13:10:39.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.842+0000 7f3f9d59c700 1 -- 172.21.15.156:0/4197612093 learned_addr learned my addr 172.21.15.156:0/4197612093 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:39.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.842+0000 7f3f9cd9b700 1 --2- 172.21.15.156:0/4197612093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f980d2320 0x7f3f98156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:39.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.842+0000 7f3f97fff700 1 --2- 172.21.15.156:0/4197612093 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f9815c9b0 0x7f3f981f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:39.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.842+0000 7f3f9d59c700 1 -- 172.21.15.156:0/4197612093 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f9815c9b0 msgr2=0x7f3f981f29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:39.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.842+0000 7f3f9d59c700 1 --2- 172.21.15.156:0/4197612093 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f9815c9b0 0x7f3f981f29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:39.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.842+0000 7f3f9d59c700 1 -- 172.21.15.156:0/4197612093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f980d2320 msgr2=0x7f3f98156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:39.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.842+0000 7f3f9d59c700 1 --2- 172.21.15.156:0/4197612093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f980d2320 0x7f3f98156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:39.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.842+0000 7f3f9d59c700 1 -- 172.21.15.156:0/4197612093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f840094a0 con 0x7f3f98161340 2023-11-28T13:10:39.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.842+0000 7f3f97fff700 1 --2- 172.21.15.156:0/4197612093 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f9815c9b0 0x7f3f981f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:39.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.842+0000 7f3f9cd9b700 1 --2- 172.21.15.156:0/4197612093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f980d2320 0x7f3f98156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:39.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.842+0000 7f3f9d59c700 1 --2- 172.21.15.156:0/4197612093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f98161340 0x7f3f980cd9a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000efc0 tx=0x7f3f9000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:39.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.843+0000 7f3f957fa700 1 -- 172.21.15.156:0/4197612093 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90009ba0 con 0x7f3f98161340 2023-11-28T13:10:39.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.843+0000 7f3f957fa700 1 -- 172.21.15.156:0/4197612093 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9000f040 con 0x7f3f98161340 2023-11-28T13:10:39.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.843+0000 7f3f957fa700 1 -- 172.21.15.156:0/4197612093 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90014480 con 0x7f3f98161340 2023-11-28T13:10:39.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.843+0000 7f3f9efff700 1 -- 172.21.15.156:0/4197612093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f980d15e0 con 0x7f3f98161340 2023-11-28T13:10:39.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.843+0000 7f3f9efff700 1 -- 172.21.15.156:0/4197612093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f980d1300 con 0x7f3f98161340 2023-11-28T13:10:39.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.844+0000 7f3f9efff700 1 -- 172.21.15.156:0/4197612093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f9815cd80 con 0x7f3f98161340 2023-11-28T13:10:39.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.845+0000 7f3f957fa700 1 -- 172.21.15.156:0/4197612093 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3f900145e0 con 0x7f3f98161340 2023-11-28T13:10:39.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.846+0000 7f3f957fa700 1 --2- 172.21.15.156:0/4197612093 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3f7c0682f0 0x7f3f7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:39.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.846+0000 7f3f957fa700 1 -- 172.21.15.156:0/4197612093 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f90086810 con 0x7f3f98161340 2023-11-28T13:10:39.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.846+0000 7f3f9cd9b700 1 --2- 172.21.15.156:0/4197612093 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3f7c0682f0 0x7f3f7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:39.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.847+0000 7f3f9cd9b700 1 --2- 172.21.15.156:0/4197612093 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3f7c0682f0 0x7f3f7c06a7a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c005910 tx=0x7f3f8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:39.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:39.850+0000 7f3f957fa700 1 -- 172.21.15.156:0/4197612093 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3f90052ef0 con 0x7f3f98161340 2023-11-28T13:10:40.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.085+0000 7f3f9efff700 1 -- 172.21.15.156:0/4197612093 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f980cb4b0 con 0x7f3f98161340 2023-11-28T13:10:40.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.085+0000 7f3f957fa700 1 -- 172.21.15.156:0/4197612093 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f3f980cb4b0 con 0x7f3f98161340 2023-11-28T13:10:40.087 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:40.087 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3828141874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3828141874},{"type":"v1","addr":"172.21.15.156:6835","nonce":3828141874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1012489762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1012489762},{"type":"v1","addr":"172.21.15.186:6837","nonce":1012489762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2894670074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2894670074},{"type":"v1","addr":"172.21.15.186:6835","nonce":2894670074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:09:52.245032+0000","modified":"2023-11-28T13:09:55.251254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5481":{"gid":5481,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6834/1930782798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1930782798},{"type":"v1","addr":"172.21.15.156:6834","nonce":1930782798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:10:40.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.087+0000 7f3f9efff700 1 -- 172.21.15.156:0/4197612093 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3f7c0682f0 msgr2=0x7f3f7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:40.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.088+0000 7f3f9efff700 1 --2- 172.21.15.156:0/4197612093 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3f7c0682f0 0x7f3f7c06a7a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c005910 tx=0x7f3f8c009040 comp rx=0 tx=0).stop 2023-11-28T13:10:40.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.088+0000 7f3f9efff700 1 -- 172.21.15.156:0/4197612093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f98161340 msgr2=0x7f3f980cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:40.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.088+0000 7f3f9efff700 1 --2- 172.21.15.156:0/4197612093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f98161340 0x7f3f980cd9a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000efc0 tx=0x7f3f9000c3a0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.088+0000 7f3f9efff700 1 -- 172.21.15.156:0/4197612093 shutdown_connections 2023-11-28T13:10:40.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.088+0000 7f3f9efff700 1 --2- 172.21.15.156:0/4197612093 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3f7c0682f0 0x7f3f7c06a7a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.088+0000 7f3f9efff700 1 --2- 172.21.15.156:0/4197612093 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f98161340 0x7f3f980cd9a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.088+0000 7f3f9efff700 1 --2- 172.21.15.156:0/4197612093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f980d2320 0x7f3f98156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.088+0000 7f3f9efff700 1 --2- 172.21.15.156:0/4197612093 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f9815c9b0 0x7f3f981f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.088+0000 7f3f9efff700 1 -- 172.21.15.156:0/4197612093 >> 172.21.15.156:0/4197612093 conn(0x7f3f98072600 msgr2=0x7f3f9806ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:40.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.088+0000 7f3f9efff700 1 -- 172.21.15.156:0/4197612093 shutdown_connections 2023-11-28T13:10:40.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.088+0000 7f3f9efff700 1 -- 172.21.15.156:0/4197612093 wait complete. 2023-11-28T13:10:40.090 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 23 2023-11-28T13:10:40.108 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:10:40.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.306+0000 7f074ffff700 1 Processor -- start 2023-11-28T13:10:40.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.306+0000 7f074ffff700 1 -- start start 2023-11-28T13:10:40.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.307+0000 7f074ffff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0748161340 0x7f07480d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:40.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.307+0000 7f074ffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f07480d2320 0x7f07480cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:40.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.307+0000 7f074ffff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07480d1950 con 0x7f074815c9b0 2023-11-28T13:10:40.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.307+0000 7f074ffff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07480b5800 con 0x7f0748161340 2023-11-28T13:10:40.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.307+0000 7f074ffff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07480b5300 con 0x7f07480d2320 2023-11-28T13:10:40.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.307+0000 7f074d59a700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0748161340 0x7f07480d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:40.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.307+0000 7f074e59c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f07480d2320 0x7f07480cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:40.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.307+0000 7f074dd9b700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f074815c9b0 0x7f07480b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:36880/0 (socket says 172.21.15.156:36880) 2023-11-28T13:10:40.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.307+0000 7f074dd9b700 1 -- 172.21.15.156:0/2801650946 learned_addr learned my addr 172.21.15.156:0/2801650946 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:40.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.308+0000 7f074cd99700 1 -- 172.21.15.156:0/2801650946 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276693485 0 0) 0x7f07480d1950 con 0x7f074815c9b0 2023-11-28T13:10:40.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.308+0000 7f074e59c700 1 -- 172.21.15.156:0/2801650946 >> v2:172.21.15.186:3300/0 conn(0x7f0748161340 msgr2=0x7f07480d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:40.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.308+0000 7f074e59c700 1 --2- 172.21.15.156:0/2801650946 >> v2:172.21.15.186:3300/0 conn(0x7f0748161340 0x7f07480d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.308+0000 7f074e59c700 1 -- 172.21.15.156:0/2801650946 >> v1:172.21.15.186:6789/0 conn(0x7f074815c9b0 legacy=0x7f07480b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:40.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.308+0000 7f074e59c700 1 -- 172.21.15.156:0/2801650946 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07480b5520 con 0x7f07480d2320 2023-11-28T13:10:40.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.308+0000 7f074d59a700 1 --2- 172.21.15.156:0/2801650946 >> v2:172.21.15.186:3300/0 conn(0x7f0748161340 0x7f07480d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:10:40.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.308+0000 7f074e59c700 1 --2- 172.21.15.156:0/2801650946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f07480d2320 0x7f07480cd7c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0740009d90 tx=0x7f074000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99eb45e962cb1b6c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:40.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.308+0000 7f074cd99700 1 -- 172.21.15.156:0/2801650946 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074000e070 con 0x7f07480d2320 2023-11-28T13:10:40.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.309+0000 7f074cd99700 1 -- 172.21.15.156:0/2801650946 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074000bb20 con 0x7f07480d2320 2023-11-28T13:10:40.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.309+0000 7f074cd99700 1 -- 172.21.15.156:0/2801650946 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740012430 con 0x7f07480d2320 2023-11-28T13:10:40.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.309+0000 7f074ffff700 1 -- 172.21.15.156:0/2801650946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f07480d2320 msgr2=0x7f07480cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:40.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.309+0000 7f074ffff700 1 --2- 172.21.15.156:0/2801650946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f07480d2320 0x7f07480cd7c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0740009d90 tx=0x7f074000b330 comp rx=0 tx=0).stop 2023-11-28T13:10:40.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.309+0000 7f074ffff700 1 -- 172.21.15.156:0/2801650946 shutdown_connections 2023-11-28T13:10:40.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.309+0000 7f074ffff700 1 --2- 172.21.15.156:0/2801650946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f07480d2320 0x7f07480cd7c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.309+0000 7f074ffff700 1 --2- 172.21.15.156:0/2801650946 >> v2:172.21.15.186:3300/0 conn(0x7f0748161340 0x7f07480d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.309+0000 7f074ffff700 1 -- 172.21.15.156:0/2801650946 >> 172.21.15.156:0/2801650946 conn(0x7f0748072600 msgr2=0x7f07480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:40.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.309+0000 7f074ffff700 1 -- 172.21.15.156:0/2801650946 shutdown_connections 2023-11-28T13:10:40.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.309+0000 7f074ffff700 1 -- 172.21.15.156:0/2801650946 wait complete. 2023-11-28T13:10:40.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.312+0000 7f074ffff700 1 Processor -- start 2023-11-28T13:10:40.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.312+0000 7f074ffff700 1 -- start start 2023-11-28T13:10:40.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.313+0000 7f074ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07480d2320 0x7f07480c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:40.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.313+0000 7f074ffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f074815c9b0 0x7f0748156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:40.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.313+0000 7f074ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0748161340 0x7f0748154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:40.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.313+0000 7f074ffff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07480c8830 con 0x7f0748161340 2023-11-28T13:10:40.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.313+0000 7f074ffff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0748154700 con 0x7f07480d2320 2023-11-28T13:10:40.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.313+0000 7f074ffff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07480ca0b0 con 0x7f074815c9b0 2023-11-28T13:10:40.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.313+0000 7f074e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0748161340 0x7f0748154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:40.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.313+0000 7f074e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0748161340 0x7f0748154b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42028/0 (socket says 172.21.15.156:42028) 2023-11-28T13:10:40.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.313+0000 7f074e59c700 1 -- 172.21.15.156:0/899132285 learned_addr learned my addr 172.21.15.156:0/899132285 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:40.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.314+0000 7f074d59a700 1 --2- 172.21.15.156:0/899132285 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f074815c9b0 0x7f0748156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:40.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.314+0000 7f074dd9b700 1 --2- 172.21.15.156:0/899132285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07480d2320 0x7f07480c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:40.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.314+0000 7f074e59c700 1 -- 172.21.15.156:0/899132285 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f074815c9b0 msgr2=0x7f0748156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:40.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.314+0000 7f074e59c700 1 --2- 172.21.15.156:0/899132285 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f074815c9b0 0x7f0748156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.314+0000 7f074e59c700 1 -- 172.21.15.156:0/899132285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07480d2320 msgr2=0x7f07480c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:40.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.314+0000 7f074e59c700 1 --2- 172.21.15.156:0/899132285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07480d2320 0x7f07480c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.314+0000 7f074e59c700 1 -- 172.21.15.156:0/899132285 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07380094a0 con 0x7f0748161340 2023-11-28T13:10:40.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.314+0000 7f074e59c700 1 --2- 172.21.15.156:0/899132285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0748161340 0x7f0748154b20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0740014040 tx=0x7f0740004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:40.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.314+0000 7f073e7fc700 1 -- 172.21.15.156:0/899132285 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740012430 con 0x7f0748161340 2023-11-28T13:10:40.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.314+0000 7f073e7fc700 1 -- 172.21.15.156:0/899132285 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0740020070 con 0x7f0748161340 2023-11-28T13:10:40.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.314+0000 7f074ffff700 1 -- 172.21.15.156:0/899132285 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0740009a70 con 0x7f0748161340 2023-11-28T13:10:40.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.315+0000 7f073e7fc700 1 -- 172.21.15.156:0/899132285 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740004280 con 0x7f0748161340 2023-11-28T13:10:40.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.315+0000 7f074ffff700 1 -- 172.21.15.156:0/899132285 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07480d15e0 con 0x7f0748161340 2023-11-28T13:10:40.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.316+0000 7f074ffff700 1 -- 172.21.15.156:0/899132285 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07480d1300 con 0x7f0748161340 2023-11-28T13:10:40.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.316+0000 7f073e7fc700 1 -- 172.21.15.156:0/899132285 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f074002e070 con 0x7f0748161340 2023-11-28T13:10:40.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.317+0000 7f073e7fc700 1 --2- 172.21.15.156:0/899132285 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f072c0682f0 0x7f072c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:40.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.317+0000 7f073e7fc700 1 -- 172.21.15.156:0/899132285 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0740022070 con 0x7f0748161340 2023-11-28T13:10:40.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.318+0000 7f074dd9b700 1 --2- 172.21.15.156:0/899132285 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f072c0682f0 0x7f072c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:40.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.318+0000 7f074dd9b700 1 --2- 172.21.15.156:0/899132285 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f072c0682f0 0x7f072c06a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0744000f60 tx=0x7f0744007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:40.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.321+0000 7f073e7fc700 1 -- 172.21.15.156:0/899132285 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f07400540a0 con 0x7f0748161340 2023-11-28T13:10:40.516 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.514+0000 7f074ffff700 1 -- 172.21.15.156:0/899132285 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f074815cd80 con 0x7f0748161340 2023-11-28T13:10:40.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.515+0000 7f073e7fc700 1 -- 172.21.15.156:0/899132285 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+17969 (secure 0 0 0) 0x7f07400576c0 con 0x7f0748161340 2023-11-28T13:10:40.517 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:40.517 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":47,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:09:53.247402+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:09:49.958043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T13:09:51.219302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:10:40.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.518+0000 7f074ffff700 1 -- 172.21.15.156:0/899132285 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f072c0682f0 msgr2=0x7f072c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:40.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.518+0000 7f074ffff700 1 --2- 172.21.15.156:0/899132285 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f072c0682f0 0x7f072c06a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0744000f60 tx=0x7f0744007e70 comp rx=0 tx=0).stop 2023-11-28T13:10:40.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.518+0000 7f074ffff700 1 -- 172.21.15.156:0/899132285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0748161340 msgr2=0x7f0748154b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:40.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.518+0000 7f074ffff700 1 --2- 172.21.15.156:0/899132285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0748161340 0x7f0748154b20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0740014040 tx=0x7f0740004ad0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.518+0000 7f074ffff700 1 -- 172.21.15.156:0/899132285 shutdown_connections 2023-11-28T13:10:40.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.518+0000 7f074ffff700 1 --2- 172.21.15.156:0/899132285 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f072c0682f0 0x7f072c06a7a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.518+0000 7f074ffff700 1 --2- 172.21.15.156:0/899132285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0748161340 0x7f0748154b20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.518+0000 7f074ffff700 1 --2- 172.21.15.156:0/899132285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07480d2320 0x7f07480c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.518+0000 7f074ffff700 1 --2- 172.21.15.156:0/899132285 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f074815c9b0 0x7f0748156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.518+0000 7f074ffff700 1 -- 172.21.15.156:0/899132285 >> 172.21.15.156:0/899132285 conn(0x7f0748072600 msgr2=0x7f0748160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:40.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.518+0000 7f074ffff700 1 -- 172.21.15.156:0/899132285 shutdown_connections 2023-11-28T13:10:40.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.518+0000 7f074ffff700 1 -- 172.21.15.156:0/899132285 wait complete. 2023-11-28T13:10:40.538 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T13:10:40.538 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:10:40.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.734+0000 7f4dd83ff700 1 Processor -- start 2023-11-28T13:10:40.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.734+0000 7f4dd83ff700 1 -- start start 2023-11-28T13:10:40.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.735+0000 7f4dd83ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4dd015c9b0 0x7f4dd00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:40.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.735+0000 7f4dd83ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dd00cd7c0 0x7f4dd00cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:40.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.735+0000 7f4dd83ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd00d1950 con 0x7f4dd0161340 2023-11-28T13:10:40.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.735+0000 7f4dd83ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd00b5800 con 0x7f4dd015c9b0 2023-11-28T13:10:40.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.735+0000 7f4dd83ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00ce140 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.736+0000 7f4dd619b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4dd015c9b0 0x7f4dd00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:40.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.736+0000 7f4dd699c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dd00cd7c0 0x7f4dd00cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:40.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.736+0000 7f4dd699c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dd00cd7c0 0x7f4dd00cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:37698/0 (socket says 172.21.15.156:37698) 2023-11-28T13:10:40.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.736+0000 7f4dd619b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4dd015c9b0 0x7f4dd00b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:60742/0 (socket says 172.21.15.156:60742) 2023-11-28T13:10:40.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.736+0000 7f4dd599a700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f4dd0161340 0x7f4dd00d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:36886/0 (socket says 172.21.15.156:36886) 2023-11-28T13:10:40.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.736+0000 7f4dd699c700 1 -- 172.21.15.156:0/1030943506 learned_addr learned my addr 172.21.15.156:0/1030943506 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:40.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.736+0000 7f4dd699c700 1 -- 172.21.15.156:0/1030943506 >> v2:172.21.15.186:3300/0 conn(0x7f4dd015c9b0 msgr2=0x7f4dd00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:40.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.736+0000 7f4dd699c700 1 --2- 172.21.15.156:0/1030943506 >> v2:172.21.15.186:3300/0 conn(0x7f4dd015c9b0 0x7f4dd00b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.736+0000 7f4dd699c700 1 -- 172.21.15.156:0/1030943506 >> v1:172.21.15.186:6789/0 conn(0x7f4dd0161340 legacy=0x7f4dd00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:40.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.736+0000 7f4dd699c700 1 -- 172.21.15.156:0/1030943506 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd00d24f0 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.736+0000 7f4dd699c700 1 --2- 172.21.15.156:0/1030943506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dd00cd7c0 0x7f4dd00cdc00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009e50 tx=0x7f4dc800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74780a5197fc6fc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:40.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.737+0000 7f4dd5199700 1 -- 172.21.15.156:0/1030943506 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc800e070 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.737+0000 7f4dd5199700 1 -- 172.21.15.156:0/1030943506 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc800b910 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.737+0000 7f4dd5199700 1 -- 172.21.15.156:0/1030943506 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc800bcc0 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.737+0000 7f4dd83ff700 1 -- 172.21.15.156:0/1030943506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dd00cd7c0 msgr2=0x7f4dd00cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:40.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.737+0000 7f4dd83ff700 1 --2- 172.21.15.156:0/1030943506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dd00cd7c0 0x7f4dd00cdc00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009e50 tx=0x7f4dc800b330 comp rx=0 tx=0).stop 2023-11-28T13:10:40.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.737+0000 7f4dd83ff700 1 -- 172.21.15.156:0/1030943506 shutdown_connections 2023-11-28T13:10:40.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.737+0000 7f4dd83ff700 1 --2- 172.21.15.156:0/1030943506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dd00cd7c0 0x7f4dd00cdc00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.737+0000 7f4dd83ff700 1 --2- 172.21.15.156:0/1030943506 >> v2:172.21.15.186:3300/0 conn(0x7f4dd015c9b0 0x7f4dd00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.737+0000 7f4dd83ff700 1 -- 172.21.15.156:0/1030943506 >> 172.21.15.156:0/1030943506 conn(0x7f4dd0072600 msgr2=0x7f4dd00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:40.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.737+0000 7f4dd83ff700 1 -- 172.21.15.156:0/1030943506 shutdown_connections 2023-11-28T13:10:40.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.737+0000 7f4dd83ff700 1 -- 172.21.15.156:0/1030943506 wait complete. 2023-11-28T13:10:40.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.739+0000 7f4dd83ff700 1 Processor -- start 2023-11-28T13:10:40.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.739+0000 7f4dd83ff700 1 -- start start 2023-11-28T13:10:40.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.740+0000 7f4dd83ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dd00cd7c0 0x7f4dd01575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:40.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.740+0000 7f4dd83ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dd015c9b0 0x7f4dd00ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:40.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.740+0000 7f4dd83ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dd0161340 0x7f4dd00cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:40.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.740+0000 7f4dd83ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd015d740 con 0x7f4dd0161340 2023-11-28T13:10:40.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.740+0000 7f4dd83ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd015e2c0 con 0x7f4dd015c9b0 2023-11-28T13:10:40.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.740+0000 7f4dd83ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00cd540 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.740+0000 7f4dd699c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dd0161340 0x7f4dd00cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:40.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.740+0000 7f4dd619b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dd00cd7c0 0x7f4dd01575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:40.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.740+0000 7f4dd619b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dd00cd7c0 0x7f4dd01575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:37702/0 (socket says 172.21.15.156:37702) 2023-11-28T13:10:40.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.740+0000 7f4dd619b700 1 -- 172.21.15.156:0/2334033130 learned_addr learned my addr 172.21.15.156:0/2334033130 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:40.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.740+0000 7f4dd599a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dd015c9b0 0x7f4dd00ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:40.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.741+0000 7f4dd619b700 1 -- 172.21.15.156:0/2334033130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dd015c9b0 msgr2=0x7f4dd00ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:40.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.741+0000 7f4dd619b700 1 --2- 172.21.15.156:0/2334033130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dd015c9b0 0x7f4dd00ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.741+0000 7f4dd619b700 1 -- 172.21.15.156:0/2334033130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dd0161340 msgr2=0x7f4dd00cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:40.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.741+0000 7f4dd619b700 1 --2- 172.21.15.156:0/2334033130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dd0161340 0x7f4dd00cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.741+0000 7f4dd619b700 1 -- 172.21.15.156:0/2334033130 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc8009b30 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.741+0000 7f4dd699c700 1 --2- 172.21.15.156:0/2334033130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dd0161340 0x7f4dd00cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:10:40.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.741+0000 7f4dd619b700 1 --2- 172.21.15.156:0/2334033130 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dd00cd7c0 0x7f4dd01575c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00e850 tx=0x7f4dcc00eb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:40.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.741+0000 7f4dc6ffd700 1 -- 172.21.15.156:0/2334033130 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0095b0 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.741+0000 7f4dc6ffd700 1 -- 172.21.15.156:0/2334033130 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc013680 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.741+0000 7f4dd83ff700 1 -- 172.21.15.156:0/2334033130 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd00d1300 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.741+0000 7f4dc6ffd700 1 -- 172.21.15.156:0/2334033130 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc013a10 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.742+0000 7f4dd83ff700 1 -- 172.21.15.156:0/2334033130 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd00b5520 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.743+0000 7f4dd83ff700 1 -- 172.21.15.156:0/2334033130 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd00d15e0 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.744+0000 7f4dc6ffd700 1 -- 172.21.15.156:0/2334033130 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4dcc019030 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.744+0000 7f4dc6ffd700 1 --2- 172.21.15.156:0/2334033130 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4db4068320 0x7f4db406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:40.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.745+0000 7f4dc6ffd700 1 -- 172.21.15.156:0/2334033130 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4dcc0859a0 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.745+0000 7f4dd599a700 1 --2- 172.21.15.156:0/2334033130 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4db4068320 0x7f4db406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:40.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.745+0000 7f4dd599a700 1 --2- 172.21.15.156:0/2334033130 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4db4068320 0x7f4db406a7d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00b8ac0 tx=0x7f4dc0007ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:40.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.750+0000 7f4dc6ffd700 1 -- 172.21.15.156:0/2334033130 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4dcc0521a0 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.981+0000 7f4dd83ff700 1 -- 172.21.15.156:0/2334033130 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dd00745c0 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.982+0000 7f4dc6ffd700 1 -- 172.21.15.156:0/2334033130 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f4dcc0557c0 con 0x7f4dd00cd7c0 2023-11-28T13:10:40.984 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:40.984 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3828141874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3828141874},{"type":"v1","addr":"172.21.15.156:6835","nonce":3828141874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1012489762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1012489762},{"type":"v1","addr":"172.21.15.186:6837","nonce":1012489762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2894670074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2894670074},{"type":"v1","addr":"172.21.15.186:6835","nonce":2894670074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:09:52.245032+0000","modified":"2023-11-28T13:09:55.251254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5481":{"gid":5481,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6834/1930782798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1930782798},{"type":"v1","addr":"172.21.15.156:6834","nonce":1930782798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:10:40.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.984+0000 7f4dd83ff700 1 -- 172.21.15.156:0/2334033130 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4db4068320 msgr2=0x7f4db406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:40.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.984+0000 7f4dd83ff700 1 --2- 172.21.15.156:0/2334033130 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4db4068320 0x7f4db406a7d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00b8ac0 tx=0x7f4dc0007ed0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.984+0000 7f4dd83ff700 1 -- 172.21.15.156:0/2334033130 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dd00cd7c0 msgr2=0x7f4dd01575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:40.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.984+0000 7f4dd83ff700 1 --2- 172.21.15.156:0/2334033130 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dd00cd7c0 0x7f4dd01575c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00e850 tx=0x7f4dcc00eb60 comp rx=0 tx=0).stop 2023-11-28T13:10:40.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.985+0000 7f4dd83ff700 1 -- 172.21.15.156:0/2334033130 shutdown_connections 2023-11-28T13:10:40.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.985+0000 7f4dd83ff700 1 --2- 172.21.15.156:0/2334033130 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4db4068320 0x7f4db406a7d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.985+0000 7f4dd83ff700 1 --2- 172.21.15.156:0/2334033130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dd0161340 0x7f4dd00cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.985+0000 7f4dd83ff700 1 --2- 172.21.15.156:0/2334033130 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dd00cd7c0 0x7f4dd01575c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.985+0000 7f4dd83ff700 1 --2- 172.21.15.156:0/2334033130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dd015c9b0 0x7f4dd00ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:40.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.985+0000 7f4dd83ff700 1 -- 172.21.15.156:0/2334033130 >> 172.21.15.156:0/2334033130 conn(0x7f4dd0072600 msgr2=0x7f4dd00b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:40.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.985+0000 7f4dd83ff700 1 -- 172.21.15.156:0/2334033130 shutdown_connections 2023-11-28T13:10:40.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:40.985+0000 7f4dd83ff700 1 -- 172.21.15.156:0/2334033130 wait complete. 2023-11-28T13:10:40.987 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 23 2023-11-28T13:10:41.004 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:10:41.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.199+0000 7f5ef47ff700 1 Processor -- start 2023-11-28T13:10:41.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.200+0000 7f5ef47ff700 1 -- start start 2023-11-28T13:10:41.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.201+0000 7f5ef47ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5eec15c9b0 0x7f5eec0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:41.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.201+0000 7f5ef47ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5eec161340 0x7f5eec0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:41.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.201+0000 7f5ef47ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5eec0d2320 0x7f5eec0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:41.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.201+0000 7f5ef47ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eec0b5800 con 0x7f5eec15c9b0 2023-11-28T13:10:41.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.201+0000 7f5ef47ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eec0d1950 con 0x7f5eec161340 2023-11-28T13:10:41.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.201+0000 7f5ef47ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0b5520 con 0x7f5eec0d2320 2023-11-28T13:10:41.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.201+0000 7f5ef259b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5eec15c9b0 0x7f5eec0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:41.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.201+0000 7f5ef259b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5eec15c9b0 0x7f5eec0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42034/0 (socket says 172.21.15.156:42034) 2023-11-28T13:10:41.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.201+0000 7f5ef259b700 1 -- 172.21.15.156:0/64975347 learned_addr learned my addr 172.21.15.156:0/64975347 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:41.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.202+0000 7f5ef2d9c700 1 --2- 172.21.15.156:0/64975347 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5eec0d2320 0x7f5eec0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:41.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.201+0000 7f5ef1d9a700 1 --2- 172.21.15.156:0/64975347 >> v2:172.21.15.186:3300/0 conn(0x7f5eec161340 0x7f5eec0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:41.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.202+0000 7f5ef259b700 1 -- 172.21.15.156:0/64975347 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5eec0d2320 msgr2=0x7f5eec0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:41.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.202+0000 7f5ef259b700 1 --2- 172.21.15.156:0/64975347 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5eec0d2320 0x7f5eec0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.202+0000 7f5ef259b700 1 -- 172.21.15.156:0/64975347 >> v2:172.21.15.186:3300/0 conn(0x7f5eec161340 msgr2=0x7f5eec0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:41.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.202+0000 7f5ef259b700 1 --2- 172.21.15.156:0/64975347 >> v2:172.21.15.186:3300/0 conn(0x7f5eec161340 0x7f5eec0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.202+0000 7f5ef259b700 1 -- 172.21.15.156:0/64975347 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec0d1300 con 0x7f5eec15c9b0 2023-11-28T13:10:41.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.202+0000 7f5ef2d9c700 1 --2- 172.21.15.156:0/64975347 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5eec0d2320 0x7f5eec0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:41.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.202+0000 7f5ef1d9a700 1 --2- 172.21.15.156:0/64975347 >> v2:172.21.15.186:3300/0 conn(0x7f5eec161340 0x7f5eec0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:41.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.202+0000 7f5ef259b700 1 --2- 172.21.15.156:0/64975347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5eec15c9b0 0x7f5eec0b4bf0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0685c0 tx=0x7f5ee8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ea9f6c33691be1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:41.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.202+0000 7f5ef1599700 1 -- 172.21.15.156:0/64975347 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee800b870 con 0x7f5eec15c9b0 2023-11-28T13:10:41.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.202+0000 7f5ef1599700 1 -- 172.21.15.156:0/64975347 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee800b9d0 con 0x7f5eec15c9b0 2023-11-28T13:10:41.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.202+0000 7f5ef1599700 1 -- 172.21.15.156:0/64975347 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee800bd60 con 0x7f5eec15c9b0 2023-11-28T13:10:41.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.203+0000 7f5ef47ff700 1 -- 172.21.15.156:0/64975347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5eec15c9b0 msgr2=0x7f5eec0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:41.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.203+0000 7f5ef47ff700 1 --2- 172.21.15.156:0/64975347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5eec15c9b0 0x7f5eec0b4bf0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0685c0 tx=0x7f5ee8009bd0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.203+0000 7f5ef47ff700 1 -- 172.21.15.156:0/64975347 shutdown_connections 2023-11-28T13:10:41.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.203+0000 7f5ef47ff700 1 --2- 172.21.15.156:0/64975347 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5eec0d2320 0x7f5eec0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.203+0000 7f5ef47ff700 1 --2- 172.21.15.156:0/64975347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5eec15c9b0 0x7f5eec0b4bf0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.203+0000 7f5ef47ff700 1 --2- 172.21.15.156:0/64975347 >> v2:172.21.15.186:3300/0 conn(0x7f5eec161340 0x7f5eec0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.203+0000 7f5ef47ff700 1 -- 172.21.15.156:0/64975347 >> 172.21.15.156:0/64975347 conn(0x7f5eec072600 msgr2=0x7f5eec0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:41.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.203+0000 7f5ef47ff700 1 -- 172.21.15.156:0/64975347 shutdown_connections 2023-11-28T13:10:41.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.203+0000 7f5ef47ff700 1 -- 172.21.15.156:0/64975347 wait complete. 2023-11-28T13:10:41.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.205+0000 7f5ef47ff700 1 Processor -- start 2023-11-28T13:10:41.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.205+0000 7f5ef47ff700 1 -- start start 2023-11-28T13:10:41.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.206+0000 7f5ef47ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eec0d2320 0x7f5eec154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:41.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.206+0000 7f5ef47ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5eec15c9b0 0x7f5eec155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:41.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.206+0000 7f5ef47ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5eec161340 0x7f5eec155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:41.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.206+0000 7f5ef47ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec152ef0 con 0x7f5eec161340 2023-11-28T13:10:41.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.206+0000 7f5ef47ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec15f7e0 con 0x7f5eec0d2320 2023-11-28T13:10:41.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.206+0000 7f5ef47ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec1553f0 con 0x7f5eec15c9b0 2023-11-28T13:10:41.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.206+0000 7f5ef2d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5eec161340 0x7f5eec155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:41.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.206+0000 7f5ef2d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5eec161340 0x7f5eec155b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42042/0 (socket says 172.21.15.156:42042) 2023-11-28T13:10:41.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.206+0000 7f5ef2d9c700 1 -- 172.21.15.156:0/2537895959 learned_addr learned my addr 172.21.15.156:0/2537895959 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:41.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.206+0000 7f5ef1d9a700 1 --2- 172.21.15.156:0/2537895959 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5eec15c9b0 0x7f5eec155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:41.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.206+0000 7f5ef259b700 1 --2- 172.21.15.156:0/2537895959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eec0d2320 0x7f5eec154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:41.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.206+0000 7f5ef2d9c700 1 -- 172.21.15.156:0/2537895959 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5eec15c9b0 msgr2=0x7f5eec155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:41.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.206+0000 7f5ef2d9c700 1 --2- 172.21.15.156:0/2537895959 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5eec15c9b0 0x7f5eec155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.206+0000 7f5ef2d9c700 1 -- 172.21.15.156:0/2537895959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eec0d2320 msgr2=0x7f5eec154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:41.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.206+0000 7f5ef2d9c700 1 --2- 172.21.15.156:0/2537895959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eec0d2320 0x7f5eec154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.207+0000 7f5ef2d9c700 1 -- 172.21.15.156:0/2537895959 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee8009550 con 0x7f5eec161340 2023-11-28T13:10:41.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.207+0000 7f5ef2d9c700 1 --2- 172.21.15.156:0/2537895959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5eec161340 0x7f5eec155b90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400cd80 tx=0x7f5ee400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:41.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.207+0000 7f5ee2ffd700 1 -- 172.21.15.156:0/2537895959 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee400f480 con 0x7f5eec161340 2023-11-28T13:10:41.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.207+0000 7f5ee2ffd700 1 -- 172.21.15.156:0/2537895959 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee400f600 con 0x7f5eec161340 2023-11-28T13:10:41.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.207+0000 7f5ee2ffd700 1 -- 172.21.15.156:0/2537895959 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee400f9b0 con 0x7f5eec161340 2023-11-28T13:10:41.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.207+0000 7f5ef47ff700 1 -- 172.21.15.156:0/2537895959 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eec0b5130 con 0x7f5eec161340 2023-11-28T13:10:41.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.207+0000 7f5ef47ff700 1 -- 172.21.15.156:0/2537895959 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eec0d15e0 con 0x7f5eec161340 2023-11-28T13:10:41.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.208+0000 7f5ef47ff700 1 -- 172.21.15.156:0/2537895959 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eec0b52c0 con 0x7f5eec161340 2023-11-28T13:10:41.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.209+0000 7f5ee2ffd700 1 -- 172.21.15.156:0/2537895959 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5ee4009030 con 0x7f5eec161340 2023-11-28T13:10:41.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.210+0000 7f5ee2ffd700 1 --2- 172.21.15.156:0/2537895959 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5ed00682f0 0x7f5ed006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:41.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.210+0000 7f5ee2ffd700 1 -- 172.21.15.156:0/2537895959 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5ee40865f0 con 0x7f5eec161340 2023-11-28T13:10:41.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.210+0000 7f5ef259b700 1 --2- 172.21.15.156:0/2537895959 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5ed00682f0 0x7f5ed006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:41.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.211+0000 7f5ef259b700 1 --2- 172.21.15.156:0/2537895959 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5ed00682f0 0x7f5ed006a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8000c00 tx=0x7f5ee8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:41.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.214+0000 7f5ee2ffd700 1 -- 172.21.15.156:0/2537895959 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5ee4052d60 con 0x7f5eec161340 2023-11-28T13:10:41.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.448+0000 7f5ef47ff700 1 -- 172.21.15.156:0/2537895959 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5eec153030 con 0x7f5eec161340 2023-11-28T13:10:41.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.448+0000 7f5ee2ffd700 1 -- 172.21.15.156:0/2537895959 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f5eec153030 con 0x7f5eec161340 2023-11-28T13:10:41.450 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:41.450 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3828141874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3828141874},{"type":"v1","addr":"172.21.15.156:6835","nonce":3828141874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1012489762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1012489762},{"type":"v1","addr":"172.21.15.186:6837","nonce":1012489762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2894670074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2894670074},{"type":"v1","addr":"172.21.15.186:6835","nonce":2894670074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:09:52.245032+0000","modified":"2023-11-28T13:09:55.251254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5481":{"gid":5481,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6834/1930782798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1930782798},{"type":"v1","addr":"172.21.15.156:6834","nonce":1930782798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:10:41.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.451+0000 7f5ef47ff700 1 -- 172.21.15.156:0/2537895959 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5ed00682f0 msgr2=0x7f5ed006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:41.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.451+0000 7f5ef47ff700 1 --2- 172.21.15.156:0/2537895959 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5ed00682f0 0x7f5ed006a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8000c00 tx=0x7f5ee8008030 comp rx=0 tx=0).stop 2023-11-28T13:10:41.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.451+0000 7f5ef47ff700 1 -- 172.21.15.156:0/2537895959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5eec161340 msgr2=0x7f5eec155b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:41.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.451+0000 7f5ef47ff700 1 --2- 172.21.15.156:0/2537895959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5eec161340 0x7f5eec155b90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400cd80 tx=0x7f5ee400a560 comp rx=0 tx=0).stop 2023-11-28T13:10:41.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.451+0000 7f5ef47ff700 1 -- 172.21.15.156:0/2537895959 shutdown_connections 2023-11-28T13:10:41.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.451+0000 7f5ef47ff700 1 --2- 172.21.15.156:0/2537895959 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5ed00682f0 0x7f5ed006a7a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.451+0000 7f5ef47ff700 1 --2- 172.21.15.156:0/2537895959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5eec161340 0x7f5eec155b90 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.451+0000 7f5ef47ff700 1 --2- 172.21.15.156:0/2537895959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eec0d2320 0x7f5eec154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.451+0000 7f5ef47ff700 1 --2- 172.21.15.156:0/2537895959 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5eec15c9b0 0x7f5eec155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.451+0000 7f5ef47ff700 1 -- 172.21.15.156:0/2537895959 >> 172.21.15.156:0/2537895959 conn(0x7f5eec072600 msgr2=0x7f5eec15d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:41.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.451+0000 7f5ef47ff700 1 -- 172.21.15.156:0/2537895959 shutdown_connections 2023-11-28T13:10:41.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.451+0000 7f5ef47ff700 1 -- 172.21.15.156:0/2537895959 wait complete. 2023-11-28T13:10:41.454 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 23 2023-11-28T13:10:41.471 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:10:41.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.666+0000 7f8944fff700 1 Processor -- start 2023-11-28T13:10:41.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.667+0000 7f8944fff700 1 -- start start 2023-11-28T13:10:41.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.668+0000 7f8944fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f89400cd7c0 0x7f89400cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:41.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.668+0000 7f8944fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89400d1950 con 0x7f894015c9b0 2023-11-28T13:10:41.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.668+0000 7f8944fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89400ce140 con 0x7f8940161340 2023-11-28T13:10:41.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.668+0000 7f8944fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89400b5800 con 0x7f89400cd7c0 2023-11-28T13:10:41.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.668+0000 7f893e59c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f894015c9b0 0x7f89400b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37882/0 (socket says 172.21.15.156:37882) 2023-11-28T13:10:41.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.668+0000 7f893e59c700 1 -- 172.21.15.156:0/2010357135 learned_addr learned my addr 172.21.15.156:0/2010357135 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:41.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.668+0000 7f893ed9d700 1 --2- 172.21.15.156:0/2010357135 >> v2:172.21.15.156:3300/0 conn(0x7f89400cd7c0 0x7f89400cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:41.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.668+0000 7f893dd9b700 1 -- 172.21.15.156:0/2010357135 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147582451 0 0) 0x7f89400d1950 con 0x7f894015c9b0 2023-11-28T13:10:41.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.668+0000 7f893dd9b700 1 -- 172.21.15.156:0/2010357135 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f892c0035b0 con 0x7f894015c9b0 2023-11-28T13:10:41.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.668+0000 7f893ed9d700 1 -- 172.21.15.156:0/2010357135 >> v1:172.21.15.186:6789/0 conn(0x7f8940161340 legacy=0x7f89400d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:41.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.669+0000 7f893ed9d700 1 -- 172.21.15.156:0/2010357135 >> v1:172.21.15.156:6789/0 conn(0x7f894015c9b0 legacy=0x7f89400b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:41.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.669+0000 7f893ed9d700 1 -- 172.21.15.156:0/2010357135 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89400d24f0 con 0x7f89400cd7c0 2023-11-28T13:10:41.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.669+0000 7f893ed9d700 1 --2- 172.21.15.156:0/2010357135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89400cd7c0 0x7f89400cdc00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8938009d90 tx=0x7f893800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af3aac3dad2fc23 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:41.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.669+0000 7f893dd9b700 1 -- 172.21.15.156:0/2010357135 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893800e070 con 0x7f89400cd7c0 2023-11-28T13:10:41.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.669+0000 7f893dd9b700 1 -- 172.21.15.156:0/2010357135 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893800bb20 con 0x7f89400cd7c0 2023-11-28T13:10:41.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.669+0000 7f893dd9b700 1 -- 172.21.15.156:0/2010357135 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938012430 con 0x7f89400cd7c0 2023-11-28T13:10:41.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.669+0000 7f8944fff700 1 -- 172.21.15.156:0/2010357135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89400cd7c0 msgr2=0x7f89400cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:41.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.670+0000 7f8944fff700 1 --2- 172.21.15.156:0/2010357135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89400cd7c0 0x7f89400cdc00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8938009d90 tx=0x7f893800b330 comp rx=0 tx=0).stop 2023-11-28T13:10:41.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.670+0000 7f8944fff700 1 -- 172.21.15.156:0/2010357135 shutdown_connections 2023-11-28T13:10:41.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.670+0000 7f8944fff700 1 --2- 172.21.15.156:0/2010357135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89400cd7c0 0x7f89400cdc00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.670+0000 7f8944fff700 1 -- 172.21.15.156:0/2010357135 >> 172.21.15.156:0/2010357135 conn(0x7f8940072600 msgr2=0x7f89400cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:41.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.670+0000 7f8944fff700 1 -- 172.21.15.156:0/2010357135 shutdown_connections 2023-11-28T13:10:41.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.670+0000 7f8944fff700 1 -- 172.21.15.156:0/2010357135 wait complete. 2023-11-28T13:10:41.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.671+0000 7f8944fff700 1 Processor -- start 2023-11-28T13:10:41.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.672+0000 7f8944fff700 1 -- start start 2023-11-28T13:10:41.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.672+0000 7f8944fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89400cd7c0 0x7f89400c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:41.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.673+0000 7f8944fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f894015c9b0 0x7f89400ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:41.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.673+0000 7f8944fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8940161340 0x7f894015e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.673+0000 7f8944fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89401f2c40 con 0x7f8940161340 2023-11-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.673+0000 7f8944fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89401f2330 con 0x7f894015c9b0 2023-11-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.673+0000 7f8944fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89400c7260 con 0x7f89400cd7c0 2023-11-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.673+0000 7f893ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8940161340 0x7f894015e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.673+0000 7f893ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8940161340 0x7f894015e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42046/0 (socket says 172.21.15.156:42046) 2023-11-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.673+0000 7f893ed9d700 1 -- 172.21.15.156:0/918031258 learned_addr learned my addr 172.21.15.156:0/918031258 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.673+0000 7f893e59c700 1 --2- 172.21.15.156:0/918031258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89400cd7c0 0x7f89400c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.673+0000 7f8937fff700 1 --2- 172.21.15.156:0/918031258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f894015c9b0 0x7f89400ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.673+0000 7f893ed9d700 1 -- 172.21.15.156:0/918031258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89400cd7c0 msgr2=0x7f89400c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.673+0000 7f893ed9d700 1 --2- 172.21.15.156:0/918031258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89400cd7c0 0x7f89400c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.673+0000 7f893ed9d700 1 -- 172.21.15.156:0/918031258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f894015c9b0 msgr2=0x7f89400ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.673+0000 7f893ed9d700 1 --2- 172.21.15.156:0/918031258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f894015c9b0 0x7f89400ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.673+0000 7f893ed9d700 1 -- 172.21.15.156:0/918031258 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8938009a70 con 0x7f8940161340 2023-11-28T13:10:41.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.674+0000 7f893ed9d700 1 --2- 172.21.15.156:0/918031258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8940161340 0x7f894015e2c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f89400675d0 tx=0x7f8938004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:41.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.674+0000 7f8936ffd700 1 -- 172.21.15.156:0/918031258 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938012430 con 0x7f8940161340 2023-11-28T13:10:41.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.674+0000 7f8936ffd700 1 -- 172.21.15.156:0/918031258 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8938020070 con 0x7f8940161340 2023-11-28T13:10:41.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.674+0000 7f8944fff700 1 -- 172.21.15.156:0/918031258 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89400d1300 con 0x7f8940161340 2023-11-28T13:10:41.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.674+0000 7f8936ffd700 1 -- 172.21.15.156:0/918031258 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938004280 con 0x7f8940161340 2023-11-28T13:10:41.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.674+0000 7f8944fff700 1 -- 172.21.15.156:0/918031258 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89400b5520 con 0x7f8940161340 2023-11-28T13:10:41.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.675+0000 7f8944fff700 1 -- 172.21.15.156:0/918031258 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89400d15e0 con 0x7f8940161340 2023-11-28T13:10:41.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.676+0000 7f8936ffd700 1 -- 172.21.15.156:0/918031258 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f893802e070 con 0x7f8940161340 2023-11-28T13:10:41.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.677+0000 7f8936ffd700 1 --2- 172.21.15.156:0/918031258 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f89200682f0 0x7f892006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:41.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.677+0000 7f8936ffd700 1 -- 172.21.15.156:0/918031258 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8938022070 con 0x7f8940161340 2023-11-28T13:10:41.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.677+0000 7f893e59c700 1 --2- 172.21.15.156:0/918031258 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f89200682f0 0x7f892006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:41.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.678+0000 7f893e59c700 1 --2- 172.21.15.156:0/918031258 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f89200682f0 0x7f892006a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8930007fd0 tx=0x7f8930007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:41.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.681+0000 7f8936ffd700 1 -- 172.21.15.156:0/918031258 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f89380540d0 con 0x7f8940161340 2023-11-28T13:10:41.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.871+0000 7f8944fff700 1 -- 172.21.15.156:0/918031258 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89400745c0 con 0x7f8940161340 2023-11-28T13:10:41.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.872+0000 7f8936ffd700 1 -- 172.21.15.156:0/918031258 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+17969 (secure 0 0 0) 0x7f89380576f0 con 0x7f8940161340 2023-11-28T13:10:41.874 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:41.874 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":47,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:09:53.247402+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:09:49.958043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T13:09:51.219302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:10:41.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.874+0000 7f8944fff700 1 -- 172.21.15.156:0/918031258 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f89200682f0 msgr2=0x7f892006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:41.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.874+0000 7f8944fff700 1 --2- 172.21.15.156:0/918031258 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f89200682f0 0x7f892006a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8930007fd0 tx=0x7f8930007dc0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.874+0000 7f8944fff700 1 -- 172.21.15.156:0/918031258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8940161340 msgr2=0x7f894015e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:41.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.874+0000 7f8944fff700 1 --2- 172.21.15.156:0/918031258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8940161340 0x7f894015e2c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f89400675d0 tx=0x7f8938004ad0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.874+0000 7f8944fff700 1 -- 172.21.15.156:0/918031258 shutdown_connections 2023-11-28T13:10:41.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.874+0000 7f8944fff700 1 --2- 172.21.15.156:0/918031258 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f89200682f0 0x7f892006a7a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.874+0000 7f8944fff700 1 --2- 172.21.15.156:0/918031258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8940161340 0x7f894015e2c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.874+0000 7f8944fff700 1 --2- 172.21.15.156:0/918031258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f89400cd7c0 0x7f89400c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.874+0000 7f8944fff700 1 --2- 172.21.15.156:0/918031258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f894015c9b0 0x7f89400ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:41.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.874+0000 7f8944fff700 1 -- 172.21.15.156:0/918031258 >> 172.21.15.156:0/918031258 conn(0x7f8940072600 msgr2=0x7f8940160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:41.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.875+0000 7f8944fff700 1 -- 172.21.15.156:0/918031258 shutdown_connections 2023-11-28T13:10:41.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:41.875+0000 7f8944fff700 1 -- 172.21.15.156:0/918031258 wait complete. 2023-11-28T13:10:41.895 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T13:10:42.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.091+0000 7f36415ff700 1 Processor -- start 2023-11-28T13:10:42.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.091+0000 7f36415ff700 1 -- start start 2023-11-28T13:10:42.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.093+0000 7f36415ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f363c161340 0x7f363c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:42.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.093+0000 7f36415ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f363c0d2320 0x7f363c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:42.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.093+0000 7f36415ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363c0d1950 con 0x7f363c15c9b0 2023-11-28T13:10:42.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.093+0000 7f36415ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c0b5800 con 0x7f363c0d2320 2023-11-28T13:10:42.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.093+0000 7f36415ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c0b5300 con 0x7f363c161340 2023-11-28T13:10:42.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.093+0000 7f363b7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f363c0d2320 0x7f363c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:42.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.093+0000 7f363b7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f363c0d2320 0x7f363c0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42056/0 (socket says 172.21.15.156:42056) 2023-11-28T13:10:42.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.093+0000 7f363b7fe700 1 -- 172.21.15.156:0/2597018466 learned_addr learned my addr 172.21.15.156:0/2597018466 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:42.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.093+0000 7f3633fff700 1 --2- 172.21.15.156:0/2597018466 >> v2:172.21.15.186:3300/0 conn(0x7f363c161340 0x7f363c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:42.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.093+0000 7f363b7fe700 1 -- 172.21.15.156:0/2597018466 >> v2:172.21.15.186:3300/0 conn(0x7f363c161340 msgr2=0x7f363c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:42.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.093+0000 7f363b7fe700 1 --2- 172.21.15.156:0/2597018466 >> v2:172.21.15.186:3300/0 conn(0x7f363c161340 0x7f363c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:42.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.093+0000 7f363b7fe700 1 -- 172.21.15.156:0/2597018466 >> v1:172.21.15.186:6789/0 conn(0x7f363c15c9b0 legacy=0x7f363c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:42.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.093+0000 7f363b7fe700 1 -- 172.21.15.156:0/2597018466 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c0b5520 con 0x7f363c0d2320 2023-11-28T13:10:42.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.094+0000 7f363b7fe700 1 --2- 172.21.15.156:0/2597018466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f363c0d2320 0x7f363c0cd7c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f363c0685c0 tx=0x7f363400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9879c433629b7955 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:42.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.094+0000 7f363a7fc700 1 -- 172.21.15.156:0/2597018466 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363400e070 con 0x7f363c0d2320 2023-11-28T13:10:42.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.094+0000 7f363a7fc700 1 -- 172.21.15.156:0/2597018466 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363400b980 con 0x7f363c0d2320 2023-11-28T13:10:42.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.094+0000 7f363a7fc700 1 -- 172.21.15.156:0/2597018466 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363400bd30 con 0x7f363c0d2320 2023-11-28T13:10:42.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.094+0000 7f36415ff700 1 -- 172.21.15.156:0/2597018466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f363c0d2320 msgr2=0x7f363c0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:42.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.094+0000 7f36415ff700 1 --2- 172.21.15.156:0/2597018466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f363c0d2320 0x7f363c0cd7c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f363c0685c0 tx=0x7f363400b330 comp rx=0 tx=0).stop 2023-11-28T13:10:42.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.095+0000 7f36415ff700 1 -- 172.21.15.156:0/2597018466 shutdown_connections 2023-11-28T13:10:42.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.095+0000 7f36415ff700 1 --2- 172.21.15.156:0/2597018466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f363c0d2320 0x7f363c0cd7c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:42.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.095+0000 7f36415ff700 1 --2- 172.21.15.156:0/2597018466 >> v2:172.21.15.186:3300/0 conn(0x7f363c161340 0x7f363c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:42.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.095+0000 7f36415ff700 1 -- 172.21.15.156:0/2597018466 >> 172.21.15.156:0/2597018466 conn(0x7f363c072600 msgr2=0x7f363c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:42.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.095+0000 7f36415ff700 1 -- 172.21.15.156:0/2597018466 shutdown_connections 2023-11-28T13:10:42.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.095+0000 7f36415ff700 1 -- 172.21.15.156:0/2597018466 wait complete. 2023-11-28T13:10:42.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.096+0000 7f36415ff700 1 Processor -- start 2023-11-28T13:10:42.098 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.096+0000 7f36415ff700 1 -- start start 2023-11-28T13:10:42.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.097+0000 7f36415ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f363c0d2320 0x7f363c0c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:42.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.097+0000 7f36415ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f363c15c9b0 0x7f363c150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:42.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.097+0000 7f36415ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f363c161340 0x7f363c0b8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:42.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.097+0000 7f36415ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c1521e0 con 0x7f363c15c9b0 2023-11-28T13:10:42.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.097+0000 7f36415ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c154dd0 con 0x7f363c0d2320 2023-11-28T13:10:42.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.097+0000 7f36415ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c150c90 con 0x7f363c161340 2023-11-28T13:10:42.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.098+0000 7f3633fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f363c15c9b0 0x7f363c150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:42.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.098+0000 7f3633fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f363c15c9b0 0x7f363c150750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42072/0 (socket says 172.21.15.156:42072) 2023-11-28T13:10:42.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.098+0000 7f3633fff700 1 -- 172.21.15.156:0/950489364 learned_addr learned my addr 172.21.15.156:0/950489364 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:42.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.098+0000 7f363affd700 1 --2- 172.21.15.156:0/950489364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f363c0d2320 0x7f363c0c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:42.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.098+0000 7f363b7fe700 1 --2- 172.21.15.156:0/950489364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f363c161340 0x7f363c0b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:42.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.098+0000 7f3633fff700 1 -- 172.21.15.156:0/950489364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f363c161340 msgr2=0x7f363c0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:42.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.098+0000 7f3633fff700 1 --2- 172.21.15.156:0/950489364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f363c161340 0x7f363c0b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:42.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.098+0000 7f3633fff700 1 -- 172.21.15.156:0/950489364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f363c0d2320 msgr2=0x7f363c0c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:42.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.098+0000 7f3633fff700 1 --2- 172.21.15.156:0/950489364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f363c0d2320 0x7f363c0c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:42.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.098+0000 7f3633fff700 1 -- 172.21.15.156:0/950489364 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3634009b30 con 0x7f363c15c9b0 2023-11-28T13:10:42.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.098+0000 7f3633fff700 1 --2- 172.21.15.156:0/950489364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f363c15c9b0 0x7f363c150750 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f362400e810 tx=0x7f362400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:42.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.099+0000 7f36337fe700 1 -- 172.21.15.156:0/950489364 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362400c070 con 0x7f363c15c9b0 2023-11-28T13:10:42.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.099+0000 7f36337fe700 1 -- 172.21.15.156:0/950489364 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3624013610 con 0x7f363c15c9b0 2023-11-28T13:10:42.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.099+0000 7f36337fe700 1 -- 172.21.15.156:0/950489364 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36240139c0 con 0x7f363c15c9b0 2023-11-28T13:10:42.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.099+0000 7f36415ff700 1 -- 172.21.15.156:0/950489364 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363c0d15e0 con 0x7f363c15c9b0 2023-11-28T13:10:42.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.099+0000 7f36415ff700 1 -- 172.21.15.156:0/950489364 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363c0d1300 con 0x7f363c15c9b0 2023-11-28T13:10:42.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.100+0000 7f36415ff700 1 -- 172.21.15.156:0/950489364 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363c15cd80 con 0x7f363c15c9b0 2023-11-28T13:10:42.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.101+0000 7f36337fe700 1 -- 172.21.15.156:0/950489364 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3624023070 con 0x7f363c15c9b0 2023-11-28T13:10:42.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.101+0000 7f36337fe700 1 --2- 172.21.15.156:0/950489364 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f361c0682f0 0x7f361c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:42.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.102+0000 7f36337fe700 1 -- 172.21.15.156:0/950489364 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3624085b50 con 0x7f363c15c9b0 2023-11-28T13:10:42.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.102+0000 7f363affd700 1 --2- 172.21.15.156:0/950489364 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f361c0682f0 0x7f361c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:42.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.103+0000 7f363affd700 1 --2- 172.21.15.156:0/950489364 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f361c0682f0 0x7f361c06a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f362c001800 tx=0x7f362c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:42.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.105+0000 7f36337fe700 1 -- 172.21.15.156:0/950489364 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3624052240 con 0x7f363c15c9b0 2023-11-28T13:10:42.345 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.343+0000 7f36415ff700 1 -- 172.21.15.156:0/950489364 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f363c0d08a0 con 0x7f363c15c9b0 2023-11-28T13:10:42.471 INFO:tasks.ceph.mon.a.smithi156.stderr:2023-11-28T13:10:42.469+0000 7f61ad1f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T13:10:42.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.477+0000 7f36337fe700 1 -- 172.21.15.156:0/950489364 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3624059e60 con 0x7f363c15c9b0 2023-11-28T13:10:42.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.479+0000 7f36415ff700 1 -- 172.21.15.156:0/950489364 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f361c0682f0 msgr2=0x7f361c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:42.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.479+0000 7f36415ff700 1 --2- 172.21.15.156:0/950489364 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f361c0682f0 0x7f361c06a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f362c001800 tx=0x7f362c009040 comp rx=0 tx=0).stop 2023-11-28T13:10:42.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.479+0000 7f36415ff700 1 -- 172.21.15.156:0/950489364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f363c15c9b0 msgr2=0x7f363c150750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:42.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.479+0000 7f36415ff700 1 --2- 172.21.15.156:0/950489364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f363c15c9b0 0x7f363c150750 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f362400e810 tx=0x7f362400eb20 comp rx=0 tx=0).stop 2023-11-28T13:10:42.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.479+0000 7f36415ff700 1 -- 172.21.15.156:0/950489364 shutdown_connections 2023-11-28T13:10:42.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.479+0000 7f36415ff700 1 --2- 172.21.15.156:0/950489364 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f361c0682f0 0x7f361c06a7a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:42.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.480+0000 7f36415ff700 1 --2- 172.21.15.156:0/950489364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f363c161340 0x7f363c0b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:42.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.480+0000 7f36415ff700 1 --2- 172.21.15.156:0/950489364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f363c0d2320 0x7f363c0c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:42.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.480+0000 7f36415ff700 1 --2- 172.21.15.156:0/950489364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f363c15c9b0 0x7f363c150750 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:42.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.480+0000 7f36415ff700 1 -- 172.21.15.156:0/950489364 >> 172.21.15.156:0/950489364 conn(0x7f363c072600 msgr2=0x7f363c15fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:42.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.480+0000 7f36415ff700 1 -- 172.21.15.156:0/950489364 shutdown_connections 2023-11-28T13:10:42.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.480+0000 7f36415ff700 1 -- 172.21.15.156:0/950489364 wait complete. 2023-11-28T13:10:42.482 INFO:teuthology.orchestra.run.smithi156.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T13:10:42.490 INFO:tasks.ceph.mds.c.smithi156.stderr: -6599> 2023-11-28T13:09:56.727+0000 7fa1217f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:10:42.496 INFO:tasks.ceph.mds.c.smithi156.stderr: -3086> 2023-11-28T13:10:02.175+0000 7fa11dfea700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5416=0-4194304@1} caps={5416=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5416 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x559ff5783700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T13:10:42.500 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T13:10:42.545 INFO:tasks.ceph.mds.c.smithi156.stdout:starting mds.c at 2023-11-28T13:10:42.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.700+0000 7f86089ff700 1 Processor -- start 2023-11-28T13:10:42.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.701+0000 7f86089ff700 1 -- start start 2023-11-28T13:10:42.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.702+0000 7f86089ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f860415c9b0 0x7f86040b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:42.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.702+0000 7f86089ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f86040cd7c0 0x7f86040cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:42.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.702+0000 7f86089ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86040ce160 con 0x7f8604161340 2023-11-28T13:10:42.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.702+0000 7f86089ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86040b5800 con 0x7f86040cd7c0 2023-11-28T13:10:42.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.702+0000 7f86089ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86040d1950 con 0x7f860415c9b0 2023-11-28T13:10:42.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.702+0000 7f8602d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f86040cd7c0 0x7f86040cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:42.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.702+0000 7f8602d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f86040cd7c0 0x7f86040cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42082/0 (socket says 172.21.15.156:42082) 2023-11-28T13:10:42.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.702+0000 7f8602d9d700 1 -- 172.21.15.156:0/2561376392 learned_addr learned my addr 172.21.15.156:0/2561376392 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.702+0000 7f860259c700 1 --2- 172.21.15.156:0/2561376392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f860415c9b0 0x7f86040b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.702+0000 7f8602d9d700 1 -- 172.21.15.156:0/2561376392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f860415c9b0 msgr2=0x7f86040b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.703+0000 7f8602d9d700 1 --2- 172.21.15.156:0/2561376392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f860415c9b0 0x7f86040b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.703+0000 7f8602d9d700 1 -- 172.21.15.156:0/2561376392 >> v1:172.21.15.186:6789/0 conn(0x7f8604161340 legacy=0x7f86040d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.703+0000 7f8602d9d700 1 -- 172.21.15.156:0/2561376392 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86040d24f0 con 0x7f86040cd7c0 2023-11-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.703+0000 7f860259c700 1 --2- 172.21.15.156:0/2561376392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f860415c9b0 0x7f86040b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.703+0000 7f8602d9d700 1 --2- 172.21.15.156:0/2561376392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86040cd7c0 0x7f86040cdc20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009e50 tx=0x7f85fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8dbac46f8ab800b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.703+0000 7f8601d9b700 1 -- 172.21.15.156:0/2561376392 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc00e070 con 0x7f86040cd7c0 2023-11-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.703+0000 7f8601d9b700 1 -- 172.21.15.156:0/2561376392 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85fc003a20 con 0x7f86040cd7c0 2023-11-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.703+0000 7f8601d9b700 1 -- 172.21.15.156:0/2561376392 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc003dd0 con 0x7f86040cd7c0 2023-11-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.703+0000 7f86089ff700 1 -- 172.21.15.156:0/2561376392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86040cd7c0 msgr2=0x7f86040cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:42.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.704+0000 7f86089ff700 1 --2- 172.21.15.156:0/2561376392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86040cd7c0 0x7f86040cdc20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009e50 tx=0x7f85fc00b330 comp rx=0 tx=0).stop 2023-11-28T13:10:42.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.704+0000 7f86089ff700 1 -- 172.21.15.156:0/2561376392 shutdown_connections 2023-11-28T13:10:42.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.704+0000 7f86089ff700 1 --2- 172.21.15.156:0/2561376392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86040cd7c0 0x7f86040cdc20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:42.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.704+0000 7f86089ff700 1 --2- 172.21.15.156:0/2561376392 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f860415c9b0 0x7f86040b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:42.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.704+0000 7f86089ff700 1 -- 172.21.15.156:0/2561376392 >> 172.21.15.156:0/2561376392 conn(0x7f8604072600 msgr2=0x7f86040cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:42.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.704+0000 7f86089ff700 1 -- 172.21.15.156:0/2561376392 shutdown_connections 2023-11-28T13:10:42.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.704+0000 7f86089ff700 1 -- 172.21.15.156:0/2561376392 wait complete. 2023-11-28T13:10:42.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.705+0000 7f86089ff700 1 Processor -- start 2023-11-28T13:10:42.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.706+0000 7f86089ff700 1 -- start start 2023-11-28T13:10:42.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.706+0000 7f86089ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86040cd7c0 0x7f8604159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:42.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.706+0000 7f86089ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f860415c9b0 0x7f86040ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:42.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.706+0000 7f86089ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8604161340 0x7f8604153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:42.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.706+0000 7f86089ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86040cc660 con 0x7f860415c9b0 2023-11-28T13:10:42.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.706+0000 7f86089ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86040cd030 con 0x7f86040cd7c0 2023-11-28T13:10:42.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.706+0000 7f86089ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86040cb770 con 0x7f8604161340 2023-11-28T13:10:42.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.707+0000 7f85fbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f860415c9b0 0x7f86040ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:42.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.707+0000 7f85fbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f860415c9b0 0x7f86040ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42094/0 (socket says 172.21.15.156:42094) 2023-11-28T13:10:42.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.707+0000 7f85fbfff700 1 -- 172.21.15.156:0/1445462142 learned_addr learned my addr 172.21.15.156:0/1445462142 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:42.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.707+0000 7f860259c700 1 --2- 172.21.15.156:0/1445462142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86040cd7c0 0x7f8604159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:42.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.707+0000 7f8602d9d700 1 --2- 172.21.15.156:0/1445462142 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8604161340 0x7f8604153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:42.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.707+0000 7f85fbfff700 1 -- 172.21.15.156:0/1445462142 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8604161340 msgr2=0x7f8604153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:42.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.707+0000 7f85fbfff700 1 --2- 172.21.15.156:0/1445462142 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8604161340 0x7f8604153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:42.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.707+0000 7f85fbfff700 1 -- 172.21.15.156:0/1445462142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86040cd7c0 msgr2=0x7f8604159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:42.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.707+0000 7f85fbfff700 1 --2- 172.21.15.156:0/1445462142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86040cd7c0 0x7f8604159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:42.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.707+0000 7f85fbfff700 1 -- 172.21.15.156:0/1445462142 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc009b30 con 0x7f860415c9b0 2023-11-28T13:10:42.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.707+0000 7f85fbfff700 1 --2- 172.21.15.156:0/1445462142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f860415c9b0 0x7f86040ccaf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00c4d0 tx=0x7f85ec00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:42.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.708+0000 7f85faffd700 1 -- 172.21.15.156:0/1445462142 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec011790 con 0x7f860415c9b0 2023-11-28T13:10:42.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.708+0000 7f85faffd700 1 -- 172.21.15.156:0/1445462142 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ec0118f0 con 0x7f860415c9b0 2023-11-28T13:10:42.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.708+0000 7f86089ff700 1 -- 172.21.15.156:0/1445462142 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86040d1300 con 0x7f860415c9b0 2023-11-28T13:10:42.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.708+0000 7f85faffd700 1 -- 172.21.15.156:0/1445462142 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec011ca0 con 0x7f860415c9b0 2023-11-28T13:10:42.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.708+0000 7f86089ff700 1 -- 172.21.15.156:0/1445462142 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86040b5520 con 0x7f860415c9b0 2023-11-28T13:10:42.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.709+0000 7f86089ff700 1 -- 172.21.15.156:0/1445462142 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86040d15e0 con 0x7f860415c9b0 2023-11-28T13:10:42.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.710+0000 7f85faffd700 1 -- 172.21.15.156:0/1445462142 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f85ec010050 con 0x7f860415c9b0 2023-11-28T13:10:42.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.710+0000 7f85faffd700 1 --2- 172.21.15.156:0/1445462142 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f85e40682f0 0x7f85e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:42.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.710+0000 7f85faffd700 1 -- 172.21.15.156:0/1445462142 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f85ec085ac0 con 0x7f860415c9b0 2023-11-28T13:10:42.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.711+0000 7f860259c700 1 --2- 172.21.15.156:0/1445462142 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f85e40682f0 0x7f85e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:42.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.711+0000 7f860259c700 1 --2- 172.21.15.156:0/1445462142 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f85e40682f0 0x7f85e406a7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f85f4006fd0 tx=0x7f85f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:42.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.715+0000 7f85faffd700 1 -- 172.21.15.156:0/1445462142 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f85ec0521d0 con 0x7f860415c9b0 2023-11-28T13:10:42.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:42.950+0000 7f86089ff700 1 -- 172.21.15.156:0/1445462142 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f86040745c0 con 0x7f860415c9b0 2023-11-28T13:10:43.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.481+0000 7f85faffd700 1 -- 172.21.15.156:0/1445462142 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f85ec0557f0 con 0x7f860415c9b0 2023-11-28T13:10:43.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.484+0000 7f86089ff700 1 -- 172.21.15.156:0/1445462142 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f85e40682f0 msgr2=0x7f85e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:43.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.484+0000 7f86089ff700 1 --2- 172.21.15.156:0/1445462142 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f85e40682f0 0x7f85e406a7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f85f4006fd0 tx=0x7f85f4008040 comp rx=0 tx=0).stop 2023-11-28T13:10:43.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.484+0000 7f86089ff700 1 -- 172.21.15.156:0/1445462142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f860415c9b0 msgr2=0x7f86040ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:43.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.484+0000 7f86089ff700 1 --2- 172.21.15.156:0/1445462142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f860415c9b0 0x7f86040ccaf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00c4d0 tx=0x7f85ec00c890 comp rx=0 tx=0).stop 2023-11-28T13:10:43.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.485+0000 7f86089ff700 1 -- 172.21.15.156:0/1445462142 shutdown_connections 2023-11-28T13:10:43.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.485+0000 7f86089ff700 1 --2- 172.21.15.156:0/1445462142 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f85e40682f0 0x7f85e406a7a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:43.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.485+0000 7f86089ff700 1 --2- 172.21.15.156:0/1445462142 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8604161340 0x7f8604153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:43.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.485+0000 7f86089ff700 1 --2- 172.21.15.156:0/1445462142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86040cd7c0 0x7f8604159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:43.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.485+0000 7f86089ff700 1 --2- 172.21.15.156:0/1445462142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f860415c9b0 0x7f86040ccaf0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:43.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.485+0000 7f86089ff700 1 -- 172.21.15.156:0/1445462142 >> 172.21.15.156:0/1445462142 conn(0x7f8604072600 msgr2=0x7f860406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:43.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.485+0000 7f86089ff700 1 -- 172.21.15.156:0/1445462142 shutdown_connections 2023-11-28T13:10:43.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.485+0000 7f86089ff700 1 -- 172.21.15.156:0/1445462142 wait complete. 2023-11-28T13:10:43.505 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T13:10:43.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.745+0000 7f9c4d5ff700 1 Processor -- start 2023-11-28T13:10:43.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.745+0000 7f9c4d5ff700 1 -- start start 2023-11-28T13:10:43.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.746+0000 7f9c4d5ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9c480cded0 0x7f9c48161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:43.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.746+0000 7f9c4d5ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c48160d70 con 0x7f9c4815a760 2023-11-28T13:10:43.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.746+0000 7f9c4d5ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c480b5390 con 0x7f9c4815dc30 2023-11-28T13:10:43.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.746+0000 7f9c4d5ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4815e0d0 con 0x7f9c480cded0 2023-11-28T13:10:43.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.747+0000 7f9c46ffd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f9c4815a760 0x7f9c480cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37900/0 (socket says 172.21.15.156:37900) 2023-11-28T13:10:43.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.747+0000 7f9c46ffd700 1 -- 172.21.15.156:0/4210083974 learned_addr learned my addr 172.21.15.156:0/4210083974 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:43.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.747+0000 7f9c3ffff700 1 --2- 172.21.15.156:0/4210083974 >> v2:172.21.15.186:3300/0 conn(0x7f9c480cded0 0x7f9c48161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:43.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.747+0000 7f9c467fc700 1 -- 172.21.15.156:0/4210083974 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3202280247 0 0) 0x7f9c48160d70 con 0x7f9c4815a760 2023-11-28T13:10:43.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.747+0000 7f9c467fc700 1 -- 172.21.15.156:0/4210083974 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c340035b0 con 0x7f9c4815a760 2023-11-28T13:10:43.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.747+0000 7f9c467fc700 1 -- 172.21.15.156:0/4210083974 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111178738 0 0) 0x7f9c480b5390 con 0x7f9c4815dc30 2023-11-28T13:10:43.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.747+0000 7f9c467fc700 1 -- 172.21.15.156:0/4210083974 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c48160d70 con 0x7f9c4815dc30 2023-11-28T13:10:43.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.747+0000 7f9c467fc700 1 -- 172.21.15.156:0/4210083974 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3206992864 0 0) 0x7f9c340035b0 con 0x7f9c4815a760 2023-11-28T13:10:43.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.747+0000 7f9c467fc700 1 -- 172.21.15.156:0/4210083974 >> v2:172.21.15.186:3300/0 conn(0x7f9c480cded0 msgr2=0x7f9c48161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:43.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.747+0000 7f9c467fc700 1 --2- 172.21.15.156:0/4210083974 >> v2:172.21.15.186:3300/0 conn(0x7f9c480cded0 0x7f9c48161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:43.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.747+0000 7f9c467fc700 1 -- 172.21.15.156:0/4210083974 >> v1:172.21.15.186:6789/0 conn(0x7f9c4815dc30 legacy=0x7f9c480d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:43.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.747+0000 7f9c467fc700 1 -- 172.21.15.156:0/4210083974 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c480b5670 con 0x7f9c4815a760 2023-11-28T13:10:43.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.747+0000 7f9c467fc700 1 -- 172.21.15.156:0/4210083974 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f9c38001670 con 0x7f9c4815a760 2023-11-28T13:10:43.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.747+0000 7f9c467fc700 1 -- 172.21.15.156:0/4210083974 >> v1:172.21.15.156:6789/0 conn(0x7f9c4815a760 legacy=0x7f9c480cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:43.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.748+0000 7f9c467fc700 1 --2- 172.21.15.156:0/4210083974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c34003b90 0x7f9c34003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:43.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.748+0000 7f9c467fc700 1 --2- 172.21.15.156:0/4210083974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c34004850 0x7f9c34004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:43.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.748+0000 7f9c467fc700 1 --2- 172.21.15.156:0/4210083974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c34005200 0x7f9c3400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:43.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.748+0000 7f9c467fc700 1 -- 172.21.15.156:0/4210083974 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4815e0d0 con 0x7f9c34004850 2023-11-28T13:10:43.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.748+0000 7f9c467fc700 1 -- 172.21.15.156:0/4210083974 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3400c110 con 0x7f9c34003b90 2023-11-28T13:10:43.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.748+0000 7f9c467fc700 1 -- 172.21.15.156:0/4210083974 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3400c2d0 con 0x7f9c34005200 2023-11-28T13:10:43.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.748+0000 7f9c3ffff700 1 --2- 172.21.15.156:0/4210083974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c34004850 0x7f9c34004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:43.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.748+0000 7f9c46ffd700 1 --2- 172.21.15.156:0/4210083974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c34003b90 0x7f9c34003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:43.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.748+0000 7f9c477fe700 1 --2- 172.21.15.156:0/4210083974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c34005200 0x7f9c3400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:43.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.748+0000 7f9c3ffff700 1 -- 172.21.15.156:0/4210083974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c34005200 msgr2=0x7f9c3400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:43.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.748+0000 7f9c3ffff700 1 --2- 172.21.15.156:0/4210083974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c34005200 0x7f9c3400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:43.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.748+0000 7f9c3ffff700 1 -- 172.21.15.156:0/4210083974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c34003b90 msgr2=0x7f9c34003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:43.753 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.748+0000 7f9c3ffff700 1 --2- 172.21.15.156:0/4210083974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c34003b90 0x7f9c34003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:43.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.748+0000 7f9c3ffff700 1 -- 172.21.15.156:0/4210083974 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c340035b0 con 0x7f9c34004850 2023-11-28T13:10:43.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.748+0000 7f9c477fe700 1 --2- 172.21.15.156:0/4210083974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c34005200 0x7f9c3400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:43.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.748+0000 7f9c477fe700 1 -- 172.21.15.156:0/4210083974 reap_dead start 2023-11-28T13:10:43.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.748+0000 7f9c3ffff700 1 --2- 172.21.15.156:0/4210083974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c34004850 0x7f9c34004cc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9c480ce2c0 tx=0x7f9c34005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4189b23c680489d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:43.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.749+0000 7f9c467fc700 1 -- 172.21.15.156:0/4210083974 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000e6c0 con 0x7f9c34004850 2023-11-28T13:10:43.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.749+0000 7f9c467fc700 1 -- 172.21.15.156:0/4210083974 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3000e820 con 0x7f9c34004850 2023-11-28T13:10:43.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.749+0000 7f9c467fc700 1 -- 172.21.15.156:0/4210083974 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000ebb0 con 0x7f9c34004850 2023-11-28T13:10:43.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.749+0000 7f9c4d5ff700 1 -- 172.21.15.156:0/4210083974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c34004850 msgr2=0x7f9c34004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:43.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.749+0000 7f9c4d5ff700 1 --2- 172.21.15.156:0/4210083974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c34004850 0x7f9c34004cc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9c480ce2c0 tx=0x7f9c34005880 comp rx=0 tx=0).stop 2023-11-28T13:10:43.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.749+0000 7f9c4d5ff700 1 -- 172.21.15.156:0/4210083974 shutdown_connections 2023-11-28T13:10:43.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.749+0000 7f9c4d5ff700 1 --2- 172.21.15.156:0/4210083974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c34004850 0x7f9c34004cc0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:43.755 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.749+0000 7f9c4d5ff700 1 -- 172.21.15.156:0/4210083974 >> 172.21.15.156:0/4210083974 conn(0x7f9c48072620 msgr2=0x7f9c480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:43.755 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.749+0000 7f9c4d5ff700 1 -- 172.21.15.156:0/4210083974 shutdown_connections 2023-11-28T13:10:43.755 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.749+0000 7f9c4d5ff700 1 -- 172.21.15.156:0/4210083974 wait complete. 2023-11-28T13:10:43.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.751+0000 7f9c4d5ff700 1 Processor -- start 2023-11-28T13:10:43.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.751+0000 7f9c4d5ff700 1 -- start start 2023-11-28T13:10:43.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.752+0000 7f9c4d5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c34004850 0x7f9c480a9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:43.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.752+0000 7f9c4d5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c481f6360 0x7f9c4814cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:43.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.752+0000 7f9c4d5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c481580a0 0x7f9c48158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:43.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.752+0000 7f9c4d5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c48158a60 con 0x7f9c481f6360 2023-11-28T13:10:43.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.752+0000 7f9c4d5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c480cb980 con 0x7f9c481580a0 2023-11-28T13:10:43.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.752+0000 7f9c4d5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c480ab250 con 0x7f9c34004850 2023-11-28T13:10:43.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.752+0000 7f9c3ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c481f6360 0x7f9c4814cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:43.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.752+0000 7f9c46ffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c34004850 0x7f9c480a9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:43.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.753+0000 7f9c3ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c481f6360 0x7f9c4814cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42108/0 (socket says 172.21.15.156:42108) 2023-11-28T13:10:43.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.753+0000 7f9c46ffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c34004850 0x7f9c480a9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:37810/0 (socket says 172.21.15.156:37810) 2023-11-28T13:10:43.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.753+0000 7f9c3ffff700 1 -- 172.21.15.156:0/1283278922 learned_addr learned my addr 172.21.15.156:0/1283278922 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:43.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.753+0000 7f9c477fe700 1 --2- 172.21.15.156:0/1283278922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c481580a0 0x7f9c48158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:43.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.753+0000 7f9c3ffff700 1 -- 172.21.15.156:0/1283278922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c34004850 msgr2=0x7f9c480a9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:43.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.753+0000 7f9c3ffff700 1 --2- 172.21.15.156:0/1283278922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c34004850 0x7f9c480a9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:43.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.753+0000 7f9c3ffff700 1 -- 172.21.15.156:0/1283278922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c481580a0 msgr2=0x7f9c48158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:43.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.753+0000 7f9c3ffff700 1 --2- 172.21.15.156:0/1283278922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c481580a0 0x7f9c48158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:43.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.753+0000 7f9c3ffff700 1 -- 172.21.15.156:0/1283278922 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c48160a00 con 0x7f9c481f6360 2023-11-28T13:10:43.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.753+0000 7f9c3ffff700 1 --2- 172.21.15.156:0/1283278922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c481f6360 0x7f9c4814cf30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400c0e0 tx=0x7f9c300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:43.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.753+0000 7f9c3f7fe700 1 -- 172.21.15.156:0/1283278922 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000e6c0 con 0x7f9c481f6360 2023-11-28T13:10:43.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.753+0000 7f9c3f7fe700 1 -- 172.21.15.156:0/1283278922 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c300038c0 con 0x7f9c481f6360 2023-11-28T13:10:43.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.753+0000 7f9c3f7fe700 1 -- 172.21.15.156:0/1283278922 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c30006e60 con 0x7f9c481f6360 2023-11-28T13:10:43.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.753+0000 7f9c4d5ff700 1 -- 172.21.15.156:0/1283278922 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c48160720 con 0x7f9c481f6360 2023-11-28T13:10:43.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.754+0000 7f9c4d5ff700 1 -- 172.21.15.156:0/1283278922 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c48161200 con 0x7f9c481f6360 2023-11-28T13:10:43.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.755+0000 7f9c4d5ff700 1 -- 172.21.15.156:0/1283278922 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c480cbb30 con 0x7f9c481f6360 2023-11-28T13:10:43.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.756+0000 7f9c3f7fe700 1 -- 172.21.15.156:0/1283278922 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9c30019030 con 0x7f9c481f6360 2023-11-28T13:10:43.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.757+0000 7f9c3f7fe700 1 --2- 172.21.15.156:0/1283278922 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9c280682f0 0x7f9c2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:43.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.757+0000 7f9c3f7fe700 1 -- 172.21.15.156:0/1283278922 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f9c30087190 con 0x7f9c481f6360 2023-11-28T13:10:43.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.757+0000 7f9c46ffd700 1 --2- 172.21.15.156:0/1283278922 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9c280682f0 0x7f9c2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:43.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.758+0000 7f9c46ffd700 1 --2- 172.21.15.156:0/1283278922 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9c280682f0 0x7f9c2806a7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9c48063f40 tx=0x7f9c38007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:43.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.761+0000 7f9c3f7fe700 1 -- 172.21.15.156:0/1283278922 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9c480cbb30 con 0x7f9c481f6360 2023-11-28T13:10:43.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:43.950+0000 7f9c4d5ff700 1 -- 172.21.15.156:0/1283278922 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9c480cbaf0 con 0x7f9c481f6360 2023-11-28T13:10:44.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.484+0000 7f9c3f7fe700 1 -- 172.21.15.156:0/1283278922 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9c30053990 con 0x7f9c481f6360 2023-11-28T13:10:44.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.489+0000 7f9c4d5ff700 1 -- 172.21.15.156:0/1283278922 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9c280682f0 msgr2=0x7f9c2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:44.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.489+0000 7f9c4d5ff700 1 --2- 172.21.15.156:0/1283278922 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9c280682f0 0x7f9c2806a7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9c48063f40 tx=0x7f9c38007040 comp rx=0 tx=0).stop 2023-11-28T13:10:44.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.489+0000 7f9c4d5ff700 1 -- 172.21.15.156:0/1283278922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c481f6360 msgr2=0x7f9c4814cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:44.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.489+0000 7f9c4d5ff700 1 --2- 172.21.15.156:0/1283278922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c481f6360 0x7f9c4814cf30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400c0e0 tx=0x7f9c300047c0 comp rx=0 tx=0).stop 2023-11-28T13:10:44.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.489+0000 7f9c4d5ff700 1 -- 172.21.15.156:0/1283278922 shutdown_connections 2023-11-28T13:10:44.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.489+0000 7f9c4d5ff700 1 --2- 172.21.15.156:0/1283278922 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9c280682f0 0x7f9c2806a7a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:44.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.489+0000 7f9c4d5ff700 1 --2- 172.21.15.156:0/1283278922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c481580a0 0x7f9c48158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:44.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.489+0000 7f9c4d5ff700 1 --2- 172.21.15.156:0/1283278922 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9c34004850 0x7f9c480a9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:44.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.489+0000 7f9c4d5ff700 1 --2- 172.21.15.156:0/1283278922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c481f6360 0x7f9c4814cf30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:44.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.489+0000 7f9c4d5ff700 1 -- 172.21.15.156:0/1283278922 >> 172.21.15.156:0/1283278922 conn(0x7f9c48072620 msgr2=0x7f9c4815ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:44.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.489+0000 7f9c4d5ff700 1 -- 172.21.15.156:0/1283278922 shutdown_connections 2023-11-28T13:10:44.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.490+0000 7f9c4d5ff700 1 -- 172.21.15.156:0/1283278922 wait complete. 2023-11-28T13:10:44.493 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_metadata' removed 2023-11-28T13:10:44.512 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T13:10:44.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.719+0000 7f0a991ff700 1 Processor -- start 2023-11-28T13:10:44.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.719+0000 7f0a991ff700 1 -- start start 2023-11-28T13:10:44.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.720+0000 7f0a991ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0a9415a760 0x7f0a94161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:44.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.720+0000 7f0a991ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a9415dc30 0x7f0a940b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:44.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.720+0000 7f0a991ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0a940cd7c0 0x7f0a940cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:44.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.720+0000 7f0a991ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9415ab30 con 0x7f0a9415a760 2023-11-28T13:10:44.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.720+0000 7f0a991ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9415e0a0 con 0x7f0a940cd7c0 2023-11-28T13:10:44.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.720+0000 7f0a991ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a940ce150 con 0x7f0a9415dc30 2023-11-28T13:10:44.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.721+0000 7f0a92d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0a9415a760 0x7f0a94161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:44.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.721+0000 7f0a92d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0a9415a760 0x7f0a94161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42112/0 (socket says 172.21.15.156:42112) 2023-11-28T13:10:44.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.721+0000 7f0a92d9d700 1 -- 172.21.15.156:0/673330501 learned_addr learned my addr 172.21.15.156:0/673330501 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:44.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.721+0000 7f0a8bfff700 1 --2- 172.21.15.156:0/673330501 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a9415dc30 0x7f0a940b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:44.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.721+0000 7f0a9359e700 1 --2- 172.21.15.156:0/673330501 >> v2:172.21.15.186:3300/0 conn(0x7f0a940cd7c0 0x7f0a940cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:44.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.721+0000 7f0a92d9d700 1 -- 172.21.15.156:0/673330501 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a9415dc30 msgr2=0x7f0a940b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:44.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.721+0000 7f0a92d9d700 1 --2- 172.21.15.156:0/673330501 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a9415dc30 0x7f0a940b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:44.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.721+0000 7f0a92d9d700 1 -- 172.21.15.156:0/673330501 >> v2:172.21.15.186:3300/0 conn(0x7f0a940cd7c0 msgr2=0x7f0a940cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:44.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.721+0000 7f0a92d9d700 1 --2- 172.21.15.156:0/673330501 >> v2:172.21.15.186:3300/0 conn(0x7f0a940cd7c0 0x7f0a940cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:44.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.721+0000 7f0a92d9d700 1 -- 172.21.15.156:0/673330501 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a94160d70 con 0x7f0a9415a760 2023-11-28T13:10:44.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.722+0000 7f0a92d9d700 1 --2- 172.21.15.156:0/673330501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9415a760 0x7f0a94161200 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0a940b5d60 tx=0x7f0a84009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f593a4c98a1579a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:44.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.722+0000 7f0a9259c700 1 -- 172.21.15.156:0/673330501 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84011830 con 0x7f0a9415a760 2023-11-28T13:10:44.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.722+0000 7f0a9259c700 1 -- 172.21.15.156:0/673330501 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a84011990 con 0x7f0a9415a760 2023-11-28T13:10:44.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.722+0000 7f0a9259c700 1 -- 172.21.15.156:0/673330501 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84011d20 con 0x7f0a9415a760 2023-11-28T13:10:44.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.722+0000 7f0a991ff700 1 -- 172.21.15.156:0/673330501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9415a760 msgr2=0x7f0a94161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:44.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.722+0000 7f0a991ff700 1 --2- 172.21.15.156:0/673330501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9415a760 0x7f0a94161200 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0a940b5d60 tx=0x7f0a84009c00 comp rx=0 tx=0).stop 2023-11-28T13:10:44.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.722+0000 7f0a991ff700 1 -- 172.21.15.156:0/673330501 shutdown_connections 2023-11-28T13:10:44.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.722+0000 7f0a991ff700 1 --2- 172.21.15.156:0/673330501 >> v2:172.21.15.186:3300/0 conn(0x7f0a940cd7c0 0x7f0a940cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:44.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.722+0000 7f0a991ff700 1 --2- 172.21.15.156:0/673330501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9415a760 0x7f0a94161200 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:44.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.722+0000 7f0a991ff700 1 --2- 172.21.15.156:0/673330501 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a9415dc30 0x7f0a940b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:44.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.722+0000 7f0a991ff700 1 -- 172.21.15.156:0/673330501 >> 172.21.15.156:0/673330501 conn(0x7f0a94072620 msgr2=0x7f0a940cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:44.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.722+0000 7f0a991ff700 1 -- 172.21.15.156:0/673330501 shutdown_connections 2023-11-28T13:10:44.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.723+0000 7f0a991ff700 1 -- 172.21.15.156:0/673330501 wait complete. 2023-11-28T13:10:44.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.724+0000 7f0a991ff700 1 Processor -- start 2023-11-28T13:10:44.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.724+0000 7f0a991ff700 1 -- start start 2023-11-28T13:10:44.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.725+0000 7f0a991ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a940cd7c0 0x7f0a941536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:44.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.725+0000 7f0a991ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9415a760 0x7f0a941f2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:44.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.725+0000 7f0a991ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a9415dc30 0x7f0a94156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:44.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.725+0000 7f0a991ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a941f33d0 con 0x7f0a9415a760 2023-11-28T13:10:44.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.725+0000 7f0a991ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a94157d20 con 0x7f0a940cd7c0 2023-11-28T13:10:44.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.725+0000 7f0a991ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a940b6d30 con 0x7f0a9415dc30 2023-11-28T13:10:44.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.726+0000 7f0a8bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9415a760 0x7f0a941f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:44.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.726+0000 7f0a8bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9415a760 0x7f0a941f2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42118/0 (socket says 172.21.15.156:42118) 2023-11-28T13:10:44.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.726+0000 7f0a8bfff700 1 -- 172.21.15.156:0/706476938 learned_addr learned my addr 172.21.15.156:0/706476938 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:44.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.726+0000 7f0a92d9d700 1 --2- 172.21.15.156:0/706476938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a940cd7c0 0x7f0a941536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:44.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.726+0000 7f0a9359e700 1 --2- 172.21.15.156:0/706476938 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a9415dc30 0x7f0a94156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:44.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.726+0000 7f0a8bfff700 1 -- 172.21.15.156:0/706476938 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a9415dc30 msgr2=0x7f0a94156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:44.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.726+0000 7f0a8bfff700 1 --2- 172.21.15.156:0/706476938 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a9415dc30 0x7f0a94156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:44.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.726+0000 7f0a8bfff700 1 -- 172.21.15.156:0/706476938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a940cd7c0 msgr2=0x7f0a941536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:44.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.726+0000 7f0a8bfff700 1 --2- 172.21.15.156:0/706476938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a940cd7c0 0x7f0a941536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:44.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.726+0000 7f0a8bfff700 1 -- 172.21.15.156:0/706476938 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a84009550 con 0x7f0a9415a760 2023-11-28T13:10:44.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.726+0000 7f0a8bfff700 1 --2- 172.21.15.156:0/706476938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9415a760 0x7f0a941f2490 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00e870 tx=0x7f0a7c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:44.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.726+0000 7f0a8b7fe700 1 -- 172.21.15.156:0/706476938 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c009590 con 0x7f0a9415a760 2023-11-28T13:10:44.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.727+0000 7f0a8b7fe700 1 -- 172.21.15.156:0/706476938 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7c013680 con 0x7f0a9415a760 2023-11-28T13:10:44.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.727+0000 7f0a8b7fe700 1 -- 172.21.15.156:0/706476938 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c0139f0 con 0x7f0a9415a760 2023-11-28T13:10:44.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.727+0000 7f0a991ff700 1 -- 172.21.15.156:0/706476938 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a940b5390 con 0x7f0a9415a760 2023-11-28T13:10:44.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.727+0000 7f0a991ff700 1 -- 172.21.15.156:0/706476938 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a940b5130 con 0x7f0a9415a760 2023-11-28T13:10:44.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.728+0000 7f0a991ff700 1 -- 172.21.15.156:0/706476938 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a940b5670 con 0x7f0a9415a760 2023-11-28T13:10:44.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.729+0000 7f0a8b7fe700 1 -- 172.21.15.156:0/706476938 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0a7c019030 con 0x7f0a9415a760 2023-11-28T13:10:44.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.729+0000 7f0a8b7fe700 1 --2- 172.21.15.156:0/706476938 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0a74068320 0x7f0a7406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:44.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.729+0000 7f0a8b7fe700 1 -- 172.21.15.156:0/706476938 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f0a7c0857e0 con 0x7f0a9415a760 2023-11-28T13:10:44.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.730+0000 7f0a92d9d700 1 --2- 172.21.15.156:0/706476938 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0a74068320 0x7f0a7406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:44.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.731+0000 7f0a92d9d700 1 --2- 172.21.15.156:0/706476938 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0a74068320 0x7f0a7406a7d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0a840098f0 tx=0x7f0a84008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:44.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.733+0000 7f0a8b7fe700 1 -- 172.21.15.156:0/706476938 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0a7c0520b0 con 0x7f0a9415a760 2023-11-28T13:10:44.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:44.923+0000 7f0a991ff700 1 -- 172.21.15.156:0/706476938 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0a94157e90 con 0x7f0a9415a760 2023-11-28T13:10:45.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.491+0000 7f0a8b7fe700 1 -- 172.21.15.156:0/706476938 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0a7c0556d0 con 0x7f0a9415a760 2023-11-28T13:10:45.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.495+0000 7f0a991ff700 1 -- 172.21.15.156:0/706476938 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0a74068320 msgr2=0x7f0a7406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:45.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.495+0000 7f0a991ff700 1 --2- 172.21.15.156:0/706476938 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0a74068320 0x7f0a7406a7d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0a840098f0 tx=0x7f0a84008030 comp rx=0 tx=0).stop 2023-11-28T13:10:45.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.495+0000 7f0a991ff700 1 -- 172.21.15.156:0/706476938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9415a760 msgr2=0x7f0a941f2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:45.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.495+0000 7f0a991ff700 1 --2- 172.21.15.156:0/706476938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9415a760 0x7f0a941f2490 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00e870 tx=0x7f0a7c00eb80 comp rx=0 tx=0).stop 2023-11-28T13:10:45.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.495+0000 7f0a991ff700 1 -- 172.21.15.156:0/706476938 shutdown_connections 2023-11-28T13:10:45.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.495+0000 7f0a991ff700 1 --2- 172.21.15.156:0/706476938 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0a74068320 0x7f0a7406a7d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:45.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.495+0000 7f0a991ff700 1 --2- 172.21.15.156:0/706476938 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a9415dc30 0x7f0a94156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:45.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.495+0000 7f0a991ff700 1 --2- 172.21.15.156:0/706476938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a940cd7c0 0x7f0a941536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:45.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.495+0000 7f0a991ff700 1 --2- 172.21.15.156:0/706476938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a9415a760 0x7f0a941f2490 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:45.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.495+0000 7f0a991ff700 1 -- 172.21.15.156:0/706476938 >> 172.21.15.156:0/706476938 conn(0x7f0a94072620 msgr2=0x7f0a940b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:45.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.495+0000 7f0a991ff700 1 -- 172.21.15.156:0/706476938 shutdown_connections 2023-11-28T13:10:45.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.495+0000 7f0a991ff700 1 -- 172.21.15.156:0/706476938 wait complete. 2023-11-28T13:10:45.498 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_data' removed 2023-11-28T13:10:45.518 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T13:10:45.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.716+0000 7f97b79ff700 1 Processor -- start 2023-11-28T13:10:45.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.717+0000 7f97b79ff700 1 -- start start 2023-11-28T13:10:45.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.718+0000 7f97b79ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97b015c9b0 0x7f97b00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:45.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.718+0000 7f97b79ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b00d1950 con 0x7f97b0161340 2023-11-28T13:10:45.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.718+0000 7f97b79ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b00d24f0 con 0x7f97b00cd7c0 2023-11-28T13:10:45.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.718+0000 7f97b79ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b00b5800 con 0x7f97b015c9b0 2023-11-28T13:10:45.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.718+0000 7f97b4f9a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f97b0161340 0x7f97b00d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37904/0 (socket says 172.21.15.156:37904) 2023-11-28T13:10:45.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.718+0000 7f97b4f9a700 1 -- 172.21.15.156:0/3990191599 learned_addr learned my addr 172.21.15.156:0/3990191599 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:45.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.718+0000 7f97b579b700 1 --2- 172.21.15.156:0/3990191599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97b015c9b0 0x7f97b00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:45.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.718+0000 7f97a7fff700 1 -- 172.21.15.156:0/3990191599 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643331157 0 0) 0x7f97b00d1950 con 0x7f97b0161340 2023-11-28T13:10:45.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.718+0000 7f97a7fff700 1 -- 172.21.15.156:0/3990191599 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f979c0035b0 con 0x7f97b0161340 2023-11-28T13:10:45.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.719+0000 7f97a7fff700 1 -- 172.21.15.156:0/3990191599 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 894670334 0 0) 0x7f97b00d24f0 con 0x7f97b00cd7c0 2023-11-28T13:10:45.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.719+0000 7f97a7fff700 1 -- 172.21.15.156:0/3990191599 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97b00d1950 con 0x7f97b00cd7c0 2023-11-28T13:10:45.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.719+0000 7f97a7fff700 1 -- 172.21.15.156:0/3990191599 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3655297790 0 0) 0x7f979c0035b0 con 0x7f97b0161340 2023-11-28T13:10:45.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.719+0000 7f97b579b700 1 -- 172.21.15.156:0/3990191599 >> v1:172.21.15.186:6789/0 conn(0x7f97b00cd7c0 legacy=0x7f97b00b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:45.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.719+0000 7f97b579b700 1 -- 172.21.15.156:0/3990191599 >> v1:172.21.15.156:6789/0 conn(0x7f97b0161340 legacy=0x7f97b00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:45.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.719+0000 7f97b579b700 1 -- 172.21.15.156:0/3990191599 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b00b5520 con 0x7f97b015c9b0 2023-11-28T13:10:45.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.719+0000 7f97b579b700 1 --2- 172.21.15.156:0/3990191599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97b015c9b0 0x7f97b00b4bf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009870 tx=0x7f97ac009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c04ffed1868832d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:45.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.719+0000 7f97a7fff700 1 -- 172.21.15.156:0/3990191599 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0001670 con 0x7f97b015c9b0 2023-11-28T13:10:45.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.719+0000 7f97a7fff700 1 -- 172.21.15.156:0/3990191599 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ac004030 con 0x7f97b015c9b0 2023-11-28T13:10:45.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.719+0000 7f97a7fff700 1 -- 172.21.15.156:0/3990191599 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac0043a0 con 0x7f97b015c9b0 2023-11-28T13:10:45.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.720+0000 7f97b79ff700 1 -- 172.21.15.156:0/3990191599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97b015c9b0 msgr2=0x7f97b00b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:45.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.720+0000 7f97b79ff700 1 --2- 172.21.15.156:0/3990191599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97b015c9b0 0x7f97b00b4bf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009870 tx=0x7f97ac009b80 comp rx=0 tx=0).stop 2023-11-28T13:10:45.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.720+0000 7f97b79ff700 1 -- 172.21.15.156:0/3990191599 shutdown_connections 2023-11-28T13:10:45.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.720+0000 7f97b79ff700 1 --2- 172.21.15.156:0/3990191599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97b015c9b0 0x7f97b00b4bf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:45.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.720+0000 7f97b79ff700 1 -- 172.21.15.156:0/3990191599 >> 172.21.15.156:0/3990191599 conn(0x7f97b0072600 msgr2=0x7f97b00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:45.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.720+0000 7f97b79ff700 1 -- 172.21.15.156:0/3990191599 shutdown_connections 2023-11-28T13:10:45.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.720+0000 7f97b79ff700 1 -- 172.21.15.156:0/3990191599 wait complete. 2023-11-28T13:10:45.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.722+0000 7f97b79ff700 1 Processor -- start 2023-11-28T13:10:45.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.723+0000 7f97b79ff700 1 -- start start 2023-11-28T13:10:45.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.724+0000 7f97b79ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97b00cd7c0 0x7f97b014a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:45.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.724+0000 7f97b79ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97b015c9b0 0x7f97b01e6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:45.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.724+0000 7f97b79ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97b0161340 0x7f97b015ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:45.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.724+0000 7f97b79ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b01f3e20 con 0x7f97b00cd7c0 2023-11-28T13:10:45.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.724+0000 7f97b79ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b01f4650 con 0x7f97b0161340 2023-11-28T13:10:45.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.724+0000 7f97b79ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b01e7630 con 0x7f97b015c9b0 2023-11-28T13:10:45.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.724+0000 7f97b579b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97b00cd7c0 0x7f97b014a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:45.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.724+0000 7f97b579b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97b00cd7c0 0x7f97b014a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42128/0 (socket says 172.21.15.156:42128) 2023-11-28T13:10:45.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.724+0000 7f97b579b700 1 -- 172.21.15.156:0/3206348930 learned_addr learned my addr 172.21.15.156:0/3206348930 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:45.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.724+0000 7f97b579b700 1 -- 172.21.15.156:0/3206348930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97b015c9b0 msgr2=0x7f97b01e6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:45.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.724+0000 7f97b579b700 1 --2- 172.21.15.156:0/3206348930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97b015c9b0 0x7f97b01e6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:45.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.724+0000 7f97b579b700 1 -- 172.21.15.156:0/3206348930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97b0161340 msgr2=0x7f97b015ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:45.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.724+0000 7f97b579b700 1 --2- 172.21.15.156:0/3206348930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97b0161340 0x7f97b015ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:45.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.724+0000 7f97b579b700 1 -- 172.21.15.156:0/3206348930 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac0094a0 con 0x7f97b00cd7c0 2023-11-28T13:10:45.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.725+0000 7f97b579b700 1 --2- 172.21.15.156:0/3206348930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97b00cd7c0 0x7f97b014a4f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f97b0067550 tx=0x7f97ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:45.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.725+0000 7f97a5ffb700 1 -- 172.21.15.156:0/3206348930 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac003bb0 con 0x7f97b00cd7c0 2023-11-28T13:10:45.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.725+0000 7f97a5ffb700 1 -- 172.21.15.156:0/3206348930 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ac01f070 con 0x7f97b00cd7c0 2023-11-28T13:10:45.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.725+0000 7f97b79ff700 1 -- 172.21.15.156:0/3206348930 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b00d15e0 con 0x7f97b00cd7c0 2023-11-28T13:10:45.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.725+0000 7f97a5ffb700 1 -- 172.21.15.156:0/3206348930 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac011630 con 0x7f97b00cd7c0 2023-11-28T13:10:45.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.725+0000 7f97b79ff700 1 -- 172.21.15.156:0/3206348930 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b00d1300 con 0x7f97b00cd7c0 2023-11-28T13:10:45.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.726+0000 7f97b79ff700 1 -- 172.21.15.156:0/3206348930 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b0160b70 con 0x7f97b00cd7c0 2023-11-28T13:10:45.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.727+0000 7f97a5ffb700 1 -- 172.21.15.156:0/3206348930 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f97ac02a020 con 0x7f97b00cd7c0 2023-11-28T13:10:45.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.728+0000 7f97a5ffb700 1 --2- 172.21.15.156:0/3206348930 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f97940682f0 0x7f979406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:45.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.728+0000 7f97a5ffb700 1 -- 172.21.15.156:0/3206348930 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f97ac086f40 con 0x7f97b00cd7c0 2023-11-28T13:10:45.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.728+0000 7f97b4f9a700 1 --2- 172.21.15.156:0/3206348930 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f97940682f0 0x7f979406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:45.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.729+0000 7f97b4f9a700 1 --2- 172.21.15.156:0/3206348930 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f97940682f0 0x7f979406a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f97b015e9c0 tx=0x7f97a0001230 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:45.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.732+0000 7f97a5ffb700 1 -- 172.21.15.156:0/3206348930 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f97ac0539d0 con 0x7f97b00cd7c0 2023-11-28T13:10:45.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.925+0000 7f97b79ff700 1 -- 172.21.15.156:0/3206348930 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f97b00745c0 con 0x7f97b00cd7c0 2023-11-28T13:10:45.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.930+0000 7f97a5ffb700 1 -- 172.21.15.156:0/3206348930 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f97ac056ff0 con 0x7f97b00cd7c0 2023-11-28T13:10:45.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.932+0000 7f97b79ff700 1 -- 172.21.15.156:0/3206348930 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f97940682f0 msgr2=0x7f979406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:45.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.932+0000 7f97b79ff700 1 --2- 172.21.15.156:0/3206348930 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f97940682f0 0x7f979406a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f97b015e9c0 tx=0x7f97a0001230 comp rx=0 tx=0).stop 2023-11-28T13:10:45.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.932+0000 7f97b79ff700 1 -- 172.21.15.156:0/3206348930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97b00cd7c0 msgr2=0x7f97b014a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:45.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.932+0000 7f97b79ff700 1 --2- 172.21.15.156:0/3206348930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97b00cd7c0 0x7f97b014a4f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f97b0067550 tx=0x7f97ac004750 comp rx=0 tx=0).stop 2023-11-28T13:10:45.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.932+0000 7f97b79ff700 1 -- 172.21.15.156:0/3206348930 shutdown_connections 2023-11-28T13:10:45.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.932+0000 7f97b79ff700 1 --2- 172.21.15.156:0/3206348930 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f97940682f0 0x7f979406a7a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:45.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.932+0000 7f97b79ff700 1 --2- 172.21.15.156:0/3206348930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97b0161340 0x7f97b015ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:45.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.932+0000 7f97b79ff700 1 --2- 172.21.15.156:0/3206348930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97b00cd7c0 0x7f97b014a4f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:45.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.932+0000 7f97b79ff700 1 --2- 172.21.15.156:0/3206348930 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f97b015c9b0 0x7f97b01e6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:45.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.932+0000 7f97b79ff700 1 -- 172.21.15.156:0/3206348930 >> 172.21.15.156:0/3206348930 conn(0x7f97b0072600 msgr2=0x7f97b0160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:45.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.932+0000 7f97b79ff700 1 -- 172.21.15.156:0/3206348930 shutdown_connections 2023-11-28T13:10:45.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:45.932+0000 7f97b79ff700 1 -- 172.21.15.156:0/3206348930 wait complete. 2023-11-28T13:10:45.960 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T13:10:46.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.158+0000 7fdaf61ff700 1 Processor -- start 2023-11-28T13:10:46.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.158+0000 7fdaf61ff700 1 -- start start 2023-11-28T13:10:46.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.159+0000 7fdaf61ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fdaf015c9b0 0x7fdaf00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:46.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.159+0000 7fdaf61ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf00d1950 con 0x7fdaf0161340 2023-11-28T13:10:46.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.159+0000 7fdaf61ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf00d24f0 con 0x7fdaf00cd7c0 2023-11-28T13:10:46.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.159+0000 7fdaf61ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf00b5800 con 0x7fdaf015c9b0 2023-11-28T13:10:46.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.159+0000 7fdae7fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fdaf0161340 0x7fdaf00d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37750/0 (socket says 172.21.15.156:37750) 2023-11-28T13:10:46.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.159+0000 7fdae7fff700 1 -- 172.21.15.156:0/96363946 learned_addr learned my addr 172.21.15.156:0/96363946 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:46.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.159+0000 7fdaef7fe700 1 --2- 172.21.15.156:0/96363946 >> v2:172.21.15.186:3300/0 conn(0x7fdaf015c9b0 0x7fdaf00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:46.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.159+0000 7fdaeeffd700 1 -- 172.21.15.156:0/96363946 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128848797 0 0) 0x7fdaf00d1950 con 0x7fdaf0161340 2023-11-28T13:10:46.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.159+0000 7fdaeeffd700 1 -- 172.21.15.156:0/96363946 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdadc0035b0 con 0x7fdaf0161340 2023-11-28T13:10:46.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.160+0000 7fdaeeffd700 1 -- 172.21.15.156:0/96363946 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1949379113 0 0) 0x7fdadc0035b0 con 0x7fdaf0161340 2023-11-28T13:10:46.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.160+0000 7fdaeeffd700 1 -- 172.21.15.156:0/96363946 >> v2:172.21.15.186:3300/0 conn(0x7fdaf015c9b0 msgr2=0x7fdaf00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:46.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.160+0000 7fdaeeffd700 1 --2- 172.21.15.156:0/96363946 >> v2:172.21.15.186:3300/0 conn(0x7fdaf015c9b0 0x7fdaf00b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:46.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.160+0000 7fdaeeffd700 1 -- 172.21.15.156:0/96363946 >> v1:172.21.15.186:6789/0 conn(0x7fdaf00cd7c0 legacy=0x7fdaf00b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:46.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.160+0000 7fdaeeffd700 1 -- 172.21.15.156:0/96363946 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf00b5520 con 0x7fdaf0161340 2023-11-28T13:10:46.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.160+0000 7fdaeeffd700 1 -- 172.21.15.156:0/96363946 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7fdad8001670 con 0x7fdaf0161340 2023-11-28T13:10:46.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.160+0000 7fdaeeffd700 1 -- 172.21.15.156:0/96363946 >> v1:172.21.15.156:6789/0 conn(0x7fdaf0161340 legacy=0x7fdaf00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:46.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.160+0000 7fdaeeffd700 1 --2- 172.21.15.156:0/96363946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdadc003a00 0x7fdadc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:46.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.160+0000 7fdaeeffd700 1 --2- 172.21.15.156:0/96363946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdadc004850 0x7fdadc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:46.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.160+0000 7fdaeeffd700 1 --2- 172.21.15.156:0/96363946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdadc005200 0x7fdadc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:46.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.160+0000 7fdaeeffd700 1 -- 172.21.15.156:0/96363946 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf00b5800 con 0x7fdadc005200 2023-11-28T13:10:46.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.160+0000 7fdaeeffd700 1 -- 172.21.15.156:0/96363946 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc00c110 con 0x7fdadc003a00 2023-11-28T13:10:46.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.160+0000 7fdaeeffd700 1 -- 172.21.15.156:0/96363946 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc00c2d0 con 0x7fdadc004850 2023-11-28T13:10:46.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.160+0000 7fdaef7fe700 1 --2- 172.21.15.156:0/96363946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdadc003a00 0x7fdadc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:46.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.160+0000 7fdaeffff700 1 --2- 172.21.15.156:0/96363946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdadc005200 0x7fdadc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:46.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.160+0000 7fdae7fff700 1 --2- 172.21.15.156:0/96363946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdadc004850 0x7fdadc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:46.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.161+0000 7fdaef7fe700 1 -- 172.21.15.156:0/96363946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdadc004850 msgr2=0x7fdadc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:46.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.161+0000 7fdaef7fe700 1 --2- 172.21.15.156:0/96363946 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdadc004850 0x7fdadc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:46.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.161+0000 7fdaef7fe700 1 -- 172.21.15.156:0/96363946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdadc005200 msgr2=0x7fdadc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:46.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.161+0000 7fdaef7fe700 1 --2- 172.21.15.156:0/96363946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdadc005200 0x7fdadc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:46.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.161+0000 7fdaef7fe700 1 -- 172.21.15.156:0/96363946 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc0035b0 con 0x7fdadc003a00 2023-11-28T13:10:46.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.161+0000 7fdaeffff700 1 --2- 172.21.15.156:0/96363946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdadc005200 0x7fdadc00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:10:46.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.161+0000 7fdaeffff700 1 -- 172.21.15.156:0/96363946 reap_dead start 2023-11-28T13:10:46.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.161+0000 7fdaef7fe700 1 --2- 172.21.15.156:0/96363946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdadc003a00 0x7fdadc003e10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009b60 tx=0x7fdadc005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeebd60fbebb202 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:46.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.161+0000 7fdaeeffd700 1 -- 172.21.15.156:0/96363946 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae000e6c0 con 0x7fdadc003a00 2023-11-28T13:10:46.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.161+0000 7fdaeeffd700 1 -- 172.21.15.156:0/96363946 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae000e820 con 0x7fdadc003a00 2023-11-28T13:10:46.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.161+0000 7fdaeeffd700 1 -- 172.21.15.156:0/96363946 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae000eb90 con 0x7fdadc003a00 2023-11-28T13:10:46.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.162+0000 7fdaf61ff700 1 -- 172.21.15.156:0/96363946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdadc003a00 msgr2=0x7fdadc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:46.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.162+0000 7fdaf61ff700 1 --2- 172.21.15.156:0/96363946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdadc003a00 0x7fdadc003e10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009b60 tx=0x7fdadc005880 comp rx=0 tx=0).stop 2023-11-28T13:10:46.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.162+0000 7fdaf61ff700 1 -- 172.21.15.156:0/96363946 shutdown_connections 2023-11-28T13:10:46.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.162+0000 7fdaf61ff700 1 --2- 172.21.15.156:0/96363946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdadc003a00 0x7fdadc003e10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:46.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.162+0000 7fdaf61ff700 1 -- 172.21.15.156:0/96363946 >> 172.21.15.156:0/96363946 conn(0x7fdaf0072600 msgr2=0x7fdaf00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:46.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.162+0000 7fdaf61ff700 1 -- 172.21.15.156:0/96363946 shutdown_connections 2023-11-28T13:10:46.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.162+0000 7fdaf61ff700 1 -- 172.21.15.156:0/96363946 wait complete. 2023-11-28T13:10:46.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.164+0000 7fdaf61ff700 1 Processor -- start 2023-11-28T13:10:46.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.165+0000 7fdaf61ff700 1 -- start start 2023-11-28T13:10:46.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.166+0000 7fdaf61ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdadc003a00 0x7fdaf014a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:46.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.166+0000 7fdaf61ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdaf01f2cf0 0x7fdaf00c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:46.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.166+0000 7fdaf61ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf01f3160 0x7fdaf00c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:46.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.166+0000 7fdaf61ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf014a990 con 0x7fdadc003a00 2023-11-28T13:10:46.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.166+0000 7fdaf61ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf00cb530 con 0x7fdaf01f3160 2023-11-28T13:10:46.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.166+0000 7fdaf61ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf00cb810 con 0x7fdaf01f2cf0 2023-11-28T13:10:46.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.166+0000 7fdaef7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdadc003a00 0x7fdaf014a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:46.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.166+0000 7fdaef7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdadc003a00 0x7fdaf014a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49204/0 (socket says 172.21.15.156:49204) 2023-11-28T13:10:46.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.166+0000 7fdaef7fe700 1 -- 172.21.15.156:0/4100107997 learned_addr learned my addr 172.21.15.156:0/4100107997 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:46.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.166+0000 7fdaeffff700 1 --2- 172.21.15.156:0/4100107997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf01f3160 0x7fdaf00c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:46.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.166+0000 7fdae7fff700 1 --2- 172.21.15.156:0/4100107997 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdaf01f2cf0 0x7fdaf00c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:46.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.166+0000 7fdaef7fe700 1 -- 172.21.15.156:0/4100107997 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdaf01f2cf0 msgr2=0x7fdaf00c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:46.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.166+0000 7fdaef7fe700 1 --2- 172.21.15.156:0/4100107997 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdaf01f2cf0 0x7fdaf00c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:46.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.166+0000 7fdaef7fe700 1 -- 172.21.15.156:0/4100107997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf01f3160 msgr2=0x7fdaf00c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:46.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.166+0000 7fdaef7fe700 1 --2- 172.21.15.156:0/4100107997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf01f3160 0x7fdaf00c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:46.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.166+0000 7fdaef7fe700 1 -- 172.21.15.156:0/4100107997 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf00d15e0 con 0x7fdadc003a00 2023-11-28T13:10:46.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.166+0000 7fdaeffff700 1 --2- 172.21.15.156:0/4100107997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf01f3160 0x7fdaf00c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:46.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.167+0000 7fdaef7fe700 1 --2- 172.21.15.156:0/4100107997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdadc003a00 0x7fdaf014a450 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00c2a0 tx=0x7fdae00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:46.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.167+0000 7fdaecff9700 1 -- 172.21.15.156:0/4100107997 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae000e6c0 con 0x7fdadc003a00 2023-11-28T13:10:46.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.167+0000 7fdaecff9700 1 -- 172.21.15.156:0/4100107997 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae0003ad0 con 0x7fdadc003a00 2023-11-28T13:10:46.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.167+0000 7fdaecff9700 1 -- 172.21.15.156:0/4100107997 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0003e80 con 0x7fdadc003a00 2023-11-28T13:10:46.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.167+0000 7fdaf61ff700 1 -- 172.21.15.156:0/4100107997 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf00d1300 con 0x7fdadc003a00 2023-11-28T13:10:46.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.167+0000 7fdaf61ff700 1 -- 172.21.15.156:0/4100107997 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf015cd80 con 0x7fdadc003a00 2023-11-28T13:10:46.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.168+0000 7fdaf61ff700 1 -- 172.21.15.156:0/4100107997 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf00745c0 con 0x7fdadc003a00 2023-11-28T13:10:46.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.169+0000 7fdaecff9700 1 -- 172.21.15.156:0/4100107997 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdae0025070 con 0x7fdadc003a00 2023-11-28T13:10:46.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.170+0000 7fdaecff9700 1 --2- 172.21.15.156:0/4100107997 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdad0068320 0x7fdad006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:46.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.170+0000 7fdaecff9700 1 -- 172.21.15.156:0/4100107997 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fdae00872f0 con 0x7fdadc003a00 2023-11-28T13:10:46.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.170+0000 7fdae7fff700 1 --2- 172.21.15.156:0/4100107997 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdad0068320 0x7fdad006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:46.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.171+0000 7fdae7fff700 1 --2- 172.21.15.156:0/4100107997 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdad0068320 0x7fdad006a7d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00c9140 tx=0x7fdad8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:46.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.174+0000 7fdaecff9700 1 -- 172.21.15.156:0/4100107997 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdae0053d80 con 0x7fdadc003a00 2023-11-28T13:10:46.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.445+0000 7fdaf61ff700 1 -- 172.21.15.156:0/4100107997 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fdaf00d2070 con 0x7fdadc003a00 2023-11-28T13:10:46.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.493+0000 7fdaecff9700 1 -- 172.21.15.156:0/4100107997 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v153) v1 ==== 149+0+0 (secure 0 0 0) 0x7fdae00573a0 con 0x7fdadc003a00 2023-11-28T13:10:46.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.496+0000 7fdaf61ff700 1 -- 172.21.15.156:0/4100107997 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdad0068320 msgr2=0x7fdad006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:46.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.496+0000 7fdaf61ff700 1 --2- 172.21.15.156:0/4100107997 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdad0068320 0x7fdad006a7d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00c9140 tx=0x7fdad8009040 comp rx=0 tx=0).stop 2023-11-28T13:10:46.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.496+0000 7fdaf61ff700 1 -- 172.21.15.156:0/4100107997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdadc003a00 msgr2=0x7fdaf014a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:46.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.496+0000 7fdaf61ff700 1 --2- 172.21.15.156:0/4100107997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdadc003a00 0x7fdaf014a450 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00c2a0 tx=0x7fdae00047c0 comp rx=0 tx=0).stop 2023-11-28T13:10:46.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.496+0000 7fdaf61ff700 1 -- 172.21.15.156:0/4100107997 shutdown_connections 2023-11-28T13:10:46.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.496+0000 7fdaf61ff700 1 --2- 172.21.15.156:0/4100107997 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdad0068320 0x7fdad006a7d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:46.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.496+0000 7fdaf61ff700 1 --2- 172.21.15.156:0/4100107997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf01f3160 0x7fdaf00c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:46.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.496+0000 7fdaf61ff700 1 --2- 172.21.15.156:0/4100107997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdadc003a00 0x7fdaf014a450 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:46.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.496+0000 7fdaf61ff700 1 --2- 172.21.15.156:0/4100107997 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdaf01f2cf0 0x7fdaf00c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:46.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.496+0000 7fdaf61ff700 1 -- 172.21.15.156:0/4100107997 >> 172.21.15.156:0/4100107997 conn(0x7fdaf0072600 msgr2=0x7fdaf015f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:46.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.496+0000 7fdaf61ff700 1 -- 172.21.15.156:0/4100107997 shutdown_connections 2023-11-28T13:10:46.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.496+0000 7fdaf61ff700 1 -- 172.21.15.156:0/4100107997 wait complete. 2023-11-28T13:10:46.516 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2023-11-28T13:10:46.516 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2023-11-28T13:10:46.516 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:10:46.517 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T13:10:46.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.713+0000 7fab4edff700 1 Processor -- start 2023-11-28T13:10:46.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.713+0000 7fab4edff700 1 -- start start 2023-11-28T13:10:46.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.715+0000 7fab4edff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab48161340 0x7fab480d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:46.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.715+0000 7fab4edff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fab480d2320 0x7fab480cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:46.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.715+0000 7fab4edff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab480d1950 con 0x7fab4815c9b0 2023-11-28T13:10:46.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.715+0000 7fab4edff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab480b5800 con 0x7fab480d2320 2023-11-28T13:10:46.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.715+0000 7fab4edff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab480b5300 con 0x7fab48161340 2023-11-28T13:10:46.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.715+0000 7fab47fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab48161340 0x7fab480d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:46.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.715+0000 7fab4cb9b700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fab4815c9b0 0x7fab480b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:44242/0 (socket says 172.21.15.156:44242) 2023-11-28T13:10:46.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.715+0000 7fab4cb9b700 1 -- 172.21.15.156:0/3885634518 learned_addr learned my addr 172.21.15.156:0/3885634518 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:46.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.715+0000 7fab4d39c700 1 --2- 172.21.15.156:0/3885634518 >> v2:172.21.15.186:3300/0 conn(0x7fab480d2320 0x7fab480cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:46.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.716+0000 7fab47fff700 1 -- 172.21.15.156:0/3885634518 >> v2:172.21.15.186:3300/0 conn(0x7fab480d2320 msgr2=0x7fab480cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:46.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.716+0000 7fab47fff700 1 --2- 172.21.15.156:0/3885634518 >> v2:172.21.15.186:3300/0 conn(0x7fab480d2320 0x7fab480cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:46.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.716+0000 7fab47fff700 1 -- 172.21.15.156:0/3885634518 >> v1:172.21.15.186:6789/0 conn(0x7fab4815c9b0 legacy=0x7fab480b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:46.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.716+0000 7fab477fe700 1 -- 172.21.15.156:0/3885634518 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359040870 0 0) 0x7fab480d1950 con 0x7fab4815c9b0 2023-11-28T13:10:46.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.716+0000 7fab47fff700 1 -- 172.21.15.156:0/3885634518 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab480b5520 con 0x7fab48161340 2023-11-28T13:10:46.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.716+0000 7fab47fff700 1 --2- 172.21.15.156:0/3885634518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab48161340 0x7fab480d1de0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fab34009840 tx=0x7fab34009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98daf91ebcc9484c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:46.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.716+0000 7fab477fe700 1 -- 172.21.15.156:0/3885634518 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34011830 con 0x7fab48161340 2023-11-28T13:10:46.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.716+0000 7fab477fe700 1 -- 172.21.15.156:0/3885634518 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab34011990 con 0x7fab48161340 2023-11-28T13:10:46.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.716+0000 7fab477fe700 1 -- 172.21.15.156:0/3885634518 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34011d20 con 0x7fab48161340 2023-11-28T13:10:46.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.717+0000 7fab4edff700 1 -- 172.21.15.156:0/3885634518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab48161340 msgr2=0x7fab480d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:46.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.717+0000 7fab4edff700 1 --2- 172.21.15.156:0/3885634518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab48161340 0x7fab480d1de0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fab34009840 tx=0x7fab34009b50 comp rx=0 tx=0).stop 2023-11-28T13:10:46.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.717+0000 7fab4edff700 1 -- 172.21.15.156:0/3885634518 shutdown_connections 2023-11-28T13:10:46.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.717+0000 7fab4edff700 1 --2- 172.21.15.156:0/3885634518 >> v2:172.21.15.186:3300/0 conn(0x7fab480d2320 0x7fab480cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:46.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.717+0000 7fab4edff700 1 --2- 172.21.15.156:0/3885634518 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab48161340 0x7fab480d1de0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:46.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.717+0000 7fab4edff700 1 -- 172.21.15.156:0/3885634518 >> 172.21.15.156:0/3885634518 conn(0x7fab48072600 msgr2=0x7fab480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:46.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.717+0000 7fab4edff700 1 -- 172.21.15.156:0/3885634518 shutdown_connections 2023-11-28T13:10:46.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.717+0000 7fab4edff700 1 -- 172.21.15.156:0/3885634518 wait complete. 2023-11-28T13:10:46.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.719+0000 7fab4edff700 1 Processor -- start 2023-11-28T13:10:46.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.719+0000 7fab4edff700 1 -- start start 2023-11-28T13:10:46.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.720+0000 7fab4edff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab480d2320 0x7fab481f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:46.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.720+0000 7fab4edff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab4815c9b0 0x7fab48156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:46.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.720+0000 7fab4edff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab48161340 0x7fab481e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:46.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.720+0000 7fab4edff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab48157040 con 0x7fab480d2320 2023-11-28T13:10:46.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.720+0000 7fab4edff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab480c6dd0 con 0x7fab4815c9b0 2023-11-28T13:10:46.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.720+0000 7fab4edff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab480c70b0 con 0x7fab48161340 2023-11-28T13:10:46.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.720+0000 7fab4cb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab480d2320 0x7fab481f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:46.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.720+0000 7fab4cb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab480d2320 0x7fab481f2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49216/0 (socket says 172.21.15.156:49216) 2023-11-28T13:10:46.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.720+0000 7fab4cb9b700 1 -- 172.21.15.156:0/4043950835 learned_addr learned my addr 172.21.15.156:0/4043950835 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:46.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.720+0000 7fab4d39c700 1 --2- 172.21.15.156:0/4043950835 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab48161340 0x7fab481e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:46.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.720+0000 7fab47fff700 1 --2- 172.21.15.156:0/4043950835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab4815c9b0 0x7fab48156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:46.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.720+0000 7fab4cb9b700 1 -- 172.21.15.156:0/4043950835 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab48161340 msgr2=0x7fab481e8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:46.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.720+0000 7fab4cb9b700 1 --2- 172.21.15.156:0/4043950835 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab48161340 0x7fab481e8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:46.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.720+0000 7fab4cb9b700 1 -- 172.21.15.156:0/4043950835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab4815c9b0 msgr2=0x7fab48156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:46.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.720+0000 7fab4cb9b700 1 --2- 172.21.15.156:0/4043950835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab4815c9b0 0x7fab48156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:46.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.720+0000 7fab4cb9b700 1 -- 172.21.15.156:0/4043950835 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab340094a0 con 0x7fab480d2320 2023-11-28T13:10:46.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.720+0000 7fab47fff700 1 --2- 172.21.15.156:0/4043950835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab4815c9b0 0x7fab48156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:46.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.721+0000 7fab4cb9b700 1 --2- 172.21.15.156:0/4043950835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab480d2320 0x7fab481f2a50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00c550 tx=0x7fab3c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:46.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.721+0000 7fab457fa700 1 -- 172.21.15.156:0/4043950835 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c004c80 con 0x7fab480d2320 2023-11-28T13:10:46.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.721+0000 7fab457fa700 1 -- 172.21.15.156:0/4043950835 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3c004e00 con 0x7fab480d2320 2023-11-28T13:10:46.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.721+0000 7fab4edff700 1 -- 172.21.15.156:0/4043950835 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab480d15e0 con 0x7fab480d2320 2023-11-28T13:10:46.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.721+0000 7fab457fa700 1 -- 172.21.15.156:0/4043950835 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c007310 con 0x7fab480d2320 2023-11-28T13:10:46.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.721+0000 7fab4edff700 1 -- 172.21.15.156:0/4043950835 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab480d1300 con 0x7fab480d2320 2023-11-28T13:10:46.726 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.722+0000 7fab4edff700 1 -- 172.21.15.156:0/4043950835 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4815cd80 con 0x7fab480d2320 2023-11-28T13:10:46.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.723+0000 7fab457fa700 1 -- 172.21.15.156:0/4043950835 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fab3c01a020 con 0x7fab480d2320 2023-11-28T13:10:46.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.724+0000 7fab457fa700 1 --2- 172.21.15.156:0/4043950835 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fab2c0682f0 0x7fab2c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:46.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.724+0000 7fab457fa700 1 -- 172.21.15.156:0/4043950835 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fab3c085610 con 0x7fab480d2320 2023-11-28T13:10:46.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.725+0000 7fab47fff700 1 --2- 172.21.15.156:0/4043950835 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fab2c0682f0 0x7fab2c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:46.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.725+0000 7fab47fff700 1 --2- 172.21.15.156:0/4043950835 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fab2c0682f0 0x7fab2c06a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fab34000c00 tx=0x7fab34008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:46.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.728+0000 7fab457fa700 1 -- 172.21.15.156:0/4043950835 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fab3c052150 con 0x7fab480d2320 2023-11-28T13:10:46.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:46.994+0000 7fab4edff700 1 -- 172.21.15.156:0/4043950835 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7fab480745c0 con 0x7fab480d2320 2023-11-28T13:10:47.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.495+0000 7fab457fa700 1 -- 172.21.15.156:0/4043950835 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v154) v1 ==== 172+0+0 (secure 0 0 0) 0x7fab3c055770 con 0x7fab480d2320 2023-11-28T13:10:47.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.497+0000 7fab4edff700 1 -- 172.21.15.156:0/4043950835 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fab2c0682f0 msgr2=0x7fab2c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:47.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.497+0000 7fab4edff700 1 --2- 172.21.15.156:0/4043950835 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fab2c0682f0 0x7fab2c06a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fab34000c00 tx=0x7fab34008030 comp rx=0 tx=0).stop 2023-11-28T13:10:47.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.497+0000 7fab4edff700 1 -- 172.21.15.156:0/4043950835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab480d2320 msgr2=0x7fab481f2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:47.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.497+0000 7fab4edff700 1 --2- 172.21.15.156:0/4043950835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab480d2320 0x7fab481f2a50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00c550 tx=0x7fab3c00c910 comp rx=0 tx=0).stop 2023-11-28T13:10:47.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.497+0000 7fab4edff700 1 -- 172.21.15.156:0/4043950835 shutdown_connections 2023-11-28T13:10:47.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.497+0000 7fab4edff700 1 --2- 172.21.15.156:0/4043950835 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fab2c0682f0 0x7fab2c06a7a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:47.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.497+0000 7fab4edff700 1 --2- 172.21.15.156:0/4043950835 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab48161340 0x7fab481e8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:47.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.497+0000 7fab4edff700 1 --2- 172.21.15.156:0/4043950835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab480d2320 0x7fab481f2a50 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:47.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.497+0000 7fab4edff700 1 --2- 172.21.15.156:0/4043950835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab4815c9b0 0x7fab48156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:47.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.497+0000 7fab4edff700 1 -- 172.21.15.156:0/4043950835 >> 172.21.15.156:0/4043950835 conn(0x7fab48072600 msgr2=0x7fab4815fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:47.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.498+0000 7fab4edff700 1 -- 172.21.15.156:0/4043950835 shutdown_connections 2023-11-28T13:10:47.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.498+0000 7fab4edff700 1 -- 172.21.15.156:0/4043950835 wait complete. 2023-11-28T13:10:47.518 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T13:10:47.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.717+0000 7f6333fff700 1 Processor -- start 2023-11-28T13:10:47.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.718+0000 7f6333fff700 1 -- start start 2023-11-28T13:10:47.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.719+0000 7f6333fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f632c15c9b0 0x7f632c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:47.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.719+0000 7f6333fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f632c161340 0x7f632c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:47.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.719+0000 7f6333fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f632c0d1950 con 0x7f632c0d2320 2023-11-28T13:10:47.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.719+0000 7f6333fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f632c0b5800 con 0x7f632c15c9b0 2023-11-28T13:10:47.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.719+0000 7f6333fff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f632c0b5520 con 0x7f632c161340 2023-11-28T13:10:47.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.719+0000 7f6331d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f632c15c9b0 0x7f632c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:47.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.719+0000 7f6331d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f632c15c9b0 0x7f632c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49230/0 (socket says 172.21.15.156:49230) 2023-11-28T13:10:47.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.719+0000 7f6331d9b700 1 -- 172.21.15.156:0/245712019 learned_addr learned my addr 172.21.15.156:0/245712019 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:47.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.719+0000 7f633159a700 1 --2- 172.21.15.156:0/245712019 >> v2:172.21.15.186:3300/0 conn(0x7f632c161340 0x7f632c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:47.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.719+0000 7f6331d9b700 1 -- 172.21.15.156:0/245712019 >> v2:172.21.15.186:3300/0 conn(0x7f632c161340 msgr2=0x7f632c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:47.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.719+0000 7f6331d9b700 1 --2- 172.21.15.156:0/245712019 >> v2:172.21.15.186:3300/0 conn(0x7f632c161340 0x7f632c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:47.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.719+0000 7f6331d9b700 1 -- 172.21.15.156:0/245712019 >> v1:172.21.15.186:6789/0 conn(0x7f632c0d2320 legacy=0x7f632c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:47.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.719+0000 7f6331d9b700 1 -- 172.21.15.156:0/245712019 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c0d1300 con 0x7f632c15c9b0 2023-11-28T13:10:47.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.720+0000 7f6331d9b700 1 --2- 172.21.15.156:0/245712019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f632c15c9b0 0x7f632c0b4bf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f632c0685c0 tx=0x7f6328009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1775b59183f04a46 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:47.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.720+0000 7f6330d99700 1 -- 172.21.15.156:0/245712019 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632800b870 con 0x7f632c15c9b0 2023-11-28T13:10:47.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.720+0000 7f6330d99700 1 -- 172.21.15.156:0/245712019 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632800b9d0 con 0x7f632c15c9b0 2023-11-28T13:10:47.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.720+0000 7f6330d99700 1 -- 172.21.15.156:0/245712019 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632800bd60 con 0x7f632c15c9b0 2023-11-28T13:10:47.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.720+0000 7f6333fff700 1 -- 172.21.15.156:0/245712019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f632c15c9b0 msgr2=0x7f632c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:47.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.720+0000 7f6333fff700 1 --2- 172.21.15.156:0/245712019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f632c15c9b0 0x7f632c0b4bf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f632c0685c0 tx=0x7f6328009bd0 comp rx=0 tx=0).stop 2023-11-28T13:10:47.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.721+0000 7f6333fff700 1 -- 172.21.15.156:0/245712019 shutdown_connections 2023-11-28T13:10:47.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.721+0000 7f6333fff700 1 --2- 172.21.15.156:0/245712019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f632c15c9b0 0x7f632c0b4bf0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:47.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.721+0000 7f6333fff700 1 --2- 172.21.15.156:0/245712019 >> v2:172.21.15.186:3300/0 conn(0x7f632c161340 0x7f632c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:47.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.721+0000 7f6333fff700 1 -- 172.21.15.156:0/245712019 >> 172.21.15.156:0/245712019 conn(0x7f632c072600 msgr2=0x7f632c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:47.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.721+0000 7f6333fff700 1 -- 172.21.15.156:0/245712019 shutdown_connections 2023-11-28T13:10:47.724 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.721+0000 7f6333fff700 1 -- 172.21.15.156:0/245712019 wait complete. 2023-11-28T13:10:47.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.722+0000 7f6333fff700 1 Processor -- start 2023-11-28T13:10:47.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.723+0000 7f6333fff700 1 -- start start 2023-11-28T13:10:47.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.724+0000 7f6333fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f632c0d2320 0x7f632c150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:47.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.724+0000 7f6333fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f632c15c9b0 0x7f632c14fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:47.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.724+0000 7f6333fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f632c161340 0x7f632c154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:47.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.724+0000 7f6333fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c150060 con 0x7f632c161340 2023-11-28T13:10:47.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.724+0000 7f6333fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c155780 con 0x7f632c15c9b0 2023-11-28T13:10:47.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.724+0000 7f6333fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632c155a60 con 0x7f632c0d2320 2023-11-28T13:10:47.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.724+0000 7f633259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f632c161340 0x7f632c154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:47.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.724+0000 7f633259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f632c161340 0x7f632c154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49238/0 (socket says 172.21.15.156:49238) 2023-11-28T13:10:47.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.724+0000 7f633259c700 1 -- 172.21.15.156:0/989598988 learned_addr learned my addr 172.21.15.156:0/989598988 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:47.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.724+0000 7f633159a700 1 --2- 172.21.15.156:0/989598988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f632c15c9b0 0x7f632c14fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:47.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.724+0000 7f6331d9b700 1 --2- 172.21.15.156:0/989598988 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f632c0d2320 0x7f632c150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:47.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.724+0000 7f633259c700 1 -- 172.21.15.156:0/989598988 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f632c0d2320 msgr2=0x7f632c150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:47.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.724+0000 7f633259c700 1 --2- 172.21.15.156:0/989598988 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f632c0d2320 0x7f632c150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:47.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.724+0000 7f633259c700 1 -- 172.21.15.156:0/989598988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f632c15c9b0 msgr2=0x7f632c14fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:47.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.724+0000 7f633259c700 1 --2- 172.21.15.156:0/989598988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f632c15c9b0 0x7f632c14fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:47.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.724+0000 7f633259c700 1 -- 172.21.15.156:0/989598988 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6328009550 con 0x7f632c161340 2023-11-28T13:10:47.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.725+0000 7f633259c700 1 --2- 172.21.15.156:0/989598988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f632c161340 0x7f632c154c20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f632400cbc0 tx=0x7f632400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:47.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.725+0000 7f63227fc700 1 -- 172.21.15.156:0/989598988 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324003c00 con 0x7f632c161340 2023-11-28T13:10:47.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.725+0000 7f63227fc700 1 -- 172.21.15.156:0/989598988 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6324003d60 con 0x7f632c161340 2023-11-28T13:10:47.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.725+0000 7f63227fc700 1 -- 172.21.15.156:0/989598988 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632400aa20 con 0x7f632c161340 2023-11-28T13:10:47.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.725+0000 7f6333fff700 1 -- 172.21.15.156:0/989598988 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632c0b5130 con 0x7f632c161340 2023-11-28T13:10:47.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.725+0000 7f6333fff700 1 -- 172.21.15.156:0/989598988 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632c0d15e0 con 0x7f632c161340 2023-11-28T13:10:47.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.726+0000 7f6333fff700 1 -- 172.21.15.156:0/989598988 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f632c0b52c0 con 0x7f632c161340 2023-11-28T13:10:47.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.727+0000 7f63227fc700 1 -- 172.21.15.156:0/989598988 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6324010050 con 0x7f632c161340 2023-11-28T13:10:47.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.728+0000 7f63227fc700 1 --2- 172.21.15.156:0/989598988 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63100682f0 0x7f631006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:47.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.728+0000 7f63227fc700 1 -- 172.21.15.156:0/989598988 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f63240857f0 con 0x7f632c161340 2023-11-28T13:10:47.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.729+0000 7f6331d9b700 1 --2- 172.21.15.156:0/989598988 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63100682f0 0x7f631006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:47.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.729+0000 7f6331d9b700 1 --2- 172.21.15.156:0/989598988 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63100682f0 0x7f631006a7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6328000c00 tx=0x7f6328008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:47.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.733+0000 7f63227fc700 1 -- 172.21.15.156:0/989598988 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6324052280 con 0x7f632c161340 2023-11-28T13:10:47.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.925+0000 7f6333fff700 1 -- 172.21.15.156:0/989598988 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f632c152bc0 con 0x7f632c161340 2023-11-28T13:10:47.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.929+0000 7f63227fc700 1 -- 172.21.15.156:0/989598988 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f63240558a0 con 0x7f632c161340 2023-11-28T13:10:47.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.931+0000 7f6333fff700 1 -- 172.21.15.156:0/989598988 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63100682f0 msgr2=0x7f631006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:47.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.931+0000 7f6333fff700 1 --2- 172.21.15.156:0/989598988 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63100682f0 0x7f631006a7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6328000c00 tx=0x7f6328008030 comp rx=0 tx=0).stop 2023-11-28T13:10:47.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.931+0000 7f6333fff700 1 -- 172.21.15.156:0/989598988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f632c161340 msgr2=0x7f632c154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:47.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.931+0000 7f6333fff700 1 --2- 172.21.15.156:0/989598988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f632c161340 0x7f632c154c20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f632400cbc0 tx=0x7f632400ced0 comp rx=0 tx=0).stop 2023-11-28T13:10:47.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.931+0000 7f6333fff700 1 -- 172.21.15.156:0/989598988 shutdown_connections 2023-11-28T13:10:47.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.931+0000 7f6333fff700 1 --2- 172.21.15.156:0/989598988 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63100682f0 0x7f631006a7a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:47.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.931+0000 7f6333fff700 1 --2- 172.21.15.156:0/989598988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f632c161340 0x7f632c154c20 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:47.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.931+0000 7f6333fff700 1 --2- 172.21.15.156:0/989598988 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f632c0d2320 0x7f632c150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:47.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.931+0000 7f6333fff700 1 --2- 172.21.15.156:0/989598988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f632c15c9b0 0x7f632c14fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:47.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.931+0000 7f6333fff700 1 -- 172.21.15.156:0/989598988 >> 172.21.15.156:0/989598988 conn(0x7f632c072600 msgr2=0x7f632c15a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:47.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.931+0000 7f6333fff700 1 -- 172.21.15.156:0/989598988 shutdown_connections 2023-11-28T13:10:47.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:47.931+0000 7f6333fff700 1 -- 172.21.15.156:0/989598988 wait complete. 2023-11-28T13:10:47.951 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:10:47.952 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:10:48.044 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:10:48.143 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:10:48.143 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:10:48.144 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:10:48.187 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:10:48.188 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:10:48.188 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:10:48.233 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:10:48.233 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:10:48.233 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:10:48.278 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:10:48.279 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:10:48.279 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:10:48.323 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:10:48.323 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T13:10:48.324 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:10:48.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.520+0000 7f9d7d7ff700 1 Processor -- start 2023-11-28T13:10:48.523 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.520+0000 7f9d7d7ff700 1 -- start start 2023-11-28T13:10:48.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.521+0000 7f9d7d7ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9d78158200 0x7f9d78070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:48.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.521+0000 7f9d7d7ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d7815e1a0 con 0x7f9d780b0d80 2023-11-28T13:10:48.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.521+0000 7f9d7d7ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d781585d0 con 0x7f9d780caa70 2023-11-28T13:10:48.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.521+0000 7f9d7d7ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d7815f070 con 0x7f9d78158200 2023-11-28T13:10:48.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.521+0000 7f9d76ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9d78158200 0x7f9d78070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:48.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.521+0000 7f9d777fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f9d780b0d80 0x7f9d7815add0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37764/0 (socket says 172.21.15.156:37764) 2023-11-28T13:10:48.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.521+0000 7f9d777fe700 1 -- 172.21.15.156:0/608296658 learned_addr learned my addr 172.21.15.156:0/608296658 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:48.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.522+0000 7f9d767fc700 1 -- 172.21.15.156:0/608296658 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440928113 0 0) 0x7f9d7815e1a0 con 0x7f9d780b0d80 2023-11-28T13:10:48.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.522+0000 7f9d767fc700 1 -- 172.21.15.156:0/608296658 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d640035b0 con 0x7f9d780b0d80 2023-11-28T13:10:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.522+0000 7f9d767fc700 1 -- 172.21.15.156:0/608296658 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741438697 0 0) 0x7f9d781585d0 con 0x7f9d780caa70 2023-11-28T13:10:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.522+0000 7f9d76ffd700 1 -- 172.21.15.156:0/608296658 >> v1:172.21.15.186:6789/0 conn(0x7f9d780caa70 legacy=0x7f9d780b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.522+0000 7f9d76ffd700 1 -- 172.21.15.156:0/608296658 >> v1:172.21.15.156:6789/0 conn(0x7f9d780b0d80 legacy=0x7f9d7815add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.522+0000 7f9d76ffd700 1 -- 172.21.15.156:0/608296658 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7815e3f0 con 0x7f9d78158200 2023-11-28T13:10:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.522+0000 7f9d76ffd700 1 --2- 172.21.15.156:0/608296658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d78158200 0x7f9d78070c50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009840 tx=0x7f9d68009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6ed48d6cd09a8b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.522+0000 7f9d767fc700 1 -- 172.21.15.156:0/608296658 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68011830 con 0x7f9d78158200 2023-11-28T13:10:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.522+0000 7f9d767fc700 1 -- 172.21.15.156:0/608296658 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d68011990 con 0x7f9d78158200 2023-11-28T13:10:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.522+0000 7f9d767fc700 1 -- 172.21.15.156:0/608296658 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68011d00 con 0x7f9d78158200 2023-11-28T13:10:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.523+0000 7f9d7d7ff700 1 -- 172.21.15.156:0/608296658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d78158200 msgr2=0x7f9d78070c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.523+0000 7f9d7d7ff700 1 --2- 172.21.15.156:0/608296658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d78158200 0x7f9d78070c50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009840 tx=0x7f9d68009b50 comp rx=0 tx=0).stop 2023-11-28T13:10:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.523+0000 7f9d7d7ff700 1 -- 172.21.15.156:0/608296658 shutdown_connections 2023-11-28T13:10:48.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.523+0000 7f9d7d7ff700 1 --2- 172.21.15.156:0/608296658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d78158200 0x7f9d78070c50 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.523+0000 7f9d7d7ff700 1 -- 172.21.15.156:0/608296658 >> 172.21.15.156:0/608296658 conn(0x7f9d780cb670 msgr2=0x7f9d780cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.523+0000 7f9d7d7ff700 1 -- 172.21.15.156:0/608296658 shutdown_connections 2023-11-28T13:10:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.523+0000 7f9d7d7ff700 1 -- 172.21.15.156:0/608296658 wait complete. 2023-11-28T13:10:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.525+0000 7f9d7d7ff700 1 Processor -- start 2023-11-28T13:10:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.525+0000 7f9d7d7ff700 1 -- start start 2023-11-28T13:10:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.526+0000 7f9d7d7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d780b0d80 0x7f9d78154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.526+0000 7f9d7d7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d780caa70 0x7f9d780c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.526+0000 7f9d7d7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9d78158200 0x7f9d781593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.526+0000 7f9d7d7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d78071190 con 0x7f9d780b0d80 2023-11-28T13:10:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.526+0000 7f9d7d7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d781f02c0 con 0x7f9d780caa70 2023-11-28T13:10:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.526+0000 7f9d7d7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d78155390 con 0x7f9d78158200 2023-11-28T13:10:48.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.526+0000 7f9d76ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d780b0d80 0x7f9d78154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.526+0000 7f9d76ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d780b0d80 0x7f9d78154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49252/0 (socket says 172.21.15.156:49252) 2023-11-28T13:10:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.526+0000 7f9d76ffd700 1 -- 172.21.15.156:0/1079425387 learned_addr learned my addr 172.21.15.156:0/1079425387 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.526+0000 7f9d777fe700 1 --2- 172.21.15.156:0/1079425387 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9d78158200 0x7f9d781593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.526+0000 7f9d6ffff700 1 --2- 172.21.15.156:0/1079425387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d780caa70 0x7f9d780c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.526+0000 7f9d76ffd700 1 -- 172.21.15.156:0/1079425387 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9d78158200 msgr2=0x7f9d781593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.526+0000 7f9d76ffd700 1 --2- 172.21.15.156:0/1079425387 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9d78158200 0x7f9d781593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.526+0000 7f9d76ffd700 1 -- 172.21.15.156:0/1079425387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d780caa70 msgr2=0x7f9d780c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:48.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.526+0000 7f9d76ffd700 1 --2- 172.21.15.156:0/1079425387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d780caa70 0x7f9d780c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:48.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.526+0000 7f9d76ffd700 1 -- 172.21.15.156:0/1079425387 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d680094a0 con 0x7f9d780b0d80 2023-11-28T13:10:48.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.526+0000 7f9d76ffd700 1 --2- 172.21.15.156:0/1079425387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d780b0d80 0x7f9d78154e50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009fd0 tx=0x7f9d68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:48.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.527+0000 7f9d6f7fe700 1 -- 172.21.15.156:0/1079425387 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d680036a0 con 0x7f9d780b0d80 2023-11-28T13:10:48.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.527+0000 7f9d6f7fe700 1 -- 172.21.15.156:0/1079425387 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6801e070 con 0x7f9d780b0d80 2023-11-28T13:10:48.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.527+0000 7f9d7d7ff700 1 -- 172.21.15.156:0/1079425387 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7815e9b0 con 0x7f9d780b0d80 2023-11-28T13:10:48.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.527+0000 7f9d6f7fe700 1 -- 172.21.15.156:0/1079425387 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68003a40 con 0x7f9d780b0d80 2023-11-28T13:10:48.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.527+0000 7f9d7d7ff700 1 -- 172.21.15.156:0/1079425387 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7815e6d0 con 0x7f9d780b0d80 2023-11-28T13:10:48.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.528+0000 7f9d7d7ff700 1 -- 172.21.15.156:0/1079425387 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7815ec90 con 0x7f9d780b0d80 2023-11-28T13:10:48.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.529+0000 7f9d6f7fe700 1 -- 172.21.15.156:0/1079425387 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9d6800f050 con 0x7f9d780b0d80 2023-11-28T13:10:48.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.530+0000 7f9d6f7fe700 1 --2- 172.21.15.156:0/1079425387 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9d580682f0 0x7f9d5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:48.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.530+0000 7f9d6f7fe700 1 -- 172.21.15.156:0/1079425387 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9d68086ad0 con 0x7f9d780b0d80 2023-11-28T13:10:48.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.530+0000 7f9d6ffff700 1 --2- 172.21.15.156:0/1079425387 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9d580682f0 0x7f9d5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:48.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.531+0000 7f9d6ffff700 1 --2- 172.21.15.156:0/1079425387 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9d580682f0 0x7f9d5806a7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9d780b4a90 tx=0x7f9d60009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:48.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.534+0000 7f9d6f7fe700 1 -- 172.21.15.156:0/1079425387 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9d68053560 con 0x7f9d780b0d80 2023-11-28T13:10:48.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.768+0000 7f9d7d7ff700 1 -- 172.21.15.156:0/1079425387 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d7806fd60 con 0x7f9d780b0d80 2023-11-28T13:10:48.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.769+0000 7f9d6f7fe700 1 -- 172.21.15.156:0/1079425387 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f9d68015090 con 0x7f9d780b0d80 2023-11-28T13:10:48.771 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:48.771 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3828141874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3828141874},{"type":"v1","addr":"172.21.15.156:6835","nonce":3828141874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1012489762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1012489762},{"type":"v1","addr":"172.21.15.186:6837","nonce":1012489762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2894670074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2894670074},{"type":"v1","addr":"172.21.15.186:6835","nonce":2894670074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5605,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1860847809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1860847809},{"type":"v1","addr":"172.21.15.156:6834","nonce":1860847809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[]} 2023-11-28T13:10:48.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.771+0000 7f9d7d7ff700 1 -- 172.21.15.156:0/1079425387 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9d580682f0 msgr2=0x7f9d5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:48.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.771+0000 7f9d7d7ff700 1 --2- 172.21.15.156:0/1079425387 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9d580682f0 0x7f9d5806a7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9d780b4a90 tx=0x7f9d60009040 comp rx=0 tx=0).stop 2023-11-28T13:10:48.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.771+0000 7f9d7d7ff700 1 -- 172.21.15.156:0/1079425387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d780b0d80 msgr2=0x7f9d78154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:48.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.771+0000 7f9d7d7ff700 1 --2- 172.21.15.156:0/1079425387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d780b0d80 0x7f9d78154e50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009fd0 tx=0x7f9d68004750 comp rx=0 tx=0).stop 2023-11-28T13:10:48.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.771+0000 7f9d7d7ff700 1 -- 172.21.15.156:0/1079425387 shutdown_connections 2023-11-28T13:10:48.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.771+0000 7f9d7d7ff700 1 --2- 172.21.15.156:0/1079425387 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9d580682f0 0x7f9d5806a7a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:48.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.771+0000 7f9d7d7ff700 1 --2- 172.21.15.156:0/1079425387 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9d78158200 0x7f9d781593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:48.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.771+0000 7f9d7d7ff700 1 --2- 172.21.15.156:0/1079425387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d780b0d80 0x7f9d78154e50 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:48.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.771+0000 7f9d7d7ff700 1 --2- 172.21.15.156:0/1079425387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d780caa70 0x7f9d780c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:48.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.771+0000 7f9d7d7ff700 1 -- 172.21.15.156:0/1079425387 >> 172.21.15.156:0/1079425387 conn(0x7f9d780cb670 msgr2=0x7f9d7815d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:48.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.771+0000 7f9d7d7ff700 1 -- 172.21.15.156:0/1079425387 shutdown_connections 2023-11-28T13:10:48.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:48.771+0000 7f9d7d7ff700 1 -- 172.21.15.156:0/1079425387 wait complete. 2023-11-28T13:10:48.774 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 25 2023-11-28T13:10:48.793 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T13:10:48.793 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T13:10:48.794 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T13:10:48.794 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:10:48.795 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T13:10:48.795 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T13:10:48.795 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T13:10:48.795 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:10:48.795 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T13:10:48.795 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T13:10:48.796 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T13:10:48.796 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:10:48.796 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T13:10:48.797 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T13:10:48.797 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T13:10:48.797 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:10:48.808 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T13:10:48.808 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T13:10:48.809 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T13:10:48.809 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T13:10:48.812 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T13:10:48.812 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T13:10:48.812 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T13:10:48.813 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T13:10:48.854 INFO:tasks.ceph.mds.a:Started 2023-11-28T13:10:48.855 DEBUG:teuthology.parallel:result is None 2023-11-28T13:10:48.856 INFO:tasks.ceph.mds.c:Started 2023-11-28T13:10:48.856 DEBUG:teuthology.parallel:result is None 2023-11-28T13:10:48.860 INFO:tasks.ceph.mds.b:Started 2023-11-28T13:10:48.861 DEBUG:teuthology.parallel:result is None 2023-11-28T13:10:48.863 INFO:tasks.ceph.mds.d:Started 2023-11-28T13:10:48.863 DEBUG:teuthology.parallel:result is None 2023-11-28T13:10:48.864 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T13:10:49.003 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2023-11-28T13:10:49.004 INFO:tasks.ceph.mds.c.smithi156.stdout:starting mds.c at 2023-11-28T13:10:49.004 INFO:tasks.ceph.mds.a.smithi156.stdout:starting mds.a at 2023-11-28T13:10:49.007 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2023-11-28T13:10:49.007 INFO:tasks.ceph.mds.a.smithi156.stderr:2023-11-28T13:10:49.006+0000 7fb4533fd700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6834/0: (98) Address already in use 2023-11-28T13:10:49.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.076+0000 7f480bbff700 1 Processor -- start 2023-11-28T13:10:49.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.076+0000 7f480bbff700 1 -- start start 2023-11-28T13:10:49.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.078+0000 7f480bbff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4804160740 0x7f48040cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:49.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.078+0000 7f480bbff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f480406b060 0x7f480406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:49.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.078+0000 7f480bbff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48040cc030 con 0x7f4804160b10 2023-11-28T13:10:49.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.078+0000 7f480bbff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48040ca720 con 0x7f4804160740 2023-11-28T13:10:49.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.078+0000 7f480bbff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48040c9e10 con 0x7f480406b060 2023-11-28T13:10:49.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.078+0000 7f480999b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4804160740 0x7f48040cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:49.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.078+0000 7f480999b700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4804160740 0x7f48040cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:57856/0 (socket says 172.21.15.156:57856) 2023-11-28T13:10:49.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.078+0000 7f480999b700 1 -- 172.21.15.156:0/2539166133 learned_addr learned my addr 172.21.15.156:0/2539166133 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:49.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.078+0000 7f480a19c700 1 --2- 172.21.15.156:0/2539166133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f480406b060 0x7f480406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:49.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.079+0000 7f480999b700 1 -- 172.21.15.156:0/2539166133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f480406b060 msgr2=0x7f480406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:49.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.079+0000 7f480999b700 1 --2- 172.21.15.156:0/2539166133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f480406b060 0x7f480406b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:49.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.079+0000 7f480999b700 1 -- 172.21.15.156:0/2539166133 >> v1:172.21.15.186:6789/0 conn(0x7f4804160b10 legacy=0x7f48040cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:49.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.079+0000 7f480999b700 1 -- 172.21.15.156:0/2539166133 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480406efc0 con 0x7f4804160740 2023-11-28T13:10:49.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.079+0000 7f4808999700 1 -- 172.21.15.156:0/2539166133 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7206812 0 0) 0x7f48040cc030 con 0x7f4804160b10 2023-11-28T13:10:49.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.079+0000 7f480999b700 1 --2- 172.21.15.156:0/2539166133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4804160740 0x7f48040cbaf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f48000098f0 tx=0x7f4800009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c43e5aaaa26ece92 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:49.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.079+0000 7f4808999700 1 -- 172.21.15.156:0/2539166133 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800011830 con 0x7f4804160740 2023-11-28T13:10:49.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.079+0000 7f4808999700 1 -- 172.21.15.156:0/2539166133 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4800011990 con 0x7f4804160740 2023-11-28T13:10:49.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.079+0000 7f4808999700 1 -- 172.21.15.156:0/2539166133 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800011d20 con 0x7f4804160740 2023-11-28T13:10:49.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.080+0000 7f480bbff700 1 -- 172.21.15.156:0/2539166133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4804160740 msgr2=0x7f48040cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:49.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.080+0000 7f480bbff700 1 --2- 172.21.15.156:0/2539166133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4804160740 0x7f48040cbaf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f48000098f0 tx=0x7f4800009c00 comp rx=0 tx=0).stop 2023-11-28T13:10:49.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.080+0000 7f480bbff700 1 -- 172.21.15.156:0/2539166133 shutdown_connections 2023-11-28T13:10:49.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.080+0000 7f480bbff700 1 --2- 172.21.15.156:0/2539166133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f480406b060 0x7f480406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:49.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.080+0000 7f480bbff700 1 --2- 172.21.15.156:0/2539166133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4804160740 0x7f48040cbaf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:49.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.080+0000 7f480bbff700 1 -- 172.21.15.156:0/2539166133 >> 172.21.15.156:0/2539166133 conn(0x7f48040b12b0 msgr2=0x7f48040b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:49.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.080+0000 7f480bbff700 1 -- 172.21.15.156:0/2539166133 shutdown_connections 2023-11-28T13:10:49.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.080+0000 7f480bbff700 1 -- 172.21.15.156:0/2539166133 wait complete. 2023-11-28T13:10:49.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.082+0000 7f480bbff700 1 Processor -- start 2023-11-28T13:10:49.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.082+0000 7f480bbff700 1 -- start start 2023-11-28T13:10:49.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480bbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f480406b060 0x7f4804157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:49.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480bbff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4804160740 0x7f48040ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:49.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480bbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4804160b10 0x7f48040ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:49.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480bbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4804158390 con 0x7f480406b060 2023-11-28T13:10:49.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480bbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480406d9d0 con 0x7f4804160b10 2023-11-28T13:10:49.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480bbff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480406dc20 con 0x7f4804160740 2023-11-28T13:10:49.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480999b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f480406b060 0x7f4804157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:49.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480999b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f480406b060 0x7f4804157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49300/0 (socket says 172.21.15.156:49300) 2023-11-28T13:10:49.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480999b700 1 -- 172.21.15.156:0/1595460821 learned_addr learned my addr 172.21.15.156:0/1595460821 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:49.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480999b700 1 -- 172.21.15.156:0/1595460821 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4804160740 msgr2=0x7f48040ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:49.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480a19c700 1 --2- 172.21.15.156:0/1595460821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4804160b10 0x7f48040ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:49.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480919a700 1 --2- 172.21.15.156:0/1595460821 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4804160740 0x7f48040ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:49.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480999b700 1 --2- 172.21.15.156:0/1595460821 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4804160740 0x7f48040ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:49.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480999b700 1 -- 172.21.15.156:0/1595460821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4804160b10 msgr2=0x7f48040ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:49.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480999b700 1 --2- 172.21.15.156:0/1595460821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4804160b10 0x7f48040ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:49.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480999b700 1 -- 172.21.15.156:0/1595460821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4800009550 con 0x7f480406b060 2023-11-28T13:10:49.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480a19c700 1 --2- 172.21.15.156:0/1595460821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4804160b10 0x7f48040ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:49.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.083+0000 7f480919a700 1 --2- 172.21.15.156:0/1595460821 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4804160740 0x7f48040ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:49.087 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.084+0000 7f480999b700 1 --2- 172.21.15.156:0/1595460821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f480406b060 0x7f4804157e50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4800009c40 tx=0x7f4800004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:49.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.084+0000 7f47fa7fc700 1 -- 172.21.15.156:0/1595460821 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800004480 con 0x7f480406b060 2023-11-28T13:10:49.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.084+0000 7f47fa7fc700 1 -- 172.21.15.156:0/1595460821 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480001f070 con 0x7f480406b060 2023-11-28T13:10:49.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.084+0000 7f480bbff700 1 -- 172.21.15.156:0/1595460821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48040ca3b0 con 0x7f480406b060 2023-11-28T13:10:49.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.084+0000 7f47fa7fc700 1 -- 172.21.15.156:0/1595460821 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48000038c0 con 0x7f480406b060 2023-11-28T13:10:49.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.084+0000 7f480bbff700 1 -- 172.21.15.156:0/1595460821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48040ca0d0 con 0x7f480406b060 2023-11-28T13:10:49.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.085+0000 7f480bbff700 1 -- 172.21.15.156:0/1595460821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4804160f60 con 0x7f480406b060 2023-11-28T13:10:49.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.086+0000 7f47fa7fc700 1 -- 172.21.15.156:0/1595460821 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f480002a020 con 0x7f480406b060 2023-11-28T13:10:49.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.087+0000 7f47fa7fc700 1 --2- 172.21.15.156:0/1595460821 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f47e80682f0 0x7f47e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:49.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.087+0000 7f47fa7fc700 1 -- 172.21.15.156:0/1595460821 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4800087060 con 0x7f480406b060 2023-11-28T13:10:49.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.087+0000 7f480919a700 1 --2- 172.21.15.156:0/1595460821 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f47e80682f0 0x7f47e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:49.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.088+0000 7f480919a700 1 --2- 172.21.15.156:0/1595460821 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f47e80682f0 0x7f47e806a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f47f4000f60 tx=0x7f47f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:49.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.091+0000 7f47fa7fc700 1 -- 172.21.15.156:0/1595460821 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f48000b85e0 con 0x7f480406b060 2023-11-28T13:10:49.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:49.299+0000 7f480bbff700 1 -- 172.21.15.156:0/1595460821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f48040745c0 con 0x7f480406b060 2023-11-28T13:10:50.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.040+0000 7f47fa7fc700 1 -- 172.21.15.156:0/1595460821 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v55) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4800015080 con 0x7f480406b060 2023-11-28T13:10:50.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.043+0000 7f480bbff700 1 -- 172.21.15.156:0/1595460821 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f47e80682f0 msgr2=0x7f47e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:50.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.043+0000 7f480bbff700 1 --2- 172.21.15.156:0/1595460821 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f47e80682f0 0x7f47e806a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f47f4000f60 tx=0x7f47f4009040 comp rx=0 tx=0).stop 2023-11-28T13:10:50.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.043+0000 7f480bbff700 1 -- 172.21.15.156:0/1595460821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f480406b060 msgr2=0x7f4804157e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:50.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.043+0000 7f480bbff700 1 --2- 172.21.15.156:0/1595460821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f480406b060 0x7f4804157e50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4800009c40 tx=0x7f4800004750 comp rx=0 tx=0).stop 2023-11-28T13:10:50.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.043+0000 7f480bbff700 1 -- 172.21.15.156:0/1595460821 shutdown_connections 2023-11-28T13:10:50.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.043+0000 7f480bbff700 1 --2- 172.21.15.156:0/1595460821 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f47e80682f0 0x7f47e806a7a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.043+0000 7f480bbff700 1 --2- 172.21.15.156:0/1595460821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4804160b10 0x7f48040ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.043+0000 7f480bbff700 1 --2- 172.21.15.156:0/1595460821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f480406b060 0x7f4804157e50 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.043+0000 7f480bbff700 1 --2- 172.21.15.156:0/1595460821 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4804160740 0x7f48040ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.043+0000 7f480bbff700 1 -- 172.21.15.156:0/1595460821 >> 172.21.15.156:0/1595460821 conn(0x7f48040b12b0 msgr2=0x7f480415c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:50.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.043+0000 7f480bbff700 1 -- 172.21.15.156:0/1595460821 shutdown_connections 2023-11-28T13:10:50.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.043+0000 7f480bbff700 1 -- 172.21.15.156:0/1595460821 wait complete. 2023-11-28T13:10:50.046 INFO:teuthology.orchestra.run.smithi156.stderr: removed all blocklist entries 2023-11-28T13:10:50.065 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T13:10:50.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.263+0000 7f7d04bff700 1 Processor -- start 2023-11-28T13:10:50.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.263+0000 7f7d04bff700 1 -- start start 2023-11-28T13:10:50.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.264+0000 7f7d04bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d0015c9b0 0x7f7d000b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:50.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.264+0000 7f7d04bff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7d000cd7c0 0x7f7d000cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:50.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.264+0000 7f7d04bff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d000ce160 con 0x7f7d00161340 2023-11-28T13:10:50.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.264+0000 7f7d04bff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d000b5800 con 0x7f7d000cd7c0 2023-11-28T13:10:50.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.264+0000 7f7d04bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d000d1950 con 0x7f7d0015c9b0 2023-11-28T13:10:50.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.265+0000 7f7cfed9d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7d000cd7c0 0x7f7d000cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:50.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.265+0000 7f7cfe59c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d0015c9b0 0x7f7d000b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:50.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.265+0000 7f7cfed9d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7d000cd7c0 0x7f7d000cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:57872/0 (socket says 172.21.15.156:57872) 2023-11-28T13:10:50.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.265+0000 7f7cfed9d700 1 -- 172.21.15.156:0/2871820740 learned_addr learned my addr 172.21.15.156:0/2871820740 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:50.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.265+0000 7f7cfe59c700 1 -- 172.21.15.156:0/2871820740 >> v2:172.21.15.186:3300/0 conn(0x7f7d000cd7c0 msgr2=0x7f7d000cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:50.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.265+0000 7f7cfe59c700 1 --2- 172.21.15.156:0/2871820740 >> v2:172.21.15.186:3300/0 conn(0x7f7d000cd7c0 0x7f7d000cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.265+0000 7f7cfe59c700 1 -- 172.21.15.156:0/2871820740 >> v1:172.21.15.186:6789/0 conn(0x7f7d00161340 legacy=0x7f7d000d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:50.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.265+0000 7f7cfe59c700 1 -- 172.21.15.156:0/2871820740 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d000d24f0 con 0x7f7d0015c9b0 2023-11-28T13:10:50.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.265+0000 7f7cfed9d700 1 --2- 172.21.15.156:0/2871820740 >> v2:172.21.15.186:3300/0 conn(0x7f7d000cd7c0 0x7f7d000cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:10:50.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.265+0000 7f7cfe59c700 1 --2- 172.21.15.156:0/2871820740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d0015c9b0 0x7f7d000b4bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009840 tx=0x7f7cf0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eea47046a1004ecd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:50.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.266+0000 7f7cfd59a700 1 -- 172.21.15.156:0/2871820740 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0011830 con 0x7f7d0015c9b0 2023-11-28T13:10:50.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.266+0000 7f7cfd59a700 1 -- 172.21.15.156:0/2871820740 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf0011990 con 0x7f7d0015c9b0 2023-11-28T13:10:50.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.266+0000 7f7cfd59a700 1 -- 172.21.15.156:0/2871820740 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0011d20 con 0x7f7d0015c9b0 2023-11-28T13:10:50.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.266+0000 7f7d04bff700 1 -- 172.21.15.156:0/2871820740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d0015c9b0 msgr2=0x7f7d000b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:50.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.266+0000 7f7d04bff700 1 --2- 172.21.15.156:0/2871820740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d0015c9b0 0x7f7d000b4bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009840 tx=0x7f7cf0009b50 comp rx=0 tx=0).stop 2023-11-28T13:10:50.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.266+0000 7f7d04bff700 1 -- 172.21.15.156:0/2871820740 shutdown_connections 2023-11-28T13:10:50.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.266+0000 7f7d04bff700 1 --2- 172.21.15.156:0/2871820740 >> v2:172.21.15.186:3300/0 conn(0x7f7d000cd7c0 0x7f7d000cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.266+0000 7f7d04bff700 1 --2- 172.21.15.156:0/2871820740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d0015c9b0 0x7f7d000b4bf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.266+0000 7f7d04bff700 1 -- 172.21.15.156:0/2871820740 >> 172.21.15.156:0/2871820740 conn(0x7f7d00072600 msgr2=0x7f7d000cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:50.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.266+0000 7f7d04bff700 1 -- 172.21.15.156:0/2871820740 shutdown_connections 2023-11-28T13:10:50.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.266+0000 7f7d04bff700 1 -- 172.21.15.156:0/2871820740 wait complete. 2023-11-28T13:10:50.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.268+0000 7f7d04bff700 1 Processor -- start 2023-11-28T13:10:50.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.268+0000 7f7d04bff700 1 -- start start 2023-11-28T13:10:50.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.269+0000 7f7d04bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d000cd7c0 0x7f7d000c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:50.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.269+0000 7f7d04bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d0015c9b0 0x7f7d000c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:50.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.269+0000 7f7d04bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d00161340 0x7f7d00153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:50.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.269+0000 7f7d04bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d000c8070 con 0x7f7d00161340 2023-11-28T13:10:50.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.269+0000 7f7d04bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d000c71e0 con 0x7f7d0015c9b0 2023-11-28T13:10:50.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.269+0000 7f7d04bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d00153f70 con 0x7f7d000cd7c0 2023-11-28T13:10:50.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.270+0000 7f7cfed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d00161340 0x7f7d00153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:50.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.270+0000 7f7cfdd9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d0015c9b0 0x7f7d000c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:50.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.270+0000 7f7cfdd9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d0015c9b0 0x7f7d000c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:57880/0 (socket says 172.21.15.156:57880) 2023-11-28T13:10:50.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.270+0000 7f7cfe59c700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d000cd7c0 0x7f7d000c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:50.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.270+0000 7f7cfdd9b700 1 -- 172.21.15.156:0/429494411 learned_addr learned my addr 172.21.15.156:0/429494411 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:50.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.270+0000 7f7cfe59c700 1 -- 172.21.15.156:0/429494411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d0015c9b0 msgr2=0x7f7d000c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:50.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.270+0000 7f7cfe59c700 1 --2- 172.21.15.156:0/429494411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d0015c9b0 0x7f7d000c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.270+0000 7f7cfe59c700 1 -- 172.21.15.156:0/429494411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d00161340 msgr2=0x7f7d00153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:50.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.270+0000 7f7cfe59c700 1 --2- 172.21.15.156:0/429494411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d00161340 0x7f7d00153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.270+0000 7f7cfe59c700 1 -- 172.21.15.156:0/429494411 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf8009b30 con 0x7f7d000cd7c0 2023-11-28T13:10:50.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.270+0000 7f7cfdd9b700 1 --2- 172.21.15.156:0/429494411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d0015c9b0 0x7f7d000c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:10:50.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.270+0000 7f7cfe59c700 1 --2- 172.21.15.156:0/429494411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d000cd7c0 0x7f7d000c8500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0016000 tx=0x7f7cf0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:50.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.271+0000 7f7ceeffd700 1 -- 172.21.15.156:0/429494411 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf00036a0 con 0x7f7d000cd7c0 2023-11-28T13:10:50.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.271+0000 7f7ceeffd700 1 -- 172.21.15.156:0/429494411 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf001e070 con 0x7f7d000cd7c0 2023-11-28T13:10:50.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.271+0000 7f7ceeffd700 1 -- 172.21.15.156:0/429494411 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0003a60 con 0x7f7d000cd7c0 2023-11-28T13:10:50.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.271+0000 7f7d04bff700 1 -- 172.21.15.156:0/429494411 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf00094a0 con 0x7f7d000cd7c0 2023-11-28T13:10:50.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.271+0000 7f7d04bff700 1 -- 172.21.15.156:0/429494411 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d000d1300 con 0x7f7d000cd7c0 2023-11-28T13:10:50.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.272+0000 7f7d04bff700 1 -- 172.21.15.156:0/429494411 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d000b5520 con 0x7f7d000cd7c0 2023-11-28T13:10:50.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.273+0000 7f7ceeffd700 1 -- 172.21.15.156:0/429494411 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7cf000f050 con 0x7f7d000cd7c0 2023-11-28T13:10:50.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.273+0000 7f7ceeffd700 1 --2- 172.21.15.156:0/429494411 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7ce00682f0 0x7f7ce006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:50.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.274+0000 7f7ceeffd700 1 -- 172.21.15.156:0/429494411 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7cf00869d0 con 0x7f7d000cd7c0 2023-11-28T13:10:50.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.274+0000 7f7cfdd9b700 1 --2- 172.21.15.156:0/429494411 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7ce00682f0 0x7f7ce006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:50.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.275+0000 7f7cfdd9b700 1 --2- 172.21.15.156:0/429494411 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7ce00682f0 0x7f7ce006a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7d001e88c0 tx=0x7f7cf4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:50.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.279+0000 7f7ceeffd700 1 -- 172.21.15.156:0/429494411 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7cf0053490 con 0x7f7d000cd7c0 2023-11-28T13:10:50.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.541+0000 7f7d04bff700 1 -- 172.21.15.156:0/429494411 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7d000d15e0 con 0x7f7d000cd7c0 2023-11-28T13:10:50.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.542+0000 7f7ceeffd700 1 -- 172.21.15.156:0/429494411 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f7cf0020090 con 0x7f7d000cd7c0 2023-11-28T13:10:50.544 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:50.544 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:10:50.544 INFO:teuthology.orchestra.run.smithi156.stdout: "auth_dump": [ 2023-11-28T13:10:50.544 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.544 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.a", 2023-11-28T13:10:50.544 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVl34w0ERAA9H2i6sMS60YUKvNCit0idg==", 2023-11-28T13:10:50.544 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.545 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:10:50.545 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T13:10:50.545 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T13:10:50.545 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:10:50.545 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.545 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.545 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.545 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.b", 2023-11-28T13:10:50.545 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVlzVSSKRAA3APxxYZXNOOOnG1t7DIIDg==", 2023-11-28T13:10:50.545 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.546 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:10:50.546 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T13:10:50.546 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T13:10:50.546 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:10:50.546 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.546 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.546 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.546 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.c", 2023-11-28T13:10:50.546 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVl++RtHRAAsenmAb/Y3eRvWw22940lkw==", 2023-11-28T13:10:50.546 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.546 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:10:50.546 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T13:10:50.547 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T13:10:50.547 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:10:50.547 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.547 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.547 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.547 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mds.d", 2023-11-28T13:10:50.547 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVlFDP1NRAASlTZM2P0RL+lcFskrr08uQ==", 2023-11-28T13:10:50.547 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.547 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:10:50.547 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T13:10:50.547 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T13:10:50.548 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:10:50.548 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.548 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.548 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.548 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.0", 2023-11-28T13:10:50.548 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD95WVlF1g2BBAAM1AkiYFhmzcy6f2xOUHuSA==", 2023-11-28T13:10:50.548 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.548 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:10:50.548 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:10:50.548 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:10:50.548 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.548 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.549 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.549 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.1", 2023-11-28T13:10:50.549 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAA5mVlMP+VChAAvBK9dslAIznG9cJCnmarfg==", 2023-11-28T13:10:50.549 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.549 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:10:50.549 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:10:50.549 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:10:50.549 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.549 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.549 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.549 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.2", 2023-11-28T13:10:50.549 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAD5mVlY5TMERAAH2W59d+/mWQ7vVOFV0uH5g==", 2023-11-28T13:10:50.549 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.550 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:10:50.550 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:10:50.550 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:10:50.550 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.550 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.550 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.550 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.3", 2023-11-28T13:10:50.550 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAG5mVlKreoGBAAHwbSd1vt4sF/Ahmz97lv5A==", 2023-11-28T13:10:50.550 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.550 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:10:50.550 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:10:50.550 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:10:50.551 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.551 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.551 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.551 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.4", 2023-11-28T13:10:50.551 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAM5mVlVb7BORAA2/l64tOgJl6WRDwI1Ee+YA==", 2023-11-28T13:10:50.551 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.551 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:10:50.551 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:10:50.551 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:10:50.551 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.551 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.552 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.552 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.5", 2023-11-28T13:10:50.552 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAQ5mVlIl0jBRAAYrEb4PGL8amZxfsGUhE0BQ==", 2023-11-28T13:10:50.552 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.552 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:10:50.552 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:10:50.552 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:10:50.552 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.552 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.552 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.552 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.6", 2023-11-28T13:10:50.552 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAT5mVlicggDRAANowLXao2cG7JHDi+ytdREg==", 2023-11-28T13:10:50.553 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.553 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:10:50.553 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:10:50.553 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:10:50.553 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.553 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.553 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.553 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "osd.7", 2023-11-28T13:10:50.553 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQAW5mVlPTz9EhAAyyq2SF/TYYStii1bExbDew==", 2023-11-28T13:10:50.553 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.553 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow profile osd", 2023-11-28T13:10:50.553 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile osd", 2023-11-28T13:10:50.554 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:10:50.554 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.554 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.554 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.554 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.0", 2023-11-28T13:10:50.554 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD75WVlhwXEBBAAu/NSgck5Joi7bKQoBne7MA==", 2023-11-28T13:10:50.554 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.554 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:10:50.554 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow r", 2023-11-28T13:10:50.554 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T13:10:50.554 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.555 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.555 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.555 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.1", 2023-11-28T13:10:50.555 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD75WVlSLGyFBAATUac8v+U0A55jCLv6SCsAA==", 2023-11-28T13:10:50.555 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.555 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:10:50.555 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T13:10:50.555 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T13:10:50.555 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T13:10:50.555 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.555 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.555 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.556 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.2", 2023-11-28T13:10:50.556 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD75WVl49+oDBAA/rsj8joMrNDo2FOlvvLvWQ==", 2023-11-28T13:10:50.556 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.556 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:10:50.556 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T13:10:50.556 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T13:10:50.556 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T13:10:50.556 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.556 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.556 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.556 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.3", 2023-11-28T13:10:50.556 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD75WVlmfuWHBAAbf1HvrNxzm3zJwTF6De2Kg==", 2023-11-28T13:10:50.557 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.557 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow", 2023-11-28T13:10:50.557 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow r", 2023-11-28T13:10:50.557 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow rw", 2023-11-28T13:10:50.557 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow rwx" 2023-11-28T13:10:50.557 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.557 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.557 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.557 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "client.admin", 2023-11-28T13:10:50.557 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD55WVld9ddFBAA6s9ybzLqB7e7Uj9gOd65/Q==", 2023-11-28T13:10:50.557 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.557 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T13:10:50.558 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": "allow *", 2023-11-28T13:10:50.558 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow *", 2023-11-28T13:10:50.558 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:10:50.558 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.558 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.558 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.559 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mgr.x", 2023-11-28T13:10:50.559 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVl3oYaCRAALjx/GgP3kX/gyD4ML7Af7Q==", 2023-11-28T13:10:50.559 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.559 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T13:10:50.559 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile mgr", 2023-11-28T13:10:50.559 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:10:50.559 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.560 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:50.560 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:50.560 INFO:teuthology.orchestra.run.smithi156.stdout: "entity": "mgr.y", 2023-11-28T13:10:50.560 INFO:teuthology.orchestra.run.smithi156.stdout: "key": "AQD65WVlhmWPABAA5F0d66op3eUWwWPKJ6Mc6w==", 2023-11-28T13:10:50.560 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": { 2023-11-28T13:10:50.560 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": "allow *", 2023-11-28T13:10:50.560 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": "allow profile mgr", 2023-11-28T13:10:50.560 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": "allow *" 2023-11-28T13:10:50.561 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.561 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:50.561 INFO:teuthology.orchestra.run.smithi156.stdout: ] 2023-11-28T13:10:50.561 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:10:50.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.544+0000 7f7d04bff700 1 -- 172.21.15.156:0/429494411 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7ce00682f0 msgr2=0x7f7ce006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:50.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.544+0000 7f7d04bff700 1 --2- 172.21.15.156:0/429494411 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7ce00682f0 0x7f7ce006a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7d001e88c0 tx=0x7f7cf4009040 comp rx=0 tx=0).stop 2023-11-28T13:10:50.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.544+0000 7f7d04bff700 1 -- 172.21.15.156:0/429494411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d000cd7c0 msgr2=0x7f7d000c8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:50.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.544+0000 7f7d04bff700 1 --2- 172.21.15.156:0/429494411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d000cd7c0 0x7f7d000c8500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0016000 tx=0x7f7cf0004750 comp rx=0 tx=0).stop 2023-11-28T13:10:50.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.545+0000 7f7d04bff700 1 -- 172.21.15.156:0/429494411 shutdown_connections 2023-11-28T13:10:50.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.545+0000 7f7d04bff700 1 --2- 172.21.15.156:0/429494411 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7ce00682f0 0x7f7ce006a7a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.545+0000 7f7d04bff700 1 --2- 172.21.15.156:0/429494411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d00161340 0x7f7d00153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.545+0000 7f7d04bff700 1 --2- 172.21.15.156:0/429494411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7d000cd7c0 0x7f7d000c8500 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.545+0000 7f7d04bff700 1 --2- 172.21.15.156:0/429494411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d0015c9b0 0x7f7d000c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.545+0000 7f7d04bff700 1 -- 172.21.15.156:0/429494411 >> 172.21.15.156:0/429494411 conn(0x7f7d00072600 msgr2=0x7f7d0015a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:50.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.545+0000 7f7d04bff700 1 -- 172.21.15.156:0/429494411 shutdown_connections 2023-11-28T13:10:50.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.545+0000 7f7d04bff700 1 -- 172.21.15.156:0/429494411 wait complete. 2023-11-28T13:10:50.565 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:10:50.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.760+0000 7f4e087ff700 1 Processor -- start 2023-11-28T13:10:50.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.760+0000 7f4e087ff700 1 -- start start 2023-11-28T13:10:50.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.762+0000 7f4e087ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4e000cd7c0 0x7f4e000cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:50.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.762+0000 7f4e087ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e000d1950 con 0x7f4e0015c9b0 2023-11-28T13:10:50.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.762+0000 7f4e087ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e000ce140 con 0x7f4e00161340 2023-11-28T13:10:50.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.762+0000 7f4e087ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e000b5800 con 0x7f4e000cd7c0 2023-11-28T13:10:50.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.762+0000 7f4e0659b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4e0015c9b0 0x7f4e000b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37790/0 (socket says 172.21.15.156:37790) 2023-11-28T13:10:50.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.762+0000 7f4e0659b700 1 -- 172.21.15.156:0/2264306677 learned_addr learned my addr 172.21.15.156:0/2264306677 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:50.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.762+0000 7f4e06d9c700 1 --2- 172.21.15.156:0/2264306677 >> v2:172.21.15.186:3300/0 conn(0x7f4e000cd7c0 0x7f4e000cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:50.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.762+0000 7f4e05599700 1 -- 172.21.15.156:0/2264306677 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3314614504 0 0) 0x7f4e000d1950 con 0x7f4e0015c9b0 2023-11-28T13:10:50.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.762+0000 7f4e05599700 1 -- 172.21.15.156:0/2264306677 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dec0035b0 con 0x7f4e0015c9b0 2023-11-28T13:10:50.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e05599700 1 -- 172.21.15.156:0/2264306677 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1513317223 0 0) 0x7f4dec0035b0 con 0x7f4e0015c9b0 2023-11-28T13:10:50.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e05599700 1 -- 172.21.15.156:0/2264306677 >> v2:172.21.15.186:3300/0 conn(0x7f4e000cd7c0 msgr2=0x7f4e000cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:50.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e05599700 1 --2- 172.21.15.156:0/2264306677 >> v2:172.21.15.186:3300/0 conn(0x7f4e000cd7c0 0x7f4e000cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e05599700 1 -- 172.21.15.156:0/2264306677 >> v1:172.21.15.186:6789/0 conn(0x7f4e00161340 legacy=0x7f4e000d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:50.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e05599700 1 -- 172.21.15.156:0/2264306677 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e000d24f0 con 0x7f4e0015c9b0 2023-11-28T13:10:50.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e05599700 1 -- 172.21.15.156:0/2264306677 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f4dfc001670 con 0x7f4e0015c9b0 2023-11-28T13:10:50.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e05599700 1 -- 172.21.15.156:0/2264306677 >> v1:172.21.15.156:6789/0 conn(0x7f4e0015c9b0 legacy=0x7f4e000b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:50.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e05599700 1 --2- 172.21.15.156:0/2264306677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dec003a00 0x7f4dec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:50.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e05599700 1 --2- 172.21.15.156:0/2264306677 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dec0043f0 0x7f4dec0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:50.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e05599700 1 --2- 172.21.15.156:0/2264306677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dec005150 0x7f4dec00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:50.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e05599700 1 -- 172.21.15.156:0/2264306677 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e000b5800 con 0x7f4dec003a00 2023-11-28T13:10:50.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e05599700 1 -- 172.21.15.156:0/2264306677 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec00c080 con 0x7f4dec005150 2023-11-28T13:10:50.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e05599700 1 -- 172.21.15.156:0/2264306677 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec00c240 con 0x7f4dec0043f0 2023-11-28T13:10:50.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e0659b700 1 --2- 172.21.15.156:0/2264306677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dec003a00 0x7f4dec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:50.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e06d9c700 1 --2- 172.21.15.156:0/2264306677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dec005150 0x7f4dec00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:50.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e05d9a700 1 --2- 172.21.15.156:0/2264306677 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dec0043f0 0x7f4dec0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:50.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e0659b700 1 -- 172.21.15.156:0/2264306677 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dec0043f0 msgr2=0x7f4dec0048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:50.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e0659b700 1 --2- 172.21.15.156:0/2264306677 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4dec0043f0 0x7f4dec0048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e0659b700 1 -- 172.21.15.156:0/2264306677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dec005150 msgr2=0x7f4dec00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:50.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e0659b700 1 --2- 172.21.15.156:0/2264306677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dec005150 0x7f4dec00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e0659b700 1 -- 172.21.15.156:0/2264306677 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec0035b0 con 0x7f4dec003a00 2023-11-28T13:10:50.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.763+0000 7f4e06d9c700 1 -- 172.21.15.156:0/2264306677 reap_dead start 2023-11-28T13:10:50.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.764+0000 7f4e0659b700 1 --2- 172.21.15.156:0/2264306677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dec003a00 0x7f4dec003dd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00c2f0 tx=0x7f4dec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdaa3aad6f59bd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:50.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.764+0000 7f4e05599700 1 -- 172.21.15.156:0/2264306677 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc00cdf0 con 0x7f4dec003a00 2023-11-28T13:10:50.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.764+0000 7f4e05599700 1 -- 172.21.15.156:0/2264306677 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dfc005960 con 0x7f4dec003a00 2023-11-28T13:10:50.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.764+0000 7f4e05599700 1 -- 172.21.15.156:0/2264306677 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc005d10 con 0x7f4dec003a00 2023-11-28T13:10:50.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.764+0000 7f4e087ff700 1 -- 172.21.15.156:0/2264306677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dec003a00 msgr2=0x7f4dec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:50.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.764+0000 7f4e087ff700 1 --2- 172.21.15.156:0/2264306677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dec003a00 0x7f4dec003dd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00c2f0 tx=0x7f4dec005880 comp rx=0 tx=0).stop 2023-11-28T13:10:50.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.764+0000 7f4e087ff700 1 -- 172.21.15.156:0/2264306677 shutdown_connections 2023-11-28T13:10:50.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.764+0000 7f4e087ff700 1 --2- 172.21.15.156:0/2264306677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dec003a00 0x7f4dec003dd0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.764+0000 7f4e087ff700 1 -- 172.21.15.156:0/2264306677 >> 172.21.15.156:0/2264306677 conn(0x7f4e00072600 msgr2=0x7f4e000cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:50.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.764+0000 7f4e087ff700 1 -- 172.21.15.156:0/2264306677 shutdown_connections 2023-11-28T13:10:50.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.764+0000 7f4e087ff700 1 -- 172.21.15.156:0/2264306677 wait complete. 2023-11-28T13:10:50.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.766+0000 7f4e087ff700 1 Processor -- start 2023-11-28T13:10:50.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.766+0000 7f4e087ff700 1 -- start start 2023-11-28T13:10:50.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.767+0000 7f4e087ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dec003a00 0x7f4e000c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:50.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.767+0000 7f4e087ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e001efdd0 0x7f4e000c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:50.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.767+0000 7f4e087ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e0015e050 0x7f4e0015e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:50.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.767+0000 7f4e087ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e000cbc40 con 0x7f4e0015e050 2023-11-28T13:10:50.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.767+0000 7f4e087ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0015ea00 con 0x7f4dec003a00 2023-11-28T13:10:50.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.767+0000 7f4e087ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e001f0240 con 0x7f4e001efdd0 2023-11-28T13:10:50.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.767+0000 7f4e06d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e0015e050 0x7f4e0015e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:50.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.767+0000 7f4e0659b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dec003a00 0x7f4e000c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:50.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.767+0000 7f4e06d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e0015e050 0x7f4e0015e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49312/0 (socket says 172.21.15.156:49312) 2023-11-28T13:10:50.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.767+0000 7f4e06d9c700 1 -- 172.21.15.156:0/3579633648 learned_addr learned my addr 172.21.15.156:0/3579633648 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.767+0000 7f4e05d9a700 1 --2- 172.21.15.156:0/3579633648 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e001efdd0 0x7f4e000c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.768+0000 7f4e06d9c700 1 -- 172.21.15.156:0/3579633648 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e001efdd0 msgr2=0x7f4e000c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.768+0000 7f4e06d9c700 1 --2- 172.21.15.156:0/3579633648 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e001efdd0 0x7f4e000c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.768+0000 7f4e06d9c700 1 -- 172.21.15.156:0/3579633648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dec003a00 msgr2=0x7f4e000c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.768+0000 7f4e06d9c700 1 --2- 172.21.15.156:0/3579633648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dec003a00 0x7f4e000c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.768+0000 7f4e06d9c700 1 -- 172.21.15.156:0/3579633648 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e000d1300 con 0x7f4e0015e050 2023-11-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.768+0000 7f4e06d9c700 1 --2- 172.21.15.156:0/3579633648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e0015e050 0x7f4e0015e4c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4e000d05d0 tx=0x7f4df80111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.768+0000 7f4df6ffd700 1 -- 172.21.15.156:0/3579633648 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df800e070 con 0x7f4e0015e050 2023-11-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.768+0000 7f4df6ffd700 1 -- 172.21.15.156:0/3579633648 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df8003cf0 con 0x7f4e0015e050 2023-11-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.768+0000 7f4e087ff700 1 -- 172.21.15.156:0/3579633648 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e000b5520 con 0x7f4e0015e050 2023-11-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.768+0000 7f4df6ffd700 1 -- 172.21.15.156:0/3579633648 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df8003e80 con 0x7f4e0015e050 2023-11-28T13:10:50.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.768+0000 7f4e087ff700 1 -- 172.21.15.156:0/3579633648 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e000d15e0 con 0x7f4e0015e050 2023-11-28T13:10:50.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.770+0000 7f4e087ff700 1 -- 172.21.15.156:0/3579633648 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e000c6ca0 con 0x7f4e0015e050 2023-11-28T13:10:50.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.770+0000 7f4df6ffd700 1 -- 172.21.15.156:0/3579633648 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4df80117d0 con 0x7f4e0015e050 2023-11-28T13:10:50.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.771+0000 7f4df6ffd700 1 --2- 172.21.15.156:0/3579633648 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4de40682f0 0x7f4de406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:50.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.771+0000 7f4df6ffd700 1 -- 172.21.15.156:0/3579633648 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4df8085da0 con 0x7f4e0015e050 2023-11-28T13:10:50.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.772+0000 7f4e0659b700 1 --2- 172.21.15.156:0/3579633648 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4de40682f0 0x7f4de406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:50.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.772+0000 7f4e0659b700 1 --2- 172.21.15.156:0/3579633648 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4de40682f0 0x7f4de406a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc000c00 tx=0x7f4dfc007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:50.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.775+0000 7f4df6ffd700 1 -- 172.21.15.156:0/3579633648 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4e000c6ca0 con 0x7f4e0015e050 2023-11-28T13:10:50.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.968+0000 7f4e087ff700 1 -- 172.21.15.156:0/3579633648 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e000c6ca0 con 0x7f4e0015e050 2023-11-28T13:10:50.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.969+0000 7f4df6ffd700 1 -- 172.21.15.156:0/3579633648 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+14257 (secure 0 0 0) 0x7f4e000c6ca0 con 0x7f4e0015e050 2023-11-28T13:10:50.971 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:50.971 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":55,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:10:50.037762+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:10:50.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.971+0000 7f4e087ff700 1 -- 172.21.15.156:0/3579633648 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4de40682f0 msgr2=0x7f4de406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:50.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.971+0000 7f4e087ff700 1 --2- 172.21.15.156:0/3579633648 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4de40682f0 0x7f4de406a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc000c00 tx=0x7f4dfc007000 comp rx=0 tx=0).stop 2023-11-28T13:10:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.971+0000 7f4e087ff700 1 -- 172.21.15.156:0/3579633648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e0015e050 msgr2=0x7f4e0015e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:50.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.971+0000 7f4e087ff700 1 --2- 172.21.15.156:0/3579633648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e0015e050 0x7f4e0015e4c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4e000d05d0 tx=0x7f4df80111d0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.971+0000 7f4e087ff700 1 -- 172.21.15.156:0/3579633648 shutdown_connections 2023-11-28T13:10:50.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.971+0000 7f4e087ff700 1 --2- 172.21.15.156:0/3579633648 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4de40682f0 0x7f4de406a7a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.971+0000 7f4e087ff700 1 --2- 172.21.15.156:0/3579633648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e0015e050 0x7f4e0015e4c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.971+0000 7f4e087ff700 1 --2- 172.21.15.156:0/3579633648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dec003a00 0x7f4e000c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.971+0000 7f4e087ff700 1 --2- 172.21.15.156:0/3579633648 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e001efdd0 0x7f4e000c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:50.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.971+0000 7f4e087ff700 1 -- 172.21.15.156:0/3579633648 >> 172.21.15.156:0/3579633648 conn(0x7f4e00072600 msgr2=0x7f4e0015a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:50.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.972+0000 7f4e087ff700 1 -- 172.21.15.156:0/3579633648 shutdown_connections 2023-11-28T13:10:50.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:50.972+0000 7f4e087ff700 1 -- 172.21.15.156:0/3579633648 wait complete. 2023-11-28T13:10:50.992 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:10:50.993 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:10:51.193 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.192+0000 7f0088bff700 1 Processor -- start 2023-11-28T13:10:51.194 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.192+0000 7f0088bff700 1 -- start start 2023-11-28T13:10:51.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.193+0000 7f0088bff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f008415c9b0 0x7f00840b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:51.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.193+0000 7f0088bff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00840d1950 con 0x7f0084161340 2023-11-28T13:10:51.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.193+0000 7f0088bff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00840d24f0 con 0x7f00840cd7c0 2023-11-28T13:10:51.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.193+0000 7f0088bff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00840b5800 con 0x7f008415c9b0 2023-11-28T13:10:51.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.193+0000 7f0081d9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f0084161340 0x7f00840d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37796/0 (socket says 172.21.15.156:37796) 2023-11-28T13:10:51.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.193+0000 7f0081d9b700 1 -- 172.21.15.156:0/4140355211 learned_addr learned my addr 172.21.15.156:0/4140355211 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:51.195 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.194+0000 7f008259c700 1 --2- 172.21.15.156:0/4140355211 >> v2:172.21.15.186:3300/0 conn(0x7f008415c9b0 0x7f00840b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:51.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.194+0000 7f008159a700 1 -- 172.21.15.156:0/4140355211 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437778226 0 0) 0x7f00840d1950 con 0x7f0084161340 2023-11-28T13:10:51.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.194+0000 7f008159a700 1 -- 172.21.15.156:0/4140355211 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f006c0035b0 con 0x7f0084161340 2023-11-28T13:10:51.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.194+0000 7f008259c700 1 -- 172.21.15.156:0/4140355211 >> v1:172.21.15.186:6789/0 conn(0x7f00840cd7c0 legacy=0x7f00840b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:51.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.194+0000 7f008159a700 1 -- 172.21.15.156:0/4140355211 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1577308012 0 0) 0x7f00840d24f0 con 0x7f00840cd7c0 2023-11-28T13:10:51.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.194+0000 7f008259c700 1 -- 172.21.15.156:0/4140355211 >> v1:172.21.15.156:6789/0 conn(0x7f0084161340 legacy=0x7f00840d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:51.197 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.194+0000 7f008259c700 1 -- 172.21.15.156:0/4140355211 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00840b5520 con 0x7f008415c9b0 2023-11-28T13:10:51.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.194+0000 7f008259c700 1 --2- 172.21.15.156:0/4140355211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f008415c9b0 0x7f00840b4bf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0074009840 tx=0x7f0074009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7568c68c0de4c925 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:51.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.195+0000 7f008159a700 1 -- 172.21.15.156:0/4140355211 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074011830 con 0x7f008415c9b0 2023-11-28T13:10:51.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.195+0000 7f008159a700 1 -- 172.21.15.156:0/4140355211 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0074011990 con 0x7f008415c9b0 2023-11-28T13:10:51.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.195+0000 7f008159a700 1 -- 172.21.15.156:0/4140355211 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074011d20 con 0x7f008415c9b0 2023-11-28T13:10:51.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.195+0000 7f0088bff700 1 -- 172.21.15.156:0/4140355211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f008415c9b0 msgr2=0x7f00840b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:51.198 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.195+0000 7f0088bff700 1 --2- 172.21.15.156:0/4140355211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f008415c9b0 0x7f00840b4bf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0074009840 tx=0x7f0074009b50 comp rx=0 tx=0).stop 2023-11-28T13:10:51.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.195+0000 7f0088bff700 1 -- 172.21.15.156:0/4140355211 shutdown_connections 2023-11-28T13:10:51.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.195+0000 7f0088bff700 1 --2- 172.21.15.156:0/4140355211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f008415c9b0 0x7f00840b4bf0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.195+0000 7f0088bff700 1 -- 172.21.15.156:0/4140355211 >> 172.21.15.156:0/4140355211 conn(0x7f0084072600 msgr2=0x7f00840cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:51.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.195+0000 7f0088bff700 1 -- 172.21.15.156:0/4140355211 shutdown_connections 2023-11-28T13:10:51.199 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.196+0000 7f0088bff700 1 -- 172.21.15.156:0/4140355211 wait complete. 2023-11-28T13:10:51.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.197+0000 7f0088bff700 1 Processor -- start 2023-11-28T13:10:51.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.197+0000 7f0088bff700 1 -- start start 2023-11-28T13:10:51.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.198+0000 7f0088bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00840cd7c0 0x7f0084156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:51.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.198+0000 7f0088bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f008415c9b0 0x7f00840cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:51.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.198+0000 7f0088bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0084161340 0x7f00840caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:51.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.198+0000 7f0088bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00840cd170 con 0x7f0084161340 2023-11-28T13:10:51.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.198+0000 7f0088bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00840ccf20 con 0x7f00840cd7c0 2023-11-28T13:10:51.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.198+0000 7f0088bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00841f6e40 con 0x7f008415c9b0 2023-11-28T13:10:51.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.198+0000 7f0082d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0084161340 0x7f00840caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:51.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.199+0000 7f0082d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0084161340 0x7f00840caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49320/0 (socket says 172.21.15.156:49320) 2023-11-28T13:10:51.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.199+0000 7f0082d9d700 1 -- 172.21.15.156:0/1880035935 learned_addr learned my addr 172.21.15.156:0/1880035935 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:51.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.199+0000 7f008259c700 1 --2- 172.21.15.156:0/1880035935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00840cd7c0 0x7f0084156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:51.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.199+0000 7f0081d9b700 1 --2- 172.21.15.156:0/1880035935 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f008415c9b0 0x7f00840cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:51.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.199+0000 7f0082d9d700 1 -- 172.21.15.156:0/1880035935 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f008415c9b0 msgr2=0x7f00840cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:51.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.199+0000 7f0082d9d700 1 --2- 172.21.15.156:0/1880035935 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f008415c9b0 0x7f00840cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.199+0000 7f0082d9d700 1 -- 172.21.15.156:0/1880035935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00840cd7c0 msgr2=0x7f0084156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:51.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.199+0000 7f0082d9d700 1 --2- 172.21.15.156:0/1880035935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00840cd7c0 0x7f0084156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.199+0000 7f0082d9d700 1 -- 172.21.15.156:0/1880035935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00740094a0 con 0x7f0084161340 2023-11-28T13:10:51.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.199+0000 7f0082d9d700 1 --2- 172.21.15.156:0/1880035935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0084161340 0x7f00840caac0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f007c00c7d0 tx=0x7f007c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:51.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.199+0000 7f0072ffd700 1 -- 172.21.15.156:0/1880035935 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c003e30 con 0x7f0084161340 2023-11-28T13:10:51.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.199+0000 7f0072ffd700 1 -- 172.21.15.156:0/1880035935 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007c005070 con 0x7f0084161340 2023-11-28T13:10:51.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.199+0000 7f0088bff700 1 -- 172.21.15.156:0/1880035935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00840d15e0 con 0x7f0084161340 2023-11-28T13:10:51.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.199+0000 7f0072ffd700 1 -- 172.21.15.156:0/1880035935 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c005420 con 0x7f0084161340 2023-11-28T13:10:51.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.200+0000 7f0088bff700 1 -- 172.21.15.156:0/1880035935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00840d1300 con 0x7f0084161340 2023-11-28T13:10:51.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.201+0000 7f0088bff700 1 -- 172.21.15.156:0/1880035935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00840745c0 con 0x7f0084161340 2023-11-28T13:10:51.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.202+0000 7f0072ffd700 1 -- 172.21.15.156:0/1880035935 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f007c019030 con 0x7f0084161340 2023-11-28T13:10:51.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.203+0000 7f0072ffd700 1 --2- 172.21.15.156:0/1880035935 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f00640682f0 0x7f006406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:51.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.203+0000 7f0072ffd700 1 -- 172.21.15.156:0/1880035935 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f007c084e70 con 0x7f0084161340 2023-11-28T13:10:51.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.203+0000 7f008259c700 1 --2- 172.21.15.156:0/1880035935 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f00640682f0 0x7f006406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:51.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.204+0000 7f008259c700 1 --2- 172.21.15.156:0/1880035935 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f00640682f0 0x7f006406a7a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0074009fd0 tx=0x7f007401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:51.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.207+0000 7f0072ffd700 1 -- 172.21.15.156:0/1880035935 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f007c051930 con 0x7f0084161340 2023-11-28T13:10:51.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.396+0000 7f0088bff700 1 -- 172.21.15.156:0/1880035935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f00840d2090 con 0x7f0084161340 2023-11-28T13:10:51.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.397+0000 7f0072ffd700 1 -- 172.21.15.156:0/1880035935 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7f007c017020 con 0x7f0084161340 2023-11-28T13:10:51.399 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:10:51.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.399+0000 7f0088bff700 1 -- 172.21.15.156:0/1880035935 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f00640682f0 msgr2=0x7f006406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:51.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.399+0000 7f0088bff700 1 --2- 172.21.15.156:0/1880035935 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f00640682f0 0x7f006406a7a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0074009fd0 tx=0x7f007401c040 comp rx=0 tx=0).stop 2023-11-28T13:10:51.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.399+0000 7f0088bff700 1 -- 172.21.15.156:0/1880035935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0084161340 msgr2=0x7f00840caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:51.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.399+0000 7f0088bff700 1 --2- 172.21.15.156:0/1880035935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0084161340 0x7f00840caac0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f007c00c7d0 tx=0x7f007c00cae0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.399+0000 7f0088bff700 1 -- 172.21.15.156:0/1880035935 shutdown_connections 2023-11-28T13:10:51.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.399+0000 7f0088bff700 1 --2- 172.21.15.156:0/1880035935 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f00640682f0 0x7f006406a7a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.399+0000 7f0088bff700 1 --2- 172.21.15.156:0/1880035935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0084161340 0x7f00840caac0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.399+0000 7f0088bff700 1 --2- 172.21.15.156:0/1880035935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00840cd7c0 0x7f0084156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.399+0000 7f0088bff700 1 --2- 172.21.15.156:0/1880035935 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f008415c9b0 0x7f00840cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.399+0000 7f0088bff700 1 -- 172.21.15.156:0/1880035935 >> 172.21.15.156:0/1880035935 conn(0x7f0084072600 msgr2=0x7f0084160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:51.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.399+0000 7f0088bff700 1 -- 172.21.15.156:0/1880035935 shutdown_connections 2023-11-28T13:10:51.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.399+0000 7f0088bff700 1 -- 172.21.15.156:0/1880035935 wait complete. 2023-11-28T13:10:51.422 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:10:51.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.620+0000 7f9cab9ff700 1 Processor -- start 2023-11-28T13:10:51.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.621+0000 7f9cab9ff700 1 -- start start 2023-11-28T13:10:51.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.622+0000 7f9cab9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca4161340 0x7f9ca40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:51.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.622+0000 7f9cab9ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9ca40d2320 0x7f9ca40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:51.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.622+0000 7f9cab9ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca40d1950 con 0x7f9ca415c9b0 2023-11-28T13:10:51.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.622+0000 7f9cab9ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca40b5800 con 0x7f9ca40d2320 2023-11-28T13:10:51.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.622+0000 7f9cab9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca40b5300 con 0x7f9ca4161340 2023-11-28T13:10:51.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.622+0000 7f9ca8f9a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca4161340 0x7f9ca40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:51.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.622+0000 7f9ca9f9c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9ca40d2320 0x7f9ca40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:51.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.622+0000 7f9ca9f9c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9ca40d2320 0x7f9ca40cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:57940/0 (socket says 172.21.15.156:57940) 2023-11-28T13:10:51.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.622+0000 7f9ca8f9a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca4161340 0x7f9ca40d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:39308/0 (socket says 172.21.15.156:39308) 2023-11-28T13:10:51.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.622+0000 7f9ca9f9c700 1 -- 172.21.15.156:0/1862967196 learned_addr learned my addr 172.21.15.156:0/1862967196 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:51.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.623+0000 7f9ca8f9a700 1 -- 172.21.15.156:0/1862967196 >> v2:172.21.15.186:3300/0 conn(0x7f9ca40d2320 msgr2=0x7f9ca40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:51.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.623+0000 7f9ca8f9a700 1 --2- 172.21.15.156:0/1862967196 >> v2:172.21.15.186:3300/0 conn(0x7f9ca40d2320 0x7f9ca40cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.623+0000 7f9ca8f9a700 1 -- 172.21.15.156:0/1862967196 >> v1:172.21.15.186:6789/0 conn(0x7f9ca415c9b0 legacy=0x7f9ca40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:51.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.623+0000 7f9ca8f9a700 1 -- 172.21.15.156:0/1862967196 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca40b5520 con 0x7f9ca4161340 2023-11-28T13:10:51.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.623+0000 7f9ca9f9c700 1 --2- 172.21.15.156:0/1862967196 >> v2:172.21.15.186:3300/0 conn(0x7f9ca40d2320 0x7f9ca40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:10:51.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.623+0000 7f9ca8f9a700 1 --2- 172.21.15.156:0/1862967196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca4161340 0x7f9ca40d1de0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009840 tx=0x7f9c94009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db5c6c1bb40d0598 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:51.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.623+0000 7f9c9bfff700 1 -- 172.21.15.156:0/1862967196 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94011830 con 0x7f9ca4161340 2023-11-28T13:10:51.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.624+0000 7f9c9bfff700 1 -- 172.21.15.156:0/1862967196 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c94011990 con 0x7f9ca4161340 2023-11-28T13:10:51.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.624+0000 7f9c9bfff700 1 -- 172.21.15.156:0/1862967196 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94011d20 con 0x7f9ca4161340 2023-11-28T13:10:51.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.624+0000 7f9cab9ff700 1 -- 172.21.15.156:0/1862967196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca4161340 msgr2=0x7f9ca40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:51.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.624+0000 7f9cab9ff700 1 --2- 172.21.15.156:0/1862967196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca4161340 0x7f9ca40d1de0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009840 tx=0x7f9c94009b50 comp rx=0 tx=0).stop 2023-11-28T13:10:51.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.624+0000 7f9cab9ff700 1 -- 172.21.15.156:0/1862967196 shutdown_connections 2023-11-28T13:10:51.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.624+0000 7f9cab9ff700 1 --2- 172.21.15.156:0/1862967196 >> v2:172.21.15.186:3300/0 conn(0x7f9ca40d2320 0x7f9ca40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.624+0000 7f9cab9ff700 1 --2- 172.21.15.156:0/1862967196 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca4161340 0x7f9ca40d1de0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.624+0000 7f9cab9ff700 1 -- 172.21.15.156:0/1862967196 >> 172.21.15.156:0/1862967196 conn(0x7f9ca4072600 msgr2=0x7f9ca40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:51.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.624+0000 7f9cab9ff700 1 -- 172.21.15.156:0/1862967196 shutdown_connections 2023-11-28T13:10:51.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.624+0000 7f9cab9ff700 1 -- 172.21.15.156:0/1862967196 wait complete. 2023-11-28T13:10:51.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.626+0000 7f9cab9ff700 1 Processor -- start 2023-11-28T13:10:51.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.626+0000 7f9cab9ff700 1 -- start start 2023-11-28T13:10:51.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.627+0000 7f9cab9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca40d2320 0x7f9ca41563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:51.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.627+0000 7f9cab9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca415c9b0 0x7f9ca40c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:51.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.627+0000 7f9cab9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca4161340 0x7f9ca4154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:51.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.627+0000 7f9cab9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4155f30 con 0x7f9ca4161340 2023-11-28T13:10:51.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.627+0000 7f9cab9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4155320 con 0x7f9ca40d2320 2023-11-28T13:10:51.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.627+0000 7f9cab9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4155600 con 0x7f9ca415c9b0 2023-11-28T13:10:51.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.627+0000 7f9ca9f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca4161340 0x7f9ca4154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:51.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.627+0000 7f9ca9f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca4161340 0x7f9ca4154860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49324/0 (socket says 172.21.15.156:49324) 2023-11-28T13:10:51.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.627+0000 7f9ca9f9c700 1 -- 172.21.15.156:0/1704850992 learned_addr learned my addr 172.21.15.156:0/1704850992 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:51.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.628+0000 7f9ca8f9a700 1 --2- 172.21.15.156:0/1704850992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca415c9b0 0x7f9ca40c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:51.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.628+0000 7f9ca979b700 1 --2- 172.21.15.156:0/1704850992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca40d2320 0x7f9ca41563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:51.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.628+0000 7f9ca9f9c700 1 -- 172.21.15.156:0/1704850992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca415c9b0 msgr2=0x7f9ca40c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.628+0000 7f9ca9f9c700 1 --2- 172.21.15.156:0/1704850992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca415c9b0 0x7f9ca40c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.628+0000 7f9ca9f9c700 1 -- 172.21.15.156:0/1704850992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca40d2320 msgr2=0x7f9ca41563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.628+0000 7f9ca9f9c700 1 --2- 172.21.15.156:0/1704850992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca40d2320 0x7f9ca41563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.628+0000 7f9ca9f9c700 1 -- 172.21.15.156:0/1704850992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9c009b30 con 0x7f9ca4161340 2023-11-28T13:10:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.628+0000 7f9ca9f9c700 1 --2- 172.21.15.156:0/1704850992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca4161340 0x7f9ca4154860 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00efc0 tx=0x7f9c9c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.628+0000 7f9c99ffb700 1 -- 172.21.15.156:0/1704850992 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c018070 con 0x7f9ca4161340 2023-11-28T13:10:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.628+0000 7f9c99ffb700 1 -- 172.21.15.156:0/1704850992 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9c00f040 con 0x7f9ca4161340 2023-11-28T13:10:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.628+0000 7f9cab9ff700 1 -- 172.21.15.156:0/1704850992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c940094a0 con 0x7f9ca4161340 2023-11-28T13:10:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.628+0000 7f9c99ffb700 1 -- 172.21.15.156:0/1704850992 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c003c40 con 0x7f9ca4161340 2023-11-28T13:10:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.629+0000 7f9cab9ff700 1 -- 172.21.15.156:0/1704850992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca40d15e0 con 0x7f9ca4161340 2023-11-28T13:10:51.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.630+0000 7f9cab9ff700 1 -- 172.21.15.156:0/1704850992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca40d1300 con 0x7f9ca4161340 2023-11-28T13:10:51.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.630+0000 7f9c99ffb700 1 -- 172.21.15.156:0/1704850992 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9c9c00c880 con 0x7f9ca4161340 2023-11-28T13:10:51.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.632+0000 7f9c99ffb700 1 --2- 172.21.15.156:0/1704850992 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9c880682f0 0x7f9c8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:51.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.632+0000 7f9c99ffb700 1 -- 172.21.15.156:0/1704850992 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9c9c085bc0 con 0x7f9ca4161340 2023-11-28T13:10:51.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.632+0000 7f9ca979b700 1 --2- 172.21.15.156:0/1704850992 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9c880682f0 0x7f9c8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:51.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.633+0000 7f9ca979b700 1 --2- 172.21.15.156:0/1704850992 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9c880682f0 0x7f9c8806a7a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0007fd0 tx=0x7f9ca0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:51.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.636+0000 7f9c99ffb700 1 -- 172.21.15.156:0/1704850992 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9c9c052710 con 0x7f9ca4161340 2023-11-28T13:10:51.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.827+0000 7f9cab9ff700 1 -- 172.21.15.156:0/1704850992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ca415cd80 con 0x7f9ca4161340 2023-11-28T13:10:51.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.827+0000 7f9c99ffb700 1 -- 172.21.15.156:0/1704850992 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c9c009ce0 con 0x7f9ca4161340 2023-11-28T13:10:51.831 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:10:51.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.830+0000 7f9cab9ff700 1 -- 172.21.15.156:0/1704850992 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9c880682f0 msgr2=0x7f9c8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:51.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.830+0000 7f9cab9ff700 1 --2- 172.21.15.156:0/1704850992 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9c880682f0 0x7f9c8806a7a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0007fd0 tx=0x7f9ca0009040 comp rx=0 tx=0).stop 2023-11-28T13:10:51.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.830+0000 7f9cab9ff700 1 -- 172.21.15.156:0/1704850992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca4161340 msgr2=0x7f9ca4154860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:51.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.830+0000 7f9cab9ff700 1 --2- 172.21.15.156:0/1704850992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca4161340 0x7f9ca4154860 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00efc0 tx=0x7f9c9c00c3a0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.830+0000 7f9cab9ff700 1 -- 172.21.15.156:0/1704850992 shutdown_connections 2023-11-28T13:10:51.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.830+0000 7f9cab9ff700 1 --2- 172.21.15.156:0/1704850992 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9c880682f0 0x7f9c8806a7a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.830+0000 7f9cab9ff700 1 --2- 172.21.15.156:0/1704850992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca4161340 0x7f9ca4154860 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.830+0000 7f9cab9ff700 1 --2- 172.21.15.156:0/1704850992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca40d2320 0x7f9ca41563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.830+0000 7f9cab9ff700 1 --2- 172.21.15.156:0/1704850992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ca415c9b0 0x7f9ca40c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:51.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.830+0000 7f9cab9ff700 1 -- 172.21.15.156:0/1704850992 >> 172.21.15.156:0/1704850992 conn(0x7f9ca4072600 msgr2=0x7f9ca415fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:51.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.830+0000 7f9cab9ff700 1 -- 172.21.15.156:0/1704850992 shutdown_connections 2023-11-28T13:10:51.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:51.830+0000 7f9cab9ff700 1 -- 172.21.15.156:0/1704850992 wait complete. 2023-11-28T13:10:51.851 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T13:10:52.050 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.049+0000 7f682d9ff700 1 Processor -- start 2023-11-28T13:10:52.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.049+0000 7f682d9ff700 1 -- start start 2023-11-28T13:10:52.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.050+0000 7f682d9ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f68280d1330 0x7f68280b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:52.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.050+0000 7f682d9ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68280b0230 con 0x7f68280d01b0 2023-11-28T13:10:52.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.050+0000 7f682d9ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68280b4570 con 0x7f68280b20b0 2023-11-28T13:10:52.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.050+0000 7f682d9ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f682815cec0 con 0x7f68280d1330 2023-11-28T13:10:52.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.050+0000 7f6826ffd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f68280d01b0 0x7f68280d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37806/0 (socket says 172.21.15.156:37806) 2023-11-28T13:10:52.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.050+0000 7f6826ffd700 1 -- 172.21.15.156:0/3382329226 learned_addr learned my addr 172.21.15.156:0/3382329226 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:52.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.051+0000 7f68267fc700 1 --2- 172.21.15.156:0/3382329226 >> v2:172.21.15.186:3300/0 conn(0x7f68280d1330 0x7f68280b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:52.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.051+0000 7f6825ffb700 1 -- 172.21.15.156:0/3382329226 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360286391 0 0) 0x7f68280b0230 con 0x7f68280d01b0 2023-11-28T13:10:52.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.051+0000 7f6825ffb700 1 -- 172.21.15.156:0/3382329226 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68100035b0 con 0x7f68280d01b0 2023-11-28T13:10:52.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.051+0000 7f6825ffb700 1 -- 172.21.15.156:0/3382329226 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 235536034 0 0) 0x7f68280b4570 con 0x7f68280b20b0 2023-11-28T13:10:52.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.051+0000 7f6825ffb700 1 -- 172.21.15.156:0/3382329226 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68280b0230 con 0x7f68280b20b0 2023-11-28T13:10:52.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.051+0000 7f68267fc700 1 -- 172.21.15.156:0/3382329226 >> v1:172.21.15.186:6789/0 conn(0x7f68280b20b0 legacy=0x7f68280b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:52.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.051+0000 7f68267fc700 1 -- 172.21.15.156:0/3382329226 >> v1:172.21.15.156:6789/0 conn(0x7f68280d01b0 legacy=0x7f68280d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:52.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.051+0000 7f68267fc700 1 -- 172.21.15.156:0/3382329226 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68280b4850 con 0x7f68280d1330 2023-11-28T13:10:52.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.051+0000 7f68267fc700 1 --2- 172.21.15.156:0/3382329226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68280d1330 0x7f68280b06c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f681c009840 tx=0x7f681c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e04caaee11bed18 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:52.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.052+0000 7f6825ffb700 1 -- 172.21.15.156:0/3382329226 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c011830 con 0x7f68280d1330 2023-11-28T13:10:52.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.052+0000 7f6825ffb700 1 -- 172.21.15.156:0/3382329226 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c011990 con 0x7f68280d1330 2023-11-28T13:10:52.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.052+0000 7f6825ffb700 1 -- 172.21.15.156:0/3382329226 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c011d20 con 0x7f68280d1330 2023-11-28T13:10:52.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.052+0000 7f682d9ff700 1 -- 172.21.15.156:0/3382329226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68280d1330 msgr2=0x7f68280b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:52.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.052+0000 7f682d9ff700 1 --2- 172.21.15.156:0/3382329226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68280d1330 0x7f68280b06c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f681c009840 tx=0x7f681c009b50 comp rx=0 tx=0).stop 2023-11-28T13:10:52.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.052+0000 7f682d9ff700 1 -- 172.21.15.156:0/3382329226 shutdown_connections 2023-11-28T13:10:52.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.052+0000 7f682d9ff700 1 --2- 172.21.15.156:0/3382329226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68280d1330 0x7f68280b06c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:52.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.052+0000 7f682d9ff700 1 -- 172.21.15.156:0/3382329226 >> 172.21.15.156:0/3382329226 conn(0x7f682809c3f0 msgr2=0x7f68280d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:52.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.052+0000 7f682d9ff700 1 -- 172.21.15.156:0/3382329226 shutdown_connections 2023-11-28T13:10:52.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.052+0000 7f682d9ff700 1 -- 172.21.15.156:0/3382329226 wait complete. 2023-11-28T13:10:52.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.054+0000 7f682d9ff700 1 Processor -- start 2023-11-28T13:10:52.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.054+0000 7f682d9ff700 1 -- start start 2023-11-28T13:10:52.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.055+0000 7f682d9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68280b20b0 0x7f68281575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:52.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.055+0000 7f682d9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68280d01b0 0x7f682815d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:52.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.055+0000 7f682d9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68280d1330 0x7f682806ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:52.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.055+0000 7f682d9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682815d970 con 0x7f68280b20b0 2023-11-28T13:10:52.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.055+0000 7f682d9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682815dc50 con 0x7f68280d1330 2023-11-28T13:10:52.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.055+0000 7f682d9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682815df30 con 0x7f68280d01b0 2023-11-28T13:10:52.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.055+0000 7f6826ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68280b20b0 0x7f68281575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:52.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.056+0000 7f6826ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68280b20b0 0x7f68281575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49328/0 (socket says 172.21.15.156:49328) 2023-11-28T13:10:52.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.056+0000 7f6826ffd700 1 -- 172.21.15.156:0/531131821 learned_addr learned my addr 172.21.15.156:0/531131821 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:52.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.056+0000 7f68267fc700 1 --2- 172.21.15.156:0/531131821 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68280d01b0 0x7f682815d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:52.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.056+0000 7f68277fe700 1 --2- 172.21.15.156:0/531131821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68280d1330 0x7f682806ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:52.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.056+0000 7f6826ffd700 1 -- 172.21.15.156:0/531131821 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68280d01b0 msgr2=0x7f682815d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:52.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.056+0000 7f6826ffd700 1 --2- 172.21.15.156:0/531131821 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68280d01b0 0x7f682815d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:52.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.056+0000 7f6826ffd700 1 -- 172.21.15.156:0/531131821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68280d1330 msgr2=0x7f682806ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:52.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.056+0000 7f6826ffd700 1 --2- 172.21.15.156:0/531131821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68280d1330 0x7f682806ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:52.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.056+0000 7f6826ffd700 1 -- 172.21.15.156:0/531131821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c0094a0 con 0x7f68280b20b0 2023-11-28T13:10:52.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.056+0000 7f6826ffd700 1 --2- 172.21.15.156:0/531131821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68280b20b0 0x7f68281575a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f681800c700 tx=0x7f681800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:52.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.056+0000 7f680ffff700 1 -- 172.21.15.156:0/531131821 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818005d70 con 0x7f68280b20b0 2023-11-28T13:10:52.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.056+0000 7f680ffff700 1 -- 172.21.15.156:0/531131821 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68180070d0 con 0x7f68280b20b0 2023-11-28T13:10:52.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.057+0000 7f682d9ff700 1 -- 172.21.15.156:0/531131821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68280afec0 con 0x7f68280b20b0 2023-11-28T13:10:52.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.057+0000 7f680ffff700 1 -- 172.21.15.156:0/531131821 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818007440 con 0x7f68280b20b0 2023-11-28T13:10:52.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.057+0000 7f682d9ff700 1 -- 172.21.15.156:0/531131821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68280afbe0 con 0x7f68280b20b0 2023-11-28T13:10:52.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.058+0000 7f682d9ff700 1 -- 172.21.15.156:0/531131821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68280b2600 con 0x7f68280b20b0 2023-11-28T13:10:52.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.058+0000 7f680ffff700 1 -- 172.21.15.156:0/531131821 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6818019030 con 0x7f68280b20b0 2023-11-28T13:10:52.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.060+0000 7f680ffff700 1 --2- 172.21.15.156:0/531131821 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f68140682f0 0x7f681406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:52.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.060+0000 7f680ffff700 1 -- 172.21.15.156:0/531131821 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6818084b00 con 0x7f68280b20b0 2023-11-28T13:10:52.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.060+0000 7f68267fc700 1 --2- 172.21.15.156:0/531131821 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f68140682f0 0x7f681406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:52.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.061+0000 7f68267fc700 1 --2- 172.21.15.156:0/531131821 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f68140682f0 0x7f681406a7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f68280d1720 tx=0x7f681c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:52.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.064+0000 7f680ffff700 1 -- 172.21.15.156:0/531131821 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f681804b5e0 con 0x7f68280b20b0 2023-11-28T13:10:52.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.261+0000 7f682d9ff700 1 -- 172.21.15.156:0/531131821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6828068690 con 0x7f68280b20b0 2023-11-28T13:10:52.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.261+0000 7f680ffff700 1 -- 172.21.15.156:0/531131821 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v55) v1 ==== 84+0+126 (secure 0 0 0) 0x7f6818017070 con 0x7f68280b20b0 2023-11-28T13:10:52.263 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:52.263 INFO:teuthology.orchestra.run.smithi156.stdout:[ 2023-11-28T13:10:52.263 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:52.263 INFO:teuthology.orchestra.run.smithi156.stdout: "poolnum": 1, 2023-11-28T13:10:52.263 INFO:teuthology.orchestra.run.smithi156.stdout: "poolname": "rbd" 2023-11-28T13:10:52.263 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:10:52.263 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:10:52.263 INFO:teuthology.orchestra.run.smithi156.stdout: "poolnum": 2, 2023-11-28T13:10:52.263 INFO:teuthology.orchestra.run.smithi156.stdout: "poolname": ".mgr" 2023-11-28T13:10:52.264 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:10:52.264 INFO:teuthology.orchestra.run.smithi156.stdout:] 2023-11-28T13:10:52.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.263+0000 7f682d9ff700 1 -- 172.21.15.156:0/531131821 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f68140682f0 msgr2=0x7f681406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:52.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.263+0000 7f682d9ff700 1 --2- 172.21.15.156:0/531131821 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f68140682f0 0x7f681406a7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f68280d1720 tx=0x7f681c01c040 comp rx=0 tx=0).stop 2023-11-28T13:10:52.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.263+0000 7f682d9ff700 1 -- 172.21.15.156:0/531131821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68280b20b0 msgr2=0x7f68281575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:52.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.263+0000 7f682d9ff700 1 --2- 172.21.15.156:0/531131821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68280b20b0 0x7f68281575a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f681800c700 tx=0x7f681800ca10 comp rx=0 tx=0).stop 2023-11-28T13:10:52.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.263+0000 7f682d9ff700 1 -- 172.21.15.156:0/531131821 shutdown_connections 2023-11-28T13:10:52.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.263+0000 7f682d9ff700 1 --2- 172.21.15.156:0/531131821 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f68140682f0 0x7f681406a7a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:52.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.263+0000 7f682d9ff700 1 --2- 172.21.15.156:0/531131821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68280d1330 0x7f682806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:52.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.263+0000 7f682d9ff700 1 --2- 172.21.15.156:0/531131821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68280b20b0 0x7f68281575a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:52.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.263+0000 7f682d9ff700 1 --2- 172.21.15.156:0/531131821 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68280d01b0 0x7f682815d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:52.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.263+0000 7f682d9ff700 1 -- 172.21.15.156:0/531131821 >> 172.21.15.156:0/531131821 conn(0x7f682809c3f0 msgr2=0x7f68280b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:52.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.263+0000 7f682d9ff700 1 -- 172.21.15.156:0/531131821 shutdown_connections 2023-11-28T13:10:52.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.263+0000 7f682d9ff700 1 -- 172.21.15.156:0/531131821 wait complete. 2023-11-28T13:10:52.284 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T13:10:52.284 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T13:10:52.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.484+0000 7f4bfc1ff700 1 Processor -- start 2023-11-28T13:10:52.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.484+0000 7f4bfc1ff700 1 -- start start 2023-11-28T13:10:52.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.485+0000 7f4bfc1ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4bf415a760 0x7f4bf4161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:52.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.485+0000 7f4bfc1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bf415dc30 0x7f4bf40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:52.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.485+0000 7f4bfc1ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4bf40cd7c0 0x7f4bf40cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:52.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.485+0000 7f4bfc1ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf415ab30 con 0x7f4bf40cd7c0 2023-11-28T13:10:52.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.485+0000 7f4bfc1ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf415e0a0 con 0x7f4bf415a760 2023-11-28T13:10:52.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.485+0000 7f4bfc1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40ce150 con 0x7f4bf415dc30 2023-11-28T13:10:52.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.485+0000 7f4bfa79c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4bf40cd7c0 0x7f4bf40cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:52.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.485+0000 7f4bfa79c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4bf40cd7c0 0x7f4bf40cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49342/0 (socket says 172.21.15.156:49342) 2023-11-28T13:10:52.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.485+0000 7f4bfa79c700 1 -- 172.21.15.156:0/105572454 learned_addr learned my addr 172.21.15.156:0/105572454 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:52.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.486+0000 7f4bf9f9b700 1 --2- 172.21.15.156:0/105572454 >> v2:172.21.15.186:3300/0 conn(0x7f4bf415a760 0x7f4bf4161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:52.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.486+0000 7f4bf979a700 1 --2- 172.21.15.156:0/105572454 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bf415dc30 0x7f4bf40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:52.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.486+0000 7f4bfa79c700 1 -- 172.21.15.156:0/105572454 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bf415dc30 msgr2=0x7f4bf40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:52.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.486+0000 7f4bfa79c700 1 --2- 172.21.15.156:0/105572454 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bf415dc30 0x7f4bf40b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:52.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.486+0000 7f4bfa79c700 1 -- 172.21.15.156:0/105572454 >> v2:172.21.15.186:3300/0 conn(0x7f4bf415a760 msgr2=0x7f4bf4161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:52.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.486+0000 7f4bfa79c700 1 --2- 172.21.15.156:0/105572454 >> v2:172.21.15.186:3300/0 conn(0x7f4bf415a760 0x7f4bf4161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:52.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.486+0000 7f4bfa79c700 1 -- 172.21.15.156:0/105572454 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf4160d70 con 0x7f4bf40cd7c0 2023-11-28T13:10:52.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.486+0000 7f4bf979a700 1 --2- 172.21.15.156:0/105572454 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bf415dc30 0x7f4bf40b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:52.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.486+0000 7f4bf9f9b700 1 --2- 172.21.15.156:0/105572454 >> v2:172.21.15.186:3300/0 conn(0x7f4bf415a760 0x7f4bf4161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:52.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.486+0000 7f4bfa79c700 1 --2- 172.21.15.156:0/105572454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40cd7c0 0x7f4bf40cdc10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4bf408bff0 tx=0x7f4bec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b3d568f431cdf6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:52.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.486+0000 7f4bf8f99700 1 -- 172.21.15.156:0/105572454 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec00e070 con 0x7f4bf40cd7c0 2023-11-28T13:10:52.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.486+0000 7f4bf8f99700 1 -- 172.21.15.156:0/105572454 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec00b980 con 0x7f4bf40cd7c0 2023-11-28T13:10:52.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.486+0000 7f4bf8f99700 1 -- 172.21.15.156:0/105572454 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec00bd30 con 0x7f4bf40cd7c0 2023-11-28T13:10:52.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.487+0000 7f4bfc1ff700 1 -- 172.21.15.156:0/105572454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40cd7c0 msgr2=0x7f4bf40cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:52.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.487+0000 7f4bfc1ff700 1 --2- 172.21.15.156:0/105572454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40cd7c0 0x7f4bf40cdc10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4bf408bff0 tx=0x7f4bec00b330 comp rx=0 tx=0).stop 2023-11-28T13:10:52.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.487+0000 7f4bfc1ff700 1 -- 172.21.15.156:0/105572454 shutdown_connections 2023-11-28T13:10:52.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.487+0000 7f4bfc1ff700 1 --2- 172.21.15.156:0/105572454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40cd7c0 0x7f4bf40cdc10 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:52.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.487+0000 7f4bfc1ff700 1 --2- 172.21.15.156:0/105572454 >> v2:172.21.15.186:3300/0 conn(0x7f4bf415a760 0x7f4bf4161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:52.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.487+0000 7f4bfc1ff700 1 --2- 172.21.15.156:0/105572454 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bf415dc30 0x7f4bf40b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:52.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.487+0000 7f4bfc1ff700 1 -- 172.21.15.156:0/105572454 >> 172.21.15.156:0/105572454 conn(0x7f4bf4072620 msgr2=0x7f4bf40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:52.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.487+0000 7f4bfc1ff700 1 -- 172.21.15.156:0/105572454 shutdown_connections 2023-11-28T13:10:52.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.487+0000 7f4bfc1ff700 1 -- 172.21.15.156:0/105572454 wait complete. 2023-11-28T13:10:52.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.489+0000 7f4bfc1ff700 1 Processor -- start 2023-11-28T13:10:52.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.489+0000 7f4bfc1ff700 1 -- start start 2023-11-28T13:10:52.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.490+0000 7f4bfc1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40cd7c0 0x7f4bf41569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:52.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.490+0000 7f4bfc1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bf415a760 0x7f4bf40b8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:52.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.490+0000 7f4bfc1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bf415dc30 0x7f4bf4158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:52.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.490+0000 7f4bfc1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40ca370 con 0x7f4bf40cd7c0 2023-11-28T13:10:52.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.490+0000 7f4bfc1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40ca650 con 0x7f4bf415dc30 2023-11-28T13:10:52.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.490+0000 7f4bfc1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf4157c60 con 0x7f4bf415a760 2023-11-28T13:10:52.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.490+0000 7f4bf9f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40cd7c0 0x7f4bf41569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:52.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.490+0000 7f4bf9f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40cd7c0 0x7f4bf41569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49358/0 (socket says 172.21.15.156:49358) 2023-11-28T13:10:52.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.490+0000 7f4bf9f9b700 1 -- 172.21.15.156:0/2430169972 learned_addr learned my addr 172.21.15.156:0/2430169972 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:52.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.490+0000 7f4bf979a700 1 --2- 172.21.15.156:0/2430169972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bf415a760 0x7f4bf40b8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:52.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.490+0000 7f4bfa79c700 1 --2- 172.21.15.156:0/2430169972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bf415dc30 0x7f4bf4158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:52.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.491+0000 7f4bf9f9b700 1 -- 172.21.15.156:0/2430169972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bf415a760 msgr2=0x7f4bf40b8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:52.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.491+0000 7f4bf9f9b700 1 --2- 172.21.15.156:0/2430169972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bf415a760 0x7f4bf40b8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:52.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.491+0000 7f4bf9f9b700 1 -- 172.21.15.156:0/2430169972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bf415dc30 msgr2=0x7f4bf4158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:52.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.491+0000 7f4bf9f9b700 1 --2- 172.21.15.156:0/2430169972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bf415dc30 0x7f4bf4158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:52.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.491+0000 7f4bf9f9b700 1 -- 172.21.15.156:0/2430169972 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec009b30 con 0x7f4bf40cd7c0 2023-11-28T13:10:52.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.491+0000 7f4bf9f9b700 1 --2- 172.21.15.156:0/2430169972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40cd7c0 0x7f4bf41569f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000d800 tx=0x7f4bf000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:52.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.491+0000 7f4bea7fc700 1 -- 172.21.15.156:0/2430169972 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0004030 con 0x7f4bf40cd7c0 2023-11-28T13:10:52.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.491+0000 7f4bea7fc700 1 -- 172.21.15.156:0/2430169972 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf0004190 con 0x7f4bf40cd7c0 2023-11-28T13:10:52.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.491+0000 7f4bfc1ff700 1 -- 172.21.15.156:0/2430169972 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf40b5390 con 0x7f4bf40cd7c0 2023-11-28T13:10:52.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.491+0000 7f4bea7fc700 1 -- 172.21.15.156:0/2430169972 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0004520 con 0x7f4bf40cd7c0 2023-11-28T13:10:52.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.491+0000 7f4bfc1ff700 1 -- 172.21.15.156:0/2430169972 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf40b5130 con 0x7f4bf40cd7c0 2023-11-28T13:10:52.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.492+0000 7f4bfc1ff700 1 -- 172.21.15.156:0/2430169972 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf40b5670 con 0x7f4bf40cd7c0 2023-11-28T13:10:52.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.493+0000 7f4bea7fc700 1 -- 172.21.15.156:0/2430169972 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4bf001b030 con 0x7f4bf40cd7c0 2023-11-28T13:10:52.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.494+0000 7f4bea7fc700 1 --2- 172.21.15.156:0/2430169972 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4bd80682f0 0x7f4bd806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:52.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.494+0000 7f4bea7fc700 1 -- 172.21.15.156:0/2430169972 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4bf0085120 con 0x7f4bf40cd7c0 2023-11-28T13:10:52.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.495+0000 7f4bf979a700 1 --2- 172.21.15.156:0/2430169972 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4bd80682f0 0x7f4bd806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:52.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.495+0000 7f4bf979a700 1 --2- 172.21.15.156:0/2430169972 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4bd80682f0 0x7f4bd806a7a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4be4006f10 tx=0x7f4be4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:52.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.499+0000 7f4bea7fc700 1 -- 172.21.15.156:0/2430169972 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4bf004d5e0 con 0x7f4bf40cd7c0 2023-11-28T13:10:52.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:52.690+0000 7f4bfc1ff700 1 -- 172.21.15.156:0/2430169972 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4bf404e620 con 0x7f4bf40cd7c0 2023-11-28T13:10:53.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.432+0000 7f4bea7fc700 1 -- 172.21.15.156:0/2430169972 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v56) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4bf0051c30 con 0x7f4bf40cd7c0 2023-11-28T13:10:53.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.435+0000 7f4bfc1ff700 1 -- 172.21.15.156:0/2430169972 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4bd80682f0 msgr2=0x7f4bd806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:53.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.435+0000 7f4bfc1ff700 1 --2- 172.21.15.156:0/2430169972 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4bd80682f0 0x7f4bd806a7a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4be4006f10 tx=0x7f4be4008040 comp rx=0 tx=0).stop 2023-11-28T13:10:53.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.435+0000 7f4bfc1ff700 1 -- 172.21.15.156:0/2430169972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40cd7c0 msgr2=0x7f4bf41569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:53.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.435+0000 7f4bfc1ff700 1 --2- 172.21.15.156:0/2430169972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40cd7c0 0x7f4bf41569f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000d800 tx=0x7f4bf000db10 comp rx=0 tx=0).stop 2023-11-28T13:10:53.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.436+0000 7f4bfc1ff700 1 -- 172.21.15.156:0/2430169972 shutdown_connections 2023-11-28T13:10:53.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.436+0000 7f4bfc1ff700 1 --2- 172.21.15.156:0/2430169972 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4bd80682f0 0x7f4bd806a7a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:53.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.436+0000 7f4bfc1ff700 1 --2- 172.21.15.156:0/2430169972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bf415dc30 0x7f4bf4158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:53.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.436+0000 7f4bfc1ff700 1 --2- 172.21.15.156:0/2430169972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40cd7c0 0x7f4bf41569f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:53.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.436+0000 7f4bfc1ff700 1 --2- 172.21.15.156:0/2430169972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4bf415a760 0x7f4bf40b8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:53.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.436+0000 7f4bfc1ff700 1 -- 172.21.15.156:0/2430169972 >> 172.21.15.156:0/2430169972 conn(0x7f4bf4072620 msgr2=0x7f4bf415e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:53.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.436+0000 7f4bfc1ff700 1 -- 172.21.15.156:0/2430169972 shutdown_connections 2023-11-28T13:10:53.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.436+0000 7f4bfc1ff700 1 -- 172.21.15.156:0/2430169972 wait complete. 2023-11-28T13:10:53.439 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_metadata' created 2023-11-28T13:10:53.461 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T13:10:53.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.665+0000 7f19739ff700 1 Processor -- start 2023-11-28T13:10:53.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.665+0000 7f19739ff700 1 -- start start 2023-11-28T13:10:53.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.666+0000 7f19739ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f196c161340 0x7f196c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:53.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.666+0000 7f19739ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f196c0d2320 0x7f196c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:53.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.666+0000 7f19739ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c0d1950 con 0x7f196c15c9b0 2023-11-28T13:10:53.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.666+0000 7f19739ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196c0b5800 con 0x7f196c161340 2023-11-28T13:10:53.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.666+0000 7f19739ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0b5300 con 0x7f196c0d2320 2023-11-28T13:10:53.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.666+0000 7f1970f9a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f196c161340 0x7f196c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:53.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.666+0000 7f1970f9a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f196c161340 0x7f196c0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49372/0 (socket says 172.21.15.156:49372) 2023-11-28T13:10:53.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.666+0000 7f1970f9a700 1 -- 172.21.15.156:0/3663641566 learned_addr learned my addr 172.21.15.156:0/3663641566 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:53.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.667+0000 7f1971f9c700 1 --2- 172.21.15.156:0/3663641566 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f196c0d2320 0x7f196c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:53.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.667+0000 7f1970f9a700 1 -- 172.21.15.156:0/3663641566 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f196c0d2320 msgr2=0x7f196c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:53.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.667+0000 7f1970f9a700 1 --2- 172.21.15.156:0/3663641566 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f196c0d2320 0x7f196c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:53.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.667+0000 7f1970f9a700 1 -- 172.21.15.156:0/3663641566 >> v1:172.21.15.156:6789/0 conn(0x7f196c15c9b0 legacy=0x7f196c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:53.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.667+0000 7f1970f9a700 1 -- 172.21.15.156:0/3663641566 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c0b5520 con 0x7f196c161340 2023-11-28T13:10:53.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.667+0000 7f1971f9c700 1 --2- 172.21.15.156:0/3663641566 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f196c0d2320 0x7f196c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:53.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.667+0000 7f1970f9a700 1 --2- 172.21.15.156:0/3663641566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f196c161340 0x7f196c0d1de0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f195c0098f0 tx=0x7f195c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8b0afb3ad3b66f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:53.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.667+0000 7f1963fff700 1 -- 172.21.15.156:0/3663641566 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c00b8a0 con 0x7f196c161340 2023-11-28T13:10:53.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.668+0000 7f1963fff700 1 -- 172.21.15.156:0/3663641566 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c00ba00 con 0x7f196c161340 2023-11-28T13:10:53.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.668+0000 7f1963fff700 1 -- 172.21.15.156:0/3663641566 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c00bd90 con 0x7f196c161340 2023-11-28T13:10:53.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.668+0000 7f19739ff700 1 -- 172.21.15.156:0/3663641566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f196c161340 msgr2=0x7f196c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:53.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.668+0000 7f19739ff700 1 --2- 172.21.15.156:0/3663641566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f196c161340 0x7f196c0d1de0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f195c0098f0 tx=0x7f195c009c00 comp rx=0 tx=0).stop 2023-11-28T13:10:53.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.668+0000 7f19739ff700 1 -- 172.21.15.156:0/3663641566 shutdown_connections 2023-11-28T13:10:53.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.668+0000 7f19739ff700 1 --2- 172.21.15.156:0/3663641566 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f196c0d2320 0x7f196c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:53.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.668+0000 7f19739ff700 1 --2- 172.21.15.156:0/3663641566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f196c161340 0x7f196c0d1de0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:53.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.668+0000 7f19739ff700 1 -- 172.21.15.156:0/3663641566 >> 172.21.15.156:0/3663641566 conn(0x7f196c072600 msgr2=0x7f196c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:53.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.668+0000 7f19739ff700 1 -- 172.21.15.156:0/3663641566 shutdown_connections 2023-11-28T13:10:53.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.668+0000 7f19739ff700 1 -- 172.21.15.156:0/3663641566 wait complete. 2023-11-28T13:10:53.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.670+0000 7f19739ff700 1 Processor -- start 2023-11-28T13:10:53.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.670+0000 7f19739ff700 1 -- start start 2023-11-28T13:10:53.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.671+0000 7f19739ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f196c0d2320 0x7f196c14c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:53.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.671+0000 7f19739ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f196c15c9b0 0x7f196c0cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:53.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.671+0000 7f19739ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f196c161340 0x7f196c0ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:53.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.671+0000 7f19739ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0cd0a0 con 0x7f196c0d2320 2023-11-28T13:10:53.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.671+0000 7f19739ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0cd2f0 con 0x7f196c161340 2023-11-28T13:10:53.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.671+0000 7f19739ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0cd5d0 con 0x7f196c15c9b0 2023-11-28T13:10:53.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.671+0000 7f197179b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f196c0d2320 0x7f196c14c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:53.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.671+0000 7f197179b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f196c0d2320 0x7f196c14c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49384/0 (socket says 172.21.15.156:49384) 2023-11-28T13:10:53.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.671+0000 7f1970f9a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f196c15c9b0 0x7f196c0cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:53.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.671+0000 7f197179b700 1 -- 172.21.15.156:0/1851398925 learned_addr learned my addr 172.21.15.156:0/1851398925 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:53.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.671+0000 7f1971f9c700 1 --2- 172.21.15.156:0/1851398925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f196c161340 0x7f196c0ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:53.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.672+0000 7f197179b700 1 -- 172.21.15.156:0/1851398925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f196c15c9b0 msgr2=0x7f196c0cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:53.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.672+0000 7f197179b700 1 --2- 172.21.15.156:0/1851398925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f196c15c9b0 0x7f196c0cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:53.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.672+0000 7f197179b700 1 -- 172.21.15.156:0/1851398925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f196c161340 msgr2=0x7f196c0ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:53.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.672+0000 7f197179b700 1 --2- 172.21.15.156:0/1851398925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f196c161340 0x7f196c0ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:53.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.672+0000 7f197179b700 1 -- 172.21.15.156:0/1851398925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c009550 con 0x7f196c0d2320 2023-11-28T13:10:53.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.672+0000 7f197179b700 1 --2- 172.21.15.156:0/1851398925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f196c0d2320 0x7f196c14c510 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f196800c450 tx=0x7f196800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:53.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.672+0000 7f1961ffb700 1 -- 172.21.15.156:0/1851398925 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19680052d0 con 0x7f196c0d2320 2023-11-28T13:10:53.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.672+0000 7f1961ffb700 1 -- 172.21.15.156:0/1851398925 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1968005430 con 0x7f196c0d2320 2023-11-28T13:10:53.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.672+0000 7f19739ff700 1 -- 172.21.15.156:0/1851398925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c0d15e0 con 0x7f196c0d2320 2023-11-28T13:10:53.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.672+0000 7f1961ffb700 1 -- 172.21.15.156:0/1851398925 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19680038c0 con 0x7f196c0d2320 2023-11-28T13:10:53.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.672+0000 7f19739ff700 1 -- 172.21.15.156:0/1851398925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c0d1300 con 0x7f196c0d2320 2023-11-28T13:10:53.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.673+0000 7f19739ff700 1 -- 172.21.15.156:0/1851398925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c15cd80 con 0x7f196c0d2320 2023-11-28T13:10:53.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.675+0000 7f1961ffb700 1 -- 172.21.15.156:0/1851398925 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1968019030 con 0x7f196c0d2320 2023-11-28T13:10:53.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.676+0000 7f1961ffb700 1 --2- 172.21.15.156:0/1851398925 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f19500682f0 0x7f195006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:53.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.676+0000 7f1961ffb700 1 -- 172.21.15.156:0/1851398925 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f1968085190 con 0x7f196c0d2320 2023-11-28T13:10:53.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.676+0000 7f1970f9a700 1 --2- 172.21.15.156:0/1851398925 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f19500682f0 0x7f195006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:53.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.677+0000 7f1970f9a700 1 --2- 172.21.15.156:0/1851398925 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f19500682f0 0x7f195006a7a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f195c009c40 tx=0x7f195c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:53.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.679+0000 7f1961ffb700 1 -- 172.21.15.156:0/1851398925 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1968051b60 con 0x7f196c0d2320 2023-11-28T13:10:53.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:53.871+0000 7f19739ff700 1 -- 172.21.15.156:0/1851398925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f196c0c7be0 con 0x7f196c0d2320 2023-11-28T13:10:54.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.439+0000 7f1961ffb700 1 -- 172.21.15.156:0/1851398925 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v57) v1 ==== 170+0+0 (secure 0 0 0) 0x7f196800ce60 con 0x7f196c0d2320 2023-11-28T13:10:54.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.444+0000 7f19739ff700 1 -- 172.21.15.156:0/1851398925 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f19500682f0 msgr2=0x7f195006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:54.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.444+0000 7f19739ff700 1 --2- 172.21.15.156:0/1851398925 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f19500682f0 0x7f195006a7a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f195c009c40 tx=0x7f195c008030 comp rx=0 tx=0).stop 2023-11-28T13:10:54.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.444+0000 7f19739ff700 1 -- 172.21.15.156:0/1851398925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f196c0d2320 msgr2=0x7f196c14c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:54.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.444+0000 7f19739ff700 1 --2- 172.21.15.156:0/1851398925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f196c0d2320 0x7f196c14c510 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f196800c450 tx=0x7f196800c760 comp rx=0 tx=0).stop 2023-11-28T13:10:54.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.446+0000 7f19739ff700 1 -- 172.21.15.156:0/1851398925 shutdown_connections 2023-11-28T13:10:54.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.446+0000 7f19739ff700 1 --2- 172.21.15.156:0/1851398925 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f19500682f0 0x7f195006a7a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:54.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.446+0000 7f19739ff700 1 --2- 172.21.15.156:0/1851398925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f196c161340 0x7f196c0ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:54.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.446+0000 7f19739ff700 1 --2- 172.21.15.156:0/1851398925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f196c0d2320 0x7f196c14c510 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:54.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.446+0000 7f19739ff700 1 --2- 172.21.15.156:0/1851398925 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f196c15c9b0 0x7f196c0cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:54.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.446+0000 7f19739ff700 1 -- 172.21.15.156:0/1851398925 >> 172.21.15.156:0/1851398925 conn(0x7f196c072600 msgr2=0x7f196c15f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:54.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.448+0000 7f19739ff700 1 -- 172.21.15.156:0/1851398925 shutdown_connections 2023-11-28T13:10:54.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.448+0000 7f19739ff700 1 -- 172.21.15.156:0/1851398925 wait complete. 2023-11-28T13:10:54.454 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_data' created 2023-11-28T13:10:54.477 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T13:10:54.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.676+0000 7fc6589ff700 1 Processor -- start 2023-11-28T13:10:54.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.677+0000 7fc6589ff700 1 -- start start 2023-11-28T13:10:54.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.678+0000 7fc6589ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc65415c9b0 0x7fc6540b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:54.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.678+0000 7fc6589ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc6540cd7c0 0x7fc6540cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:54.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.678+0000 7fc6589ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6540d1950 con 0x7fc654161340 2023-11-28T13:10:54.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.678+0000 7fc6589ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6540b5800 con 0x7fc6540cd7c0 2023-11-28T13:10:54.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.678+0000 7fc6589ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6540ce140 con 0x7fc65415c9b0 2023-11-28T13:10:54.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.678+0000 7fc652d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc6540cd7c0 0x7fc6540cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:54.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.678+0000 7fc652d9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc6540cd7c0 0x7fc6540cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49400/0 (socket says 172.21.15.156:49400) 2023-11-28T13:10:54.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.678+0000 7fc652d9d700 1 -- 172.21.15.156:0/3388788727 learned_addr learned my addr 172.21.15.156:0/3388788727 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:54.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.678+0000 7fc65259c700 1 --2- 172.21.15.156:0/3388788727 >> v2:172.21.15.186:3300/0 conn(0x7fc65415c9b0 0x7fc6540b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:54.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.679+0000 7fc652d9d700 1 -- 172.21.15.156:0/3388788727 >> v2:172.21.15.186:3300/0 conn(0x7fc65415c9b0 msgr2=0x7fc6540b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:54.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.679+0000 7fc652d9d700 1 --2- 172.21.15.156:0/3388788727 >> v2:172.21.15.186:3300/0 conn(0x7fc65415c9b0 0x7fc6540b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:54.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.679+0000 7fc652d9d700 1 -- 172.21.15.156:0/3388788727 >> v1:172.21.15.156:6789/0 conn(0x7fc654161340 legacy=0x7fc6540d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:54.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.679+0000 7fc652d9d700 1 -- 172.21.15.156:0/3388788727 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6540d24f0 con 0x7fc6540cd7c0 2023-11-28T13:10:54.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.679+0000 7fc652d9d700 1 --2- 172.21.15.156:0/3388788727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6540cd7c0 0x7fc6540cdc00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc654067460 tx=0x7fc64c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1e31707e089431 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:54.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.679+0000 7fc651d9b700 1 -- 172.21.15.156:0/3388788727 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c00e070 con 0x7fc6540cd7c0 2023-11-28T13:10:54.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.679+0000 7fc651d9b700 1 -- 172.21.15.156:0/3388788727 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c00b9c0 con 0x7fc6540cd7c0 2023-11-28T13:10:54.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.679+0000 7fc651d9b700 1 -- 172.21.15.156:0/3388788727 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c00bd70 con 0x7fc6540cd7c0 2023-11-28T13:10:54.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.679+0000 7fc6589ff700 1 -- 172.21.15.156:0/3388788727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6540cd7c0 msgr2=0x7fc6540cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:54.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.679+0000 7fc6589ff700 1 --2- 172.21.15.156:0/3388788727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6540cd7c0 0x7fc6540cdc00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc654067460 tx=0x7fc64c00b330 comp rx=0 tx=0).stop 2023-11-28T13:10:54.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.680+0000 7fc6589ff700 1 -- 172.21.15.156:0/3388788727 shutdown_connections 2023-11-28T13:10:54.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.680+0000 7fc6589ff700 1 --2- 172.21.15.156:0/3388788727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6540cd7c0 0x7fc6540cdc00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:54.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.680+0000 7fc6589ff700 1 --2- 172.21.15.156:0/3388788727 >> v2:172.21.15.186:3300/0 conn(0x7fc65415c9b0 0x7fc6540b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:54.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.680+0000 7fc6589ff700 1 -- 172.21.15.156:0/3388788727 >> 172.21.15.156:0/3388788727 conn(0x7fc654072600 msgr2=0x7fc6540cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:54.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.680+0000 7fc6589ff700 1 -- 172.21.15.156:0/3388788727 shutdown_connections 2023-11-28T13:10:54.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.680+0000 7fc6589ff700 1 -- 172.21.15.156:0/3388788727 wait complete. 2023-11-28T13:10:54.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.681+0000 7fc6589ff700 1 Processor -- start 2023-11-28T13:10:54.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.682+0000 7fc6589ff700 1 -- start start 2023-11-28T13:10:54.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc6589ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6540cd7c0 0x7fc65414c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:54.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc6589ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc65415c9b0 0x7fc6540b7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:54.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc6589ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc654161340 0x7fc6540b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:54.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc6589ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6541e7b50 con 0x7fc654161340 2023-11-28T13:10:54.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc6589ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540c79d0 con 0x7fc65415c9b0 2023-11-28T13:10:54.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc6589ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540c7cb0 con 0x7fc6540cd7c0 2023-11-28T13:10:54.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc652d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc654161340 0x7fc6540b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:54.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc652d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc654161340 0x7fc6540b7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49404/0 (socket says 172.21.15.156:49404) 2023-11-28T13:10:54.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc652d9d700 1 -- 172.21.15.156:0/3492869787 learned_addr learned my addr 172.21.15.156:0/3492869787 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:54.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc65259c700 1 --2- 172.21.15.156:0/3492869787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6540cd7c0 0x7fc65414c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:54.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc64bfff700 1 --2- 172.21.15.156:0/3492869787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc65415c9b0 0x7fc6540b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:54.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc652d9d700 1 -- 172.21.15.156:0/3492869787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6540cd7c0 msgr2=0x7fc65414c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:54.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc652d9d700 1 --2- 172.21.15.156:0/3492869787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6540cd7c0 0x7fc65414c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:54.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc652d9d700 1 -- 172.21.15.156:0/3492869787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc65415c9b0 msgr2=0x7fc6540b7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:54.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc652d9d700 1 --2- 172.21.15.156:0/3492869787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc65415c9b0 0x7fc6540b7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:54.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc652d9d700 1 -- 172.21.15.156:0/3492869787 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c009b30 con 0x7fc654161340 2023-11-28T13:10:54.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc64bfff700 1 --2- 172.21.15.156:0/3492869787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc65415c9b0 0x7fc6540b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:54.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.683+0000 7fc652d9d700 1 --2- 172.21.15.156:0/3492869787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc654161340 0x7fc6540b7ac0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc6540675d0 tx=0x7fc64c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:54.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.684+0000 7fc64affd700 1 -- 172.21.15.156:0/3492869787 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c004930 con 0x7fc654161340 2023-11-28T13:10:54.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.684+0000 7fc64affd700 1 -- 172.21.15.156:0/3492869787 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c0099b0 con 0x7fc654161340 2023-11-28T13:10:54.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.684+0000 7fc64affd700 1 -- 172.21.15.156:0/3492869787 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c01b670 con 0x7fc654161340 2023-11-28T13:10:54.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.684+0000 7fc6589ff700 1 -- 172.21.15.156:0/3492869787 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6540d1300 con 0x7fc654161340 2023-11-28T13:10:54.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.684+0000 7fc6589ff700 1 -- 172.21.15.156:0/3492869787 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6540b5520 con 0x7fc654161340 2023-11-28T13:10:54.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.685+0000 7fc6589ff700 1 -- 172.21.15.156:0/3492869787 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6540d15e0 con 0x7fc654161340 2023-11-28T13:10:54.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.686+0000 7fc64affd700 1 -- 172.21.15.156:0/3492869787 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc64c02b070 con 0x7fc654161340 2023-11-28T13:10:54.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.687+0000 7fc64affd700 1 --2- 172.21.15.156:0/3492869787 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6340682f0 0x7fc63406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:54.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.687+0000 7fc64affd700 1 -- 172.21.15.156:0/3492869787 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fc64c087490 con 0x7fc654161340 2023-11-28T13:10:54.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.687+0000 7fc65259c700 1 --2- 172.21.15.156:0/3492869787 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6340682f0 0x7fc63406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:54.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.688+0000 7fc65259c700 1 --2- 172.21.15.156:0/3492869787 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6340682f0 0x7fc63406a7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc644006b20 tx=0x7fc644008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:54.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.692+0000 7fc64affd700 1 -- 172.21.15.156:0/3492869787 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc64c053c60 con 0x7fc654161340 2023-11-28T13:10:54.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.926+0000 7fc6589ff700 1 -- 172.21.15.156:0/3492869787 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc6541e7c90 con 0x7fc654161340 2023-11-28T13:10:54.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.932+0000 7fc64affd700 1 -- 172.21.15.156:0/3492869787 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v27) v1 ==== 173+0+0 (secure 0 0 0) 0x7fc6541e7c90 con 0x7fc654161340 2023-11-28T13:10:54.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.940+0000 7fc6589ff700 1 -- 172.21.15.156:0/3492869787 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6340682f0 msgr2=0x7fc63406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:54.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.940+0000 7fc6589ff700 1 --2- 172.21.15.156:0/3492869787 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6340682f0 0x7fc63406a7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc644006b20 tx=0x7fc644008040 comp rx=0 tx=0).stop 2023-11-28T13:10:54.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.940+0000 7fc6589ff700 1 -- 172.21.15.156:0/3492869787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc654161340 msgr2=0x7fc6540b7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:54.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.940+0000 7fc6589ff700 1 --2- 172.21.15.156:0/3492869787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc654161340 0x7fc6540b7ac0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc6540675d0 tx=0x7fc64c004ad0 comp rx=0 tx=0).stop 2023-11-28T13:10:54.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.941+0000 7fc6589ff700 1 -- 172.21.15.156:0/3492869787 shutdown_connections 2023-11-28T13:10:54.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.941+0000 7fc6589ff700 1 --2- 172.21.15.156:0/3492869787 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6340682f0 0x7fc63406a7a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:54.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.941+0000 7fc6589ff700 1 --2- 172.21.15.156:0/3492869787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc654161340 0x7fc6540b7ac0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:54.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.941+0000 7fc6589ff700 1 --2- 172.21.15.156:0/3492869787 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6540cd7c0 0x7fc65414c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:54.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.941+0000 7fc6589ff700 1 --2- 172.21.15.156:0/3492869787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc65415c9b0 0x7fc6540b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:54.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.941+0000 7fc6589ff700 1 -- 172.21.15.156:0/3492869787 >> 172.21.15.156:0/3492869787 conn(0x7fc654072600 msgr2=0x7fc6540b6320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:54.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.944+0000 7fc6589ff700 1 -- 172.21.15.156:0/3492869787 shutdown_connections 2023-11-28T13:10:54.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:54.944+0000 7fc6589ff700 1 -- 172.21.15.156:0/3492869787 wait complete. 2023-11-28T13:10:54.955 INFO:teuthology.orchestra.run.smithi156.stderr:new fs with metadata pool 9 and data pool 10 2023-11-28T13:10:54.982 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:10:55.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.178+0000 7f09f27ff700 1 Processor -- start 2023-11-28T13:10:55.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.179+0000 7f09f27ff700 1 -- start start 2023-11-28T13:10:55.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.180+0000 7f09f27ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f09ec161340 0x7f09ec0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:55.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.180+0000 7f09f27ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ec0d1950 con 0x7f09ec0d2320 2023-11-28T13:10:55.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.180+0000 7f09f27ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ec0b5520 con 0x7f09ec15c9b0 2023-11-28T13:10:55.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.180+0000 7f09f27ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09ec0b5800 con 0x7f09ec161340 2023-11-28T13:10:55.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.180+0000 7f09eb7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f09ec161340 0x7f09ec0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:55.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.180+0000 7f09eb7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f09ec161340 0x7f09ec0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49418/0 (socket says 172.21.15.156:49418) 2023-11-28T13:10:55.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.180+0000 7f09eb7fe700 1 -- 172.21.15.156:0/2653553113 learned_addr learned my addr 172.21.15.156:0/2653553113 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:55.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.180+0000 7f09f0d9c700 1 --1- 172.21.15.156:0/2653553113 >> v1:172.21.15.156:6789/0 conn(0x7f09ec0d2320 0x7f09ec0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37828/0 (socket says 172.21.15.156:37828) 2023-11-28T13:10:55.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.181+0000 7f09eb7fe700 1 -- 172.21.15.156:0/2653553113 >> v1:172.21.15.186:6789/0 conn(0x7f09ec15c9b0 legacy=0x7f09ec0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:55.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.181+0000 7f09eb7fe700 1 -- 172.21.15.156:0/2653553113 >> v1:172.21.15.156:6789/0 conn(0x7f09ec0d2320 legacy=0x7f09ec0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:55.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.181+0000 7f09eb7fe700 1 -- 172.21.15.156:0/2653553113 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec0d1300 con 0x7f09ec161340 2023-11-28T13:10:55.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.181+0000 7f09eb7fe700 1 --2- 172.21.15.156:0/2653553113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09ec161340 0x7f09ec0d1de0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f09e00098f0 tx=0x7f09e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d345841d24b67095 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:55.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.181+0000 7f09eaffd700 1 -- 172.21.15.156:0/2653553113 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e000b8a0 con 0x7f09ec161340 2023-11-28T13:10:55.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.181+0000 7f09eaffd700 1 -- 172.21.15.156:0/2653553113 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e000ba00 con 0x7f09ec161340 2023-11-28T13:10:55.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.181+0000 7f09eaffd700 1 -- 172.21.15.156:0/2653553113 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e000bd90 con 0x7f09ec161340 2023-11-28T13:10:55.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.182+0000 7f09f27ff700 1 -- 172.21.15.156:0/2653553113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09ec161340 msgr2=0x7f09ec0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:55.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.182+0000 7f09f27ff700 1 --2- 172.21.15.156:0/2653553113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09ec161340 0x7f09ec0d1de0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f09e00098f0 tx=0x7f09e0009c00 comp rx=0 tx=0).stop 2023-11-28T13:10:55.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.182+0000 7f09f27ff700 1 -- 172.21.15.156:0/2653553113 shutdown_connections 2023-11-28T13:10:55.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.182+0000 7f09f27ff700 1 --2- 172.21.15.156:0/2653553113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09ec161340 0x7f09ec0d1de0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.182+0000 7f09f27ff700 1 -- 172.21.15.156:0/2653553113 >> 172.21.15.156:0/2653553113 conn(0x7f09ec072600 msgr2=0x7f09ec0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:55.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.182+0000 7f09f27ff700 1 -- 172.21.15.156:0/2653553113 shutdown_connections 2023-11-28T13:10:55.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.182+0000 7f09f27ff700 1 -- 172.21.15.156:0/2653553113 wait complete. 2023-11-28T13:10:55.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.184+0000 7f09f27ff700 1 Processor -- start 2023-11-28T13:10:55.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.184+0000 7f09f27ff700 1 -- start start 2023-11-28T13:10:55.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.185+0000 7f09f27ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09ec0d2320 0x7f09ec1e8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:55.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.185+0000 7f09f27ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09ec15c9b0 0x7f09ec155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:55.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.185+0000 7f09f27ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09ec161340 0x7f09ec14e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:55.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.186+0000 7f09f27ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec157c70 con 0x7f09ec15c9b0 2023-11-28T13:10:55.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.186+0000 7f09f27ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec156b20 con 0x7f09ec161340 2023-11-28T13:10:55.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.186+0000 7f09f27ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec156280 con 0x7f09ec0d2320 2023-11-28T13:10:55.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.186+0000 7f09eb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09ec15c9b0 0x7f09ec155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:55.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.186+0000 7f09eb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09ec15c9b0 0x7f09ec155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49422/0 (socket says 172.21.15.156:49422) 2023-11-28T13:10:55.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.186+0000 7f09eb7fe700 1 -- 172.21.15.156:0/1381223800 learned_addr learned my addr 172.21.15.156:0/1381223800 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:55.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.186+0000 7f09ebfff700 1 --2- 172.21.15.156:0/1381223800 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09ec0d2320 0x7f09ec1e8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:55.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.186+0000 7f09f0d9c700 1 --2- 172.21.15.156:0/1381223800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09ec161340 0x7f09ec14e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:55.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.186+0000 7f09eb7fe700 1 -- 172.21.15.156:0/1381223800 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09ec0d2320 msgr2=0x7f09ec1e8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:55.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.186+0000 7f09eb7fe700 1 --2- 172.21.15.156:0/1381223800 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09ec0d2320 0x7f09ec1e8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.186+0000 7f09eb7fe700 1 -- 172.21.15.156:0/1381223800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09ec161340 msgr2=0x7f09ec14e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:55.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.186+0000 7f09eb7fe700 1 --2- 172.21.15.156:0/1381223800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09ec161340 0x7f09ec14e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.186+0000 7f09eb7fe700 1 -- 172.21.15.156:0/1381223800 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e0009550 con 0x7f09ec15c9b0 2023-11-28T13:10:55.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.186+0000 7f09ebfff700 1 --2- 172.21.15.156:0/1381223800 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09ec0d2320 0x7f09ec1e8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:55.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.186+0000 7f09f0d9c700 1 --2- 172.21.15.156:0/1381223800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09ec161340 0x7f09ec14e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:55.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.186+0000 7f09eb7fe700 1 --2- 172.21.15.156:0/1381223800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09ec15c9b0 0x7f09ec155bc0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009130 tx=0x7f09e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:55.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.187+0000 7f09e8ff9700 1 -- 172.21.15.156:0/1381223800 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e00036a0 con 0x7f09ec15c9b0 2023-11-28T13:10:55.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.187+0000 7f09e8ff9700 1 -- 172.21.15.156:0/1381223800 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e001e070 con 0x7f09ec15c9b0 2023-11-28T13:10:55.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.187+0000 7f09f27ff700 1 -- 172.21.15.156:0/1381223800 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ec15cd80 con 0x7f09ec15c9b0 2023-11-28T13:10:55.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.187+0000 7f09e8ff9700 1 -- 172.21.15.156:0/1381223800 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e0003a60 con 0x7f09ec15c9b0 2023-11-28T13:10:55.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.187+0000 7f09f27ff700 1 -- 172.21.15.156:0/1381223800 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ec0d15e0 con 0x7f09ec15c9b0 2023-11-28T13:10:55.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.188+0000 7f09f27ff700 1 -- 172.21.15.156:0/1381223800 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ec15cf10 con 0x7f09ec15c9b0 2023-11-28T13:10:55.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.189+0000 7f09e8ff9700 1 -- 172.21.15.156:0/1381223800 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f09e000f050 con 0x7f09ec15c9b0 2023-11-28T13:10:55.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.190+0000 7f09e8ff9700 1 --2- 172.21.15.156:0/1381223800 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f09d80682f0 0x7f09d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:55.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.190+0000 7f09e8ff9700 1 -- 172.21.15.156:0/1381223800 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f09e0020070 con 0x7f09ec15c9b0 2023-11-28T13:10:55.192 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.190+0000 7f09ebfff700 1 --2- 172.21.15.156:0/1381223800 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f09d80682f0 0x7f09d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:55.193 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.191+0000 7f09ebfff700 1 --2- 172.21.15.156:0/1381223800 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f09d80682f0 0x7f09d806a7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f09dc000f60 tx=0x7f09dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:55.196 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.195+0000 7f09e8ff9700 1 -- 172.21.15.156:0/1381223800 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f09e0053a70 con 0x7f09ec15c9b0 2023-11-28T13:10:55.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.386+0000 7f09f27ff700 1 -- 172.21.15.156:0/1381223800 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09ec0745c0 con 0x7f09ec15c9b0 2023-11-28T13:10:55.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.388+0000 7f09e8ff9700 1 -- 172.21.15.156:0/1381223800 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+17924 (secure 0 0 0) 0x7f09e0057090 con 0x7f09ec15c9b0 2023-11-28T13:10:55.390 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:55.390 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":58,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:10:54.928207+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:10:52.732399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T13:10:53.912753+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:10:55.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.390+0000 7f09f27ff700 1 -- 172.21.15.156:0/1381223800 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f09d80682f0 msgr2=0x7f09d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:55.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.390+0000 7f09f27ff700 1 --2- 172.21.15.156:0/1381223800 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f09d80682f0 0x7f09d806a7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f09dc000f60 tx=0x7f09dc009040 comp rx=0 tx=0).stop 2023-11-28T13:10:55.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.390+0000 7f09f27ff700 1 -- 172.21.15.156:0/1381223800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09ec15c9b0 msgr2=0x7f09ec155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:55.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.390+0000 7f09f27ff700 1 --2- 172.21.15.156:0/1381223800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09ec15c9b0 0x7f09ec155bc0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009130 tx=0x7f09e0004750 comp rx=0 tx=0).stop 2023-11-28T13:10:55.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.390+0000 7f09f27ff700 1 -- 172.21.15.156:0/1381223800 shutdown_connections 2023-11-28T13:10:55.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.390+0000 7f09f27ff700 1 --2- 172.21.15.156:0/1381223800 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f09d80682f0 0x7f09d806a7a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.390+0000 7f09f27ff700 1 --2- 172.21.15.156:0/1381223800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09ec161340 0x7f09ec14e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.390+0000 7f09f27ff700 1 --2- 172.21.15.156:0/1381223800 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f09ec0d2320 0x7f09ec1e8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.390+0000 7f09f27ff700 1 --2- 172.21.15.156:0/1381223800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09ec15c9b0 0x7f09ec155bc0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.390+0000 7f09f27ff700 1 -- 172.21.15.156:0/1381223800 >> 172.21.15.156:0/1381223800 conn(0x7f09ec072600 msgr2=0x7f09ec15a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:55.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.390+0000 7f09f27ff700 1 -- 172.21.15.156:0/1381223800 shutdown_connections 2023-11-28T13:10:55.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.390+0000 7f09f27ff700 1 -- 172.21.15.156:0/1381223800 wait complete. 2023-11-28T13:10:55.411 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:10:55.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.607+0000 7f1bf7fff700 1 Processor -- start 2023-11-28T13:10:55.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.607+0000 7f1bf7fff700 1 -- start start 2023-11-28T13:10:55.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.608+0000 7f1bf7fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bf00d01b0 0x7f1bf00d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:55.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.608+0000 7f1bf7fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1bf00b20b0 0x7f1bf00d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:55.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.608+0000 7f1bf7fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1bf00b29b0 0x7f1bf00b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:55.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.608+0000 7f1bf7fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bf015cec0 con 0x7f1bf00b29b0 2023-11-28T13:10:55.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.608+0000 7f1bf7fff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bf00b0230 con 0x7f1bf00b20b0 2023-11-28T13:10:55.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.608+0000 7f1bf7fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00b4570 con 0x7f1bf00d01b0 2023-11-28T13:10:55.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.608+0000 7f1bf659c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1bf00b29b0 0x7f1bf00b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:55.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.608+0000 7f1bf659c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1bf00b29b0 0x7f1bf00b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49428/0 (socket says 172.21.15.156:49428) 2023-11-28T13:10:55.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.608+0000 7f1bf659c700 1 -- 172.21.15.156:0/3213079669 learned_addr learned my addr 172.21.15.156:0/3213079669 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:55.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.608+0000 7f1bf5d9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bf00d01b0 0x7f1bf00d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:55.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.608+0000 7f1bf559a700 1 --2- 172.21.15.156:0/3213079669 >> v2:172.21.15.186:3300/0 conn(0x7f1bf00b20b0 0x7f1bf00d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:55.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.609+0000 7f1bf659c700 1 -- 172.21.15.156:0/3213079669 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bf00d01b0 msgr2=0x7f1bf00d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:55.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.609+0000 7f1bf659c700 1 --2- 172.21.15.156:0/3213079669 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bf00d01b0 0x7f1bf00d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.609+0000 7f1bf659c700 1 -- 172.21.15.156:0/3213079669 >> v2:172.21.15.186:3300/0 conn(0x7f1bf00b20b0 msgr2=0x7f1bf00d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:55.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.609+0000 7f1bf659c700 1 --2- 172.21.15.156:0/3213079669 >> v2:172.21.15.186:3300/0 conn(0x7f1bf00b20b0 0x7f1bf00d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.609+0000 7f1bf659c700 1 -- 172.21.15.156:0/3213079669 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf00b4850 con 0x7f1bf00b29b0 2023-11-28T13:10:55.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.609+0000 7f1bf659c700 1 --2- 172.21.15.156:0/3213079669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bf00b29b0 0x7f1bf00b2e30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00b5c10 tx=0x7f1be800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fb7dd23a178705 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:55.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.609+0000 7f1bf4d99700 1 -- 172.21.15.156:0/3213079669 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be800e070 con 0x7f1bf00b29b0 2023-11-28T13:10:55.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.609+0000 7f1bf4d99700 1 -- 172.21.15.156:0/3213079669 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be800b910 con 0x7f1bf00b29b0 2023-11-28T13:10:55.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.609+0000 7f1bf4d99700 1 -- 172.21.15.156:0/3213079669 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be800bcc0 con 0x7f1bf00b29b0 2023-11-28T13:10:55.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.610+0000 7f1bf7fff700 1 -- 172.21.15.156:0/3213079669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bf00b29b0 msgr2=0x7f1bf00b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:55.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.610+0000 7f1bf7fff700 1 --2- 172.21.15.156:0/3213079669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bf00b29b0 0x7f1bf00b2e30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00b5c10 tx=0x7f1be800b330 comp rx=0 tx=0).stop 2023-11-28T13:10:55.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.610+0000 7f1bf7fff700 1 -- 172.21.15.156:0/3213079669 shutdown_connections 2023-11-28T13:10:55.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.610+0000 7f1bf7fff700 1 --2- 172.21.15.156:0/3213079669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bf00b29b0 0x7f1bf00b2e30 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.610+0000 7f1bf7fff700 1 --2- 172.21.15.156:0/3213079669 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bf00d01b0 0x7f1bf00d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.610+0000 7f1bf7fff700 1 --2- 172.21.15.156:0/3213079669 >> v2:172.21.15.186:3300/0 conn(0x7f1bf00b20b0 0x7f1bf00d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.610+0000 7f1bf7fff700 1 -- 172.21.15.156:0/3213079669 >> 172.21.15.156:0/3213079669 conn(0x7f1bf009c3f0 msgr2=0x7f1bf00d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:55.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.610+0000 7f1bf7fff700 1 -- 172.21.15.156:0/3213079669 shutdown_connections 2023-11-28T13:10:55.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.610+0000 7f1bf7fff700 1 -- 172.21.15.156:0/3213079669 wait complete. 2023-11-28T13:10:55.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.612+0000 7f1bf7fff700 1 Processor -- start 2023-11-28T13:10:55.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.612+0000 7f1bf7fff700 1 -- start start 2023-11-28T13:10:55.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.613+0000 7f1bf7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bf00b20b0 0x7f1bf00ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:55.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.613+0000 7f1bf7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bf00b29b0 0x7f1bf0143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:55.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.613+0000 7f1bf7fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bf00d01b0 0x7f1bf00cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:55.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.613+0000 7f1bf7fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00c94a0 con 0x7f1bf00b20b0 2023-11-28T13:10:55.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.613+0000 7f1bf7fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00c9740 con 0x7f1bf00b29b0 2023-11-28T13:10:55.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.613+0000 7f1bf7fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00cb060 con 0x7f1bf00d01b0 2023-11-28T13:10:55.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.613+0000 7f1bf5d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bf00b20b0 0x7f1bf00ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:55.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.613+0000 7f1bf5d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bf00b20b0 0x7f1bf00ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49434/0 (socket says 172.21.15.156:49434) 2023-11-28T13:10:55.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.613+0000 7f1bf5d9b700 1 -- 172.21.15.156:0/2146246242 learned_addr learned my addr 172.21.15.156:0/2146246242 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:55.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.614+0000 7f1bf659c700 1 --2- 172.21.15.156:0/2146246242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bf00d01b0 0x7f1bf00cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:55.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.614+0000 7f1bf559a700 1 --2- 172.21.15.156:0/2146246242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bf00b29b0 0x7f1bf0143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:55.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.614+0000 7f1bf5d9b700 1 -- 172.21.15.156:0/2146246242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bf00d01b0 msgr2=0x7f1bf00cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:55.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.614+0000 7f1bf5d9b700 1 --2- 172.21.15.156:0/2146246242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bf00d01b0 0x7f1bf00cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.614+0000 7f1bf5d9b700 1 -- 172.21.15.156:0/2146246242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bf00b29b0 msgr2=0x7f1bf0143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:55.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.614+0000 7f1bf5d9b700 1 --2- 172.21.15.156:0/2146246242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bf00b29b0 0x7f1bf0143c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.614+0000 7f1bf5d9b700 1 -- 172.21.15.156:0/2146246242 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be8009b30 con 0x7f1bf00b20b0 2023-11-28T13:10:55.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.614+0000 7f1bf659c700 1 --2- 172.21.15.156:0/2146246242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bf00d01b0 0x7f1bf00cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:55.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.614+0000 7f1bf559a700 1 --2- 172.21.15.156:0/2146246242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bf00b29b0 0x7f1bf0143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:55.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.614+0000 7f1bf5d9b700 1 --2- 172.21.15.156:0/2146246242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bf00b20b0 0x7f1bf00ca640 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00e870 tx=0x7f1bec00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:55.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.614+0000 7f1be67fc700 1 -- 172.21.15.156:0/2146246242 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec009590 con 0x7f1bf00b20b0 2023-11-28T13:10:55.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.614+0000 7f1be67fc700 1 -- 172.21.15.156:0/2146246242 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bec00ee70 con 0x7f1bf00b20b0 2023-11-28T13:10:55.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.614+0000 7f1bf7fff700 1 -- 172.21.15.156:0/2146246242 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf00afec0 con 0x7f1bf00b20b0 2023-11-28T13:10:55.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.614+0000 7f1be67fc700 1 -- 172.21.15.156:0/2146246242 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec00ca50 con 0x7f1bf00b20b0 2023-11-28T13:10:55.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.614+0000 7f1bf7fff700 1 -- 172.21.15.156:0/2146246242 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf00afbe0 con 0x7f1bf00b20b0 2023-11-28T13:10:55.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.615+0000 7f1bf7fff700 1 -- 172.21.15.156:0/2146246242 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bf00b2600 con 0x7f1bf00b20b0 2023-11-28T13:10:55.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.616+0000 7f1be67fc700 1 -- 172.21.15.156:0/2146246242 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1bec019030 con 0x7f1bf00b20b0 2023-11-28T13:10:55.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.617+0000 7f1be67fc700 1 --2- 172.21.15.156:0/2146246242 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1bd40682f0 0x7f1bd406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:55.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.617+0000 7f1be67fc700 1 -- 172.21.15.156:0/2146246242 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1bec085c10 con 0x7f1bf00b20b0 2023-11-28T13:10:55.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.618+0000 7f1bf559a700 1 --2- 172.21.15.156:0/2146246242 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1bd40682f0 0x7f1bd406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:55.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.618+0000 7f1bf559a700 1 --2- 172.21.15.156:0/2146246242 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1bd40682f0 0x7f1bd406a7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00ccc70 tx=0x7f1be0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:55.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.621+0000 7f1be67fc700 1 -- 172.21.15.156:0/2146246242 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1bec052380 con 0x7f1bf00b20b0 2023-11-28T13:10:55.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.821+0000 7f1bf7fff700 1 -- 172.21.15.156:0/2146246242 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bf00cb3d0 con 0x7f1bf00b20b0 2023-11-28T13:10:55.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.822+0000 7f1be67fc700 1 -- 172.21.15.156:0/2146246242 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+17924 (secure 0 0 0) 0x7f1bec0559a0 con 0x7f1bf00b20b0 2023-11-28T13:10:55.824 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:55.824 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":58,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:10:54.928207+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:10:52.732399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T13:10:53.912753+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:10:55.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.825+0000 7f1bf7fff700 1 -- 172.21.15.156:0/2146246242 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1bd40682f0 msgr2=0x7f1bd406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:55.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.825+0000 7f1bf7fff700 1 --2- 172.21.15.156:0/2146246242 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1bd40682f0 0x7f1bd406a7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00ccc70 tx=0x7f1be0008040 comp rx=0 tx=0).stop 2023-11-28T13:10:55.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.825+0000 7f1bf7fff700 1 -- 172.21.15.156:0/2146246242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bf00b20b0 msgr2=0x7f1bf00ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:55.828 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.825+0000 7f1bf7fff700 1 --2- 172.21.15.156:0/2146246242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bf00b20b0 0x7f1bf00ca640 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00e870 tx=0x7f1bec00ec30 comp rx=0 tx=0).stop 2023-11-28T13:10:55.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.825+0000 7f1bf7fff700 1 -- 172.21.15.156:0/2146246242 shutdown_connections 2023-11-28T13:10:55.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.825+0000 7f1bf7fff700 1 --2- 172.21.15.156:0/2146246242 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1bd40682f0 0x7f1bd406a7a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.825+0000 7f1bf7fff700 1 --2- 172.21.15.156:0/2146246242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bf00d01b0 0x7f1bf00cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.825+0000 7f1bf7fff700 1 --2- 172.21.15.156:0/2146246242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bf00b20b0 0x7f1bf00ca640 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.825+0000 7f1bf7fff700 1 --2- 172.21.15.156:0/2146246242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bf00b29b0 0x7f1bf0143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:55.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.825+0000 7f1bf7fff700 1 -- 172.21.15.156:0/2146246242 >> 172.21.15.156:0/2146246242 conn(0x7f1bf009c3f0 msgr2=0x7f1bf00b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:55.829 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.825+0000 7f1bf7fff700 1 -- 172.21.15.156:0/2146246242 shutdown_connections 2023-11-28T13:10:55.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:55.826+0000 7f1bf7fff700 1 -- 172.21.15.156:0/2146246242 wait complete. 2023-11-28T13:10:55.847 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T13:10:56.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.049+0000 7fb41f1ff700 1 Processor -- start 2023-11-28T13:10:56.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.049+0000 7fb41f1ff700 1 -- start start 2023-11-28T13:10:56.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.050+0000 7fb41f1ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb41815c9b0 0x7fb4180b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:56.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.050+0000 7fb41f1ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb4180cd7c0 0x7fb4180cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:56.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.050+0000 7fb41f1ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4180d1950 con 0x7fb418161340 2023-11-28T13:10:56.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.050+0000 7fb41f1ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4180b5800 con 0x7fb4180cd7c0 2023-11-28T13:10:56.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.050+0000 7fb41f1ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4180ce140 con 0x7fb41815c9b0 2023-11-28T13:10:56.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.051+0000 7fb41d79c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb4180cd7c0 0x7fb4180cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:56.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.051+0000 7fb41d79c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb4180cd7c0 0x7fb4180cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:52974/0 (socket says 172.21.15.156:52974) 2023-11-28T13:10:56.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.051+0000 7fb41d79c700 1 -- 172.21.15.156:0/3432571309 learned_addr learned my addr 172.21.15.156:0/3432571309 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:56.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.051+0000 7fb41cf9b700 1 --2- 172.21.15.156:0/3432571309 >> v2:172.21.15.186:3300/0 conn(0x7fb41815c9b0 0x7fb4180b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:56.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.051+0000 7fb41d79c700 1 -- 172.21.15.156:0/3432571309 >> v2:172.21.15.186:3300/0 conn(0x7fb41815c9b0 msgr2=0x7fb4180b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:56.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.051+0000 7fb41d79c700 1 --2- 172.21.15.156:0/3432571309 >> v2:172.21.15.186:3300/0 conn(0x7fb41815c9b0 0x7fb4180b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:56.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.051+0000 7fb41d79c700 1 -- 172.21.15.156:0/3432571309 >> v1:172.21.15.186:6789/0 conn(0x7fb418161340 legacy=0x7fb4180d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:56.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.051+0000 7fb41d79c700 1 -- 172.21.15.156:0/3432571309 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4180d24f0 con 0x7fb4180cd7c0 2023-11-28T13:10:56.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.051+0000 7fb41d79c700 1 --2- 172.21.15.156:0/3432571309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180cd7c0 0x7fb4180cdc00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb418067460 tx=0x7fb41000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c8599187bfaf87 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:56.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.052+0000 7fb4177fe700 1 -- 172.21.15.156:0/3432571309 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41000e070 con 0x7fb4180cd7c0 2023-11-28T13:10:56.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.052+0000 7fb4177fe700 1 -- 172.21.15.156:0/3432571309 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41000b9c0 con 0x7fb4180cd7c0 2023-11-28T13:10:56.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.052+0000 7fb4177fe700 1 -- 172.21.15.156:0/3432571309 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41000bd70 con 0x7fb4180cd7c0 2023-11-28T13:10:56.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.052+0000 7fb41f1ff700 1 -- 172.21.15.156:0/3432571309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180cd7c0 msgr2=0x7fb4180cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:56.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.052+0000 7fb41f1ff700 1 --2- 172.21.15.156:0/3432571309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180cd7c0 0x7fb4180cdc00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb418067460 tx=0x7fb41000b330 comp rx=0 tx=0).stop 2023-11-28T13:10:56.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.052+0000 7fb41f1ff700 1 -- 172.21.15.156:0/3432571309 shutdown_connections 2023-11-28T13:10:56.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.052+0000 7fb41f1ff700 1 --2- 172.21.15.156:0/3432571309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180cd7c0 0x7fb4180cdc00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:56.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.052+0000 7fb41f1ff700 1 --2- 172.21.15.156:0/3432571309 >> v2:172.21.15.186:3300/0 conn(0x7fb41815c9b0 0x7fb4180b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:56.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.052+0000 7fb41f1ff700 1 -- 172.21.15.156:0/3432571309 >> 172.21.15.156:0/3432571309 conn(0x7fb418072600 msgr2=0x7fb4180cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:56.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.052+0000 7fb41f1ff700 1 -- 172.21.15.156:0/3432571309 shutdown_connections 2023-11-28T13:10:56.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.052+0000 7fb41f1ff700 1 -- 172.21.15.156:0/3432571309 wait complete. 2023-11-28T13:10:56.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.055+0000 7fb41f1ff700 1 Processor -- start 2023-11-28T13:10:56.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.055+0000 7fb41f1ff700 1 -- start start 2023-11-28T13:10:56.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.056+0000 7fb41f1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180cd7c0 0x7fb41814c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:56.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.056+0000 7fb41f1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb41815c9b0 0x7fb4180b7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:56.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.056+0000 7fb41f1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb418161340 0x7fb4180b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:56.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.056+0000 7fb41f1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4181e7b50 con 0x7fb4180cd7c0 2023-11-28T13:10:56.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.056+0000 7fb41f1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4180c79d0 con 0x7fb41815c9b0 2023-11-28T13:10:56.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.056+0000 7fb41f1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4180c7cb0 con 0x7fb418161340 2023-11-28T13:10:56.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.056+0000 7fb41cf9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180cd7c0 0x7fb41814c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:56.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.056+0000 7fb41cf9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180cd7c0 0x7fb41814c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:52988/0 (socket says 172.21.15.156:52988) 2023-11-28T13:10:56.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.056+0000 7fb41cf9b700 1 -- 172.21.15.156:0/1728051765 learned_addr learned my addr 172.21.15.156:0/1728051765 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:56.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.057+0000 7fb417fff700 1 --2- 172.21.15.156:0/1728051765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb41815c9b0 0x7fb4180b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:56.059 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.057+0000 7fb41d79c700 1 --2- 172.21.15.156:0/1728051765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb418161340 0x7fb4180b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:56.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.057+0000 7fb41cf9b700 1 -- 172.21.15.156:0/1728051765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb418161340 msgr2=0x7fb4180b7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:56.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.057+0000 7fb41cf9b700 1 --2- 172.21.15.156:0/1728051765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb418161340 0x7fb4180b7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:56.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.057+0000 7fb41cf9b700 1 -- 172.21.15.156:0/1728051765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb41815c9b0 msgr2=0x7fb4180b7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:56.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.057+0000 7fb41cf9b700 1 --2- 172.21.15.156:0/1728051765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb41815c9b0 0x7fb4180b7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:56.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.057+0000 7fb41cf9b700 1 -- 172.21.15.156:0/1728051765 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb410009b30 con 0x7fb4180cd7c0 2023-11-28T13:10:56.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.057+0000 7fb417fff700 1 --2- 172.21.15.156:0/1728051765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb41815c9b0 0x7fb4180b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:56.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.057+0000 7fb41cf9b700 1 --2- 172.21.15.156:0/1728051765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180cd7c0 0x7fb41814c730 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00d800 tx=0x7fb40c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:56.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.058+0000 7fb4157fa700 1 -- 172.21.15.156:0/1728051765 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c0125f0 con 0x7fb4180cd7c0 2023-11-28T13:10:56.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.058+0000 7fb4157fa700 1 -- 172.21.15.156:0/1728051765 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40c012750 con 0x7fb4180cd7c0 2023-11-28T13:10:56.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.058+0000 7fb41f1ff700 1 -- 172.21.15.156:0/1728051765 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4180d1300 con 0x7fb4180cd7c0 2023-11-28T13:10:56.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.058+0000 7fb4157fa700 1 -- 172.21.15.156:0/1728051765 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c012b00 con 0x7fb4180cd7c0 2023-11-28T13:10:56.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.058+0000 7fb41f1ff700 1 -- 172.21.15.156:0/1728051765 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4180b5520 con 0x7fb4180cd7c0 2023-11-28T13:10:56.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.059+0000 7fb41f1ff700 1 -- 172.21.15.156:0/1728051765 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4180d15e0 con 0x7fb4180cd7c0 2023-11-28T13:10:56.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.061+0000 7fb4157fa700 1 -- 172.21.15.156:0/1728051765 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb40c012c60 con 0x7fb4180cd7c0 2023-11-28T13:10:56.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.062+0000 7fb4157fa700 1 --2- 172.21.15.156:0/1728051765 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3fc0682f0 0x7fb3fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:56.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.063+0000 7fb4157fa700 1 -- 172.21.15.156:0/1728051765 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb40c086220 con 0x7fb4180cd7c0 2023-11-28T13:10:56.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.063+0000 7fb417fff700 1 --2- 172.21.15.156:0/1728051765 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3fc0682f0 0x7fb3fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:56.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.064+0000 7fb417fff700 1 --2- 172.21.15.156:0/1728051765 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3fc0682f0 0x7fb3fc06a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb41815a2f0 tx=0x7fb404009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:56.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.066+0000 7fb4157fa700 1 -- 172.21.15.156:0/1728051765 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb40c052990 con 0x7fb4180cd7c0 2023-11-28T13:10:56.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.299+0000 7fb41f1ff700 1 -- 172.21.15.156:0/1728051765 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb4180745c0 con 0x7fb4180cd7c0 2023-11-28T13:10:56.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.939+0000 7fb4157fa700 1 -- 172.21.15.156:0/1728051765 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb40c055fb0 con 0x7fb4180cd7c0 2023-11-28T13:10:56.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.941+0000 7fb41f1ff700 1 -- 172.21.15.156:0/1728051765 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3fc0682f0 msgr2=0x7fb3fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:56.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.941+0000 7fb41f1ff700 1 --2- 172.21.15.156:0/1728051765 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3fc0682f0 0x7fb3fc06a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb41815a2f0 tx=0x7fb404009040 comp rx=0 tx=0).stop 2023-11-28T13:10:56.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.941+0000 7fb41f1ff700 1 -- 172.21.15.156:0/1728051765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180cd7c0 msgr2=0x7fb41814c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:56.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.941+0000 7fb41f1ff700 1 --2- 172.21.15.156:0/1728051765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180cd7c0 0x7fb41814c730 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00d800 tx=0x7fb40c00dbc0 comp rx=0 tx=0).stop 2023-11-28T13:10:56.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.941+0000 7fb41f1ff700 1 -- 172.21.15.156:0/1728051765 shutdown_connections 2023-11-28T13:10:56.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.941+0000 7fb41f1ff700 1 --2- 172.21.15.156:0/1728051765 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3fc0682f0 0x7fb3fc06a7a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:56.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.941+0000 7fb41f1ff700 1 --2- 172.21.15.156:0/1728051765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb418161340 0x7fb4180b7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:56.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.941+0000 7fb41f1ff700 1 --2- 172.21.15.156:0/1728051765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180cd7c0 0x7fb41814c730 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:56.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.941+0000 7fb41f1ff700 1 --2- 172.21.15.156:0/1728051765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb41815c9b0 0x7fb4180b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:56.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.942+0000 7fb41f1ff700 1 -- 172.21.15.156:0/1728051765 >> 172.21.15.156:0/1728051765 conn(0x7fb418072600 msgr2=0x7fb4180b6320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:56.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.942+0000 7fb41f1ff700 1 -- 172.21.15.156:0/1728051765 shutdown_connections 2023-11-28T13:10:56.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:56.942+0000 7fb41f1ff700 1 -- 172.21.15.156:0/1728051765 wait complete. 2023-11-28T13:10:56.965 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T13:10:56.966 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T13:10:57.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.168+0000 7f6edb3ff700 1 Processor -- start 2023-11-28T13:10:57.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.168+0000 7f6edb3ff700 1 -- start start 2023-11-28T13:10:57.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.169+0000 7f6edb3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ed4161340 0x7f6ed40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:57.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.169+0000 7f6edb3ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6ed40d2320 0x7f6ed40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:57.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.169+0000 7f6edb3ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed40d1950 con 0x7f6ed415c9b0 2023-11-28T13:10:57.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.169+0000 7f6edb3ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ed40b5800 con 0x7f6ed40d2320 2023-11-28T13:10:57.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.169+0000 7f6edb3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed40b5300 con 0x7f6ed4161340 2023-11-28T13:10:57.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.170+0000 7f6ed919b700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f6ed415c9b0 0x7f6ed40b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:56320/0 (socket says 172.21.15.156:56320) 2023-11-28T13:10:57.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.170+0000 7f6ed919b700 1 -- 172.21.15.156:0/377552857 learned_addr learned my addr 172.21.15.156:0/377552857 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:57.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.170+0000 7f6ed899a700 1 --2- 172.21.15.156:0/377552857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ed4161340 0x7f6ed40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:57.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.170+0000 7f6ed999c700 1 --2- 172.21.15.156:0/377552857 >> v2:172.21.15.186:3300/0 conn(0x7f6ed40d2320 0x7f6ed40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:57.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.170+0000 7f6ed999c700 1 -- 172.21.15.156:0/377552857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ed4161340 msgr2=0x7f6ed40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:57.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.170+0000 7f6ed999c700 1 --2- 172.21.15.156:0/377552857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ed4161340 0x7f6ed40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:57.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.170+0000 7f6ed999c700 1 -- 172.21.15.156:0/377552857 >> v1:172.21.15.186:6789/0 conn(0x7f6ed415c9b0 legacy=0x7f6ed40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:57.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.171+0000 7f6ed999c700 1 -- 172.21.15.156:0/377552857 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed40b5520 con 0x7f6ed40d2320 2023-11-28T13:10:57.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.171+0000 7f6ed899a700 1 --2- 172.21.15.156:0/377552857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ed4161340 0x7f6ed40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:10:57.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.171+0000 7f6ed999c700 1 --2- 172.21.15.156:0/377552857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ed40d2320 0x7f6ed40cd7c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009dc0 tx=0x7f6ecc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c755727a585bcf1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:57.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.171+0000 7f6ecbfff700 1 -- 172.21.15.156:0/377552857 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc00e070 con 0x7f6ed40d2320 2023-11-28T13:10:57.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.171+0000 7f6ecbfff700 1 -- 172.21.15.156:0/377552857 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ecc00bae0 con 0x7f6ed40d2320 2023-11-28T13:10:57.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.171+0000 7f6ecbfff700 1 -- 172.21.15.156:0/377552857 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc00be90 con 0x7f6ed40d2320 2023-11-28T13:10:57.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.172+0000 7f6edb3ff700 1 -- 172.21.15.156:0/377552857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ed40d2320 msgr2=0x7f6ed40cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:57.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.172+0000 7f6edb3ff700 1 --2- 172.21.15.156:0/377552857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ed40d2320 0x7f6ed40cd7c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009dc0 tx=0x7f6ecc00b330 comp rx=0 tx=0).stop 2023-11-28T13:10:57.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.172+0000 7f6edb3ff700 1 -- 172.21.15.156:0/377552857 shutdown_connections 2023-11-28T13:10:57.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.172+0000 7f6edb3ff700 1 --2- 172.21.15.156:0/377552857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ed40d2320 0x7f6ed40cd7c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:57.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.172+0000 7f6edb3ff700 1 --2- 172.21.15.156:0/377552857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ed4161340 0x7f6ed40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:57.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.172+0000 7f6edb3ff700 1 -- 172.21.15.156:0/377552857 >> 172.21.15.156:0/377552857 conn(0x7f6ed4072600 msgr2=0x7f6ed40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:57.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.172+0000 7f6edb3ff700 1 -- 172.21.15.156:0/377552857 shutdown_connections 2023-11-28T13:10:57.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.172+0000 7f6edb3ff700 1 -- 172.21.15.156:0/377552857 wait complete. 2023-11-28T13:10:57.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.174+0000 7f6edb3ff700 1 Processor -- start 2023-11-28T13:10:57.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.174+0000 7f6edb3ff700 1 -- start start 2023-11-28T13:10:57.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.175+0000 7f6edb3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ed40d2320 0x7f6ed4152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:57.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.175+0000 7f6edb3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ed415c9b0 0x7f6ed4150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:57.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.175+0000 7f6edb3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ed4161340 0x7f6ed414e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:57.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.175+0000 7f6edb3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed4153860 con 0x7f6ed4161340 2023-11-28T13:10:57.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.175+0000 7f6edb3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed4155680 con 0x7f6ed415c9b0 2023-11-28T13:10:57.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.175+0000 7f6edb3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed4154d50 con 0x7f6ed40d2320 2023-11-28T13:10:57.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.175+0000 7f6ed999c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ed4161340 0x7f6ed414e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:57.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.175+0000 7f6ed999c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ed4161340 0x7f6ed414e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53004/0 (socket says 172.21.15.156:53004) 2023-11-28T13:10:57.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.175+0000 7f6ed999c700 1 -- 172.21.15.156:0/770286092 learned_addr learned my addr 172.21.15.156:0/770286092 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:57.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.175+0000 7f6ed899a700 1 --2- 172.21.15.156:0/770286092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ed415c9b0 0x7f6ed4150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:57.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.175+0000 7f6ed919b700 1 --2- 172.21.15.156:0/770286092 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ed40d2320 0x7f6ed4152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:57.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.175+0000 7f6ed999c700 1 -- 172.21.15.156:0/770286092 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ed40d2320 msgr2=0x7f6ed4152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:57.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.175+0000 7f6ed999c700 1 --2- 172.21.15.156:0/770286092 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ed40d2320 0x7f6ed4152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:57.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.175+0000 7f6ed999c700 1 -- 172.21.15.156:0/770286092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ed415c9b0 msgr2=0x7f6ed4150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:57.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.175+0000 7f6ed999c700 1 --2- 172.21.15.156:0/770286092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ed415c9b0 0x7f6ed4150460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:57.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.176+0000 7f6ed999c700 1 -- 172.21.15.156:0/770286092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec40094a0 con 0x7f6ed4161340 2023-11-28T13:10:57.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.176+0000 7f6ed999c700 1 --2- 172.21.15.156:0/770286092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ed4161340 0x7f6ed414e4e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00b370 tx=0x7f6ecc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:57.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.176+0000 7f6ec9ffb700 1 -- 172.21.15.156:0/770286092 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc004550 con 0x7f6ed4161340 2023-11-28T13:10:57.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.176+0000 7f6ec9ffb700 1 -- 172.21.15.156:0/770286092 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ecc020070 con 0x7f6ed4161340 2023-11-28T13:10:57.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.176+0000 7f6edb3ff700 1 -- 172.21.15.156:0/770286092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ecc009a70 con 0x7f6ed4161340 2023-11-28T13:10:57.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.176+0000 7f6ec9ffb700 1 -- 172.21.15.156:0/770286092 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc004910 con 0x7f6ed4161340 2023-11-28T13:10:57.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.176+0000 7f6edb3ff700 1 -- 172.21.15.156:0/770286092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed40d15e0 con 0x7f6ed4161340 2023-11-28T13:10:57.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.177+0000 7f6edb3ff700 1 -- 172.21.15.156:0/770286092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed40d1300 con 0x7f6ed4161340 2023-11-28T13:10:57.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.178+0000 7f6ec9ffb700 1 -- 172.21.15.156:0/770286092 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6ecc02e070 con 0x7f6ed4161340 2023-11-28T13:10:57.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.179+0000 7f6ec9ffb700 1 --2- 172.21.15.156:0/770286092 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6eb80682f0 0x7f6eb806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:57.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.179+0000 7f6ec9ffb700 1 -- 172.21.15.156:0/770286092 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6ecc022070 con 0x7f6ed4161340 2023-11-28T13:10:57.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.180+0000 7f6ed919b700 1 --2- 172.21.15.156:0/770286092 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6eb80682f0 0x7f6eb806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:57.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.180+0000 7f6ed919b700 1 --2- 172.21.15.156:0/770286092 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6eb80682f0 0x7f6eb806a7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0005910 tx=0x7f6ed0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:57.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.183+0000 7f6ec9ffb700 1 -- 172.21.15.156:0/770286092 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6ecc0540b0 con 0x7f6ed4161340 2023-11-28T13:10:57.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.415+0000 7f6edb3ff700 1 -- 172.21.15.156:0/770286092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6ed415cd80 con 0x7f6ed4161340 2023-11-28T13:10:57.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.943+0000 7f6ec9ffb700 1 -- 172.21.15.156:0/770286092 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6ecc02c0c0 con 0x7f6ed4161340 2023-11-28T13:10:57.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.946+0000 7f6edb3ff700 1 -- 172.21.15.156:0/770286092 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6eb80682f0 msgr2=0x7f6eb806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:57.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.946+0000 7f6edb3ff700 1 --2- 172.21.15.156:0/770286092 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6eb80682f0 0x7f6eb806a7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0005910 tx=0x7f6ed0009040 comp rx=0 tx=0).stop 2023-11-28T13:10:57.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.946+0000 7f6edb3ff700 1 -- 172.21.15.156:0/770286092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ed4161340 msgr2=0x7f6ed414e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:57.948 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.946+0000 7f6edb3ff700 1 --2- 172.21.15.156:0/770286092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ed4161340 0x7f6ed414e4e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00b370 tx=0x7f6ecc004ad0 comp rx=0 tx=0).stop 2023-11-28T13:10:57.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.946+0000 7f6edb3ff700 1 -- 172.21.15.156:0/770286092 shutdown_connections 2023-11-28T13:10:57.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.946+0000 7f6edb3ff700 1 --2- 172.21.15.156:0/770286092 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6eb80682f0 0x7f6eb806a7a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:57.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.946+0000 7f6edb3ff700 1 --2- 172.21.15.156:0/770286092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ed4161340 0x7f6ed414e4e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:57.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.946+0000 7f6edb3ff700 1 --2- 172.21.15.156:0/770286092 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ed40d2320 0x7f6ed4152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:57.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.946+0000 7f6edb3ff700 1 --2- 172.21.15.156:0/770286092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ed415c9b0 0x7f6ed4150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:57.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.946+0000 7f6edb3ff700 1 -- 172.21.15.156:0/770286092 >> 172.21.15.156:0/770286092 conn(0x7f6ed4072600 msgr2=0x7f6ed40b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:57.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.946+0000 7f6edb3ff700 1 -- 172.21.15.156:0/770286092 shutdown_connections 2023-11-28T13:10:57.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:57.946+0000 7f6edb3ff700 1 -- 172.21.15.156:0/770286092 wait complete. 2023-11-28T13:10:57.967 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T13:10:57.968 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:10:58.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.163+0000 7ff2af9ff700 1 Processor -- start 2023-11-28T13:10:58.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.163+0000 7ff2af9ff700 1 -- start start 2023-11-28T13:10:58.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.164+0000 7ff2af9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a815cbb0 0x7ff2a80d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:58.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.164+0000 7ff2af9ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff2a80b47b0 0x7ff2a80d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:58.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.164+0000 7ff2af9ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff2a80b1080 0x7ff2a80b1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:58.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.164+0000 7ff2af9ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a80d3060 con 0x7ff2a80b1080 2023-11-28T13:10:58.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.164+0000 7ff2af9ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a80afd90 con 0x7ff2a80b47b0 2023-11-28T13:10:58.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.164+0000 7ff2af9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80affe0 con 0x7ff2a815cbb0 2023-11-28T13:10:58.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.165+0000 7ff2adf9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff2a80b1080 0x7ff2a80b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:58.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.165+0000 7ff2adf9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff2a80b1080 0x7ff2a80b1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53010/0 (socket says 172.21.15.156:53010) 2023-11-28T13:10:58.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.165+0000 7ff2adf9c700 1 -- 172.21.15.156:0/1696308362 learned_addr learned my addr 172.21.15.156:0/1696308362 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:58.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.165+0000 7ff2acf9a700 1 --2- 172.21.15.156:0/1696308362 >> v2:172.21.15.186:3300/0 conn(0x7ff2a80b47b0 0x7ff2a80d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:58.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.165+0000 7ff2ad79b700 1 --2- 172.21.15.156:0/1696308362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a815cbb0 0x7ff2a80d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:58.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.165+0000 7ff2adf9c700 1 -- 172.21.15.156:0/1696308362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a815cbb0 msgr2=0x7ff2a80d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:58.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.165+0000 7ff2adf9c700 1 --2- 172.21.15.156:0/1696308362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a815cbb0 0x7ff2a80d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.165+0000 7ff2adf9c700 1 -- 172.21.15.156:0/1696308362 >> v2:172.21.15.186:3300/0 conn(0x7ff2a80b47b0 msgr2=0x7ff2a80d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:58.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.165+0000 7ff2adf9c700 1 --2- 172.21.15.156:0/1696308362 >> v2:172.21.15.186:3300/0 conn(0x7ff2a80b47b0 0x7ff2a80d12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.165+0000 7ff2adf9c700 1 -- 172.21.15.156:0/1696308362 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a80b02c0 con 0x7ff2a80b1080 2023-11-28T13:10:58.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.165+0000 7ff2ad79b700 1 --2- 172.21.15.156:0/1696308362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a815cbb0 0x7ff2a80d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:58.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.165+0000 7ff2acf9a700 1 --2- 172.21.15.156:0/1696308362 >> v2:172.21.15.186:3300/0 conn(0x7ff2a80b47b0 0x7ff2a80d12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:58.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.165+0000 7ff2adf9c700 1 --2- 172.21.15.156:0/1696308362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a80b1080 0x7ff2a80b1500 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff2a806a480 tx=0x7ff2a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564410d9c44dcc1a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:58.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.166+0000 7ff29ffff700 1 -- 172.21.15.156:0/1696308362 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a000e070 con 0x7ff2a80b1080 2023-11-28T13:10:58.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.166+0000 7ff29ffff700 1 -- 172.21.15.156:0/1696308362 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a000b9c0 con 0x7ff2a80b1080 2023-11-28T13:10:58.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.166+0000 7ff29ffff700 1 -- 172.21.15.156:0/1696308362 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a000bd70 con 0x7ff2a80b1080 2023-11-28T13:10:58.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.166+0000 7ff2af9ff700 1 -- 172.21.15.156:0/1696308362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a80b1080 msgr2=0x7ff2a80b1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:58.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.166+0000 7ff2af9ff700 1 --2- 172.21.15.156:0/1696308362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a80b1080 0x7ff2a80b1500 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff2a806a480 tx=0x7ff2a000b330 comp rx=0 tx=0).stop 2023-11-28T13:10:58.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.166+0000 7ff2af9ff700 1 -- 172.21.15.156:0/1696308362 shutdown_connections 2023-11-28T13:10:58.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.166+0000 7ff2af9ff700 1 --2- 172.21.15.156:0/1696308362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a80b1080 0x7ff2a80b1500 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.166+0000 7ff2af9ff700 1 --2- 172.21.15.156:0/1696308362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a815cbb0 0x7ff2a80d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.166+0000 7ff2af9ff700 1 --2- 172.21.15.156:0/1696308362 >> v2:172.21.15.186:3300/0 conn(0x7ff2a80b47b0 0x7ff2a80d12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.166+0000 7ff2af9ff700 1 -- 172.21.15.156:0/1696308362 >> 172.21.15.156:0/1696308362 conn(0x7ff2a809c430 msgr2=0x7ff2a80d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:58.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.167+0000 7ff2af9ff700 1 -- 172.21.15.156:0/1696308362 shutdown_connections 2023-11-28T13:10:58.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.167+0000 7ff2af9ff700 1 -- 172.21.15.156:0/1696308362 wait complete. 2023-11-28T13:10:58.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.168+0000 7ff2af9ff700 1 Processor -- start 2023-11-28T13:10:58.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.168+0000 7ff2af9ff700 1 -- start start 2023-11-28T13:10:58.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.169+0000 7ff2af9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a80b1080 0x7ff2a81e66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:58.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.169+0000 7ff2af9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a80b47b0 0x7ff2a81e4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:58.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.169+0000 7ff2af9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a815cbb0 0x7ff2a80b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:58.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.169+0000 7ff2af9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a8155f30 con 0x7ff2a80b47b0 2023-11-28T13:10:58.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.169+0000 7ff2af9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a8156210 con 0x7ff2a80b1080 2023-11-28T13:10:58.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.169+0000 7ff2af9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a81e4680 con 0x7ff2a815cbb0 2023-11-28T13:10:58.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.169+0000 7ff2acf9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a80b47b0 0x7ff2a81e4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:58.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.170+0000 7ff2acf9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a80b47b0 0x7ff2a81e4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53016/0 (socket says 172.21.15.156:53016) 2023-11-28T13:10:58.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.170+0000 7ff2acf9a700 1 -- 172.21.15.156:0/4267138394 learned_addr learned my addr 172.21.15.156:0/4267138394 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:58.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.170+0000 7ff2ad79b700 1 --2- 172.21.15.156:0/4267138394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a80b1080 0x7ff2a81e66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:58.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.170+0000 7ff2adf9c700 1 --2- 172.21.15.156:0/4267138394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a815cbb0 0x7ff2a80b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:58.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.170+0000 7ff2acf9a700 1 -- 172.21.15.156:0/4267138394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a815cbb0 msgr2=0x7ff2a80b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:58.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.170+0000 7ff2acf9a700 1 --2- 172.21.15.156:0/4267138394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a815cbb0 0x7ff2a80b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.170+0000 7ff2acf9a700 1 -- 172.21.15.156:0/4267138394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a80b1080 msgr2=0x7ff2a81e66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:58.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.170+0000 7ff2acf9a700 1 --2- 172.21.15.156:0/4267138394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a80b1080 0x7ff2a81e66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.170+0000 7ff2acf9a700 1 -- 172.21.15.156:0/4267138394 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a0009b30 con 0x7ff2a80b47b0 2023-11-28T13:10:58.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.170+0000 7ff2acf9a700 1 --2- 172.21.15.156:0/4267138394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a80b47b0 0x7ff2a81e4140 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff29800d830 tx=0x7ff29800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:58.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.170+0000 7ff29dffb700 1 -- 172.21.15.156:0/4267138394 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298004030 con 0x7ff2a80b47b0 2023-11-28T13:10:58.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.170+0000 7ff29dffb700 1 -- 172.21.15.156:0/4267138394 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff298004190 con 0x7ff2a80b47b0 2023-11-28T13:10:58.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.170+0000 7ff29dffb700 1 -- 172.21.15.156:0/4267138394 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298004500 con 0x7ff2a80b47b0 2023-11-28T13:10:58.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.171+0000 7ff2af9ff700 1 -- 172.21.15.156:0/4267138394 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a80b0880 con 0x7ff2a80b47b0 2023-11-28T13:10:58.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.171+0000 7ff2af9ff700 1 -- 172.21.15.156:0/4267138394 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a80b05a0 con 0x7ff2a80b47b0 2023-11-28T13:10:58.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.172+0000 7ff2af9ff700 1 -- 172.21.15.156:0/4267138394 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a80b0b60 con 0x7ff2a80b47b0 2023-11-28T13:10:58.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.172+0000 7ff29dffb700 1 -- 172.21.15.156:0/4267138394 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff29801b030 con 0x7ff2a80b47b0 2023-11-28T13:10:58.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.174+0000 7ff29dffb700 1 --2- 172.21.15.156:0/4267138394 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff28c0682f0 0x7ff28c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:58.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.174+0000 7ff29dffb700 1 -- 172.21.15.156:0/4267138394 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff298085420 con 0x7ff2a80b47b0 2023-11-28T13:10:58.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.174+0000 7ff2ad79b700 1 --2- 172.21.15.156:0/4267138394 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff28c0682f0 0x7ff28c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:58.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.175+0000 7ff2ad79b700 1 --2- 172.21.15.156:0/4267138394 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff28c0682f0 0x7ff28c06a7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400c960 tx=0x7ff2a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:58.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.178+0000 7ff29dffb700 1 -- 172.21.15.156:0/4267138394 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff298051c40 con 0x7ff2a80b47b0 2023-11-28T13:10:58.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.409+0000 7ff2af9ff700 1 -- 172.21.15.156:0/4267138394 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2a80745c0 con 0x7ff2a80b47b0 2023-11-28T13:10:58.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.410+0000 7ff29dffb700 1 -- 172.21.15.156:0/4267138394 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4658 (secure 0 0 0) 0x7ff298019090 con 0x7ff2a80b47b0 2023-11-28T13:10:58.412 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:58.412 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3942362276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3942362276},{"type":"v1","addr":"172.21.15.186:6835","nonce":3942362276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/722198303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":722198303},{"type":"v1","addr":"172.21.15.186:6837","nonce":722198303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5916,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3601975324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3601975324},{"type":"v1","addr":"172.21.15.156:6835","nonce":3601975324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:10:54.928601+0000","modified":"2023-11-28T13:10:57.942034+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5919":{"gid":5919,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6834/3314379816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3314379816},{"type":"v1","addr":"172.21.15.156:6834","nonce":3314379816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:10:58.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.412+0000 7ff2af9ff700 1 -- 172.21.15.156:0/4267138394 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff28c0682f0 msgr2=0x7ff28c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:58.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.412+0000 7ff2af9ff700 1 --2- 172.21.15.156:0/4267138394 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff28c0682f0 0x7ff28c06a7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400c960 tx=0x7ff2a4008040 comp rx=0 tx=0).stop 2023-11-28T13:10:58.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.412+0000 7ff2af9ff700 1 -- 172.21.15.156:0/4267138394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a80b47b0 msgr2=0x7ff2a81e4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:58.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.412+0000 7ff2af9ff700 1 --2- 172.21.15.156:0/4267138394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a80b47b0 0x7ff2a81e4140 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff29800d830 tx=0x7ff29800db40 comp rx=0 tx=0).stop 2023-11-28T13:10:58.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.412+0000 7ff2af9ff700 1 -- 172.21.15.156:0/4267138394 shutdown_connections 2023-11-28T13:10:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.412+0000 7ff2af9ff700 1 --2- 172.21.15.156:0/4267138394 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff28c0682f0 0x7ff28c06a7a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.412+0000 7ff2af9ff700 1 --2- 172.21.15.156:0/4267138394 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2a815cbb0 0x7ff2a80b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.412+0000 7ff2af9ff700 1 --2- 172.21.15.156:0/4267138394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a80b1080 0x7ff2a81e66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.412+0000 7ff2af9ff700 1 --2- 172.21.15.156:0/4267138394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a80b47b0 0x7ff2a81e4140 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.412+0000 7ff2af9ff700 1 -- 172.21.15.156:0/4267138394 >> 172.21.15.156:0/4267138394 conn(0x7ff2a809c430 msgr2=0x7ff2a80b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.413+0000 7ff2af9ff700 1 -- 172.21.15.156:0/4267138394 shutdown_connections 2023-11-28T13:10:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.413+0000 7ff2af9ff700 1 -- 172.21.15.156:0/4267138394 wait complete. 2023-11-28T13:10:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 31 2023-11-28T13:10:58.433 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:10:58.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.629+0000 7f1df89ff700 1 Processor -- start 2023-11-28T13:10:58.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.629+0000 7f1df89ff700 1 -- start start 2023-11-28T13:10:58.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.630+0000 7f1df89ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1df4161340 0x7f1df40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:58.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.630+0000 7f1df89ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1df40d2320 0x7f1df40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:58.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.630+0000 7f1df89ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df40d1950 con 0x7f1df415c9b0 2023-11-28T13:10:58.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.630+0000 7f1df89ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df40b5800 con 0x7f1df4161340 2023-11-28T13:10:58.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.630+0000 7f1df89ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df40b5300 con 0x7f1df40d2320 2023-11-28T13:10:58.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.630+0000 7f1df1d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1df4161340 0x7f1df40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:58.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.630+0000 7f1df1d9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1df4161340 0x7f1df40d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53032/0 (socket says 172.21.15.156:53032) 2023-11-28T13:10:58.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.630+0000 7f1df1d9b700 1 -- 172.21.15.156:0/2726795390 learned_addr learned my addr 172.21.15.156:0/2726795390 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:58.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.630+0000 7f1df2d9d700 1 --2- 172.21.15.156:0/2726795390 >> v2:172.21.15.186:3300/0 conn(0x7f1df40d2320 0x7f1df40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:58.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.631+0000 7f1df1d9b700 1 -- 172.21.15.156:0/2726795390 >> v2:172.21.15.186:3300/0 conn(0x7f1df40d2320 msgr2=0x7f1df40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:10:58.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.631+0000 7f1df1d9b700 1 --2- 172.21.15.156:0/2726795390 >> v2:172.21.15.186:3300/0 conn(0x7f1df40d2320 0x7f1df40cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.631+0000 7f1df1d9b700 1 -- 172.21.15.156:0/2726795390 >> v1:172.21.15.186:6789/0 conn(0x7f1df415c9b0 legacy=0x7f1df40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:58.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.631+0000 7f1df1d9b700 1 -- 172.21.15.156:0/2726795390 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df40b5520 con 0x7f1df4161340 2023-11-28T13:10:58.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.631+0000 7f1df2d9d700 1 --2- 172.21.15.156:0/2726795390 >> v2:172.21.15.186:3300/0 conn(0x7f1df40d2320 0x7f1df40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:10:58.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.631+0000 7f1df1d9b700 1 --2- 172.21.15.156:0/2726795390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1df4161340 0x7f1df40d1de0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1df406c170 tx=0x7f1de8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbbf11a04b4c223 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:58.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.631+0000 7f1df159a700 1 -- 172.21.15.156:0/2726795390 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de800b860 con 0x7f1df4161340 2023-11-28T13:10:58.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.631+0000 7f1df159a700 1 -- 172.21.15.156:0/2726795390 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de800b9c0 con 0x7f1df4161340 2023-11-28T13:10:58.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.631+0000 7f1df159a700 1 -- 172.21.15.156:0/2726795390 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de800bd50 con 0x7f1df4161340 2023-11-28T13:10:58.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.632+0000 7f1df89ff700 1 -- 172.21.15.156:0/2726795390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1df4161340 msgr2=0x7f1df40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:58.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.632+0000 7f1df89ff700 1 --2- 172.21.15.156:0/2726795390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1df4161340 0x7f1df40d1de0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1df406c170 tx=0x7f1de8009c00 comp rx=0 tx=0).stop 2023-11-28T13:10:58.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.632+0000 7f1df89ff700 1 -- 172.21.15.156:0/2726795390 shutdown_connections 2023-11-28T13:10:58.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.632+0000 7f1df89ff700 1 --2- 172.21.15.156:0/2726795390 >> v2:172.21.15.186:3300/0 conn(0x7f1df40d2320 0x7f1df40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.632+0000 7f1df89ff700 1 --2- 172.21.15.156:0/2726795390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1df4161340 0x7f1df40d1de0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.632+0000 7f1df89ff700 1 -- 172.21.15.156:0/2726795390 >> 172.21.15.156:0/2726795390 conn(0x7f1df4072600 msgr2=0x7f1df40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:58.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.632+0000 7f1df89ff700 1 -- 172.21.15.156:0/2726795390 shutdown_connections 2023-11-28T13:10:58.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.632+0000 7f1df89ff700 1 -- 172.21.15.156:0/2726795390 wait complete. 2023-11-28T13:10:58.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.634+0000 7f1df89ff700 1 Processor -- start 2023-11-28T13:10:58.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.634+0000 7f1df89ff700 1 -- start start 2023-11-28T13:10:58.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.635+0000 7f1df89ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1df40d2320 0x7f1df415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:58.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.635+0000 7f1df89ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1df415c9b0 0x7f1df41f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:58.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.635+0000 7f1df89ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1df4161340 0x7f1df41f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:58.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.635+0000 7f1df89ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df414cb60 con 0x7f1df415c9b0 2023-11-28T13:10:58.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.635+0000 7f1df89ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df41f3340 con 0x7f1df40d2320 2023-11-28T13:10:58.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.635+0000 7f1df89ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df415e240 con 0x7f1df4161340 2023-11-28T13:10:58.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.635+0000 7f1df1d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1df415c9b0 0x7f1df41f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:58.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.635+0000 7f1df1d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1df415c9b0 0x7f1df41f28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53048/0 (socket says 172.21.15.156:53048) 2023-11-28T13:10:58.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.635+0000 7f1df1d9b700 1 -- 172.21.15.156:0/281741545 learned_addr learned my addr 172.21.15.156:0/281741545 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:58.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.635+0000 7f1df2d9d700 1 --2- 172.21.15.156:0/281741545 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1df4161340 0x7f1df41f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:58.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.635+0000 7f1df259c700 1 --2- 172.21.15.156:0/281741545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1df40d2320 0x7f1df415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:58.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.635+0000 7f1df1d9b700 1 -- 172.21.15.156:0/281741545 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1df4161340 msgr2=0x7f1df41f2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:58.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.635+0000 7f1df1d9b700 1 --2- 172.21.15.156:0/281741545 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1df4161340 0x7f1df41f2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.635+0000 7f1df1d9b700 1 -- 172.21.15.156:0/281741545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1df40d2320 msgr2=0x7f1df415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:58.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.635+0000 7f1df1d9b700 1 --2- 172.21.15.156:0/281741545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1df40d2320 0x7f1df415f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.635+0000 7f1df1d9b700 1 -- 172.21.15.156:0/281741545 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de8009550 con 0x7f1df415c9b0 2023-11-28T13:10:58.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.635+0000 7f1df259c700 1 --2- 172.21.15.156:0/281741545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1df40d2320 0x7f1df415f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:10:58.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.636+0000 7f1df1d9b700 1 --2- 172.21.15.156:0/281741545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1df415c9b0 0x7f1df41f28c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009c40 tx=0x7f1de8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:58.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.636+0000 7f1de2ffd700 1 -- 172.21.15.156:0/281741545 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de80036a0 con 0x7f1df415c9b0 2023-11-28T13:10:58.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.636+0000 7f1de2ffd700 1 -- 172.21.15.156:0/281741545 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de8003820 con 0x7f1df415c9b0 2023-11-28T13:10:58.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.636+0000 7f1df89ff700 1 -- 172.21.15.156:0/281741545 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df40d15e0 con 0x7f1df415c9b0 2023-11-28T13:10:58.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.636+0000 7f1de2ffd700 1 -- 172.21.15.156:0/281741545 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8003bd0 con 0x7f1df415c9b0 2023-11-28T13:10:58.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.636+0000 7f1df89ff700 1 -- 172.21.15.156:0/281741545 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df40d1300 con 0x7f1df415c9b0 2023-11-28T13:10:58.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.637+0000 7f1df89ff700 1 -- 172.21.15.156:0/281741545 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df415cd80 con 0x7f1df415c9b0 2023-11-28T13:10:58.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.638+0000 7f1de2ffd700 1 -- 172.21.15.156:0/281741545 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1de800f050 con 0x7f1df415c9b0 2023-11-28T13:10:58.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.639+0000 7f1de2ffd700 1 --2- 172.21.15.156:0/281741545 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1dd40682f0 0x7f1dd406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:58.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.639+0000 7f1de2ffd700 1 -- 172.21.15.156:0/281741545 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1de80874c0 con 0x7f1df415c9b0 2023-11-28T13:10:58.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.639+0000 7f1df259c700 1 --2- 172.21.15.156:0/281741545 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1dd40682f0 0x7f1dd406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:58.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.640+0000 7f1df259c700 1 --2- 172.21.15.156:0/281741545 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1dd40682f0 0x7f1dd406a7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1de4000f60 tx=0x7f1de4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:58.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.643+0000 7f1de2ffd700 1 -- 172.21.15.156:0/281741545 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1de8053ca0 con 0x7f1df415c9b0 2023-11-28T13:10:58.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.835+0000 7f1df89ff700 1 -- 172.21.15.156:0/281741545 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1df40cf0d0 con 0x7f1df415c9b0 2023-11-28T13:10:58.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.836+0000 7f1de2ffd700 1 -- 172.21.15.156:0/281741545 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+17911 (secure 0 0 0) 0x7f1de80572c0 con 0x7f1df415c9b0 2023-11-28T13:10:58.839 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:58.839 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":59,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:10:55.931198+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:10:52.732399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T13:10:53.912753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:10:58.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.839+0000 7f1df89ff700 1 -- 172.21.15.156:0/281741545 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1dd40682f0 msgr2=0x7f1dd406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:58.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.839+0000 7f1df89ff700 1 --2- 172.21.15.156:0/281741545 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1dd40682f0 0x7f1dd406a7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1de4000f60 tx=0x7f1de4009040 comp rx=0 tx=0).stop 2023-11-28T13:10:58.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.839+0000 7f1df89ff700 1 -- 172.21.15.156:0/281741545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1df415c9b0 msgr2=0x7f1df41f28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:58.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.839+0000 7f1df89ff700 1 --2- 172.21.15.156:0/281741545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1df415c9b0 0x7f1df41f28c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009c40 tx=0x7f1de8004750 comp rx=0 tx=0).stop 2023-11-28T13:10:58.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.839+0000 7f1df89ff700 1 -- 172.21.15.156:0/281741545 shutdown_connections 2023-11-28T13:10:58.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.839+0000 7f1df89ff700 1 --2- 172.21.15.156:0/281741545 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1dd40682f0 0x7f1dd406a7a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.839+0000 7f1df89ff700 1 --2- 172.21.15.156:0/281741545 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1df4161340 0x7f1df41f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.839+0000 7f1df89ff700 1 --2- 172.21.15.156:0/281741545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1df40d2320 0x7f1df415f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.839+0000 7f1df89ff700 1 --2- 172.21.15.156:0/281741545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1df415c9b0 0x7f1df41f28c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:58.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.839+0000 7f1df89ff700 1 -- 172.21.15.156:0/281741545 >> 172.21.15.156:0/281741545 conn(0x7f1df4072600 msgr2=0x7f1df40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:58.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.839+0000 7f1df89ff700 1 -- 172.21.15.156:0/281741545 shutdown_connections 2023-11-28T13:10:58.842 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:58.839+0000 7f1df89ff700 1 -- 172.21.15.156:0/281741545 wait complete. 2023-11-28T13:10:58.859 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T13:10:58.860 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:10:59.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.056+0000 7f937dbff700 1 Processor -- start 2023-11-28T13:10:59.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.057+0000 7f937dbff700 1 -- start start 2023-11-28T13:10:59.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.058+0000 7f937dbff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f937815c9b0 0x7f93780b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:59.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.058+0000 7f937dbff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93780d1950 con 0x7f93780cd7c0 2023-11-28T13:10:59.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.058+0000 7f937dbff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93780d24f0 con 0x7f9378161340 2023-11-28T13:10:59.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.058+0000 7f937dbff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93780b5800 con 0x7f937815c9b0 2023-11-28T13:10:59.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.058+0000 7f93777fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f937815c9b0 0x7f93780b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:59.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.058+0000 7f93777fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f937815c9b0 0x7f93780b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53058/0 (socket says 172.21.15.156:53058) 2023-11-28T13:10:59.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.058+0000 7f93777fe700 1 -- 172.21.15.156:0/3578880566 learned_addr learned my addr 172.21.15.156:0/3578880566 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:59.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.059+0000 7f93777fe700 1 -- 172.21.15.156:0/3578880566 >> v1:172.21.15.186:6789/0 conn(0x7f9378161340 legacy=0x7f93780d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:59.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.059+0000 7f93777fe700 1 -- 172.21.15.156:0/3578880566 >> v1:172.21.15.156:6789/0 conn(0x7f93780cd7c0 legacy=0x7f93780b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:59.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.059+0000 7f93777fe700 1 -- 172.21.15.156:0/3578880566 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93780b5520 con 0x7f937815c9b0 2023-11-28T13:10:59.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.059+0000 7f93777fe700 1 --2- 172.21.15.156:0/3578880566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f937815c9b0 0x7f93780b4bf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f93680098f0 tx=0x7f9368009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13e8a50bcdf3802 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:59.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.059+0000 7f93767fc700 1 -- 172.21.15.156:0/3578880566 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936800b8a0 con 0x7f937815c9b0 2023-11-28T13:10:59.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.059+0000 7f93767fc700 1 -- 172.21.15.156:0/3578880566 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936800ba00 con 0x7f937815c9b0 2023-11-28T13:10:59.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.059+0000 7f93767fc700 1 -- 172.21.15.156:0/3578880566 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936800bd90 con 0x7f937815c9b0 2023-11-28T13:10:59.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.060+0000 7f937dbff700 1 -- 172.21.15.156:0/3578880566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f937815c9b0 msgr2=0x7f93780b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:59.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.060+0000 7f937dbff700 1 --2- 172.21.15.156:0/3578880566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f937815c9b0 0x7f93780b4bf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f93680098f0 tx=0x7f9368009c00 comp rx=0 tx=0).stop 2023-11-28T13:10:59.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.060+0000 7f937dbff700 1 -- 172.21.15.156:0/3578880566 shutdown_connections 2023-11-28T13:10:59.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.060+0000 7f937dbff700 1 --2- 172.21.15.156:0/3578880566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f937815c9b0 0x7f93780b4bf0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:59.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.060+0000 7f937dbff700 1 -- 172.21.15.156:0/3578880566 >> 172.21.15.156:0/3578880566 conn(0x7f9378072600 msgr2=0x7f93780cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:59.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.060+0000 7f937dbff700 1 -- 172.21.15.156:0/3578880566 shutdown_connections 2023-11-28T13:10:59.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.060+0000 7f937dbff700 1 -- 172.21.15.156:0/3578880566 wait complete. 2023-11-28T13:10:59.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.062+0000 7f937dbff700 1 Processor -- start 2023-11-28T13:10:59.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.062+0000 7f937dbff700 1 -- start start 2023-11-28T13:10:59.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.064+0000 7f937dbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93780cd7c0 0x7f9378153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:59.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.064+0000 7f937dbff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f937815c9b0 0x7f937815e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:59.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.064+0000 7f937dbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9378161340 0x7f9378153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:59.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.064+0000 7f937dbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9378155740 con 0x7f93780cd7c0 2023-11-28T13:10:59.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.064+0000 7f937dbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9378155990 con 0x7f9378161340 2023-11-28T13:10:59.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.064+0000 7f937dbff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9378155c70 con 0x7f937815c9b0 2023-11-28T13:10:59.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.064+0000 7f93777fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93780cd7c0 0x7f9378153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:59.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.064+0000 7f93777fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93780cd7c0 0x7f9378153f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53074/0 (socket says 172.21.15.156:53074) 2023-11-28T13:10:59.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.064+0000 7f93777fe700 1 -- 172.21.15.156:0/4038849007 learned_addr learned my addr 172.21.15.156:0/4038849007 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:59.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.064+0000 7f93777fe700 1 -- 172.21.15.156:0/4038849007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f937815c9b0 msgr2=0x7f937815e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:59.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.064+0000 7f93777fe700 1 --2- 172.21.15.156:0/4038849007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f937815c9b0 0x7f937815e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.064+0000 7f93777fe700 1 -- 172.21.15.156:0/4038849007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9378161340 msgr2=0x7f9378153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.064+0000 7f93777fe700 1 --2- 172.21.15.156:0/4038849007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9378161340 0x7f9378153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.064+0000 7f93777fe700 1 -- 172.21.15.156:0/4038849007 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9368009550 con 0x7f93780cd7c0 2023-11-28T13:10:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.064+0000 7f93777fe700 1 --2- 172.21.15.156:0/4038849007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93780cd7c0 0x7f9378153f30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9368009130 tx=0x7f9368004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.065+0000 7f935ffff700 1 -- 172.21.15.156:0/4038849007 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93680036a0 con 0x7f93780cd7c0 2023-11-28T13:10:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.065+0000 7f935ffff700 1 -- 172.21.15.156:0/4038849007 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936801e070 con 0x7f93780cd7c0 2023-11-28T13:10:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.065+0000 7f935ffff700 1 -- 172.21.15.156:0/4038849007 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368003a60 con 0x7f93780cd7c0 2023-11-28T13:10:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.065+0000 7f937dbff700 1 -- 172.21.15.156:0/4038849007 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93780d15e0 con 0x7f93780cd7c0 2023-11-28T13:10:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.065+0000 7f937dbff700 1 -- 172.21.15.156:0/4038849007 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93780d1300 con 0x7f93780cd7c0 2023-11-28T13:10:59.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.066+0000 7f937dbff700 1 -- 172.21.15.156:0/4038849007 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93780745c0 con 0x7f93780cd7c0 2023-11-28T13:10:59.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.067+0000 7f935ffff700 1 -- 172.21.15.156:0/4038849007 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f936800f050 con 0x7f93780cd7c0 2023-11-28T13:10:59.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.068+0000 7f935ffff700 1 --2- 172.21.15.156:0/4038849007 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9364068340 0x7f936406a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:59.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.068+0000 7f935ffff700 1 -- 172.21.15.156:0/4038849007 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9368020070 con 0x7f93780cd7c0 2023-11-28T13:10:59.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.068+0000 7f9376ffd700 1 --2- 172.21.15.156:0/4038849007 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9364068340 0x7f936406a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:59.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.069+0000 7f9376ffd700 1 --2- 172.21.15.156:0/4038849007 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9364068340 0x7f936406a7f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f936c000f60 tx=0x7f936c007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:59.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.072+0000 7f935ffff700 1 -- 172.21.15.156:0/4038849007 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9368053b40 con 0x7f93780cd7c0 2023-11-28T13:10:59.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.258+0000 7f937dbff700 1 -- 172.21.15.156:0/4038849007 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93780b50b0 con 0x7f9364068340 2023-11-28T13:10:59.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.268+0000 7f935ffff700 1 -- 172.21.15.156:0/4038849007 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311706 (secure 0 0 0) 0x7f93780b50b0 con 0x7f9364068340 2023-11-28T13:10:59.274 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:59.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.273+0000 7f937dbff700 1 -- 172.21.15.156:0/4038849007 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9364068340 msgr2=0x7f936406a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:59.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.273+0000 7f937dbff700 1 --2- 172.21.15.156:0/4038849007 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9364068340 0x7f936406a7f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f936c000f60 tx=0x7f936c007a40 comp rx=0 tx=0).stop 2023-11-28T13:10:59.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.273+0000 7f937dbff700 1 -- 172.21.15.156:0/4038849007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93780cd7c0 msgr2=0x7f9378153f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:59.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.273+0000 7f937dbff700 1 --2- 172.21.15.156:0/4038849007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93780cd7c0 0x7f9378153f30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9368009130 tx=0x7f9368004750 comp rx=0 tx=0).stop 2023-11-28T13:10:59.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.273+0000 7f937dbff700 1 -- 172.21.15.156:0/4038849007 shutdown_connections 2023-11-28T13:10:59.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.273+0000 7f937dbff700 1 --2- 172.21.15.156:0/4038849007 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9364068340 0x7f936406a7f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:59.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.273+0000 7f937dbff700 1 --2- 172.21.15.156:0/4038849007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9378161340 0x7f9378153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:59.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.273+0000 7f937dbff700 1 --2- 172.21.15.156:0/4038849007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93780cd7c0 0x7f9378153f30 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:59.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.273+0000 7f937dbff700 1 --2- 172.21.15.156:0/4038849007 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f937815c9b0 0x7f937815e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:59.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.273+0000 7f937dbff700 1 -- 172.21.15.156:0/4038849007 >> 172.21.15.156:0/4038849007 conn(0x7f9378072600 msgr2=0x7f937815a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:59.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.273+0000 7f937dbff700 1 -- 172.21.15.156:0/4038849007 shutdown_connections 2023-11-28T13:10:59.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.273+0000 7f937dbff700 1 -- 172.21.15.156:0/4038849007 wait complete. 2023-11-28T13:10:59.278 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T13:10:59.293 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2023-11-28T13:10:58.058053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":97436,"kb_used_data":6428,"kb_used_omap":0,"kb_used_meta":90944,"kb_avail":754877284,"statfs":{"total":773094113280,"available":772994338816,"internally_reserved":0,"allocated":6582272,"data_stored":3515662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":93126656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.624633"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.965381+0000","last_change":"2023-11-28T13:10:54.961652+0000","last_active":"2023-11-28T13:10:54.965381+0000","last_peered":"2023-11-28T13:10:54.965381+0000","last_clean":"2023-11-28T13:10:54.965381+0000","last_became_active":"2023-11-28T13:10:54.961404+0000","last_became_peered":"2023-11-28T13:10:54.961404+0000","last_unstale":"2023-11-28T13:10:54.965381+0000","last_undegraded":"2023-11-28T13:10:54.965381+0000","last_fullsized":"2023-11-28T13:10:54.965381+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:00:26.663675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.965480+0000","last_change":"2023-11-28T13:10:54.960157+0000","last_active":"2023-11-28T13:10:54.965480+0000","last_peered":"2023-11-28T13:10:54.965480+0000","last_clean":"2023-11-28T13:10:54.965480+0000","last_became_active":"2023-11-28T13:10:54.959690+0000","last_became_peered":"2023-11-28T13:10:54.959690+0000","last_unstale":"2023-11-28T13:10:54.965480+0000","last_undegraded":"2023-11-28T13:10:54.965480+0000","last_fullsized":"2023-11-28T13:10:54.965480+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:56.493889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.959022+0000","last_change":"2023-11-28T13:10:54.953490+0000","last_active":"2023-11-28T13:10:54.959022+0000","last_peered":"2023-11-28T13:10:54.959022+0000","last_clean":"2023-11-28T13:10:54.959022+0000","last_became_active":"2023-11-28T13:10:54.953224+0000","last_became_peered":"2023-11-28T13:10:54.953224+0000","last_unstale":"2023-11-28T13:10:54.959022+0000","last_undegraded":"2023-11-28T13:10:54.959022+0000","last_fullsized":"2023-11-28T13:10:54.959022+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:44:47.603128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.957814+0000","last_change":"2023-11-28T13:10:54.955625+0000","last_active":"2023-11-28T13:10:54.957814+0000","last_peered":"2023-11-28T13:10:54.957814+0000","last_clean":"2023-11-28T13:10:54.957814+0000","last_became_active":"2023-11-28T13:10:54.955376+0000","last_became_peered":"2023-11-28T13:10:54.955376+0000","last_unstale":"2023-11-28T13:10:54.957814+0000","last_undegraded":"2023-11-28T13:10:54.957814+0000","last_fullsized":"2023-11-28T13:10:54.957814+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:50:49.582092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.962470+0000","last_change":"2023-11-28T13:10:54.954060+0000","last_active":"2023-11-28T13:10:54.962470+0000","last_peered":"2023-11-28T13:10:54.962470+0000","last_clean":"2023-11-28T13:10:54.962470+0000","last_became_active":"2023-11-28T13:10:54.953479+0000","last_became_peered":"2023-11-28T13:10:54.953479+0000","last_unstale":"2023-11-28T13:10:54.962470+0000","last_undegraded":"2023-11-28T13:10:54.962470+0000","last_fullsized":"2023-11-28T13:10:54.962470+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:59.938061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.954623+0000","last_change":"2023-11-28T13:10:54.953353+0000","last_active":"2023-11-28T13:10:54.954623+0000","last_peered":"2023-11-28T13:10:54.954623+0000","last_clean":"2023-11-28T13:10:54.954623+0000","last_became_active":"2023-11-28T13:10:54.952999+0000","last_became_peered":"2023-11-28T13:10:54.952999+0000","last_unstale":"2023-11-28T13:10:54.954623+0000","last_undegraded":"2023-11-28T13:10:54.954623+0000","last_fullsized":"2023-11-28T13:10:54.954623+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:20.842433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.958991+0000","last_change":"2023-11-28T13:10:54.957499+0000","last_active":"2023-11-28T13:10:54.958991+0000","last_peered":"2023-11-28T13:10:54.958991+0000","last_clean":"2023-11-28T13:10:54.958991+0000","last_became_active":"2023-11-28T13:10:54.957165+0000","last_became_peered":"2023-11-28T13:10:54.957165+0000","last_unstale":"2023-11-28T13:10:54.958991+0000","last_undegraded":"2023-11-28T13:10:54.958991+0000","last_fullsized":"2023-11-28T13:10:54.958991+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:41:02.504630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966739+0000","last_change":"2023-11-28T13:10:54.962260+0000","last_active":"2023-11-28T13:10:54.966739+0000","last_peered":"2023-11-28T13:10:54.966739+0000","last_clean":"2023-11-28T13:10:54.966739+0000","last_became_active":"2023-11-28T13:10:54.958638+0000","last_became_peered":"2023-11-28T13:10:54.958638+0000","last_unstale":"2023-11-28T13:10:54.966739+0000","last_undegraded":"2023-11-28T13:10:54.966739+0000","last_fullsized":"2023-11-28T13:10:54.966739+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:13.622196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.965812+0000","last_change":"2023-11-28T13:10:54.958591+0000","last_active":"2023-11-28T13:10:54.965812+0000","last_peered":"2023-11-28T13:10:54.965812+0000","last_clean":"2023-11-28T13:10:54.965812+0000","last_became_active":"2023-11-28T13:10:54.958293+0000","last_became_peered":"2023-11-28T13:10:54.958293+0000","last_unstale":"2023-11-28T13:10:54.965812+0000","last_undegraded":"2023-11-28T13:10:54.965812+0000","last_fullsized":"2023-11-28T13:10:54.965812+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:29.939656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.960430+0000","last_change":"2023-11-28T13:10:54.956373+0000","last_active":"2023-11-28T13:10:54.960430+0000","last_peered":"2023-11-28T13:10:54.960430+0000","last_clean":"2023-11-28T13:10:54.960430+0000","last_became_active":"2023-11-28T13:10:54.956076+0000","last_became_peered":"2023-11-28T13:10:54.956076+0000","last_unstale":"2023-11-28T13:10:54.960430+0000","last_undegraded":"2023-11-28T13:10:54.960430+0000","last_fullsized":"2023-11-28T13:10:54.960430+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:50:49.582092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.971832+0000","last_change":"2023-11-28T13:10:54.968577+0000","last_active":"2023-11-28T13:10:54.971832+0000","last_peered":"2023-11-28T13:10:54.971832+0000","last_clean":"2023-11-28T13:10:54.971832+0000","last_became_active":"2023-11-28T13:10:54.968309+0000","last_became_peered":"2023-11-28T13:10:54.968309+0000","last_unstale":"2023-11-28T13:10:54.971832+0000","last_undegraded":"2023-11-28T13:10:54.971832+0000","last_fullsized":"2023-11-28T13:10:54.971832+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:51.738865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966757+0000","last_change":"2023-11-28T13:10:54.963008+0000","last_active":"2023-11-28T13:10:54.966757+0000","last_peered":"2023-11-28T13:10:54.966757+0000","last_clean":"2023-11-28T13:10:54.966757+0000","last_became_active":"2023-11-28T13:10:54.962684+0000","last_became_peered":"2023-11-28T13:10:54.962684+0000","last_unstale":"2023-11-28T13:10:54.966757+0000","last_undegraded":"2023-11-28T13:10:54.966757+0000","last_fullsized":"2023-11-28T13:10:54.966757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:16:24.754084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966599+0000","last_change":"2023-11-28T13:10:54.957772+0000","last_active":"2023-11-28T13:10:54.966599+0000","last_peered":"2023-11-28T13:10:54.966599+0000","last_clean":"2023-11-28T13:10:54.966599+0000","last_became_active":"2023-11-28T13:10:54.957143+0000","last_became_peered":"2023-11-28T13:10:54.957143+0000","last_unstale":"2023-11-28T13:10:54.966599+0000","last_undegraded":"2023-11-28T13:10:54.966599+0000","last_fullsized":"2023-11-28T13:10:54.966599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:26:38.445148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.962079+0000","last_change":"2023-11-28T13:10:54.957271+0000","last_active":"2023-11-28T13:10:54.962079+0000","last_peered":"2023-11-28T13:10:54.962079+0000","last_clean":"2023-11-28T13:10:54.962079+0000","last_became_active":"2023-11-28T13:10:54.956920+0000","last_became_peered":"2023-11-28T13:10:54.956920+0000","last_unstale":"2023-11-28T13:10:54.962079+0000","last_undegraded":"2023-11-28T13:10:54.962079+0000","last_fullsized":"2023-11-28T13:10:54.962079+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:40.521358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.963195+0000","last_change":"2023-11-28T13:10:54.958827+0000","last_active":"2023-11-28T13:10:54.963195+0000","last_peered":"2023-11-28T13:10:54.963195+0000","last_clean":"2023-11-28T13:10:54.963195+0000","last_became_active":"2023-11-28T13:10:54.958450+0000","last_became_peered":"2023-11-28T13:10:54.958450+0000","last_unstale":"2023-11-28T13:10:54.963195+0000","last_undegraded":"2023-11-28T13:10:54.963195+0000","last_fullsized":"2023-11-28T13:10:54.963195+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:53.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.956401+0000","last_change":"2023-11-28T13:10:54.951602+0000","last_active":"2023-11-28T13:10:54.956401+0000","last_peered":"2023-11-28T13:10:54.956401+0000","last_clean":"2023-11-28T13:10:54.956401+0000","last_became_active":"2023-11-28T13:10:54.949451+0000","last_became_peered":"2023-11-28T13:10:54.949451+0000","last_unstale":"2023-11-28T13:10:54.956401+0000","last_undegraded":"2023-11-28T13:10:54.956401+0000","last_fullsized":"2023-11-28T13:10:54.956401+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:32.485741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.965405+0000","last_change":"2023-11-28T13:10:54.961686+0000","last_active":"2023-11-28T13:10:54.965405+0000","last_peered":"2023-11-28T13:10:54.965405+0000","last_clean":"2023-11-28T13:10:54.965405+0000","last_became_active":"2023-11-28T13:10:54.961426+0000","last_became_peered":"2023-11-28T13:10:54.961426+0000","last_unstale":"2023-11-28T13:10:54.965405+0000","last_undegraded":"2023-11-28T13:10:54.965405+0000","last_fullsized":"2023-11-28T13:10:54.965405+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:27:19.503456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.959935+0000","last_change":"2023-11-28T13:10:54.956246+0000","last_active":"2023-11-28T13:10:54.959935+0000","last_peered":"2023-11-28T13:10:54.959935+0000","last_clean":"2023-11-28T13:10:54.959935+0000","last_became_active":"2023-11-28T13:10:54.955986+0000","last_became_peered":"2023-11-28T13:10:54.955986+0000","last_unstale":"2023-11-28T13:10:54.959935+0000","last_undegraded":"2023-11-28T13:10:54.959935+0000","last_fullsized":"2023-11-28T13:10:54.959935+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:40.521358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.944616+0000","last_change":"2023-11-28T13:10:54.942803+0000","last_active":"2023-11-28T13:10:54.944616+0000","last_peered":"2023-11-28T13:10:54.944616+0000","last_clean":"2023-11-28T13:10:54.944616+0000","last_became_active":"2023-11-28T13:10:54.942494+0000","last_became_peered":"2023-11-28T13:10:54.942494+0000","last_unstale":"2023-11-28T13:10:54.944616+0000","last_undegraded":"2023-11-28T13:10:54.944616+0000","last_fullsized":"2023-11-28T13:10:54.944616+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:26.486619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.963083+0000","last_change":"2023-11-28T13:10:54.959131+0000","last_active":"2023-11-28T13:10:54.963083+0000","last_peered":"2023-11-28T13:10:54.963083+0000","last_clean":"2023-11-28T13:10:54.963083+0000","last_became_active":"2023-11-28T13:10:54.958867+0000","last_became_peered":"2023-11-28T13:10:54.958867+0000","last_unstale":"2023-11-28T13:10:54.963083+0000","last_undegraded":"2023-11-28T13:10:54.963083+0000","last_fullsized":"2023-11-28T13:10:54.963083+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:55.169875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966619+0000","last_change":"2023-11-28T13:10:54.957967+0000","last_active":"2023-11-28T13:10:54.966619+0000","last_peered":"2023-11-28T13:10:54.966619+0000","last_clean":"2023-11-28T13:10:54.966619+0000","last_became_active":"2023-11-28T13:10:54.957506+0000","last_became_peered":"2023-11-28T13:10:54.957506+0000","last_unstale":"2023-11-28T13:10:54.966619+0000","last_undegraded":"2023-11-28T13:10:54.966619+0000","last_fullsized":"2023-11-28T13:10:54.966619+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:05.224374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.958892+0000","last_change":"2023-11-28T13:10:54.957256+0000","last_active":"2023-11-28T13:10:54.958892+0000","last_peered":"2023-11-28T13:10:54.958892+0000","last_clean":"2023-11-28T13:10:54.958892+0000","last_became_active":"2023-11-28T13:10:54.957008+0000","last_became_peered":"2023-11-28T13:10:54.957008+0000","last_unstale":"2023-11-28T13:10:54.958892+0000","last_undegraded":"2023-11-28T13:10:54.958892+0000","last_fullsized":"2023-11-28T13:10:54.958892+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:59:49.860376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.959090+0000","last_change":"2023-11-28T13:10:54.957866+0000","last_active":"2023-11-28T13:10:54.959090+0000","last_peered":"2023-11-28T13:10:54.959090+0000","last_clean":"2023-11-28T13:10:54.959090+0000","last_became_active":"2023-11-28T13:10:54.957610+0000","last_became_peered":"2023-11-28T13:10:54.957610+0000","last_unstale":"2023-11-28T13:10:54.959090+0000","last_undegraded":"2023-11-28T13:10:54.959090+0000","last_fullsized":"2023-11-28T13:10:54.959090+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:05:56.985098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.960314+0000","last_change":"2023-11-28T13:10:54.953957+0000","last_active":"2023-11-28T13:10:54.960314+0000","last_peered":"2023-11-28T13:10:54.960314+0000","last_clean":"2023-11-28T13:10:54.960314+0000","last_became_active":"2023-11-28T13:10:54.953668+0000","last_became_peered":"2023-11-28T13:10:54.953668+0000","last_unstale":"2023-11-28T13:10:54.960314+0000","last_undegraded":"2023-11-28T13:10:54.960314+0000","last_fullsized":"2023-11-28T13:10:54.960314+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:30.770582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.955601+0000","last_change":"2023-11-28T13:10:54.950615+0000","last_active":"2023-11-28T13:10:54.955601+0000","last_peered":"2023-11-28T13:10:54.955601+0000","last_clean":"2023-11-28T13:10:54.955601+0000","last_became_active":"2023-11-28T13:10:54.947092+0000","last_became_peered":"2023-11-28T13:10:54.947092+0000","last_unstale":"2023-11-28T13:10:54.955601+0000","last_undegraded":"2023-11-28T13:10:54.955601+0000","last_fullsized":"2023-11-28T13:10:54.955601+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:01:41.568218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.963154+0000","last_change":"2023-11-28T13:10:54.948805+0000","last_active":"2023-11-28T13:10:54.963154+0000","last_peered":"2023-11-28T13:10:54.963154+0000","last_clean":"2023-11-28T13:10:54.963154+0000","last_became_active":"2023-11-28T13:10:54.948529+0000","last_became_peered":"2023-11-28T13:10:54.948529+0000","last_unstale":"2023-11-28T13:10:54.963154+0000","last_undegraded":"2023-11-28T13:10:54.963154+0000","last_fullsized":"2023-11-28T13:10:54.963154+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:59:07.213157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.969948+0000","last_change":"2023-11-28T13:10:54.965970+0000","last_active":"2023-11-28T13:10:54.969948+0000","last_peered":"2023-11-28T13:10:54.969948+0000","last_clean":"2023-11-28T13:10:54.969948+0000","last_became_active":"2023-11-28T13:10:54.960090+0000","last_became_peered":"2023-11-28T13:10:54.960090+0000","last_unstale":"2023-11-28T13:10:54.969948+0000","last_undegraded":"2023-11-28T13:10:54.969948+0000","last_fullsized":"2023-11-28T13:10:54.969948+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:46:36.216924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966021+0000","last_change":"2023-11-28T13:10:54.962779+0000","last_active":"2023-11-28T13:10:54.966021+0000","last_peered":"2023-11-28T13:10:54.966021+0000","last_clean":"2023-11-28T13:10:54.966021+0000","last_became_active":"2023-11-28T13:10:54.962512+0000","last_became_peered":"2023-11-28T13:10:54.962512+0000","last_unstale":"2023-11-28T13:10:54.966021+0000","last_undegraded":"2023-11-28T13:10:54.966021+0000","last_fullsized":"2023-11-28T13:10:54.966021+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:28.144954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.960357+0000","last_change":"2023-11-28T13:10:54.954735+0000","last_active":"2023-11-28T13:10:54.960357+0000","last_peered":"2023-11-28T13:10:54.960357+0000","last_clean":"2023-11-28T13:10:54.960357+0000","last_became_active":"2023-11-28T13:10:54.954175+0000","last_became_peered":"2023-11-28T13:10:54.954175+0000","last_unstale":"2023-11-28T13:10:54.960357+0000","last_undegraded":"2023-11-28T13:10:54.960357+0000","last_fullsized":"2023-11-28T13:10:54.960357+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:51.738865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.967118+0000","last_change":"2023-11-28T13:10:54.958693+0000","last_active":"2023-11-28T13:10:54.967118+0000","last_peered":"2023-11-28T13:10:54.967118+0000","last_clean":"2023-11-28T13:10:54.967118+0000","last_became_active":"2023-11-28T13:10:54.958350+0000","last_became_peered":"2023-11-28T13:10:54.958350+0000","last_unstale":"2023-11-28T13:10:54.967118+0000","last_undegraded":"2023-11-28T13:10:54.967118+0000","last_fullsized":"2023-11-28T13:10:54.967118+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:55.169875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.968080+0000","last_change":"2023-11-28T13:10:54.958583+0000","last_active":"2023-11-28T13:10:54.968080+0000","last_peered":"2023-11-28T13:10:54.968080+0000","last_clean":"2023-11-28T13:10:54.968080+0000","last_became_active":"2023-11-28T13:10:54.953594+0000","last_became_peered":"2023-11-28T13:10:54.953594+0000","last_unstale":"2023-11-28T13:10:54.968080+0000","last_undegraded":"2023-11-28T13:10:54.968080+0000","last_fullsized":"2023-11-28T13:10:54.968080+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:53.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.949682+0000","last_change":"2023-11-28T13:10:54.945572+0000","last_active":"2023-11-28T13:10:54.949682+0000","last_peered":"2023-11-28T13:10:54.949682+0000","last_clean":"2023-11-28T13:10:54.949682+0000","last_became_active":"2023-11-28T13:10:54.945292+0000","last_became_peered":"2023-11-28T13:10:54.945292+0000","last_unstale":"2023-11-28T13:10:54.949682+0000","last_undegraded":"2023-11-28T13:10:54.949682+0000","last_fullsized":"2023-11-28T13:10:54.949682+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:59:49.860376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.968399+0000","last_change":"2023-11-28T13:10:54.964604+0000","last_active":"2023-11-28T13:10:54.968399+0000","last_peered":"2023-11-28T13:10:54.968399+0000","last_clean":"2023-11-28T13:10:54.968399+0000","last_became_active":"2023-11-28T13:10:54.964014+0000","last_became_peered":"2023-11-28T13:10:54.964014+0000","last_unstale":"2023-11-28T13:10:54.968399+0000","last_undegraded":"2023-11-28T13:10:54.968399+0000","last_fullsized":"2023-11-28T13:10:54.968399+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:52.131617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"58'4","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.968281+0000","last_change":"2023-11-28T13:10:54.468707+0000","last_active":"2023-11-28T13:10:54.968281+0000","last_peered":"2023-11-28T13:10:54.968281+0000","last_clean":"2023-11-28T13:10:54.968281+0000","last_became_active":"2023-11-28T13:10:54.468432+0000","last_became_peered":"2023-11-28T13:10:54.468432+0000","last_unstale":"2023-11-28T13:10:54.968281+0000","last_undegraded":"2023-11-28T13:10:54.968281+0000","last_fullsized":"2023-11-28T13:10:54.968281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:58.935101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.971814+0000","last_change":"2023-11-28T13:10:54.968294+0000","last_active":"2023-11-28T13:10:54.971814+0000","last_peered":"2023-11-28T13:10:54.971814+0000","last_clean":"2023-11-28T13:10:54.971814+0000","last_became_active":"2023-11-28T13:10:54.966924+0000","last_became_peered":"2023-11-28T13:10:54.966924+0000","last_unstale":"2023-11-28T13:10:54.971814+0000","last_undegraded":"2023-11-28T13:10:54.971814+0000","last_fullsized":"2023-11-28T13:10:54.971814+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:30.770582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.958710+0000","last_change":"2023-11-28T13:10:54.465578+0000","last_active":"2023-11-28T13:10:54.958710+0000","last_peered":"2023-11-28T13:10:54.958710+0000","last_clean":"2023-11-28T13:10:54.958710+0000","last_became_active":"2023-11-28T13:10:54.465318+0000","last_became_peered":"2023-11-28T13:10:54.465318+0000","last_unstale":"2023-11-28T13:10:54.958710+0000","last_undegraded":"2023-11-28T13:10:54.958710+0000","last_fullsized":"2023-11-28T13:10:54.958710+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:04.221414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.953725+0000","last_change":"2023-11-28T13:10:54.948522+0000","last_active":"2023-11-28T13:10:54.953725+0000","last_peered":"2023-11-28T13:10:54.953725+0000","last_clean":"2023-11-28T13:10:54.953725+0000","last_became_active":"2023-11-28T13:10:54.946138+0000","last_became_peered":"2023-11-28T13:10:54.946138+0000","last_unstale":"2023-11-28T13:10:54.953725+0000","last_undegraded":"2023-11-28T13:10:54.953725+0000","last_fullsized":"2023-11-28T13:10:54.953725+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:55.169875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.957505+0000","last_change":"2023-11-28T13:10:54.461525+0000","last_active":"2023-11-28T13:10:54.957505+0000","last_peered":"2023-11-28T13:10:54.957505+0000","last_clean":"2023-11-28T13:10:54.957505+0000","last_became_active":"2023-11-28T13:10:54.461118+0000","last_became_peered":"2023-11-28T13:10:54.461118+0000","last_unstale":"2023-11-28T13:10:54.957505+0000","last_undegraded":"2023-11-28T13:10:54.957505+0000","last_fullsized":"2023-11-28T13:10:54.957505+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:27:34.442096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.967805+0000","last_change":"2023-11-28T13:10:54.965654+0000","last_active":"2023-11-28T13:10:54.967805+0000","last_peered":"2023-11-28T13:10:54.967805+0000","last_clean":"2023-11-28T13:10:54.967805+0000","last_became_active":"2023-11-28T13:10:54.962899+0000","last_became_peered":"2023-11-28T13:10:54.962899+0000","last_unstale":"2023-11-28T13:10:54.967805+0000","last_undegraded":"2023-11-28T13:10:54.967805+0000","last_fullsized":"2023-11-28T13:10:54.967805+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:38.720594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.963160+0000","last_change":"2023-11-28T13:10:54.456716+0000","last_active":"2023-11-28T13:10:54.963160+0000","last_peered":"2023-11-28T13:10:54.963160+0000","last_clean":"2023-11-28T13:10:54.963160+0000","last_became_active":"2023-11-28T13:10:54.456446+0000","last_became_peered":"2023-11-28T13:10:54.456446+0000","last_unstale":"2023-11-28T13:10:54.963160+0000","last_undegraded":"2023-11-28T13:10:54.963160+0000","last_fullsized":"2023-11-28T13:10:54.963160+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:09:42.866301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.961976+0000","last_change":"2023-11-28T13:10:54.960029+0000","last_active":"2023-11-28T13:10:54.961976+0000","last_peered":"2023-11-28T13:10:54.961976+0000","last_clean":"2023-11-28T13:10:54.961976+0000","last_became_active":"2023-11-28T13:10:54.959749+0000","last_became_peered":"2023-11-28T13:10:54.959749+0000","last_unstale":"2023-11-28T13:10:54.961976+0000","last_undegraded":"2023-11-28T13:10:54.961976+0000","last_fullsized":"2023-11-28T13:10:54.961976+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:45.692752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.950722+0000","last_change":"2023-11-28T13:10:54.453537+0000","last_active":"2023-11-28T13:10:54.950722+0000","last_peered":"2023-11-28T13:10:54.950722+0000","last_clean":"2023-11-28T13:10:54.950722+0000","last_became_active":"2023-11-28T13:10:54.453242+0000","last_became_peered":"2023-11-28T13:10:54.453242+0000","last_unstale":"2023-11-28T13:10:54.950722+0000","last_undegraded":"2023-11-28T13:10:54.950722+0000","last_fullsized":"2023-11-28T13:10:54.950722+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:04.221414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.959336+0000","last_change":"2023-11-28T13:10:54.956498+0000","last_active":"2023-11-28T13:10:54.959336+0000","last_peered":"2023-11-28T13:10:54.959336+0000","last_clean":"2023-11-28T13:10:54.959336+0000","last_became_active":"2023-11-28T13:10:54.956142+0000","last_became_peered":"2023-11-28T13:10:54.956142+0000","last_unstale":"2023-11-28T13:10:54.959336+0000","last_undegraded":"2023-11-28T13:10:54.959336+0000","last_fullsized":"2023-11-28T13:10:54.959336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:05.580755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.947733+0000","last_change":"2023-11-28T13:10:54.456732+0000","last_active":"2023-11-28T13:10:54.947733+0000","last_peered":"2023-11-28T13:10:54.947733+0000","last_clean":"2023-11-28T13:10:54.947733+0000","last_became_active":"2023-11-28T13:10:54.456467+0000","last_became_peered":"2023-11-28T13:10:54.456467+0000","last_unstale":"2023-11-28T13:10:54.947733+0000","last_undegraded":"2023-11-28T13:10:54.947733+0000","last_fullsized":"2023-11-28T13:10:54.947733+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:37.717634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.944816+0000","last_change":"2023-11-28T13:10:54.942792+0000","last_active":"2023-11-28T13:10:54.944816+0000","last_peered":"2023-11-28T13:10:54.944816+0000","last_clean":"2023-11-28T13:10:54.944816+0000","last_became_active":"2023-11-28T13:10:54.942375+0000","last_became_peered":"2023-11-28T13:10:54.942375+0000","last_unstale":"2023-11-28T13:10:54.944816+0000","last_undegraded":"2023-11-28T13:10:54.944816+0000","last_fullsized":"2023-11-28T13:10:54.944816+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:29:47.742190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.940822+0000","last_change":"2023-11-28T13:10:54.464086+0000","last_active":"2023-11-28T13:10:54.940822+0000","last_peered":"2023-11-28T13:10:54.940822+0000","last_clean":"2023-11-28T13:10:54.940822+0000","last_became_active":"2023-11-28T13:10:54.463634+0000","last_became_peered":"2023-11-28T13:10:54.463634+0000","last_unstale":"2023-11-28T13:10:54.940822+0000","last_undegraded":"2023-11-28T13:10:54.940822+0000","last_fullsized":"2023-11-28T13:10:54.940822+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:57:33.861682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.958496+0000","last_change":"2023-11-28T13:10:54.954775+0000","last_active":"2023-11-28T13:10:54.958496+0000","last_peered":"2023-11-28T13:10:54.958496+0000","last_clean":"2023-11-28T13:10:54.958496+0000","last_became_active":"2023-11-28T13:10:54.954312+0000","last_became_peered":"2023-11-28T13:10:54.954312+0000","last_unstale":"2023-11-28T13:10:54.958496+0000","last_undegraded":"2023-11-28T13:10:54.958496+0000","last_fullsized":"2023-11-28T13:10:54.958496+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:13.622196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.935241+0000","last_change":"2023-11-28T13:10:54.463702+0000","last_active":"2023-11-28T13:10:54.935241+0000","last_peered":"2023-11-28T13:10:54.935241+0000","last_clean":"2023-11-28T13:10:54.935241+0000","last_became_active":"2023-11-28T13:10:54.463260+0000","last_became_peered":"2023-11-28T13:10:54.463260+0000","last_unstale":"2023-11-28T13:10:54.935241+0000","last_undegraded":"2023-11-28T13:10:54.935241+0000","last_fullsized":"2023-11-28T13:10:54.935241+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:02.071198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966002+0000","last_change":"2023-11-28T13:10:54.962816+0000","last_active":"2023-11-28T13:10:54.966002+0000","last_peered":"2023-11-28T13:10:54.966002+0000","last_clean":"2023-11-28T13:10:54.966002+0000","last_became_active":"2023-11-28T13:10:54.962525+0000","last_became_peered":"2023-11-28T13:10:54.962525+0000","last_unstale":"2023-11-28T13:10:54.966002+0000","last_undegraded":"2023-11-28T13:10:54.966002+0000","last_fullsized":"2023-11-28T13:10:54.966002+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:56.343056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.944187+0000","last_change":"2023-11-28T13:10:54.469918+0000","last_active":"2023-11-28T13:10:54.944187+0000","last_peered":"2023-11-28T13:10:54.944187+0000","last_clean":"2023-11-28T13:10:54.944187+0000","last_became_active":"2023-11-28T13:10:54.469536+0000","last_became_peered":"2023-11-28T13:10:54.469536+0000","last_unstale":"2023-11-28T13:10:54.944187+0000","last_undegraded":"2023-11-28T13:10:54.944187+0000","last_fullsized":"2023-11-28T13:10:54.944187+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:25.483658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.950103+0000","last_change":"2023-11-28T13:10:54.949029+0000","last_active":"2023-11-28T13:10:54.950103+0000","last_peered":"2023-11-28T13:10:54.950103+0000","last_clean":"2023-11-28T13:10:54.950103+0000","last_became_active":"2023-11-28T13:10:54.948774+0000","last_became_peered":"2023-11-28T13:10:54.948774+0000","last_unstale":"2023-11-28T13:10:54.950103+0000","last_undegraded":"2023-11-28T13:10:54.950103+0000","last_fullsized":"2023-11-28T13:10:54.950103+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:16:24.754084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.961370+0000","last_change":"2023-11-28T13:10:54.453709+0000","last_active":"2023-11-28T13:10:54.961370+0000","last_peered":"2023-11-28T13:10:54.961370+0000","last_clean":"2023-11-28T13:10:54.961370+0000","last_became_active":"2023-11-28T13:10:54.453066+0000","last_became_peered":"2023-11-28T13:10:54.453066+0000","last_unstale":"2023-11-28T13:10:54.961370+0000","last_undegraded":"2023-11-28T13:10:54.961370+0000","last_fullsized":"2023-11-28T13:10:54.961370+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:10.970647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.955243+0000","last_change":"2023-11-28T13:10:54.948499+0000","last_active":"2023-11-28T13:10:54.955243+0000","last_peered":"2023-11-28T13:10:54.955243+0000","last_clean":"2023-11-28T13:10:54.955243+0000","last_became_active":"2023-11-28T13:10:54.948174+0000","last_became_peered":"2023-11-28T13:10:54.948174+0000","last_unstale":"2023-11-28T13:10:54.955243+0000","last_undegraded":"2023-11-28T13:10:54.955243+0000","last_fullsized":"2023-11-28T13:10:54.955243+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:05.580755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"58'3","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966338+0000","last_change":"2023-11-28T13:10:54.468809+0000","last_active":"2023-11-28T13:10:54.966338+0000","last_peered":"2023-11-28T13:10:54.966338+0000","last_clean":"2023-11-28T13:10:54.966338+0000","last_became_active":"2023-11-28T13:10:54.468426+0000","last_became_peered":"2023-11-28T13:10:54.468426+0000","last_unstale":"2023-11-28T13:10:54.966338+0000","last_undegraded":"2023-11-28T13:10:54.966338+0000","last_fullsized":"2023-11-28T13:10:54.966338+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:10.970647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.968749+0000","last_change":"2023-11-28T13:10:54.961768+0000","last_active":"2023-11-28T13:10:54.968749+0000","last_peered":"2023-11-28T13:10:54.968749+0000","last_clean":"2023-11-28T13:10:54.968749+0000","last_became_active":"2023-11-28T13:10:54.961375+0000","last_became_peered":"2023-11-28T13:10:54.961375+0000","last_unstale":"2023-11-28T13:10:54.968749+0000","last_undegraded":"2023-11-28T13:10:54.968749+0000","last_fullsized":"2023-11-28T13:10:54.968749+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:57:34.864642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"58'6","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.974518+0000","last_change":"2023-11-28T13:10:54.458857+0000","last_active":"2023-11-28T13:10:54.974518+0000","last_peered":"2023-11-28T13:10:54.974518+0000","last_clean":"2023-11-28T13:10:54.974518+0000","last_became_active":"2023-11-28T13:10:54.458581+0000","last_became_peered":"2023-11-28T13:10:54.458581+0000","last_unstale":"2023-11-28T13:10:54.974518+0000","last_undegraded":"2023-11-28T13:10:54.974518+0000","last_fullsized":"2023-11-28T13:10:54.974518+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:29:46.739230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.969567+0000","last_change":"2023-11-28T13:10:54.963935+0000","last_active":"2023-11-28T13:10:54.969567+0000","last_peered":"2023-11-28T13:10:54.969567+0000","last_clean":"2023-11-28T13:10:54.969567+0000","last_became_active":"2023-11-28T13:10:54.963638+0000","last_became_peered":"2023-11-28T13:10:54.963638+0000","last_unstale":"2023-11-28T13:10:54.969567+0000","last_undegraded":"2023-11-28T13:10:54.969567+0000","last_fullsized":"2023-11-28T13:10:54.969567+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:29.939656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.963539+0000","last_change":"2023-11-28T13:10:54.470122+0000","last_active":"2023-11-28T13:10:54.963539+0000","last_peered":"2023-11-28T13:10:54.963539+0000","last_clean":"2023-11-28T13:10:54.963539+0000","last_became_active":"2023-11-28T13:10:54.469267+0000","last_became_peered":"2023-11-28T13:10:54.469267+0000","last_unstale":"2023-11-28T13:10:54.963539+0000","last_undegraded":"2023-11-28T13:10:54.963539+0000","last_fullsized":"2023-11-28T13:10:54.963539+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:51.128657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.956058+0000","last_change":"2023-11-28T13:10:54.951065+0000","last_active":"2023-11-28T13:10:54.956058+0000","last_peered":"2023-11-28T13:10:54.956058+0000","last_clean":"2023-11-28T13:10:54.956058+0000","last_became_active":"2023-11-28T13:10:54.950729+0000","last_became_peered":"2023-11-28T13:10:54.950729+0000","last_unstale":"2023-11-28T13:10:54.956058+0000","last_undegraded":"2023-11-28T13:10:54.956058+0000","last_fullsized":"2023-11-28T13:10:54.956058+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:09:43.869262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.943565+0000","last_change":"2023-11-28T13:10:54.459460+0000","last_active":"2023-11-28T13:10:54.943565+0000","last_peered":"2023-11-28T13:10:54.943565+0000","last_clean":"2023-11-28T13:10:54.943565+0000","last_became_active":"2023-11-28T13:10:54.458402+0000","last_became_peered":"2023-11-28T13:10:54.458402+0000","last_unstale":"2023-11-28T13:10:54.943565+0000","last_undegraded":"2023-11-28T13:10:54.943565+0000","last_fullsized":"2023-11-28T13:10:54.943565+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:55.340096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.944789+0000","last_change":"2023-11-28T13:10:54.941321+0000","last_active":"2023-11-28T13:10:54.944789+0000","last_peered":"2023-11-28T13:10:54.944789+0000","last_clean":"2023-11-28T13:10:54.944789+0000","last_became_active":"2023-11-28T13:10:54.940728+0000","last_became_peered":"2023-11-28T13:10:54.940728+0000","last_unstale":"2023-11-28T13:10:54.944789+0000","last_undegraded":"2023-11-28T13:10:54.944789+0000","last_fullsized":"2023-11-28T13:10:54.944789+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:30.670841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.943204+0000","last_change":"2023-11-28T13:10:54.468803+0000","last_active":"2023-11-28T13:10:54.943204+0000","last_peered":"2023-11-28T13:10:54.943204+0000","last_clean":"2023-11-28T13:10:54.943204+0000","last_became_active":"2023-11-28T13:10:54.468475+0000","last_became_peered":"2023-11-28T13:10:54.468475+0000","last_unstale":"2023-11-28T13:10:54.943204+0000","last_undegraded":"2023-11-28T13:10:54.943204+0000","last_fullsized":"2023-11-28T13:10:54.943204+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:33:10.154448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.965643+0000","last_change":"2023-11-28T13:10:54.960919+0000","last_active":"2023-11-28T13:10:54.965643+0000","last_peered":"2023-11-28T13:10:54.965643+0000","last_clean":"2023-11-28T13:10:54.965643+0000","last_became_active":"2023-11-28T13:10:54.959814+0000","last_became_peered":"2023-11-28T13:10:54.959814+0000","last_unstale":"2023-11-28T13:10:54.965643+0000","last_undegraded":"2023-11-28T13:10:54.965643+0000","last_fullsized":"2023-11-28T13:10:54.965643+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:54.513040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"58'8","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.974404+0000","last_change":"2023-11-28T13:10:54.459509+0000","last_active":"2023-11-28T13:10:54.974404+0000","last_peered":"2023-11-28T13:10:54.974404+0000","last_clean":"2023-11-28T13:10:54.974404+0000","last_became_active":"2023-11-28T13:10:54.459232+0000","last_became_peered":"2023-11-28T13:10:54.459232+0000","last_unstale":"2023-11-28T13:10:54.974404+0000","last_undegraded":"2023-11-28T13:10:54.974404+0000","last_fullsized":"2023-11-28T13:10:54.974404+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:25.483658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.963207+0000","last_change":"2023-11-28T13:10:54.962276+0000","last_active":"2023-11-28T13:10:54.963207+0000","last_peered":"2023-11-28T13:10:54.963207+0000","last_clean":"2023-11-28T13:10:54.963207+0000","last_became_active":"2023-11-28T13:10:54.961826+0000","last_became_peered":"2023-11-28T13:10:54.961826+0000","last_unstale":"2023-11-28T13:10:54.963207+0000","last_undegraded":"2023-11-28T13:10:54.963207+0000","last_fullsized":"2023-11-28T13:10:54.963207+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:26:06.317970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.967015+0000","last_change":"2023-11-28T13:10:54.468898+0000","last_active":"2023-11-28T13:10:54.967015+0000","last_peered":"2023-11-28T13:10:54.967015+0000","last_clean":"2023-11-28T13:10:54.967015+0000","last_became_active":"2023-11-28T13:10:54.468498+0000","last_became_peered":"2023-11-28T13:10:54.468498+0000","last_unstale":"2023-11-28T13:10:54.967015+0000","last_undegraded":"2023-11-28T13:10:54.967015+0000","last_fullsized":"2023-11-28T13:10:54.967015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:53.510080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.954470+0000","last_change":"2023-11-28T13:10:54.949875+0000","last_active":"2023-11-28T13:10:54.954470+0000","last_peered":"2023-11-28T13:10:54.954470+0000","last_clean":"2023-11-28T13:10:54.954470+0000","last_became_active":"2023-11-28T13:10:54.949595+0000","last_became_peered":"2023-11-28T13:10:54.949595+0000","last_unstale":"2023-11-28T13:10:54.954470+0000","last_undegraded":"2023-11-28T13:10:54.954470+0000","last_fullsized":"2023-11-28T13:10:54.954470+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:03.074158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.944204+0000","last_change":"2023-11-28T13:10:54.458249+0000","last_active":"2023-11-28T13:10:54.944204+0000","last_peered":"2023-11-28T13:10:54.944204+0000","last_clean":"2023-11-28T13:10:54.944204+0000","last_became_active":"2023-11-28T13:10:54.457974+0000","last_became_peered":"2023-11-28T13:10:54.457974+0000","last_unstale":"2023-11-28T13:10:54.944204+0000","last_undegraded":"2023-11-28T13:10:54.944204+0000","last_fullsized":"2023-11-28T13:10:54.944204+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:27.141994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966547+0000","last_change":"2023-11-28T13:10:54.957391+0000","last_active":"2023-11-28T13:10:54.966547+0000","last_peered":"2023-11-28T13:10:54.966547+0000","last_clean":"2023-11-28T13:10:54.966547+0000","last_became_active":"2023-11-28T13:10:54.956682+0000","last_became_peered":"2023-11-28T13:10:54.956682+0000","last_unstale":"2023-11-28T13:10:54.966547+0000","last_undegraded":"2023-11-28T13:10:54.966547+0000","last_fullsized":"2023-11-28T13:10:54.966547+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:27:35.445056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.941399+0000","last_change":"2023-11-28T13:10:54.454857+0000","last_active":"2023-11-28T13:10:54.941399+0000","last_peered":"2023-11-28T13:10:54.941399+0000","last_clean":"2023-11-28T13:10:54.941399+0000","last_became_active":"2023-11-28T13:10:54.454594+0000","last_became_peered":"2023-11-28T13:10:54.454594+0000","last_unstale":"2023-11-28T13:10:54.941399+0000","last_undegraded":"2023-11-28T13:10:54.941399+0000","last_fullsized":"2023-11-28T13:10:54.941399+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:37.717634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.954847+0000","last_change":"2023-11-28T13:10:54.947275+0000","last_active":"2023-11-28T13:10:54.954847+0000","last_peered":"2023-11-28T13:10:54.954847+0000","last_clean":"2023-11-28T13:10:54.954847+0000","last_became_active":"2023-11-28T13:10:54.947011+0000","last_became_peered":"2023-11-28T13:10:54.947011+0000","last_unstale":"2023-11-28T13:10:54.954847+0000","last_undegraded":"2023-11-28T13:10:54.954847+0000","last_fullsized":"2023-11-28T13:10:54.954847+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:33:11.157408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.955529+0000","last_change":"2023-11-28T13:10:54.457853+0000","last_active":"2023-11-28T13:10:54.955529+0000","last_peered":"2023-11-28T13:10:54.955529+0000","last_clean":"2023-11-28T13:10:54.955529+0000","last_became_active":"2023-11-28T13:10:54.457562+0000","last_became_peered":"2023-11-28T13:10:54.457562+0000","last_unstale":"2023-11-28T13:10:54.955529+0000","last_undegraded":"2023-11-28T13:10:54.955529+0000","last_fullsized":"2023-11-28T13:10:54.955529+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:24:31.630491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.964185+0000","last_change":"2023-11-28T13:10:54.959743+0000","last_active":"2023-11-28T13:10:54.964185+0000","last_peered":"2023-11-28T13:10:54.964185+0000","last_clean":"2023-11-28T13:10:54.964185+0000","last_became_active":"2023-11-28T13:10:54.959458+0000","last_became_peered":"2023-11-28T13:10:54.959458+0000","last_unstale":"2023-11-28T13:10:54.964185+0000","last_undegraded":"2023-11-28T13:10:54.964185+0000","last_fullsized":"2023-11-28T13:10:54.964185+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:16.780414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.942455+0000","last_change":"2023-11-28T13:10:54.469964+0000","last_active":"2023-11-28T13:10:54.942455+0000","last_peered":"2023-11-28T13:10:54.942455+0000","last_clean":"2023-11-28T13:10:54.942455+0000","last_became_active":"2023-11-28T13:10:54.469510+0000","last_became_peered":"2023-11-28T13:10:54.469510+0000","last_unstale":"2023-11-28T13:10:54.942455+0000","last_undegraded":"2023-11-28T13:10:54.942455+0000","last_fullsized":"2023-11-28T13:10:54.942455+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:00:25.660715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.969983+0000","last_change":"2023-11-28T13:10:54.966114+0000","last_active":"2023-11-28T13:10:54.969983+0000","last_peered":"2023-11-28T13:10:54.969983+0000","last_clean":"2023-11-28T13:10:54.969983+0000","last_became_active":"2023-11-28T13:10:54.965818+0000","last_became_peered":"2023-11-28T13:10:54.965818+0000","last_unstale":"2023-11-28T13:10:54.969983+0000","last_undegraded":"2023-11-28T13:10:54.969983+0000","last_fullsized":"2023-11-28T13:10:54.969983+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:38.720594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.941378+0000","last_change":"2023-11-28T13:10:54.450412+0000","last_active":"2023-11-28T13:10:54.941378+0000","last_peered":"2023-11-28T13:10:54.941378+0000","last_clean":"2023-11-28T13:10:54.941378+0000","last_became_active":"2023-11-28T13:10:54.449928+0000","last_became_peered":"2023-11-28T13:10:54.449928+0000","last_unstale":"2023-11-28T13:10:54.941378+0000","last_undegraded":"2023-11-28T13:10:54.941378+0000","last_fullsized":"2023-11-28T13:10:54.941378+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:39.518398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.959390+0000","last_change":"2023-11-28T13:10:54.957416+0000","last_active":"2023-11-28T13:10:54.959390+0000","last_peered":"2023-11-28T13:10:54.959390+0000","last_clean":"2023-11-28T13:10:54.959390+0000","last_became_active":"2023-11-28T13:10:54.957130+0000","last_became_peered":"2023-11-28T13:10:54.957130+0000","last_unstale":"2023-11-28T13:10:54.959390+0000","last_undegraded":"2023-11-28T13:10:54.959390+0000","last_fullsized":"2023-11-28T13:10:54.959390+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:24:18.289975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.940921+0000","last_change":"2023-11-28T13:10:54.461128+0000","last_active":"2023-11-28T13:10:54.940921+0000","last_peered":"2023-11-28T13:10:54.940921+0000","last_clean":"2023-11-28T13:10:54.940921+0000","last_became_active":"2023-11-28T13:10:54.460844+0000","last_became_peered":"2023-11-28T13:10:54.460844+0000","last_unstale":"2023-11-28T13:10:54.940921+0000","last_undegraded":"2023-11-28T13:10:54.940921+0000","last_fullsized":"2023-11-28T13:10:54.940921+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:50.735905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.954584+0000","last_change":"2023-11-28T13:10:54.949080+0000","last_active":"2023-11-28T13:10:54.954584+0000","last_peered":"2023-11-28T13:10:54.954584+0000","last_clean":"2023-11-28T13:10:54.954584+0000","last_became_active":"2023-11-28T13:10:54.948824+0000","last_became_peered":"2023-11-28T13:10:54.948824+0000","last_unstale":"2023-11-28T13:10:54.954584+0000","last_undegraded":"2023-11-28T13:10:54.954584+0000","last_fullsized":"2023-11-28T13:10:54.954584+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:55:52.451792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.949499+0000","last_change":"2023-11-28T13:10:54.462528+0000","last_active":"2023-11-28T13:10:54.949499+0000","last_peered":"2023-11-28T13:10:54.949499+0000","last_clean":"2023-11-28T13:10:54.949499+0000","last_became_active":"2023-11-28T13:10:54.462211+0000","last_became_peered":"2023-11-28T13:10:54.462211+0000","last_unstale":"2023-11-28T13:10:54.949499+0000","last_undegraded":"2023-11-28T13:10:54.949499+0000","last_fullsized":"2023-11-28T13:10:54.949499+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:19.933480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.968765+0000","last_change":"2023-11-28T13:10:54.961500+0000","last_active":"2023-11-28T13:10:54.968765+0000","last_peered":"2023-11-28T13:10:54.968765+0000","last_clean":"2023-11-28T13:10:54.968765+0000","last_became_active":"2023-11-28T13:10:54.961033+0000","last_became_peered":"2023-11-28T13:10:54.961033+0000","last_unstale":"2023-11-28T13:10:54.968765+0000","last_undegraded":"2023-11-28T13:10:54.968765+0000","last_fullsized":"2023-11-28T13:10:54.968765+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:08.361205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966100+0000","last_change":"2023-11-28T13:10:54.469511+0000","last_active":"2023-11-28T13:10:54.966100+0000","last_peered":"2023-11-28T13:10:54.966100+0000","last_clean":"2023-11-28T13:10:54.966100+0000","last_became_active":"2023-11-28T13:10:54.469187+0000","last_became_peered":"2023-11-28T13:10:54.469187+0000","last_unstale":"2023-11-28T13:10:54.966100+0000","last_undegraded":"2023-11-28T13:10:54.966100+0000","last_fullsized":"2023-11-28T13:10:54.966100+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:15.777454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":49,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.449088+0000","last_change":"2023-11-28T13:07:56.890378+0000","last_active":"2023-11-28T13:10:54.449088+0000","last_peered":"2023-11-28T13:10:54.449088+0000","last_clean":"2023-11-28T13:10:54.449088+0000","last_became_active":"2023-11-28T13:07:56.890058+0000","last_became_peered":"2023-11-28T13:07:56.890058+0000","last_unstale":"2023-11-28T13:10:54.449088+0000","last_undegraded":"2023-11-28T13:10:54.449088+0000","last_fullsized":"2023-11-28T13:10:54.449088+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:05.344602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.953664+0000","last_change":"2023-11-28T13:10:54.945727+0000","last_active":"2023-11-28T13:10:54.953664+0000","last_peered":"2023-11-28T13:10:54.953664+0000","last_clean":"2023-11-28T13:10:54.953664+0000","last_became_active":"2023-11-28T13:10:54.945234+0000","last_became_peered":"2023-11-28T13:10:54.945234+0000","last_unstale":"2023-11-28T13:10:54.953664+0000","last_undegraded":"2023-11-28T13:10:54.953664+0000","last_fullsized":"2023-11-28T13:10:54.953664+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:00:26.663675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.941403+0000","last_change":"2023-11-28T13:10:54.454063+0000","last_active":"2023-11-28T13:10:54.941403+0000","last_peered":"2023-11-28T13:10:54.941403+0000","last_clean":"2023-11-28T13:10:54.941403+0000","last_became_active":"2023-11-28T13:10:54.452898+0000","last_became_peered":"2023-11-28T13:10:54.452898+0000","last_unstale":"2023-11-28T13:10:54.941403+0000","last_undegraded":"2023-11-28T13:10:54.941403+0000","last_fullsized":"2023-11-28T13:10:54.941403+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:28.936696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":48,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.440220+0000","last_change":"2023-11-28T13:07:56.844509+0000","last_active":"2023-11-28T13:10:54.440220+0000","last_peered":"2023-11-28T13:10:54.440220+0000","last_clean":"2023-11-28T13:10:54.440220+0000","last_became_active":"2023-11-28T13:07:56.844081+0000","last_became_peered":"2023-11-28T13:07:56.844081+0000","last_unstale":"2023-11-28T13:10:54.440220+0000","last_undegraded":"2023-11-28T13:10:54.440220+0000","last_fullsized":"2023-11-28T13:10:54.440220+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.948086+0000","last_change":"2023-11-28T13:10:54.942502+0000","last_active":"2023-11-28T13:10:54.948086+0000","last_peered":"2023-11-28T13:10:54.948086+0000","last_clean":"2023-11-28T13:10:54.948086+0000","last_became_active":"2023-11-28T13:10:54.942244+0000","last_became_peered":"2023-11-28T13:10:54.942244+0000","last_unstale":"2023-11-28T13:10:54.948086+0000","last_undegraded":"2023-11-28T13:10:54.948086+0000","last_fullsized":"2023-11-28T13:10:54.948086+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:05:56.985098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.963615+0000","last_change":"2023-11-28T13:10:54.463674+0000","last_active":"2023-11-28T13:10:54.963615+0000","last_peered":"2023-11-28T13:10:54.963615+0000","last_clean":"2023-11-28T13:10:54.963615+0000","last_became_active":"2023-11-28T13:10:54.463196+0000","last_became_peered":"2023-11-28T13:10:54.463196+0000","last_unstale":"2023-11-28T13:10:54.963615+0000","last_undegraded":"2023-11-28T13:10:54.963615+0000","last_fullsized":"2023-11-28T13:10:54.963615+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:28.936696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":46,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.446832+0000","last_change":"2023-11-28T13:07:56.844699+0000","last_active":"2023-11-28T13:10:54.446832+0000","last_peered":"2023-11-28T13:10:54.446832+0000","last_clean":"2023-11-28T13:10:54.446832+0000","last_became_active":"2023-11-28T13:07:56.844391+0000","last_became_peered":"2023-11-28T13:07:56.844391+0000","last_unstale":"2023-11-28T13:10:54.446832+0000","last_undegraded":"2023-11-28T13:10:54.446832+0000","last_fullsized":"2023-11-28T13:10:54.446832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":197,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.461726+0000","last_change":"2023-11-28T13:07:57.844020+0000","last_active":"2023-11-28T13:10:54.461726+0000","last_peered":"2023-11-28T13:10:54.461726+0000","last_clean":"2023-11-28T13:10:54.461726+0000","last_became_active":"2023-11-28T13:07:57.842981+0000","last_became_peered":"2023-11-28T13:07:57.842981+0000","last_unstale":"2023-11-28T13:10:54.461726+0000","last_undegraded":"2023-11-28T13:10:54.461726+0000","last_fullsized":"2023-11-28T13:10:54.461726+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_clean_scrub_stamp":"2023-11-28T13:07:56.825711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:44.973218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.950294+0000","last_change":"2023-11-28T13:10:54.946601+0000","last_active":"2023-11-28T13:10:54.950294+0000","last_peered":"2023-11-28T13:10:54.950294+0000","last_clean":"2023-11-28T13:10:54.950294+0000","last_became_active":"2023-11-28T13:10:54.944813+0000","last_became_peered":"2023-11-28T13:10:54.944813+0000","last_unstale":"2023-11-28T13:10:54.950294+0000","last_undegraded":"2023-11-28T13:10:54.950294+0000","last_fullsized":"2023-11-28T13:10:54.950294+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:16.097414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.943668+0000","last_change":"2023-11-28T13:10:54.454273+0000","last_active":"2023-11-28T13:10:54.943668+0000","last_peered":"2023-11-28T13:10:54.943668+0000","last_clean":"2023-11-28T13:10:54.943668+0000","last_became_active":"2023-11-28T13:10:54.453694+0000","last_became_peered":"2023-11-28T13:10:54.453694+0000","last_unstale":"2023-11-28T13:10:54.943668+0000","last_undegraded":"2023-11-28T13:10:54.943668+0000","last_fullsized":"2023-11-28T13:10:54.943668+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:27:18.500496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":48,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.461847+0000","last_change":"2023-11-28T13:07:57.306318+0000","last_active":"2023-11-28T13:10:54.461847+0000","last_peered":"2023-11-28T13:10:54.461847+0000","last_clean":"2023-11-28T13:10:54.461847+0000","last_became_active":"2023-11-28T13:07:57.305937+0000","last_became_peered":"2023-11-28T13:07:57.305937+0000","last_unstale":"2023-11-28T13:10:54.461847+0000","last_undegraded":"2023-11-28T13:10:54.461847+0000","last_fullsized":"2023-11-28T13:10:54.461847+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.963349+0000","last_change":"2023-11-28T13:10:54.959871+0000","last_active":"2023-11-28T13:10:54.963349+0000","last_peered":"2023-11-28T13:10:54.963349+0000","last_clean":"2023-11-28T13:10:54.963349+0000","last_became_active":"2023-11-28T13:10:54.959569+0000","last_became_peered":"2023-11-28T13:10:54.959569+0000","last_unstale":"2023-11-28T13:10:54.963349+0000","last_undegraded":"2023-11-28T13:10:54.963349+0000","last_fullsized":"2023-11-28T13:10:54.963349+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:42:26.200782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.939896+0000","last_change":"2023-11-28T13:10:54.454973+0000","last_active":"2023-11-28T13:10:54.939896+0000","last_peered":"2023-11-28T13:10:54.939896+0000","last_clean":"2023-11-28T13:10:54.939896+0000","last_became_active":"2023-11-28T13:10:54.454692+0000","last_became_peered":"2023-11-28T13:10:54.454692+0000","last_unstale":"2023-11-28T13:10:54.939896+0000","last_undegraded":"2023-11-28T13:10:54.939896+0000","last_fullsized":"2023-11-28T13:10:54.939896+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:27.171232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":48,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.462057+0000","last_change":"2023-11-28T13:07:57.306277+0000","last_active":"2023-11-28T13:10:54.462057+0000","last_peered":"2023-11-28T13:10:54.462057+0000","last_clean":"2023-11-28T13:10:54.462057+0000","last_became_active":"2023-11-28T13:07:57.305983+0000","last_became_peered":"2023-11-28T13:07:57.305983+0000","last_unstale":"2023-11-28T13:10:54.462057+0000","last_undegraded":"2023-11-28T13:10:54.462057+0000","last_fullsized":"2023-11-28T13:10:54.462057+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.953442+0000","last_change":"2023-11-28T13:10:54.945967+0000","last_active":"2023-11-28T13:10:54.953442+0000","last_peered":"2023-11-28T13:10:54.953442+0000","last_clean":"2023-11-28T13:10:54.953442+0000","last_became_active":"2023-11-28T13:10:54.945538+0000","last_became_peered":"2023-11-28T13:10:54.945538+0000","last_unstale":"2023-11-28T13:10:54.953442+0000","last_undegraded":"2023-11-28T13:10:54.953442+0000","last_fullsized":"2023-11-28T13:10:54.953442+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:57:52.631897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.940219+0000","last_change":"2023-11-28T13:10:54.460189+0000","last_active":"2023-11-28T13:10:54.940219+0000","last_peered":"2023-11-28T13:10:54.940219+0000","last_clean":"2023-11-28T13:10:54.940219+0000","last_became_active":"2023-11-28T13:10:54.459402+0000","last_became_peered":"2023-11-28T13:10:54.459402+0000","last_unstale":"2023-11-28T13:10:54.940219+0000","last_undegraded":"2023-11-28T13:10:54.940219+0000","last_fullsized":"2023-11-28T13:10:54.940219+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:27.171232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":53,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.446920+0000","last_change":"2023-11-28T13:07:57.306177+0000","last_active":"2023-11-28T13:10:54.446920+0000","last_peered":"2023-11-28T13:10:54.446920+0000","last_clean":"2023-11-28T13:10:54.446920+0000","last_became_active":"2023-11-28T13:07:57.305853+0000","last_became_peered":"2023-11-28T13:07:57.305853+0000","last_unstale":"2023-11-28T13:10:54.446920+0000","last_undegraded":"2023-11-28T13:10:54.446920+0000","last_fullsized":"2023-11-28T13:10:54.446920+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.967085+0000","last_change":"2023-11-28T13:10:54.961438+0000","last_active":"2023-11-28T13:10:54.967085+0000","last_peered":"2023-11-28T13:10:54.967085+0000","last_clean":"2023-11-28T13:10:54.967085+0000","last_became_active":"2023-11-28T13:10:54.961162+0000","last_became_peered":"2023-11-28T13:10:54.961162+0000","last_unstale":"2023-11-28T13:10:54.967085+0000","last_undegraded":"2023-11-28T13:10:54.967085+0000","last_fullsized":"2023-11-28T13:10:54.967085+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:11.973607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.958467+0000","last_change":"2023-11-28T13:10:54.468802+0000","last_active":"2023-11-28T13:10:54.958467+0000","last_peered":"2023-11-28T13:10:54.958467+0000","last_clean":"2023-11-28T13:10:54.958467+0000","last_became_active":"2023-11-28T13:10:54.468478+0000","last_became_peered":"2023-11-28T13:10:54.468478+0000","last_unstale":"2023-11-28T13:10:54.958467+0000","last_undegraded":"2023-11-28T13:10:54.958467+0000","last_fullsized":"2023-11-28T13:10:54.958467+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:13:04.302760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":48,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.458770+0000","last_change":"2023-11-28T13:07:57.306321+0000","last_active":"2023-11-28T13:10:54.458770+0000","last_peered":"2023-11-28T13:10:54.458770+0000","last_clean":"2023-11-28T13:10:54.458770+0000","last_became_active":"2023-11-28T13:07:57.305923+0000","last_became_peered":"2023-11-28T13:07:57.305923+0000","last_unstale":"2023-11-28T13:10:54.458770+0000","last_undegraded":"2023-11-28T13:10:54.458770+0000","last_fullsized":"2023-11-28T13:10:54.458770+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:41.104679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966197+0000","last_change":"2023-11-28T13:10:54.959336+0000","last_active":"2023-11-28T13:10:54.966197+0000","last_peered":"2023-11-28T13:10:54.966197+0000","last_clean":"2023-11-28T13:10:54.966197+0000","last_became_active":"2023-11-28T13:10:54.959004+0000","last_became_peered":"2023-11-28T13:10:54.959004+0000","last_unstale":"2023-11-28T13:10:54.966197+0000","last_undegraded":"2023-11-28T13:10:54.966197+0000","last_fullsized":"2023-11-28T13:10:54.966197+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:54.513040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.964204+0000","last_change":"2023-11-28T13:10:54.453286+0000","last_active":"2023-11-28T13:10:54.964204+0000","last_peered":"2023-11-28T13:10:54.964204+0000","last_clean":"2023-11-28T13:10:54.964204+0000","last_became_active":"2023-11-28T13:10:54.452994+0000","last_became_peered":"2023-11-28T13:10:54.452994+0000","last_unstale":"2023-11-28T13:10:54.964204+0000","last_undegraded":"2023-11-28T13:10:54.964204+0000","last_fullsized":"2023-11-28T13:10:54.964204+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:55.490929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":48,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.459102+0000","last_change":"2023-11-28T13:07:56.844305+0000","last_active":"2023-11-28T13:10:54.459102+0000","last_peered":"2023-11-28T13:10:54.459102+0000","last_clean":"2023-11-28T13:10:54.459102+0000","last_became_active":"2023-11-28T13:07:56.843980+0000","last_became_peered":"2023-11-28T13:07:56.843980+0000","last_unstale":"2023-11-28T13:10:54.459102+0000","last_undegraded":"2023-11-28T13:10:54.459102+0000","last_fullsized":"2023-11-28T13:10:54.459102+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14696,"kb_used_data":1248,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94357144,"statfs":{"total":96636764160,"available":96621715456,"internally_reserved":0,"allocated":1277952,"data_stored":887322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66200000000000003,"15min":0.65900000000000003},"min":{"1min":0.46700000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90300000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.49099999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88100000000000001,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.433}]},{"osd":1,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71899999999999997,"15min":0.71699999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.91900000000000004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69999999999999996,"15min":0.70099999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93600000000000005,"5min":1.095,"15min":1.095},"last":0.54100000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.52600000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91700000000000004,"5min":0.996,"15min":0.996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.86899999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.82599999999999996}]},{"osd":3,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.56699999999999995,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85399999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.627},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76900000000000002,"15min":0.77700000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.996,"5min":1.079,"15min":1.079},"last":0.88600000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.69699999999999995,"15min":0.71099999999999997},"min":{"1min":0.44600000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89300000000000002,"5min":1.008,"15min":1.008},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90100000000000002,"5min":1.071,"15min":1.071},"last":0.70399999999999996}]},{"osd":5,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77200000000000002,"15min":0.78500000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.044,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.80300000000000005,"15min":0.81599999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.97799999999999998,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.73499999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.84099999999999997,"15min":0.85399999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97799999999999998,"5min":1.165,"15min":1.165},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.78900000000000003,"15min":0.80000000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94899999999999995,"5min":1.141,"15min":1.141},"last":0.84699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9112,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362728,"statfs":{"total":96636764160,"available":96627433472,"internally_reserved":0,"allocated":671744,"data_stored":290334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.46700000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.64600000000000002,"15min":0.64300000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.91900000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.45900000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67700000000000005,"15min":0.67600000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95899999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65200000000000002,"15min":0.65300000000000002},"min":{"1min":0.48499999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67300000000000004,"15min":0.67100000000000004},"min":{"1min":0.52200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.48199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.56200000000000006}]},{"osd":3,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.48499999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.94799999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51200000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0109999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.67300000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67600000000000005,"15min":0.67900000000000005},"min":{"1min":0.40100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88200000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.81499999999999995,"15min":0.82099999999999995},"min":{"1min":0.621,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.96599999999999997,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.76300000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"min":{"1min":0.55100000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.877,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.71899999999999997,"15min":0.72499999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92500000000000004,"5min":0.996,"15min":0.996},"last":0.74199999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.73899999999999999,"15min":0.74199999999999999},"min":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.93999999999999995,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.59299999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94099999999999995,"5min":1.107,"15min":1.107},"last":0.78500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14364,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357476,"statfs":{"total":96636764160,"available":96622055424,"internally_reserved":0,"allocated":1265664,"data_stored":880702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:10:00 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.436,"15min":0.438},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0549999999999999,"5min":1.28,"15min":1.28},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.44,"5min":0.44400000000000001,"15min":0.44500000000000001},"min":{"1min":0.29799999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61399999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.46700000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:10:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67300000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.38200000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:10:00 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54100000000000004,"15min":0.54700000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83999999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.54100000000000004,"15min":0.55000000000000004},"min":{"1min":0.37,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68999999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.53800000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:10:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.56399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70999999999999996,"15min":0.70599999999999996},"min":{"1min":0.52100000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.60499999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:10:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.51600000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.94499999999999995,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64900000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:10:00 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.73199999999999998,"15min":0.72999999999999998},"min":{"1min":0.54400000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.997,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.77600000000000002,"15min":0.78000000000000003},"min":{"1min":0.56499999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0860000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.85999999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:10:00 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81200000000000006,"15min":0.81599999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1000000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.78900000000000003,"15min":0.78700000000000003},"min":{"1min":0.61299999999999999,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0860000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56299999999999994}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13460,"kb_used_data":652,"kb_used_omap":0,"kb_used_meta":12800,"kb_avail":94358380,"statfs":{"total":96636764160,"available":96622981120,"internally_reserved":0,"allocated":667648,"data_stored":289348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13107200},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:10:03 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.49199999999999999,"15min":0.504},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68400000000000005,"5min":1.48,"15min":1.48},"last":0.503}]},{"osd":2,"last update":"Tue Nov 28 13:10:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.59799999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.503,"15min":0.50700000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.436}]},{"osd":3,"last update":"Tue Nov 28 13:10:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47299999999999998,"15min":0.47399999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70599999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.59399999999999997}]},{"osd":4,"last update":"Tue Nov 28 13:10:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.78300000000000003},"min":{"1min":0.625,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.85799999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:10:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.78900000000000003,"15min":0.79800000000000004},"min":{"1min":0.51700000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97099999999999997,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79000000000000004,"15min":0.79100000000000004},"min":{"1min":0.65400000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91000000000000003,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.66700000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:10:03 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.754,"15min":0.755},"min":{"1min":0.54600000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.81699999999999995,"15min":0.82099999999999995},"min":{"1min":0.623,"5min":0.62,"15min":0.62},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.81499999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:10:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.58799999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.96199999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.80000000000000004,"15min":0.80600000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.73199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13336,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":12672,"kb_avail":94358504,"statfs":{"total":96636764160,"available":96623108096,"internally_reserved":0,"allocated":671744,"data_stored":290334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12976128},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42599999999999999,"15min":0.42799999999999999},"min":{"1min":0.28899999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64700000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.314},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46200000000000002,"15min":0.46100000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66600000000000004,"5min":0.75,"15min":0.75},"last":0.52200000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32700000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69199999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51000000000000001,"15min":0.51300000000000001},"min":{"1min":0.375,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.64300000000000002,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.434}]},{"osd":3,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.36099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.60299999999999998,"5min":0.75,"15min":0.75},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.503}]},{"osd":4,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.67600000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0149999999999999,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.087,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.63500000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.044,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.58999999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96399999999999997,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.54600000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.75600000000000001,"15min":0.746},"min":{"1min":0.56100000000000005,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.03,"5min":1.083,"15min":1.083},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79400000000000004,"15min":0.79200000000000004},"min":{"1min":0.61099999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.071,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.55800000000000005}]},{"osd":7,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.76100000000000001,"15min":0.75900000000000001},"min":{"1min":0.56399999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.99299999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80200000000000005,"15min":0.80000000000000004},"min":{"1min":0.58699999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.998,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.58599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13656,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":12992,"kb_avail":94358184,"statfs":{"total":96636764160,"available":96622780416,"internally_reserved":0,"allocated":671744,"data_stored":290334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13303808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:09:57 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.41699999999999998,"15min":0.42099999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.44400000000000001,"15min":0.44500000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.55600000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.35399999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:09:57 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.47899999999999998,"15min":0.49099999999999999},"min":{"1min":0.317,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.59499999999999997,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.66000000000000003,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.30499999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:09:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.67600000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.39000000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.40799999999999997}]},{"osd":4,"last update":"Tue Nov 28 13:09:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69699999999999995,"15min":0.69599999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.93799999999999994,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.67200000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:09:57 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.76700000000000002,"15min":0.77000000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0049999999999999,"5min":1.091,"15min":1.091},"last":0.623},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.75700000000000001,"15min":0.76200000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.94899999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.50600000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:09:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.71999999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70099999999999996,"15min":0.69899999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:09:57 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.754,"15min":0.755},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0569999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.70899999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9444,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362396,"statfs":{"total":96636764160,"available":96627093504,"internally_reserved":0,"allocated":684032,"data_stored":296954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.42799999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.876},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66800000000000004,"15min":0.67000000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85999999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.58899999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67600000000000005,"15min":0.67900000000000005},"min":{"1min":0.50600000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.87,"5min":1.161,"15min":1.161},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60399999999999998,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.624}]},{"osd":2,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.64900000000000002},"min":{"1min":0.49299999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.441,"15min":0.441},"max":{"1min":0.84099999999999997,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.50800000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.86399999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.89500000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54800000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67100000000000004,"15min":0.67900000000000005},"min":{"1min":0.41799999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88200000000000001,"5min":1.264,"15min":1.264},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.72499999999999998,"15min":0.73499999999999999},"min":{"1min":0.39900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90300000000000002,"5min":1.079,"15min":1.079},"last":0.77700000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.72899999999999998,"15min":0.73499999999999999},"min":{"1min":0.47299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.93899999999999995,"5min":1.022,"15min":1.022},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.74399999999999999,"15min":0.748},"min":{"1min":0.48499999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92400000000000004,"5min":1.22,"15min":1.22},"last":0.82499999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.76500000000000001,"15min":0.77500000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92700000000000005,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78300000000000003,"15min":0.78700000000000003},"min":{"1min":0.52000000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.96599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9368,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362472,"statfs":{"total":96636764160,"available":96627171328,"internally_reserved":0,"allocated":671744,"data_stored":290334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.61599999999999999,"15min":0.59899999999999998},"min":{"1min":0.46200000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.61399999999999999,"15min":0.59999999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.76000000000000001}]},{"osd":1,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.63500000000000001,"15min":0.61399999999999999},"min":{"1min":0.443,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.028}]},{"osd":2,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.64900000000000002,"15min":0.63300000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.3860000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.47699999999999998,"5min":0.432,"15min":0.432},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.88100000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.64400000000000002,"15min":0.626},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.70999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.91300000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.745,"15min":0.73699999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.0529999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73199999999999998,"15min":0.72699999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.012}]},{"osd":7,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.77700000000000002,"15min":0.76800000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.4249999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":0.98099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:10:59.299 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:10:59.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.530+0000 7f5b869ff700 1 Processor -- start 2023-11-28T13:10:59.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.530+0000 7f5b869ff700 1 -- start start 2023-11-28T13:10:59.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.531+0000 7f5b869ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5b8015c9b0 0x7f5b800b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:59.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.531+0000 7f5b869ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b800d1950 con 0x7f5b800cd7c0 2023-11-28T13:10:59.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.531+0000 7f5b869ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b800d24f0 con 0x7f5b80161340 2023-11-28T13:10:59.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.531+0000 7f5b869ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b800b5800 con 0x7f5b8015c9b0 2023-11-28T13:10:59.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.532+0000 7f5b84f9c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f5b800cd7c0 0x7f5b800b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37780/0 (socket says 172.21.15.156:37780) 2023-11-28T13:10:59.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.532+0000 7f5b84f9c700 1 -- 172.21.15.156:0/1793222931 learned_addr learned my addr 172.21.15.156:0/1793222931 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:10:59.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.532+0000 7f5b7ffff700 1 --2- 172.21.15.156:0/1793222931 >> v2:172.21.15.186:3300/0 conn(0x7f5b8015c9b0 0x7f5b800b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:59.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.532+0000 7f5b7effd700 1 -- 172.21.15.156:0/1793222931 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575699436 0 0) 0x7f5b800d1950 con 0x7f5b800cd7c0 2023-11-28T13:10:59.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.532+0000 7f5b7effd700 1 -- 172.21.15.156:0/1793222931 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b680035b0 con 0x7f5b800cd7c0 2023-11-28T13:10:59.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.532+0000 7f5b7effd700 1 -- 172.21.15.156:0/1793222931 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813148636 0 0) 0x7f5b800d24f0 con 0x7f5b80161340 2023-11-28T13:10:59.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.532+0000 7f5b7effd700 1 -- 172.21.15.156:0/1793222931 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b800d1950 con 0x7f5b80161340 2023-11-28T13:10:59.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.532+0000 7f5b7ffff700 1 -- 172.21.15.156:0/1793222931 >> v1:172.21.15.186:6789/0 conn(0x7f5b80161340 legacy=0x7f5b800d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:59.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.532+0000 7f5b7ffff700 1 -- 172.21.15.156:0/1793222931 >> v1:172.21.15.156:6789/0 conn(0x7f5b800cd7c0 legacy=0x7f5b800b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:59.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.532+0000 7f5b7effd700 1 -- 172.21.15.156:0/1793222931 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3120757056 0 0) 0x7f5b680035b0 con 0x7f5b800cd7c0 2023-11-28T13:10:59.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.532+0000 7f5b7ffff700 1 -- 172.21.15.156:0/1793222931 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b800b5520 con 0x7f5b8015c9b0 2023-11-28T13:10:59.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.533+0000 7f5b7ffff700 1 --2- 172.21.15.156:0/1793222931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b8015c9b0 0x7f5b800b4bf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009840 tx=0x7f5b70009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8834d2e7e7340652 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:59.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.533+0000 7f5b7effd700 1 -- 172.21.15.156:0/1793222931 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b780019f0 con 0x7f5b8015c9b0 2023-11-28T13:10:59.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.533+0000 7f5b7effd700 1 -- 172.21.15.156:0/1793222931 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b70004030 con 0x7f5b8015c9b0 2023-11-28T13:10:59.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.533+0000 7f5b7effd700 1 -- 172.21.15.156:0/1793222931 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b700043c0 con 0x7f5b8015c9b0 2023-11-28T13:10:59.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.533+0000 7f5b869ff700 1 -- 172.21.15.156:0/1793222931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b8015c9b0 msgr2=0x7f5b800b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:59.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.533+0000 7f5b869ff700 1 --2- 172.21.15.156:0/1793222931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b8015c9b0 0x7f5b800b4bf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009840 tx=0x7f5b70009b50 comp rx=0 tx=0).stop 2023-11-28T13:10:59.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.534+0000 7f5b869ff700 1 -- 172.21.15.156:0/1793222931 shutdown_connections 2023-11-28T13:10:59.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.534+0000 7f5b869ff700 1 --2- 172.21.15.156:0/1793222931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b8015c9b0 0x7f5b800b4bf0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:59.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.534+0000 7f5b869ff700 1 -- 172.21.15.156:0/1793222931 >> 172.21.15.156:0/1793222931 conn(0x7f5b80072600 msgr2=0x7f5b800cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:59.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.534+0000 7f5b869ff700 1 -- 172.21.15.156:0/1793222931 shutdown_connections 2023-11-28T13:10:59.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.534+0000 7f5b869ff700 1 -- 172.21.15.156:0/1793222931 wait complete. 2023-11-28T13:10:59.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.535+0000 7f5b869ff700 1 Processor -- start 2023-11-28T13:10:59.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.536+0000 7f5b869ff700 1 -- start start 2023-11-28T13:10:59.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.536+0000 7f5b869ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b800cd7c0 0x7f5b800cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:59.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.536+0000 7f5b869ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b8015c9b0 0x7f5b80154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:59.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.536+0000 7f5b869ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b80161340 0x7f5b801f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:59.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.536+0000 7f5b869ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b801f64c0 con 0x7f5b8015c9b0 2023-11-28T13:10:59.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.536+0000 7f5b869ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b801f67a0 con 0x7f5b800cd7c0 2023-11-28T13:10:59.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.536+0000 7f5b869ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b801f6a80 con 0x7f5b80161340 2023-11-28T13:10:59.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.537+0000 7f5b7f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b8015c9b0 0x7f5b80154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:59.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.537+0000 7f5b7f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b8015c9b0 0x7f5b80154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53088/0 (socket says 172.21.15.156:53088) 2023-11-28T13:10:59.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.537+0000 7f5b7f7fe700 1 -- 172.21.15.156:0/2125415281 learned_addr learned my addr 172.21.15.156:0/2125415281 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:10:59.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.537+0000 7f5b7ffff700 1 --2- 172.21.15.156:0/2125415281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b800cd7c0 0x7f5b800cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:59.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.537+0000 7f5b84f9c700 1 --2- 172.21.15.156:0/2125415281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b80161340 0x7f5b801f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:59.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.537+0000 7f5b7f7fe700 1 -- 172.21.15.156:0/2125415281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b80161340 msgr2=0x7f5b801f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:59.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.537+0000 7f5b7f7fe700 1 --2- 172.21.15.156:0/2125415281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b80161340 0x7f5b801f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:59.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.537+0000 7f5b7f7fe700 1 -- 172.21.15.156:0/2125415281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b800cd7c0 msgr2=0x7f5b800cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:59.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.537+0000 7f5b7f7fe700 1 --2- 172.21.15.156:0/2125415281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b800cd7c0 0x7f5b800cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:59.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.537+0000 7f5b7f7fe700 1 -- 172.21.15.156:0/2125415281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b700094a0 con 0x7f5b8015c9b0 2023-11-28T13:10:59.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.538+0000 7f5b7f7fe700 1 --2- 172.21.15.156:0/2125415281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b8015c9b0 0x7f5b80154240 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400c520 tx=0x7f5b7400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:59.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.538+0000 7f5b7cff9700 1 -- 172.21.15.156:0/2125415281 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b740052d0 con 0x7f5b8015c9b0 2023-11-28T13:10:59.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.538+0000 7f5b7cff9700 1 -- 172.21.15.156:0/2125415281 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b74005430 con 0x7f5b8015c9b0 2023-11-28T13:10:59.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.538+0000 7f5b869ff700 1 -- 172.21.15.156:0/2125415281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b800d15e0 con 0x7f5b8015c9b0 2023-11-28T13:10:59.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.538+0000 7f5b7cff9700 1 -- 172.21.15.156:0/2125415281 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b740038c0 con 0x7f5b8015c9b0 2023-11-28T13:10:59.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.538+0000 7f5b869ff700 1 -- 172.21.15.156:0/2125415281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b800d1300 con 0x7f5b8015c9b0 2023-11-28T13:10:59.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.539+0000 7f5b869ff700 1 -- 172.21.15.156:0/2125415281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b800b5130 con 0x7f5b8015c9b0 2023-11-28T13:10:59.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.540+0000 7f5b7cff9700 1 -- 172.21.15.156:0/2125415281 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5b74019030 con 0x7f5b8015c9b0 2023-11-28T13:10:59.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.541+0000 7f5b7cff9700 1 --2- 172.21.15.156:0/2125415281 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5b6c0682f0 0x7f5b6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:10:59.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.541+0000 7f5b7cff9700 1 -- 172.21.15.156:0/2125415281 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5b74085070 con 0x7f5b8015c9b0 2023-11-28T13:10:59.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.541+0000 7f5b7ffff700 1 --2- 172.21.15.156:0/2125415281 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5b6c0682f0 0x7f5b6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:10:59.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.542+0000 7f5b7ffff700 1 --2- 172.21.15.156:0/2125415281 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5b6c0682f0 0x7f5b6c06a7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5b700090b0 tx=0x7f5b70008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:10:59.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.544+0000 7f5b7cff9700 1 -- 172.21.15.156:0/2125415281 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5b740517e0 con 0x7f5b8015c9b0 2023-11-28T13:10:59.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.732+0000 7f5b869ff700 1 -- 172.21.15.156:0/2125415281 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5b801f68e0 con 0x7f5b6c0682f0 2023-11-28T13:10:59.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.741+0000 7f5b7cff9700 1 -- 172.21.15.156:0/2125415281 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311706 (secure 0 0 0) 0x7f5b801f68e0 con 0x7f5b6c0682f0 2023-11-28T13:10:59.747 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:10:59.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.746+0000 7f5b869ff700 1 -- 172.21.15.156:0/2125415281 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5b6c0682f0 msgr2=0x7f5b6c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:59.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.746+0000 7f5b869ff700 1 --2- 172.21.15.156:0/2125415281 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5b6c0682f0 0x7f5b6c06a7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5b700090b0 tx=0x7f5b70008030 comp rx=0 tx=0).stop 2023-11-28T13:10:59.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.746+0000 7f5b869ff700 1 -- 172.21.15.156:0/2125415281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b8015c9b0 msgr2=0x7f5b80154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:10:59.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.746+0000 7f5b869ff700 1 --2- 172.21.15.156:0/2125415281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b8015c9b0 0x7f5b80154240 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400c520 tx=0x7f5b7400c830 comp rx=0 tx=0).stop 2023-11-28T13:10:59.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.746+0000 7f5b869ff700 1 -- 172.21.15.156:0/2125415281 shutdown_connections 2023-11-28T13:10:59.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.746+0000 7f5b869ff700 1 --2- 172.21.15.156:0/2125415281 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5b6c0682f0 0x7f5b6c06a7a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:59.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.746+0000 7f5b869ff700 1 --2- 172.21.15.156:0/2125415281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b80161340 0x7f5b801f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:59.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.746+0000 7f5b869ff700 1 --2- 172.21.15.156:0/2125415281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b800cd7c0 0x7f5b800cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:59.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.746+0000 7f5b869ff700 1 --2- 172.21.15.156:0/2125415281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b8015c9b0 0x7f5b80154240 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:10:59.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.746+0000 7f5b869ff700 1 -- 172.21.15.156:0/2125415281 >> 172.21.15.156:0/2125415281 conn(0x7f5b80072600 msgr2=0x7f5b8015ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:10:59.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.746+0000 7f5b869ff700 1 -- 172.21.15.156:0/2125415281 shutdown_connections 2023-11-28T13:10:59.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:10:59.746+0000 7f5b869ff700 1 -- 172.21.15.156:0/2125415281 wait complete. 2023-11-28T13:10:59.751 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T13:10:59.767 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2023-11-28T13:10:58.058053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":97436,"kb_used_data":6428,"kb_used_omap":0,"kb_used_meta":90944,"kb_avail":754877284,"statfs":{"total":773094113280,"available":772994338816,"internally_reserved":0,"allocated":6582272,"data_stored":3515662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":93126656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.624633"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.965381+0000","last_change":"2023-11-28T13:10:54.961652+0000","last_active":"2023-11-28T13:10:54.965381+0000","last_peered":"2023-11-28T13:10:54.965381+0000","last_clean":"2023-11-28T13:10:54.965381+0000","last_became_active":"2023-11-28T13:10:54.961404+0000","last_became_peered":"2023-11-28T13:10:54.961404+0000","last_unstale":"2023-11-28T13:10:54.965381+0000","last_undegraded":"2023-11-28T13:10:54.965381+0000","last_fullsized":"2023-11-28T13:10:54.965381+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:00:26.663675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.965480+0000","last_change":"2023-11-28T13:10:54.960157+0000","last_active":"2023-11-28T13:10:54.965480+0000","last_peered":"2023-11-28T13:10:54.965480+0000","last_clean":"2023-11-28T13:10:54.965480+0000","last_became_active":"2023-11-28T13:10:54.959690+0000","last_became_peered":"2023-11-28T13:10:54.959690+0000","last_unstale":"2023-11-28T13:10:54.965480+0000","last_undegraded":"2023-11-28T13:10:54.965480+0000","last_fullsized":"2023-11-28T13:10:54.965480+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:56.493889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.959022+0000","last_change":"2023-11-28T13:10:54.953490+0000","last_active":"2023-11-28T13:10:54.959022+0000","last_peered":"2023-11-28T13:10:54.959022+0000","last_clean":"2023-11-28T13:10:54.959022+0000","last_became_active":"2023-11-28T13:10:54.953224+0000","last_became_peered":"2023-11-28T13:10:54.953224+0000","last_unstale":"2023-11-28T13:10:54.959022+0000","last_undegraded":"2023-11-28T13:10:54.959022+0000","last_fullsized":"2023-11-28T13:10:54.959022+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:44:47.603128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.957814+0000","last_change":"2023-11-28T13:10:54.955625+0000","last_active":"2023-11-28T13:10:54.957814+0000","last_peered":"2023-11-28T13:10:54.957814+0000","last_clean":"2023-11-28T13:10:54.957814+0000","last_became_active":"2023-11-28T13:10:54.955376+0000","last_became_peered":"2023-11-28T13:10:54.955376+0000","last_unstale":"2023-11-28T13:10:54.957814+0000","last_undegraded":"2023-11-28T13:10:54.957814+0000","last_fullsized":"2023-11-28T13:10:54.957814+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:50:49.582092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.962470+0000","last_change":"2023-11-28T13:10:54.954060+0000","last_active":"2023-11-28T13:10:54.962470+0000","last_peered":"2023-11-28T13:10:54.962470+0000","last_clean":"2023-11-28T13:10:54.962470+0000","last_became_active":"2023-11-28T13:10:54.953479+0000","last_became_peered":"2023-11-28T13:10:54.953479+0000","last_unstale":"2023-11-28T13:10:54.962470+0000","last_undegraded":"2023-11-28T13:10:54.962470+0000","last_fullsized":"2023-11-28T13:10:54.962470+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:59.938061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.954623+0000","last_change":"2023-11-28T13:10:54.953353+0000","last_active":"2023-11-28T13:10:54.954623+0000","last_peered":"2023-11-28T13:10:54.954623+0000","last_clean":"2023-11-28T13:10:54.954623+0000","last_became_active":"2023-11-28T13:10:54.952999+0000","last_became_peered":"2023-11-28T13:10:54.952999+0000","last_unstale":"2023-11-28T13:10:54.954623+0000","last_undegraded":"2023-11-28T13:10:54.954623+0000","last_fullsized":"2023-11-28T13:10:54.954623+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:20.842433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.958991+0000","last_change":"2023-11-28T13:10:54.957499+0000","last_active":"2023-11-28T13:10:54.958991+0000","last_peered":"2023-11-28T13:10:54.958991+0000","last_clean":"2023-11-28T13:10:54.958991+0000","last_became_active":"2023-11-28T13:10:54.957165+0000","last_became_peered":"2023-11-28T13:10:54.957165+0000","last_unstale":"2023-11-28T13:10:54.958991+0000","last_undegraded":"2023-11-28T13:10:54.958991+0000","last_fullsized":"2023-11-28T13:10:54.958991+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:41:02.504630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966739+0000","last_change":"2023-11-28T13:10:54.962260+0000","last_active":"2023-11-28T13:10:54.966739+0000","last_peered":"2023-11-28T13:10:54.966739+0000","last_clean":"2023-11-28T13:10:54.966739+0000","last_became_active":"2023-11-28T13:10:54.958638+0000","last_became_peered":"2023-11-28T13:10:54.958638+0000","last_unstale":"2023-11-28T13:10:54.966739+0000","last_undegraded":"2023-11-28T13:10:54.966739+0000","last_fullsized":"2023-11-28T13:10:54.966739+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:13.622196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.965812+0000","last_change":"2023-11-28T13:10:54.958591+0000","last_active":"2023-11-28T13:10:54.965812+0000","last_peered":"2023-11-28T13:10:54.965812+0000","last_clean":"2023-11-28T13:10:54.965812+0000","last_became_active":"2023-11-28T13:10:54.958293+0000","last_became_peered":"2023-11-28T13:10:54.958293+0000","last_unstale":"2023-11-28T13:10:54.965812+0000","last_undegraded":"2023-11-28T13:10:54.965812+0000","last_fullsized":"2023-11-28T13:10:54.965812+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:29.939656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.960430+0000","last_change":"2023-11-28T13:10:54.956373+0000","last_active":"2023-11-28T13:10:54.960430+0000","last_peered":"2023-11-28T13:10:54.960430+0000","last_clean":"2023-11-28T13:10:54.960430+0000","last_became_active":"2023-11-28T13:10:54.956076+0000","last_became_peered":"2023-11-28T13:10:54.956076+0000","last_unstale":"2023-11-28T13:10:54.960430+0000","last_undegraded":"2023-11-28T13:10:54.960430+0000","last_fullsized":"2023-11-28T13:10:54.960430+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:50:49.582092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.971832+0000","last_change":"2023-11-28T13:10:54.968577+0000","last_active":"2023-11-28T13:10:54.971832+0000","last_peered":"2023-11-28T13:10:54.971832+0000","last_clean":"2023-11-28T13:10:54.971832+0000","last_became_active":"2023-11-28T13:10:54.968309+0000","last_became_peered":"2023-11-28T13:10:54.968309+0000","last_unstale":"2023-11-28T13:10:54.971832+0000","last_undegraded":"2023-11-28T13:10:54.971832+0000","last_fullsized":"2023-11-28T13:10:54.971832+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:51.738865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966757+0000","last_change":"2023-11-28T13:10:54.963008+0000","last_active":"2023-11-28T13:10:54.966757+0000","last_peered":"2023-11-28T13:10:54.966757+0000","last_clean":"2023-11-28T13:10:54.966757+0000","last_became_active":"2023-11-28T13:10:54.962684+0000","last_became_peered":"2023-11-28T13:10:54.962684+0000","last_unstale":"2023-11-28T13:10:54.966757+0000","last_undegraded":"2023-11-28T13:10:54.966757+0000","last_fullsized":"2023-11-28T13:10:54.966757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:16:24.754084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966599+0000","last_change":"2023-11-28T13:10:54.957772+0000","last_active":"2023-11-28T13:10:54.966599+0000","last_peered":"2023-11-28T13:10:54.966599+0000","last_clean":"2023-11-28T13:10:54.966599+0000","last_became_active":"2023-11-28T13:10:54.957143+0000","last_became_peered":"2023-11-28T13:10:54.957143+0000","last_unstale":"2023-11-28T13:10:54.966599+0000","last_undegraded":"2023-11-28T13:10:54.966599+0000","last_fullsized":"2023-11-28T13:10:54.966599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:26:38.445148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.962079+0000","last_change":"2023-11-28T13:10:54.957271+0000","last_active":"2023-11-28T13:10:54.962079+0000","last_peered":"2023-11-28T13:10:54.962079+0000","last_clean":"2023-11-28T13:10:54.962079+0000","last_became_active":"2023-11-28T13:10:54.956920+0000","last_became_peered":"2023-11-28T13:10:54.956920+0000","last_unstale":"2023-11-28T13:10:54.962079+0000","last_undegraded":"2023-11-28T13:10:54.962079+0000","last_fullsized":"2023-11-28T13:10:54.962079+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:40.521358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.963195+0000","last_change":"2023-11-28T13:10:54.958827+0000","last_active":"2023-11-28T13:10:54.963195+0000","last_peered":"2023-11-28T13:10:54.963195+0000","last_clean":"2023-11-28T13:10:54.963195+0000","last_became_active":"2023-11-28T13:10:54.958450+0000","last_became_peered":"2023-11-28T13:10:54.958450+0000","last_unstale":"2023-11-28T13:10:54.963195+0000","last_undegraded":"2023-11-28T13:10:54.963195+0000","last_fullsized":"2023-11-28T13:10:54.963195+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:53.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.956401+0000","last_change":"2023-11-28T13:10:54.951602+0000","last_active":"2023-11-28T13:10:54.956401+0000","last_peered":"2023-11-28T13:10:54.956401+0000","last_clean":"2023-11-28T13:10:54.956401+0000","last_became_active":"2023-11-28T13:10:54.949451+0000","last_became_peered":"2023-11-28T13:10:54.949451+0000","last_unstale":"2023-11-28T13:10:54.956401+0000","last_undegraded":"2023-11-28T13:10:54.956401+0000","last_fullsized":"2023-11-28T13:10:54.956401+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:32.485741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.965405+0000","last_change":"2023-11-28T13:10:54.961686+0000","last_active":"2023-11-28T13:10:54.965405+0000","last_peered":"2023-11-28T13:10:54.965405+0000","last_clean":"2023-11-28T13:10:54.965405+0000","last_became_active":"2023-11-28T13:10:54.961426+0000","last_became_peered":"2023-11-28T13:10:54.961426+0000","last_unstale":"2023-11-28T13:10:54.965405+0000","last_undegraded":"2023-11-28T13:10:54.965405+0000","last_fullsized":"2023-11-28T13:10:54.965405+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:27:19.503456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.959935+0000","last_change":"2023-11-28T13:10:54.956246+0000","last_active":"2023-11-28T13:10:54.959935+0000","last_peered":"2023-11-28T13:10:54.959935+0000","last_clean":"2023-11-28T13:10:54.959935+0000","last_became_active":"2023-11-28T13:10:54.955986+0000","last_became_peered":"2023-11-28T13:10:54.955986+0000","last_unstale":"2023-11-28T13:10:54.959935+0000","last_undegraded":"2023-11-28T13:10:54.959935+0000","last_fullsized":"2023-11-28T13:10:54.959935+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:40.521358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.944616+0000","last_change":"2023-11-28T13:10:54.942803+0000","last_active":"2023-11-28T13:10:54.944616+0000","last_peered":"2023-11-28T13:10:54.944616+0000","last_clean":"2023-11-28T13:10:54.944616+0000","last_became_active":"2023-11-28T13:10:54.942494+0000","last_became_peered":"2023-11-28T13:10:54.942494+0000","last_unstale":"2023-11-28T13:10:54.944616+0000","last_undegraded":"2023-11-28T13:10:54.944616+0000","last_fullsized":"2023-11-28T13:10:54.944616+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:26.486619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.963083+0000","last_change":"2023-11-28T13:10:54.959131+0000","last_active":"2023-11-28T13:10:54.963083+0000","last_peered":"2023-11-28T13:10:54.963083+0000","last_clean":"2023-11-28T13:10:54.963083+0000","last_became_active":"2023-11-28T13:10:54.958867+0000","last_became_peered":"2023-11-28T13:10:54.958867+0000","last_unstale":"2023-11-28T13:10:54.963083+0000","last_undegraded":"2023-11-28T13:10:54.963083+0000","last_fullsized":"2023-11-28T13:10:54.963083+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:55.169875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966619+0000","last_change":"2023-11-28T13:10:54.957967+0000","last_active":"2023-11-28T13:10:54.966619+0000","last_peered":"2023-11-28T13:10:54.966619+0000","last_clean":"2023-11-28T13:10:54.966619+0000","last_became_active":"2023-11-28T13:10:54.957506+0000","last_became_peered":"2023-11-28T13:10:54.957506+0000","last_unstale":"2023-11-28T13:10:54.966619+0000","last_undegraded":"2023-11-28T13:10:54.966619+0000","last_fullsized":"2023-11-28T13:10:54.966619+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:05.224374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.958892+0000","last_change":"2023-11-28T13:10:54.957256+0000","last_active":"2023-11-28T13:10:54.958892+0000","last_peered":"2023-11-28T13:10:54.958892+0000","last_clean":"2023-11-28T13:10:54.958892+0000","last_became_active":"2023-11-28T13:10:54.957008+0000","last_became_peered":"2023-11-28T13:10:54.957008+0000","last_unstale":"2023-11-28T13:10:54.958892+0000","last_undegraded":"2023-11-28T13:10:54.958892+0000","last_fullsized":"2023-11-28T13:10:54.958892+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:59:49.860376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.959090+0000","last_change":"2023-11-28T13:10:54.957866+0000","last_active":"2023-11-28T13:10:54.959090+0000","last_peered":"2023-11-28T13:10:54.959090+0000","last_clean":"2023-11-28T13:10:54.959090+0000","last_became_active":"2023-11-28T13:10:54.957610+0000","last_became_peered":"2023-11-28T13:10:54.957610+0000","last_unstale":"2023-11-28T13:10:54.959090+0000","last_undegraded":"2023-11-28T13:10:54.959090+0000","last_fullsized":"2023-11-28T13:10:54.959090+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:05:56.985098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.960314+0000","last_change":"2023-11-28T13:10:54.953957+0000","last_active":"2023-11-28T13:10:54.960314+0000","last_peered":"2023-11-28T13:10:54.960314+0000","last_clean":"2023-11-28T13:10:54.960314+0000","last_became_active":"2023-11-28T13:10:54.953668+0000","last_became_peered":"2023-11-28T13:10:54.953668+0000","last_unstale":"2023-11-28T13:10:54.960314+0000","last_undegraded":"2023-11-28T13:10:54.960314+0000","last_fullsized":"2023-11-28T13:10:54.960314+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:30.770582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.955601+0000","last_change":"2023-11-28T13:10:54.950615+0000","last_active":"2023-11-28T13:10:54.955601+0000","last_peered":"2023-11-28T13:10:54.955601+0000","last_clean":"2023-11-28T13:10:54.955601+0000","last_became_active":"2023-11-28T13:10:54.947092+0000","last_became_peered":"2023-11-28T13:10:54.947092+0000","last_unstale":"2023-11-28T13:10:54.955601+0000","last_undegraded":"2023-11-28T13:10:54.955601+0000","last_fullsized":"2023-11-28T13:10:54.955601+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:01:41.568218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.963154+0000","last_change":"2023-11-28T13:10:54.948805+0000","last_active":"2023-11-28T13:10:54.963154+0000","last_peered":"2023-11-28T13:10:54.963154+0000","last_clean":"2023-11-28T13:10:54.963154+0000","last_became_active":"2023-11-28T13:10:54.948529+0000","last_became_peered":"2023-11-28T13:10:54.948529+0000","last_unstale":"2023-11-28T13:10:54.963154+0000","last_undegraded":"2023-11-28T13:10:54.963154+0000","last_fullsized":"2023-11-28T13:10:54.963154+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:59:07.213157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.969948+0000","last_change":"2023-11-28T13:10:54.965970+0000","last_active":"2023-11-28T13:10:54.969948+0000","last_peered":"2023-11-28T13:10:54.969948+0000","last_clean":"2023-11-28T13:10:54.969948+0000","last_became_active":"2023-11-28T13:10:54.960090+0000","last_became_peered":"2023-11-28T13:10:54.960090+0000","last_unstale":"2023-11-28T13:10:54.969948+0000","last_undegraded":"2023-11-28T13:10:54.969948+0000","last_fullsized":"2023-11-28T13:10:54.969948+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:46:36.216924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966021+0000","last_change":"2023-11-28T13:10:54.962779+0000","last_active":"2023-11-28T13:10:54.966021+0000","last_peered":"2023-11-28T13:10:54.966021+0000","last_clean":"2023-11-28T13:10:54.966021+0000","last_became_active":"2023-11-28T13:10:54.962512+0000","last_became_peered":"2023-11-28T13:10:54.962512+0000","last_unstale":"2023-11-28T13:10:54.966021+0000","last_undegraded":"2023-11-28T13:10:54.966021+0000","last_fullsized":"2023-11-28T13:10:54.966021+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:28.144954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.960357+0000","last_change":"2023-11-28T13:10:54.954735+0000","last_active":"2023-11-28T13:10:54.960357+0000","last_peered":"2023-11-28T13:10:54.960357+0000","last_clean":"2023-11-28T13:10:54.960357+0000","last_became_active":"2023-11-28T13:10:54.954175+0000","last_became_peered":"2023-11-28T13:10:54.954175+0000","last_unstale":"2023-11-28T13:10:54.960357+0000","last_undegraded":"2023-11-28T13:10:54.960357+0000","last_fullsized":"2023-11-28T13:10:54.960357+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:51.738865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.967118+0000","last_change":"2023-11-28T13:10:54.958693+0000","last_active":"2023-11-28T13:10:54.967118+0000","last_peered":"2023-11-28T13:10:54.967118+0000","last_clean":"2023-11-28T13:10:54.967118+0000","last_became_active":"2023-11-28T13:10:54.958350+0000","last_became_peered":"2023-11-28T13:10:54.958350+0000","last_unstale":"2023-11-28T13:10:54.967118+0000","last_undegraded":"2023-11-28T13:10:54.967118+0000","last_fullsized":"2023-11-28T13:10:54.967118+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:55.169875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.968080+0000","last_change":"2023-11-28T13:10:54.958583+0000","last_active":"2023-11-28T13:10:54.968080+0000","last_peered":"2023-11-28T13:10:54.968080+0000","last_clean":"2023-11-28T13:10:54.968080+0000","last_became_active":"2023-11-28T13:10:54.953594+0000","last_became_peered":"2023-11-28T13:10:54.953594+0000","last_unstale":"2023-11-28T13:10:54.968080+0000","last_undegraded":"2023-11-28T13:10:54.968080+0000","last_fullsized":"2023-11-28T13:10:54.968080+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:53.545672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.949682+0000","last_change":"2023-11-28T13:10:54.945572+0000","last_active":"2023-11-28T13:10:54.949682+0000","last_peered":"2023-11-28T13:10:54.949682+0000","last_clean":"2023-11-28T13:10:54.949682+0000","last_became_active":"2023-11-28T13:10:54.945292+0000","last_became_peered":"2023-11-28T13:10:54.945292+0000","last_unstale":"2023-11-28T13:10:54.949682+0000","last_undegraded":"2023-11-28T13:10:54.949682+0000","last_fullsized":"2023-11-28T13:10:54.949682+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:59:49.860376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.968399+0000","last_change":"2023-11-28T13:10:54.964604+0000","last_active":"2023-11-28T13:10:54.968399+0000","last_peered":"2023-11-28T13:10:54.968399+0000","last_clean":"2023-11-28T13:10:54.968399+0000","last_became_active":"2023-11-28T13:10:54.964014+0000","last_became_peered":"2023-11-28T13:10:54.964014+0000","last_unstale":"2023-11-28T13:10:54.968399+0000","last_undegraded":"2023-11-28T13:10:54.968399+0000","last_fullsized":"2023-11-28T13:10:54.968399+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:52.131617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"58'4","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.968281+0000","last_change":"2023-11-28T13:10:54.468707+0000","last_active":"2023-11-28T13:10:54.968281+0000","last_peered":"2023-11-28T13:10:54.968281+0000","last_clean":"2023-11-28T13:10:54.968281+0000","last_became_active":"2023-11-28T13:10:54.468432+0000","last_became_peered":"2023-11-28T13:10:54.468432+0000","last_unstale":"2023-11-28T13:10:54.968281+0000","last_undegraded":"2023-11-28T13:10:54.968281+0000","last_fullsized":"2023-11-28T13:10:54.968281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:58.935101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.971814+0000","last_change":"2023-11-28T13:10:54.968294+0000","last_active":"2023-11-28T13:10:54.971814+0000","last_peered":"2023-11-28T13:10:54.971814+0000","last_clean":"2023-11-28T13:10:54.971814+0000","last_became_active":"2023-11-28T13:10:54.966924+0000","last_became_peered":"2023-11-28T13:10:54.966924+0000","last_unstale":"2023-11-28T13:10:54.971814+0000","last_undegraded":"2023-11-28T13:10:54.971814+0000","last_fullsized":"2023-11-28T13:10:54.971814+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:30.770582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.958710+0000","last_change":"2023-11-28T13:10:54.465578+0000","last_active":"2023-11-28T13:10:54.958710+0000","last_peered":"2023-11-28T13:10:54.958710+0000","last_clean":"2023-11-28T13:10:54.958710+0000","last_became_active":"2023-11-28T13:10:54.465318+0000","last_became_peered":"2023-11-28T13:10:54.465318+0000","last_unstale":"2023-11-28T13:10:54.958710+0000","last_undegraded":"2023-11-28T13:10:54.958710+0000","last_fullsized":"2023-11-28T13:10:54.958710+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:04.221414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.953725+0000","last_change":"2023-11-28T13:10:54.948522+0000","last_active":"2023-11-28T13:10:54.953725+0000","last_peered":"2023-11-28T13:10:54.953725+0000","last_clean":"2023-11-28T13:10:54.953725+0000","last_became_active":"2023-11-28T13:10:54.946138+0000","last_became_peered":"2023-11-28T13:10:54.946138+0000","last_unstale":"2023-11-28T13:10:54.953725+0000","last_undegraded":"2023-11-28T13:10:54.953725+0000","last_fullsized":"2023-11-28T13:10:54.953725+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:55.169875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.957505+0000","last_change":"2023-11-28T13:10:54.461525+0000","last_active":"2023-11-28T13:10:54.957505+0000","last_peered":"2023-11-28T13:10:54.957505+0000","last_clean":"2023-11-28T13:10:54.957505+0000","last_became_active":"2023-11-28T13:10:54.461118+0000","last_became_peered":"2023-11-28T13:10:54.461118+0000","last_unstale":"2023-11-28T13:10:54.957505+0000","last_undegraded":"2023-11-28T13:10:54.957505+0000","last_fullsized":"2023-11-28T13:10:54.957505+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:27:34.442096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.967805+0000","last_change":"2023-11-28T13:10:54.965654+0000","last_active":"2023-11-28T13:10:54.967805+0000","last_peered":"2023-11-28T13:10:54.967805+0000","last_clean":"2023-11-28T13:10:54.967805+0000","last_became_active":"2023-11-28T13:10:54.962899+0000","last_became_peered":"2023-11-28T13:10:54.962899+0000","last_unstale":"2023-11-28T13:10:54.967805+0000","last_undegraded":"2023-11-28T13:10:54.967805+0000","last_fullsized":"2023-11-28T13:10:54.967805+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:38.720594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.963160+0000","last_change":"2023-11-28T13:10:54.456716+0000","last_active":"2023-11-28T13:10:54.963160+0000","last_peered":"2023-11-28T13:10:54.963160+0000","last_clean":"2023-11-28T13:10:54.963160+0000","last_became_active":"2023-11-28T13:10:54.456446+0000","last_became_peered":"2023-11-28T13:10:54.456446+0000","last_unstale":"2023-11-28T13:10:54.963160+0000","last_undegraded":"2023-11-28T13:10:54.963160+0000","last_fullsized":"2023-11-28T13:10:54.963160+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:09:42.866301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.961976+0000","last_change":"2023-11-28T13:10:54.960029+0000","last_active":"2023-11-28T13:10:54.961976+0000","last_peered":"2023-11-28T13:10:54.961976+0000","last_clean":"2023-11-28T13:10:54.961976+0000","last_became_active":"2023-11-28T13:10:54.959749+0000","last_became_peered":"2023-11-28T13:10:54.959749+0000","last_unstale":"2023-11-28T13:10:54.961976+0000","last_undegraded":"2023-11-28T13:10:54.961976+0000","last_fullsized":"2023-11-28T13:10:54.961976+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:45.692752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.950722+0000","last_change":"2023-11-28T13:10:54.453537+0000","last_active":"2023-11-28T13:10:54.950722+0000","last_peered":"2023-11-28T13:10:54.950722+0000","last_clean":"2023-11-28T13:10:54.950722+0000","last_became_active":"2023-11-28T13:10:54.453242+0000","last_became_peered":"2023-11-28T13:10:54.453242+0000","last_unstale":"2023-11-28T13:10:54.950722+0000","last_undegraded":"2023-11-28T13:10:54.950722+0000","last_fullsized":"2023-11-28T13:10:54.950722+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:04.221414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.959336+0000","last_change":"2023-11-28T13:10:54.956498+0000","last_active":"2023-11-28T13:10:54.959336+0000","last_peered":"2023-11-28T13:10:54.959336+0000","last_clean":"2023-11-28T13:10:54.959336+0000","last_became_active":"2023-11-28T13:10:54.956142+0000","last_became_peered":"2023-11-28T13:10:54.956142+0000","last_unstale":"2023-11-28T13:10:54.959336+0000","last_undegraded":"2023-11-28T13:10:54.959336+0000","last_fullsized":"2023-11-28T13:10:54.959336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:05.580755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.947733+0000","last_change":"2023-11-28T13:10:54.456732+0000","last_active":"2023-11-28T13:10:54.947733+0000","last_peered":"2023-11-28T13:10:54.947733+0000","last_clean":"2023-11-28T13:10:54.947733+0000","last_became_active":"2023-11-28T13:10:54.456467+0000","last_became_peered":"2023-11-28T13:10:54.456467+0000","last_unstale":"2023-11-28T13:10:54.947733+0000","last_undegraded":"2023-11-28T13:10:54.947733+0000","last_fullsized":"2023-11-28T13:10:54.947733+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:37.717634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.944816+0000","last_change":"2023-11-28T13:10:54.942792+0000","last_active":"2023-11-28T13:10:54.944816+0000","last_peered":"2023-11-28T13:10:54.944816+0000","last_clean":"2023-11-28T13:10:54.944816+0000","last_became_active":"2023-11-28T13:10:54.942375+0000","last_became_peered":"2023-11-28T13:10:54.942375+0000","last_unstale":"2023-11-28T13:10:54.944816+0000","last_undegraded":"2023-11-28T13:10:54.944816+0000","last_fullsized":"2023-11-28T13:10:54.944816+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:29:47.742190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.940822+0000","last_change":"2023-11-28T13:10:54.464086+0000","last_active":"2023-11-28T13:10:54.940822+0000","last_peered":"2023-11-28T13:10:54.940822+0000","last_clean":"2023-11-28T13:10:54.940822+0000","last_became_active":"2023-11-28T13:10:54.463634+0000","last_became_peered":"2023-11-28T13:10:54.463634+0000","last_unstale":"2023-11-28T13:10:54.940822+0000","last_undegraded":"2023-11-28T13:10:54.940822+0000","last_fullsized":"2023-11-28T13:10:54.940822+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:57:33.861682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.958496+0000","last_change":"2023-11-28T13:10:54.954775+0000","last_active":"2023-11-28T13:10:54.958496+0000","last_peered":"2023-11-28T13:10:54.958496+0000","last_clean":"2023-11-28T13:10:54.958496+0000","last_became_active":"2023-11-28T13:10:54.954312+0000","last_became_peered":"2023-11-28T13:10:54.954312+0000","last_unstale":"2023-11-28T13:10:54.958496+0000","last_undegraded":"2023-11-28T13:10:54.958496+0000","last_fullsized":"2023-11-28T13:10:54.958496+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:13.622196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.935241+0000","last_change":"2023-11-28T13:10:54.463702+0000","last_active":"2023-11-28T13:10:54.935241+0000","last_peered":"2023-11-28T13:10:54.935241+0000","last_clean":"2023-11-28T13:10:54.935241+0000","last_became_active":"2023-11-28T13:10:54.463260+0000","last_became_peered":"2023-11-28T13:10:54.463260+0000","last_unstale":"2023-11-28T13:10:54.935241+0000","last_undegraded":"2023-11-28T13:10:54.935241+0000","last_fullsized":"2023-11-28T13:10:54.935241+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:02.071198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966002+0000","last_change":"2023-11-28T13:10:54.962816+0000","last_active":"2023-11-28T13:10:54.966002+0000","last_peered":"2023-11-28T13:10:54.966002+0000","last_clean":"2023-11-28T13:10:54.966002+0000","last_became_active":"2023-11-28T13:10:54.962525+0000","last_became_peered":"2023-11-28T13:10:54.962525+0000","last_unstale":"2023-11-28T13:10:54.966002+0000","last_undegraded":"2023-11-28T13:10:54.966002+0000","last_fullsized":"2023-11-28T13:10:54.966002+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:56.343056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.944187+0000","last_change":"2023-11-28T13:10:54.469918+0000","last_active":"2023-11-28T13:10:54.944187+0000","last_peered":"2023-11-28T13:10:54.944187+0000","last_clean":"2023-11-28T13:10:54.944187+0000","last_became_active":"2023-11-28T13:10:54.469536+0000","last_became_peered":"2023-11-28T13:10:54.469536+0000","last_unstale":"2023-11-28T13:10:54.944187+0000","last_undegraded":"2023-11-28T13:10:54.944187+0000","last_fullsized":"2023-11-28T13:10:54.944187+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:25.483658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.950103+0000","last_change":"2023-11-28T13:10:54.949029+0000","last_active":"2023-11-28T13:10:54.950103+0000","last_peered":"2023-11-28T13:10:54.950103+0000","last_clean":"2023-11-28T13:10:54.950103+0000","last_became_active":"2023-11-28T13:10:54.948774+0000","last_became_peered":"2023-11-28T13:10:54.948774+0000","last_unstale":"2023-11-28T13:10:54.950103+0000","last_undegraded":"2023-11-28T13:10:54.950103+0000","last_fullsized":"2023-11-28T13:10:54.950103+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:16:24.754084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.961370+0000","last_change":"2023-11-28T13:10:54.453709+0000","last_active":"2023-11-28T13:10:54.961370+0000","last_peered":"2023-11-28T13:10:54.961370+0000","last_clean":"2023-11-28T13:10:54.961370+0000","last_became_active":"2023-11-28T13:10:54.453066+0000","last_became_peered":"2023-11-28T13:10:54.453066+0000","last_unstale":"2023-11-28T13:10:54.961370+0000","last_undegraded":"2023-11-28T13:10:54.961370+0000","last_fullsized":"2023-11-28T13:10:54.961370+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:10.970647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.955243+0000","last_change":"2023-11-28T13:10:54.948499+0000","last_active":"2023-11-28T13:10:54.955243+0000","last_peered":"2023-11-28T13:10:54.955243+0000","last_clean":"2023-11-28T13:10:54.955243+0000","last_became_active":"2023-11-28T13:10:54.948174+0000","last_became_peered":"2023-11-28T13:10:54.948174+0000","last_unstale":"2023-11-28T13:10:54.955243+0000","last_undegraded":"2023-11-28T13:10:54.955243+0000","last_fullsized":"2023-11-28T13:10:54.955243+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:05.580755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"58'3","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966338+0000","last_change":"2023-11-28T13:10:54.468809+0000","last_active":"2023-11-28T13:10:54.966338+0000","last_peered":"2023-11-28T13:10:54.966338+0000","last_clean":"2023-11-28T13:10:54.966338+0000","last_became_active":"2023-11-28T13:10:54.468426+0000","last_became_peered":"2023-11-28T13:10:54.468426+0000","last_unstale":"2023-11-28T13:10:54.966338+0000","last_undegraded":"2023-11-28T13:10:54.966338+0000","last_fullsized":"2023-11-28T13:10:54.966338+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:10.970647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.968749+0000","last_change":"2023-11-28T13:10:54.961768+0000","last_active":"2023-11-28T13:10:54.968749+0000","last_peered":"2023-11-28T13:10:54.968749+0000","last_clean":"2023-11-28T13:10:54.968749+0000","last_became_active":"2023-11-28T13:10:54.961375+0000","last_became_peered":"2023-11-28T13:10:54.961375+0000","last_unstale":"2023-11-28T13:10:54.968749+0000","last_undegraded":"2023-11-28T13:10:54.968749+0000","last_fullsized":"2023-11-28T13:10:54.968749+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:57:34.864642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"58'6","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.974518+0000","last_change":"2023-11-28T13:10:54.458857+0000","last_active":"2023-11-28T13:10:54.974518+0000","last_peered":"2023-11-28T13:10:54.974518+0000","last_clean":"2023-11-28T13:10:54.974518+0000","last_became_active":"2023-11-28T13:10:54.458581+0000","last_became_peered":"2023-11-28T13:10:54.458581+0000","last_unstale":"2023-11-28T13:10:54.974518+0000","last_undegraded":"2023-11-28T13:10:54.974518+0000","last_fullsized":"2023-11-28T13:10:54.974518+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:29:46.739230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.969567+0000","last_change":"2023-11-28T13:10:54.963935+0000","last_active":"2023-11-28T13:10:54.969567+0000","last_peered":"2023-11-28T13:10:54.969567+0000","last_clean":"2023-11-28T13:10:54.969567+0000","last_became_active":"2023-11-28T13:10:54.963638+0000","last_became_peered":"2023-11-28T13:10:54.963638+0000","last_unstale":"2023-11-28T13:10:54.969567+0000","last_undegraded":"2023-11-28T13:10:54.969567+0000","last_fullsized":"2023-11-28T13:10:54.969567+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:29.939656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.963539+0000","last_change":"2023-11-28T13:10:54.470122+0000","last_active":"2023-11-28T13:10:54.963539+0000","last_peered":"2023-11-28T13:10:54.963539+0000","last_clean":"2023-11-28T13:10:54.963539+0000","last_became_active":"2023-11-28T13:10:54.469267+0000","last_became_peered":"2023-11-28T13:10:54.469267+0000","last_unstale":"2023-11-28T13:10:54.963539+0000","last_undegraded":"2023-11-28T13:10:54.963539+0000","last_fullsized":"2023-11-28T13:10:54.963539+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:51.128657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.956058+0000","last_change":"2023-11-28T13:10:54.951065+0000","last_active":"2023-11-28T13:10:54.956058+0000","last_peered":"2023-11-28T13:10:54.956058+0000","last_clean":"2023-11-28T13:10:54.956058+0000","last_became_active":"2023-11-28T13:10:54.950729+0000","last_became_peered":"2023-11-28T13:10:54.950729+0000","last_unstale":"2023-11-28T13:10:54.956058+0000","last_undegraded":"2023-11-28T13:10:54.956058+0000","last_fullsized":"2023-11-28T13:10:54.956058+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:09:43.869262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.943565+0000","last_change":"2023-11-28T13:10:54.459460+0000","last_active":"2023-11-28T13:10:54.943565+0000","last_peered":"2023-11-28T13:10:54.943565+0000","last_clean":"2023-11-28T13:10:54.943565+0000","last_became_active":"2023-11-28T13:10:54.458402+0000","last_became_peered":"2023-11-28T13:10:54.458402+0000","last_unstale":"2023-11-28T13:10:54.943565+0000","last_undegraded":"2023-11-28T13:10:54.943565+0000","last_fullsized":"2023-11-28T13:10:54.943565+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:55.340096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.944789+0000","last_change":"2023-11-28T13:10:54.941321+0000","last_active":"2023-11-28T13:10:54.944789+0000","last_peered":"2023-11-28T13:10:54.944789+0000","last_clean":"2023-11-28T13:10:54.944789+0000","last_became_active":"2023-11-28T13:10:54.940728+0000","last_became_peered":"2023-11-28T13:10:54.940728+0000","last_unstale":"2023-11-28T13:10:54.944789+0000","last_undegraded":"2023-11-28T13:10:54.944789+0000","last_fullsized":"2023-11-28T13:10:54.944789+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:30.670841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.943204+0000","last_change":"2023-11-28T13:10:54.468803+0000","last_active":"2023-11-28T13:10:54.943204+0000","last_peered":"2023-11-28T13:10:54.943204+0000","last_clean":"2023-11-28T13:10:54.943204+0000","last_became_active":"2023-11-28T13:10:54.468475+0000","last_became_peered":"2023-11-28T13:10:54.468475+0000","last_unstale":"2023-11-28T13:10:54.943204+0000","last_undegraded":"2023-11-28T13:10:54.943204+0000","last_fullsized":"2023-11-28T13:10:54.943204+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:33:10.154448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.965643+0000","last_change":"2023-11-28T13:10:54.960919+0000","last_active":"2023-11-28T13:10:54.965643+0000","last_peered":"2023-11-28T13:10:54.965643+0000","last_clean":"2023-11-28T13:10:54.965643+0000","last_became_active":"2023-11-28T13:10:54.959814+0000","last_became_peered":"2023-11-28T13:10:54.959814+0000","last_unstale":"2023-11-28T13:10:54.965643+0000","last_undegraded":"2023-11-28T13:10:54.965643+0000","last_fullsized":"2023-11-28T13:10:54.965643+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:54.513040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"58'8","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.974404+0000","last_change":"2023-11-28T13:10:54.459509+0000","last_active":"2023-11-28T13:10:54.974404+0000","last_peered":"2023-11-28T13:10:54.974404+0000","last_clean":"2023-11-28T13:10:54.974404+0000","last_became_active":"2023-11-28T13:10:54.459232+0000","last_became_peered":"2023-11-28T13:10:54.459232+0000","last_unstale":"2023-11-28T13:10:54.974404+0000","last_undegraded":"2023-11-28T13:10:54.974404+0000","last_fullsized":"2023-11-28T13:10:54.974404+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:25.483658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.963207+0000","last_change":"2023-11-28T13:10:54.962276+0000","last_active":"2023-11-28T13:10:54.963207+0000","last_peered":"2023-11-28T13:10:54.963207+0000","last_clean":"2023-11-28T13:10:54.963207+0000","last_became_active":"2023-11-28T13:10:54.961826+0000","last_became_peered":"2023-11-28T13:10:54.961826+0000","last_unstale":"2023-11-28T13:10:54.963207+0000","last_undegraded":"2023-11-28T13:10:54.963207+0000","last_fullsized":"2023-11-28T13:10:54.963207+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:26:06.317970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.967015+0000","last_change":"2023-11-28T13:10:54.468898+0000","last_active":"2023-11-28T13:10:54.967015+0000","last_peered":"2023-11-28T13:10:54.967015+0000","last_clean":"2023-11-28T13:10:54.967015+0000","last_became_active":"2023-11-28T13:10:54.468498+0000","last_became_peered":"2023-11-28T13:10:54.468498+0000","last_unstale":"2023-11-28T13:10:54.967015+0000","last_undegraded":"2023-11-28T13:10:54.967015+0000","last_fullsized":"2023-11-28T13:10:54.967015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:53.510080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.954470+0000","last_change":"2023-11-28T13:10:54.949875+0000","last_active":"2023-11-28T13:10:54.954470+0000","last_peered":"2023-11-28T13:10:54.954470+0000","last_clean":"2023-11-28T13:10:54.954470+0000","last_became_active":"2023-11-28T13:10:54.949595+0000","last_became_peered":"2023-11-28T13:10:54.949595+0000","last_unstale":"2023-11-28T13:10:54.954470+0000","last_undegraded":"2023-11-28T13:10:54.954470+0000","last_fullsized":"2023-11-28T13:10:54.954470+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:03.074158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.944204+0000","last_change":"2023-11-28T13:10:54.458249+0000","last_active":"2023-11-28T13:10:54.944204+0000","last_peered":"2023-11-28T13:10:54.944204+0000","last_clean":"2023-11-28T13:10:54.944204+0000","last_became_active":"2023-11-28T13:10:54.457974+0000","last_became_peered":"2023-11-28T13:10:54.457974+0000","last_unstale":"2023-11-28T13:10:54.944204+0000","last_undegraded":"2023-11-28T13:10:54.944204+0000","last_fullsized":"2023-11-28T13:10:54.944204+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:27.141994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966547+0000","last_change":"2023-11-28T13:10:54.957391+0000","last_active":"2023-11-28T13:10:54.966547+0000","last_peered":"2023-11-28T13:10:54.966547+0000","last_clean":"2023-11-28T13:10:54.966547+0000","last_became_active":"2023-11-28T13:10:54.956682+0000","last_became_peered":"2023-11-28T13:10:54.956682+0000","last_unstale":"2023-11-28T13:10:54.966547+0000","last_undegraded":"2023-11-28T13:10:54.966547+0000","last_fullsized":"2023-11-28T13:10:54.966547+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:27:35.445056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.941399+0000","last_change":"2023-11-28T13:10:54.454857+0000","last_active":"2023-11-28T13:10:54.941399+0000","last_peered":"2023-11-28T13:10:54.941399+0000","last_clean":"2023-11-28T13:10:54.941399+0000","last_became_active":"2023-11-28T13:10:54.454594+0000","last_became_peered":"2023-11-28T13:10:54.454594+0000","last_unstale":"2023-11-28T13:10:54.941399+0000","last_undegraded":"2023-11-28T13:10:54.941399+0000","last_fullsized":"2023-11-28T13:10:54.941399+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:37.717634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.954847+0000","last_change":"2023-11-28T13:10:54.947275+0000","last_active":"2023-11-28T13:10:54.954847+0000","last_peered":"2023-11-28T13:10:54.954847+0000","last_clean":"2023-11-28T13:10:54.954847+0000","last_became_active":"2023-11-28T13:10:54.947011+0000","last_became_peered":"2023-11-28T13:10:54.947011+0000","last_unstale":"2023-11-28T13:10:54.954847+0000","last_undegraded":"2023-11-28T13:10:54.954847+0000","last_fullsized":"2023-11-28T13:10:54.954847+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:33:11.157408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.955529+0000","last_change":"2023-11-28T13:10:54.457853+0000","last_active":"2023-11-28T13:10:54.955529+0000","last_peered":"2023-11-28T13:10:54.955529+0000","last_clean":"2023-11-28T13:10:54.955529+0000","last_became_active":"2023-11-28T13:10:54.457562+0000","last_became_peered":"2023-11-28T13:10:54.457562+0000","last_unstale":"2023-11-28T13:10:54.955529+0000","last_undegraded":"2023-11-28T13:10:54.955529+0000","last_fullsized":"2023-11-28T13:10:54.955529+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:24:31.630491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.964185+0000","last_change":"2023-11-28T13:10:54.959743+0000","last_active":"2023-11-28T13:10:54.964185+0000","last_peered":"2023-11-28T13:10:54.964185+0000","last_clean":"2023-11-28T13:10:54.964185+0000","last_became_active":"2023-11-28T13:10:54.959458+0000","last_became_peered":"2023-11-28T13:10:54.959458+0000","last_unstale":"2023-11-28T13:10:54.964185+0000","last_undegraded":"2023-11-28T13:10:54.964185+0000","last_fullsized":"2023-11-28T13:10:54.964185+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:16.780414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.942455+0000","last_change":"2023-11-28T13:10:54.469964+0000","last_active":"2023-11-28T13:10:54.942455+0000","last_peered":"2023-11-28T13:10:54.942455+0000","last_clean":"2023-11-28T13:10:54.942455+0000","last_became_active":"2023-11-28T13:10:54.469510+0000","last_became_peered":"2023-11-28T13:10:54.469510+0000","last_unstale":"2023-11-28T13:10:54.942455+0000","last_undegraded":"2023-11-28T13:10:54.942455+0000","last_fullsized":"2023-11-28T13:10:54.942455+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:00:25.660715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.969983+0000","last_change":"2023-11-28T13:10:54.966114+0000","last_active":"2023-11-28T13:10:54.969983+0000","last_peered":"2023-11-28T13:10:54.969983+0000","last_clean":"2023-11-28T13:10:54.969983+0000","last_became_active":"2023-11-28T13:10:54.965818+0000","last_became_peered":"2023-11-28T13:10:54.965818+0000","last_unstale":"2023-11-28T13:10:54.969983+0000","last_undegraded":"2023-11-28T13:10:54.969983+0000","last_fullsized":"2023-11-28T13:10:54.969983+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:38.720594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.941378+0000","last_change":"2023-11-28T13:10:54.450412+0000","last_active":"2023-11-28T13:10:54.941378+0000","last_peered":"2023-11-28T13:10:54.941378+0000","last_clean":"2023-11-28T13:10:54.941378+0000","last_became_active":"2023-11-28T13:10:54.449928+0000","last_became_peered":"2023-11-28T13:10:54.449928+0000","last_unstale":"2023-11-28T13:10:54.941378+0000","last_undegraded":"2023-11-28T13:10:54.941378+0000","last_fullsized":"2023-11-28T13:10:54.941378+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:39.518398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.959390+0000","last_change":"2023-11-28T13:10:54.957416+0000","last_active":"2023-11-28T13:10:54.959390+0000","last_peered":"2023-11-28T13:10:54.959390+0000","last_clean":"2023-11-28T13:10:54.959390+0000","last_became_active":"2023-11-28T13:10:54.957130+0000","last_became_peered":"2023-11-28T13:10:54.957130+0000","last_unstale":"2023-11-28T13:10:54.959390+0000","last_undegraded":"2023-11-28T13:10:54.959390+0000","last_fullsized":"2023-11-28T13:10:54.959390+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:24:18.289975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.940921+0000","last_change":"2023-11-28T13:10:54.461128+0000","last_active":"2023-11-28T13:10:54.940921+0000","last_peered":"2023-11-28T13:10:54.940921+0000","last_clean":"2023-11-28T13:10:54.940921+0000","last_became_active":"2023-11-28T13:10:54.460844+0000","last_became_peered":"2023-11-28T13:10:54.460844+0000","last_unstale":"2023-11-28T13:10:54.940921+0000","last_undegraded":"2023-11-28T13:10:54.940921+0000","last_fullsized":"2023-11-28T13:10:54.940921+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:50.735905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.954584+0000","last_change":"2023-11-28T13:10:54.949080+0000","last_active":"2023-11-28T13:10:54.954584+0000","last_peered":"2023-11-28T13:10:54.954584+0000","last_clean":"2023-11-28T13:10:54.954584+0000","last_became_active":"2023-11-28T13:10:54.948824+0000","last_became_peered":"2023-11-28T13:10:54.948824+0000","last_unstale":"2023-11-28T13:10:54.954584+0000","last_undegraded":"2023-11-28T13:10:54.954584+0000","last_fullsized":"2023-11-28T13:10:54.954584+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:55:52.451792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.949499+0000","last_change":"2023-11-28T13:10:54.462528+0000","last_active":"2023-11-28T13:10:54.949499+0000","last_peered":"2023-11-28T13:10:54.949499+0000","last_clean":"2023-11-28T13:10:54.949499+0000","last_became_active":"2023-11-28T13:10:54.462211+0000","last_became_peered":"2023-11-28T13:10:54.462211+0000","last_unstale":"2023-11-28T13:10:54.949499+0000","last_undegraded":"2023-11-28T13:10:54.949499+0000","last_fullsized":"2023-11-28T13:10:54.949499+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:19.933480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.968765+0000","last_change":"2023-11-28T13:10:54.961500+0000","last_active":"2023-11-28T13:10:54.968765+0000","last_peered":"2023-11-28T13:10:54.968765+0000","last_clean":"2023-11-28T13:10:54.968765+0000","last_became_active":"2023-11-28T13:10:54.961033+0000","last_became_peered":"2023-11-28T13:10:54.961033+0000","last_unstale":"2023-11-28T13:10:54.968765+0000","last_undegraded":"2023-11-28T13:10:54.968765+0000","last_fullsized":"2023-11-28T13:10:54.968765+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:08.361205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966100+0000","last_change":"2023-11-28T13:10:54.469511+0000","last_active":"2023-11-28T13:10:54.966100+0000","last_peered":"2023-11-28T13:10:54.966100+0000","last_clean":"2023-11-28T13:10:54.966100+0000","last_became_active":"2023-11-28T13:10:54.469187+0000","last_became_peered":"2023-11-28T13:10:54.469187+0000","last_unstale":"2023-11-28T13:10:54.966100+0000","last_undegraded":"2023-11-28T13:10:54.966100+0000","last_fullsized":"2023-11-28T13:10:54.966100+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:15.777454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":49,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.449088+0000","last_change":"2023-11-28T13:07:56.890378+0000","last_active":"2023-11-28T13:10:54.449088+0000","last_peered":"2023-11-28T13:10:54.449088+0000","last_clean":"2023-11-28T13:10:54.449088+0000","last_became_active":"2023-11-28T13:07:56.890058+0000","last_became_peered":"2023-11-28T13:07:56.890058+0000","last_unstale":"2023-11-28T13:10:54.449088+0000","last_undegraded":"2023-11-28T13:10:54.449088+0000","last_fullsized":"2023-11-28T13:10:54.449088+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:05.344602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.953664+0000","last_change":"2023-11-28T13:10:54.945727+0000","last_active":"2023-11-28T13:10:54.953664+0000","last_peered":"2023-11-28T13:10:54.953664+0000","last_clean":"2023-11-28T13:10:54.953664+0000","last_became_active":"2023-11-28T13:10:54.945234+0000","last_became_peered":"2023-11-28T13:10:54.945234+0000","last_unstale":"2023-11-28T13:10:54.953664+0000","last_undegraded":"2023-11-28T13:10:54.953664+0000","last_fullsized":"2023-11-28T13:10:54.953664+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:00:26.663675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.941403+0000","last_change":"2023-11-28T13:10:54.454063+0000","last_active":"2023-11-28T13:10:54.941403+0000","last_peered":"2023-11-28T13:10:54.941403+0000","last_clean":"2023-11-28T13:10:54.941403+0000","last_became_active":"2023-11-28T13:10:54.452898+0000","last_became_peered":"2023-11-28T13:10:54.452898+0000","last_unstale":"2023-11-28T13:10:54.941403+0000","last_undegraded":"2023-11-28T13:10:54.941403+0000","last_fullsized":"2023-11-28T13:10:54.941403+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:28.936696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":48,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.440220+0000","last_change":"2023-11-28T13:07:56.844509+0000","last_active":"2023-11-28T13:10:54.440220+0000","last_peered":"2023-11-28T13:10:54.440220+0000","last_clean":"2023-11-28T13:10:54.440220+0000","last_became_active":"2023-11-28T13:07:56.844081+0000","last_became_peered":"2023-11-28T13:07:56.844081+0000","last_unstale":"2023-11-28T13:10:54.440220+0000","last_undegraded":"2023-11-28T13:10:54.440220+0000","last_fullsized":"2023-11-28T13:10:54.440220+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.948086+0000","last_change":"2023-11-28T13:10:54.942502+0000","last_active":"2023-11-28T13:10:54.948086+0000","last_peered":"2023-11-28T13:10:54.948086+0000","last_clean":"2023-11-28T13:10:54.948086+0000","last_became_active":"2023-11-28T13:10:54.942244+0000","last_became_peered":"2023-11-28T13:10:54.942244+0000","last_unstale":"2023-11-28T13:10:54.948086+0000","last_undegraded":"2023-11-28T13:10:54.948086+0000","last_fullsized":"2023-11-28T13:10:54.948086+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:05:56.985098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.963615+0000","last_change":"2023-11-28T13:10:54.463674+0000","last_active":"2023-11-28T13:10:54.963615+0000","last_peered":"2023-11-28T13:10:54.963615+0000","last_clean":"2023-11-28T13:10:54.963615+0000","last_became_active":"2023-11-28T13:10:54.463196+0000","last_became_peered":"2023-11-28T13:10:54.463196+0000","last_unstale":"2023-11-28T13:10:54.963615+0000","last_undegraded":"2023-11-28T13:10:54.963615+0000","last_fullsized":"2023-11-28T13:10:54.963615+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:28.936696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":46,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.446832+0000","last_change":"2023-11-28T13:07:56.844699+0000","last_active":"2023-11-28T13:10:54.446832+0000","last_peered":"2023-11-28T13:10:54.446832+0000","last_clean":"2023-11-28T13:10:54.446832+0000","last_became_active":"2023-11-28T13:07:56.844391+0000","last_became_peered":"2023-11-28T13:07:56.844391+0000","last_unstale":"2023-11-28T13:10:54.446832+0000","last_undegraded":"2023-11-28T13:10:54.446832+0000","last_fullsized":"2023-11-28T13:10:54.446832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":197,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.461726+0000","last_change":"2023-11-28T13:07:57.844020+0000","last_active":"2023-11-28T13:10:54.461726+0000","last_peered":"2023-11-28T13:10:54.461726+0000","last_clean":"2023-11-28T13:10:54.461726+0000","last_became_active":"2023-11-28T13:07:57.842981+0000","last_became_peered":"2023-11-28T13:07:57.842981+0000","last_unstale":"2023-11-28T13:10:54.461726+0000","last_undegraded":"2023-11-28T13:10:54.461726+0000","last_fullsized":"2023-11-28T13:10:54.461726+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_clean_scrub_stamp":"2023-11-28T13:07:56.825711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:44.973218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.950294+0000","last_change":"2023-11-28T13:10:54.946601+0000","last_active":"2023-11-28T13:10:54.950294+0000","last_peered":"2023-11-28T13:10:54.950294+0000","last_clean":"2023-11-28T13:10:54.950294+0000","last_became_active":"2023-11-28T13:10:54.944813+0000","last_became_peered":"2023-11-28T13:10:54.944813+0000","last_unstale":"2023-11-28T13:10:54.950294+0000","last_undegraded":"2023-11-28T13:10:54.950294+0000","last_fullsized":"2023-11-28T13:10:54.950294+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:05:16.097414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.943668+0000","last_change":"2023-11-28T13:10:54.454273+0000","last_active":"2023-11-28T13:10:54.943668+0000","last_peered":"2023-11-28T13:10:54.943668+0000","last_clean":"2023-11-28T13:10:54.943668+0000","last_became_active":"2023-11-28T13:10:54.453694+0000","last_became_peered":"2023-11-28T13:10:54.453694+0000","last_unstale":"2023-11-28T13:10:54.943668+0000","last_undegraded":"2023-11-28T13:10:54.943668+0000","last_fullsized":"2023-11-28T13:10:54.943668+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:27:18.500496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":48,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.461847+0000","last_change":"2023-11-28T13:07:57.306318+0000","last_active":"2023-11-28T13:10:54.461847+0000","last_peered":"2023-11-28T13:10:54.461847+0000","last_clean":"2023-11-28T13:10:54.461847+0000","last_became_active":"2023-11-28T13:07:57.305937+0000","last_became_peered":"2023-11-28T13:07:57.305937+0000","last_unstale":"2023-11-28T13:10:54.461847+0000","last_undegraded":"2023-11-28T13:10:54.461847+0000","last_fullsized":"2023-11-28T13:10:54.461847+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.963349+0000","last_change":"2023-11-28T13:10:54.959871+0000","last_active":"2023-11-28T13:10:54.963349+0000","last_peered":"2023-11-28T13:10:54.963349+0000","last_clean":"2023-11-28T13:10:54.963349+0000","last_became_active":"2023-11-28T13:10:54.959569+0000","last_became_peered":"2023-11-28T13:10:54.959569+0000","last_unstale":"2023-11-28T13:10:54.963349+0000","last_undegraded":"2023-11-28T13:10:54.963349+0000","last_fullsized":"2023-11-28T13:10:54.963349+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:42:26.200782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.939896+0000","last_change":"2023-11-28T13:10:54.454973+0000","last_active":"2023-11-28T13:10:54.939896+0000","last_peered":"2023-11-28T13:10:54.939896+0000","last_clean":"2023-11-28T13:10:54.939896+0000","last_became_active":"2023-11-28T13:10:54.454692+0000","last_became_peered":"2023-11-28T13:10:54.454692+0000","last_unstale":"2023-11-28T13:10:54.939896+0000","last_undegraded":"2023-11-28T13:10:54.939896+0000","last_fullsized":"2023-11-28T13:10:54.939896+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:27.171232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":48,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.462057+0000","last_change":"2023-11-28T13:07:57.306277+0000","last_active":"2023-11-28T13:10:54.462057+0000","last_peered":"2023-11-28T13:10:54.462057+0000","last_clean":"2023-11-28T13:10:54.462057+0000","last_became_active":"2023-11-28T13:07:57.305983+0000","last_became_peered":"2023-11-28T13:07:57.305983+0000","last_unstale":"2023-11-28T13:10:54.462057+0000","last_undegraded":"2023-11-28T13:10:54.462057+0000","last_fullsized":"2023-11-28T13:10:54.462057+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.953442+0000","last_change":"2023-11-28T13:10:54.945967+0000","last_active":"2023-11-28T13:10:54.953442+0000","last_peered":"2023-11-28T13:10:54.953442+0000","last_clean":"2023-11-28T13:10:54.953442+0000","last_became_active":"2023-11-28T13:10:54.945538+0000","last_became_peered":"2023-11-28T13:10:54.945538+0000","last_unstale":"2023-11-28T13:10:54.953442+0000","last_undegraded":"2023-11-28T13:10:54.953442+0000","last_fullsized":"2023-11-28T13:10:54.953442+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:57:52.631897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.940219+0000","last_change":"2023-11-28T13:10:54.460189+0000","last_active":"2023-11-28T13:10:54.940219+0000","last_peered":"2023-11-28T13:10:54.940219+0000","last_clean":"2023-11-28T13:10:54.940219+0000","last_became_active":"2023-11-28T13:10:54.459402+0000","last_became_peered":"2023-11-28T13:10:54.459402+0000","last_unstale":"2023-11-28T13:10:54.940219+0000","last_undegraded":"2023-11-28T13:10:54.940219+0000","last_fullsized":"2023-11-28T13:10:54.940219+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:16:27.171232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":53,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.446920+0000","last_change":"2023-11-28T13:07:57.306177+0000","last_active":"2023-11-28T13:10:54.446920+0000","last_peered":"2023-11-28T13:10:54.446920+0000","last_clean":"2023-11-28T13:10:54.446920+0000","last_became_active":"2023-11-28T13:07:57.305853+0000","last_became_peered":"2023-11-28T13:07:57.305853+0000","last_unstale":"2023-11-28T13:10:54.446920+0000","last_undegraded":"2023-11-28T13:10:54.446920+0000","last_fullsized":"2023-11-28T13:10:54.446920+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.967085+0000","last_change":"2023-11-28T13:10:54.961438+0000","last_active":"2023-11-28T13:10:54.967085+0000","last_peered":"2023-11-28T13:10:54.967085+0000","last_clean":"2023-11-28T13:10:54.967085+0000","last_became_active":"2023-11-28T13:10:54.961162+0000","last_became_peered":"2023-11-28T13:10:54.961162+0000","last_unstale":"2023-11-28T13:10:54.967085+0000","last_undegraded":"2023-11-28T13:10:54.967085+0000","last_fullsized":"2023-11-28T13:10:54.967085+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:11.973607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.958467+0000","last_change":"2023-11-28T13:10:54.468802+0000","last_active":"2023-11-28T13:10:54.958467+0000","last_peered":"2023-11-28T13:10:54.958467+0000","last_clean":"2023-11-28T13:10:54.958467+0000","last_became_active":"2023-11-28T13:10:54.468478+0000","last_became_peered":"2023-11-28T13:10:54.468478+0000","last_unstale":"2023-11-28T13:10:54.958467+0000","last_undegraded":"2023-11-28T13:10:54.958467+0000","last_fullsized":"2023-11-28T13:10:54.958467+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:13:04.302760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":48,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.458770+0000","last_change":"2023-11-28T13:07:57.306321+0000","last_active":"2023-11-28T13:10:54.458770+0000","last_peered":"2023-11-28T13:10:54.458770+0000","last_clean":"2023-11-28T13:10:54.458770+0000","last_became_active":"2023-11-28T13:07:57.305923+0000","last_became_peered":"2023-11-28T13:07:57.305923+0000","last_unstale":"2023-11-28T13:10:54.458770+0000","last_undegraded":"2023-11-28T13:10:54.458770+0000","last_fullsized":"2023-11-28T13:10:54.458770+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:41.104679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.966197+0000","last_change":"2023-11-28T13:10:54.959336+0000","last_active":"2023-11-28T13:10:54.966197+0000","last_peered":"2023-11-28T13:10:54.966197+0000","last_clean":"2023-11-28T13:10:54.966197+0000","last_became_active":"2023-11-28T13:10:54.959004+0000","last_became_peered":"2023-11-28T13:10:54.959004+0000","last_unstale":"2023-11-28T13:10:54.966197+0000","last_undegraded":"2023-11-28T13:10:54.966197+0000","last_fullsized":"2023-11-28T13:10:54.966197+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:54.432660+0000","last_clean_scrub_stamp":"2023-11-28T13:10:54.432660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:54.513040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.964204+0000","last_change":"2023-11-28T13:10:54.453286+0000","last_active":"2023-11-28T13:10:54.964204+0000","last_peered":"2023-11-28T13:10:54.964204+0000","last_clean":"2023-11-28T13:10:54.964204+0000","last_became_active":"2023-11-28T13:10:54.452994+0000","last_became_peered":"2023-11-28T13:10:54.452994+0000","last_unstale":"2023-11-28T13:10:54.964204+0000","last_undegraded":"2023-11-28T13:10:54.964204+0000","last_fullsized":"2023-11-28T13:10:54.964204+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:10:53.429700+0000","last_clean_scrub_stamp":"2023-11-28T13:10:53.429700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:55.490929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":48,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-28T13:10:54.459102+0000","last_change":"2023-11-28T13:07:56.844305+0000","last_active":"2023-11-28T13:10:54.459102+0000","last_peered":"2023-11-28T13:10:54.459102+0000","last_clean":"2023-11-28T13:10:54.459102+0000","last_became_active":"2023-11-28T13:07:56.843980+0000","last_became_peered":"2023-11-28T13:07:56.843980+0000","last_unstale":"2023-11-28T13:10:54.459102+0000","last_undegraded":"2023-11-28T13:10:54.459102+0000","last_fullsized":"2023-11-28T13:10:54.459102+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14696,"kb_used_data":1248,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94357144,"statfs":{"total":96636764160,"available":96621715456,"internally_reserved":0,"allocated":1277952,"data_stored":887322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66200000000000003,"15min":0.65900000000000003},"min":{"1min":0.46700000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90300000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.49099999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88100000000000001,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.433}]},{"osd":1,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71899999999999997,"15min":0.71699999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.91900000000000004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69999999999999996,"15min":0.70099999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93600000000000005,"5min":1.095,"15min":1.095},"last":0.54100000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.52600000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91700000000000004,"5min":0.996,"15min":0.996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.86899999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.82599999999999996}]},{"osd":3,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.56699999999999995,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85399999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.627},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76900000000000002,"15min":0.77700000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.996,"5min":1.079,"15min":1.079},"last":0.88600000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.69699999999999995,"15min":0.71099999999999997},"min":{"1min":0.44600000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89300000000000002,"5min":1.008,"15min":1.008},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90100000000000002,"5min":1.071,"15min":1.071},"last":0.70399999999999996}]},{"osd":5,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77200000000000002,"15min":0.78500000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.044,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.80300000000000005,"15min":0.81599999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.97799999999999998,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.73499999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.84099999999999997,"15min":0.85399999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97799999999999998,"5min":1.165,"15min":1.165},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.78900000000000003,"15min":0.80000000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94899999999999995,"5min":1.141,"15min":1.141},"last":0.84699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9112,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362728,"statfs":{"total":96636764160,"available":96627433472,"internally_reserved":0,"allocated":671744,"data_stored":290334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.46700000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.64600000000000002,"15min":0.64300000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.91900000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.45900000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67700000000000005,"15min":0.67600000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95899999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65200000000000002,"15min":0.65300000000000002},"min":{"1min":0.48499999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67300000000000004,"15min":0.67100000000000004},"min":{"1min":0.52200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.48199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.56200000000000006}]},{"osd":3,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.48499999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.94799999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51200000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0109999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.67300000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67600000000000005,"15min":0.67900000000000005},"min":{"1min":0.40100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88200000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.81499999999999995,"15min":0.82099999999999995},"min":{"1min":0.621,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.96599999999999997,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.76300000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"min":{"1min":0.55100000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.877,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.71899999999999997,"15min":0.72499999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92500000000000004,"5min":0.996,"15min":0.996},"last":0.74199999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.73899999999999999,"15min":0.74199999999999999},"min":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.93999999999999995,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.59299999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94099999999999995,"5min":1.107,"15min":1.107},"last":0.78500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14364,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357476,"statfs":{"total":96636764160,"available":96622055424,"internally_reserved":0,"allocated":1265664,"data_stored":880702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:10:00 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.436,"15min":0.438},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0549999999999999,"5min":1.28,"15min":1.28},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.44,"5min":0.44400000000000001,"15min":0.44500000000000001},"min":{"1min":0.29799999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61399999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.46700000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:10:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67300000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.38200000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:10:00 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54100000000000004,"15min":0.54700000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83999999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.54100000000000004,"15min":0.55000000000000004},"min":{"1min":0.37,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68999999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.53800000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:10:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.56399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70999999999999996,"15min":0.70599999999999996},"min":{"1min":0.52100000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.60499999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:10:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.51600000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.94499999999999995,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64900000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:10:00 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.73199999999999998,"15min":0.72999999999999998},"min":{"1min":0.54400000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.997,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.77600000000000002,"15min":0.78000000000000003},"min":{"1min":0.56499999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0860000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.85999999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:10:00 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81200000000000006,"15min":0.81599999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1000000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.78900000000000003,"15min":0.78700000000000003},"min":{"1min":0.61299999999999999,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0860000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56299999999999994}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13460,"kb_used_data":652,"kb_used_omap":0,"kb_used_meta":12800,"kb_avail":94358380,"statfs":{"total":96636764160,"available":96622981120,"internally_reserved":0,"allocated":667648,"data_stored":289348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13107200},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:10:03 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.49199999999999999,"15min":0.504},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68400000000000005,"5min":1.48,"15min":1.48},"last":0.503}]},{"osd":2,"last update":"Tue Nov 28 13:10:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.59799999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.503,"15min":0.50700000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.436}]},{"osd":3,"last update":"Tue Nov 28 13:10:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47299999999999998,"15min":0.47399999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70599999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.59399999999999997}]},{"osd":4,"last update":"Tue Nov 28 13:10:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.78300000000000003},"min":{"1min":0.625,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.85799999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:10:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.78900000000000003,"15min":0.79800000000000004},"min":{"1min":0.51700000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97099999999999997,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79000000000000004,"15min":0.79100000000000004},"min":{"1min":0.65400000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91000000000000003,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.66700000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:10:03 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.754,"15min":0.755},"min":{"1min":0.54600000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.81699999999999995,"15min":0.82099999999999995},"min":{"1min":0.623,"5min":0.62,"15min":0.62},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.81499999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:10:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.58799999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.96199999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.80000000000000004,"15min":0.80600000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.73199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13336,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":12672,"kb_avail":94358504,"statfs":{"total":96636764160,"available":96623108096,"internally_reserved":0,"allocated":671744,"data_stored":290334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12976128},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42599999999999999,"15min":0.42799999999999999},"min":{"1min":0.28899999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64700000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.314},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46200000000000002,"15min":0.46100000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66600000000000004,"5min":0.75,"15min":0.75},"last":0.52200000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32700000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69199999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51000000000000001,"15min":0.51300000000000001},"min":{"1min":0.375,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.64300000000000002,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.434}]},{"osd":3,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.36099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.60299999999999998,"5min":0.75,"15min":0.75},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.503}]},{"osd":4,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.67600000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0149999999999999,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.087,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.63500000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.044,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.58999999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96399999999999997,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.54600000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.75600000000000001,"15min":0.746},"min":{"1min":0.56100000000000005,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.03,"5min":1.083,"15min":1.083},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79400000000000004,"15min":0.79200000000000004},"min":{"1min":0.61099999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.071,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.55800000000000005}]},{"osd":7,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.76100000000000001,"15min":0.75900000000000001},"min":{"1min":0.56399999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.99299999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80200000000000005,"15min":0.80000000000000004},"min":{"1min":0.58699999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.998,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.58599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13656,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":12992,"kb_avail":94358184,"statfs":{"total":96636764160,"available":96622780416,"internally_reserved":0,"allocated":671744,"data_stored":290334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13303808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:09:57 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.41699999999999998,"15min":0.42099999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.44400000000000001,"15min":0.44500000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.55600000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.35399999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:09:57 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.47899999999999998,"15min":0.49099999999999999},"min":{"1min":0.317,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.59499999999999997,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.66000000000000003,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.30499999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:09:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.67600000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.39000000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.40799999999999997}]},{"osd":4,"last update":"Tue Nov 28 13:09:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69699999999999995,"15min":0.69599999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.93799999999999994,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.67200000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:09:57 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.76700000000000002,"15min":0.77000000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0049999999999999,"5min":1.091,"15min":1.091},"last":0.623},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.75700000000000001,"15min":0.76200000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.94899999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.50600000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:09:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.71999999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70099999999999996,"15min":0.69899999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:09:57 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.754,"15min":0.755},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0569999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.70899999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9444,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362396,"statfs":{"total":96636764160,"available":96627093504,"internally_reserved":0,"allocated":684032,"data_stored":296954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.42799999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.876},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66800000000000004,"15min":0.67000000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85999999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.58899999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67600000000000005,"15min":0.67900000000000005},"min":{"1min":0.50600000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.87,"5min":1.161,"15min":1.161},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60399999999999998,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.624}]},{"osd":2,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.64900000000000002},"min":{"1min":0.49299999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.441,"15min":0.441},"max":{"1min":0.84099999999999997,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.50800000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.86399999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.89500000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54800000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67100000000000004,"15min":0.67900000000000005},"min":{"1min":0.41799999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88200000000000001,"5min":1.264,"15min":1.264},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.72499999999999998,"15min":0.73499999999999999},"min":{"1min":0.39900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90300000000000002,"5min":1.079,"15min":1.079},"last":0.77700000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.72899999999999998,"15min":0.73499999999999999},"min":{"1min":0.47299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.93899999999999995,"5min":1.022,"15min":1.022},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.74399999999999999,"15min":0.748},"min":{"1min":0.48499999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92400000000000004,"5min":1.22,"15min":1.22},"last":0.82499999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.76500000000000001,"15min":0.77500000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92700000000000005,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78300000000000003,"15min":0.78700000000000003},"min":{"1min":0.52000000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.96599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9368,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362472,"statfs":{"total":96636764160,"available":96627171328,"internally_reserved":0,"allocated":671744,"data_stored":290334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.61599999999999999,"15min":0.59899999999999998},"min":{"1min":0.46200000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.61399999999999999,"15min":0.59999999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.76000000000000001}]},{"osd":1,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.63500000000000001,"15min":0.61399999999999999},"min":{"1min":0.443,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.028}]},{"osd":2,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.64900000000000002,"15min":0.63300000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.3860000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.47699999999999998,"5min":0.432,"15min":0.432},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.88100000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.64400000000000002,"15min":0.626},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.70999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.91300000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.745,"15min":0.73699999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.0529999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73199999999999998,"15min":0.72699999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.012}]},{"osd":7,"last update":"Tue Nov 28 13:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.77700000000000002,"15min":0.76800000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.4249999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":0.98099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:10:59.772 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T13:10:59.773 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T13:10:59.941 INFO:tasks.ceph.mds.c.smithi156.stderr:2023-11-28T13:10:59.939+0000 7f6a319f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:11:00.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.009+0000 7fead9bff700 1 Processor -- start 2023-11-28T13:11:00.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.009+0000 7fead9bff700 1 -- start start 2023-11-28T13:11:00.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.010+0000 7fead9bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fead415c9b0 0x7fead40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:00.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.011+0000 7fead9bff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fead4161340 0x7fead40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:00.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.011+0000 7fead9bff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead40d1950 con 0x7fead40d2320 2023-11-28T13:11:00.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.011+0000 7fead9bff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead40b5800 con 0x7fead4161340 2023-11-28T13:11:00.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.011+0000 7fead9bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead40b5520 con 0x7fead415c9b0 2023-11-28T13:11:00.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.011+0000 7fead37fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fead415c9b0 0x7fead40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:00.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.011+0000 7feacbfff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fead4161340 0x7fead40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:00.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.011+0000 7fead37fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fead415c9b0 0x7fead40b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:45786/0 (socket says 172.21.15.156:45786) 2023-11-28T13:11:00.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.011+0000 7fead37fe700 1 -- 172.21.15.156:0/3044233597 learned_addr learned my addr 172.21.15.156:0/3044233597 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:00.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.012+0000 7fead2ffd700 1 -- 172.21.15.156:0/3044233597 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3031849631 0 0) 0x7fead40d1950 con 0x7fead40d2320 2023-11-28T13:11:00.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.012+0000 7fead2ffd700 1 -- 172.21.15.156:0/3044233597 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feac00035b0 con 0x7fead40d2320 2023-11-28T13:11:00.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.012+0000 7fead37fe700 1 -- 172.21.15.156:0/3044233597 >> v2:172.21.15.186:3300/0 conn(0x7fead4161340 msgr2=0x7fead40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:00.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.012+0000 7fead37fe700 1 --2- 172.21.15.156:0/3044233597 >> v2:172.21.15.186:3300/0 conn(0x7fead4161340 0x7fead40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.012+0000 7fead37fe700 1 -- 172.21.15.156:0/3044233597 >> v1:172.21.15.186:6789/0 conn(0x7fead40d2320 legacy=0x7fead40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:00.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.012+0000 7fead37fe700 1 -- 172.21.15.156:0/3044233597 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead40d1300 con 0x7fead415c9b0 2023-11-28T13:11:00.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.012+0000 7feacbfff700 1 --2- 172.21.15.156:0/3044233597 >> v2:172.21.15.186:3300/0 conn(0x7fead4161340 0x7fead40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:11:00.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.012+0000 7fead37fe700 1 --2- 172.21.15.156:0/3044233597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fead415c9b0 0x7fead40b4bf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feac40098f0 tx=0x7feac4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a565c514e36cdea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:00.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.012+0000 7fead2ffd700 1 -- 172.21.15.156:0/3044233597 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4011830 con 0x7fead415c9b0 2023-11-28T13:11:00.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.012+0000 7fead2ffd700 1 -- 172.21.15.156:0/3044233597 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac4011990 con 0x7fead415c9b0 2023-11-28T13:11:00.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.012+0000 7fead2ffd700 1 -- 172.21.15.156:0/3044233597 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4011d00 con 0x7fead415c9b0 2023-11-28T13:11:00.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.013+0000 7fead9bff700 1 -- 172.21.15.156:0/3044233597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fead415c9b0 msgr2=0x7fead40b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:00.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.013+0000 7fead9bff700 1 --2- 172.21.15.156:0/3044233597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fead415c9b0 0x7fead40b4bf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feac40098f0 tx=0x7feac4009c00 comp rx=0 tx=0).stop 2023-11-28T13:11:00.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.013+0000 7fead9bff700 1 -- 172.21.15.156:0/3044233597 shutdown_connections 2023-11-28T13:11:00.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.013+0000 7fead9bff700 1 --2- 172.21.15.156:0/3044233597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fead415c9b0 0x7fead40b4bf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.013+0000 7fead9bff700 1 --2- 172.21.15.156:0/3044233597 >> v2:172.21.15.186:3300/0 conn(0x7fead4161340 0x7fead40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.013+0000 7fead9bff700 1 -- 172.21.15.156:0/3044233597 >> 172.21.15.156:0/3044233597 conn(0x7fead4072600 msgr2=0x7fead40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:00.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.013+0000 7fead9bff700 1 -- 172.21.15.156:0/3044233597 shutdown_connections 2023-11-28T13:11:00.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.013+0000 7fead9bff700 1 -- 172.21.15.156:0/3044233597 wait complete. 2023-11-28T13:11:00.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.015+0000 7fead9bff700 1 Processor -- start 2023-11-28T13:11:00.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.015+0000 7fead9bff700 1 -- start start 2023-11-28T13:11:00.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.016+0000 7fead9bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fead40d2320 0x7fead4146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:00.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.016+0000 7fead9bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fead415c9b0 0x7fead40c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:00.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.016+0000 7fead9bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fead4161340 0x7fead4159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:00.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.016+0000 7fead9bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead4151e40 con 0x7fead415c9b0 2023-11-28T13:11:00.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.016+0000 7fead9bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead4146cd0 con 0x7fead4161340 2023-11-28T13:11:00.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.016+0000 7fead9bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead4151510 con 0x7fead40d2320 2023-11-28T13:11:00.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.016+0000 7feacbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fead415c9b0 0x7fead40c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:00.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.016+0000 7feacbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fead415c9b0 0x7fead40c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53100/0 (socket says 172.21.15.156:53100) 2023-11-28T13:11:00.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.016+0000 7feacbfff700 1 -- 172.21.15.156:0/3544553340 learned_addr learned my addr 172.21.15.156:0/3544553340 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:00.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.016+0000 7fead37fe700 1 --2- 172.21.15.156:0/3544553340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fead40d2320 0x7fead4146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:00.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.016+0000 7fead3fff700 1 --2- 172.21.15.156:0/3544553340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fead4161340 0x7fead4159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:00.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.017+0000 7feacbfff700 1 -- 172.21.15.156:0/3544553340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fead40d2320 msgr2=0x7fead4146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:00.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.017+0000 7feacbfff700 1 --2- 172.21.15.156:0/3544553340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fead40d2320 0x7fead4146790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.017+0000 7feacbfff700 1 -- 172.21.15.156:0/3544553340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fead4161340 msgr2=0x7fead4159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:00.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.017+0000 7feacbfff700 1 --2- 172.21.15.156:0/3544553340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fead4161340 0x7fead4159800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.017+0000 7feacbfff700 1 -- 172.21.15.156:0/3544553340 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc0094a0 con 0x7fead415c9b0 2023-11-28T13:11:00.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.017+0000 7feacbfff700 1 --2- 172.21.15.156:0/3544553340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fead415c9b0 0x7fead40c7670 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7feabc00ebe0 tx=0x7feabc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:00.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.017+0000 7fead0ff9700 1 -- 172.21.15.156:0/3544553340 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc00c850 con 0x7fead415c9b0 2023-11-28T13:11:00.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.017+0000 7fead0ff9700 1 -- 172.21.15.156:0/3544553340 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc00c9b0 con 0x7fead415c9b0 2023-11-28T13:11:00.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.017+0000 7fead0ff9700 1 -- 172.21.15.156:0/3544553340 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc00cd40 con 0x7fead415c9b0 2023-11-28T13:11:00.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.017+0000 7fead9bff700 1 -- 172.21.15.156:0/3544553340 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac4009550 con 0x7fead415c9b0 2023-11-28T13:11:00.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.017+0000 7fead9bff700 1 -- 172.21.15.156:0/3544553340 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead40b5130 con 0x7fead415c9b0 2023-11-28T13:11:00.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.019+0000 7fead9bff700 1 -- 172.21.15.156:0/3544553340 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead40d15e0 con 0x7fead415c9b0 2023-11-28T13:11:00.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.019+0000 7fead0ff9700 1 -- 172.21.15.156:0/3544553340 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feabc019030 con 0x7fead415c9b0 2023-11-28T13:11:00.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.020+0000 7fead0ff9700 1 --2- 172.21.15.156:0/3544553340 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feab40682f0 0x7feab406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:00.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.020+0000 7fead0ff9700 1 -- 172.21.15.156:0/3544553340 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feabc085bc0 con 0x7fead415c9b0 2023-11-28T13:11:00.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.020+0000 7fead37fe700 1 --2- 172.21.15.156:0/3544553340 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feab40682f0 0x7feab406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:00.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.021+0000 7fead37fe700 1 --2- 172.21.15.156:0/3544553340 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feab40682f0 0x7feab406a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7feac4014040 tx=0x7feac4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:00.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.024+0000 7fead0ff9700 1 -- 172.21.15.156:0/3544553340 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feabc052420 con 0x7fead415c9b0 2023-11-28T13:11:00.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.289+0000 7fead9bff700 1 -- 172.21.15.156:0/3544553340 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7fead40b52c0 con 0x7fead415c9b0 2023-11-28T13:11:00.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.294+0000 7fead0ff9700 1 -- 172.21.15.156:0/3544553340 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7feabc00cea0 con 0x7fead415c9b0 2023-11-28T13:11:00.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.296+0000 7fead9bff700 1 -- 172.21.15.156:0/3544553340 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feab40682f0 msgr2=0x7feab406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:00.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.296+0000 7fead9bff700 1 --2- 172.21.15.156:0/3544553340 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feab40682f0 0x7feab406a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7feac4014040 tx=0x7feac4008030 comp rx=0 tx=0).stop 2023-11-28T13:11:00.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.296+0000 7fead9bff700 1 -- 172.21.15.156:0/3544553340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fead415c9b0 msgr2=0x7fead40c7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:00.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.296+0000 7fead9bff700 1 --2- 172.21.15.156:0/3544553340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fead415c9b0 0x7fead40c7670 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7feabc00ebe0 tx=0x7feabc00eef0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.296+0000 7fead9bff700 1 -- 172.21.15.156:0/3544553340 shutdown_connections 2023-11-28T13:11:00.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.296+0000 7fead9bff700 1 --2- 172.21.15.156:0/3544553340 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feab40682f0 0x7feab406a7a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.296+0000 7fead9bff700 1 --2- 172.21.15.156:0/3544553340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fead4161340 0x7fead4159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.296+0000 7fead9bff700 1 --2- 172.21.15.156:0/3544553340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fead40d2320 0x7fead4146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.296+0000 7fead9bff700 1 --2- 172.21.15.156:0/3544553340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fead415c9b0 0x7fead40c7670 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.296+0000 7fead9bff700 1 -- 172.21.15.156:0/3544553340 >> 172.21.15.156:0/3544553340 conn(0x7fead4072600 msgr2=0x7fead40b8940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:00.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.296+0000 7fead9bff700 1 -- 172.21.15.156:0/3544553340 shutdown_connections 2023-11-28T13:11:00.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.296+0000 7fead9bff700 1 -- 172.21.15.156:0/3544553340 wait complete. 2023-11-28T13:11:00.299 INFO:teuthology.orchestra.run.smithi156.stderr:updated caps for client.0 2023-11-28T13:11:00.317 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:11:00.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.512+0000 7f5e80dff700 1 Processor -- start 2023-11-28T13:11:00.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.512+0000 7f5e80dff700 1 -- start start 2023-11-28T13:11:00.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.514+0000 7f5e80dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e7c161340 0x7f5e7c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:00.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.514+0000 7f5e80dff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5e7c0d2320 0x7f5e7c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:00.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.514+0000 7f5e80dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7c0d1950 con 0x7f5e7c15c9b0 2023-11-28T13:11:00.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.514+0000 7f5e80dff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7c0b5800 con 0x7f5e7c0d2320 2023-11-28T13:11:00.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.514+0000 7f5e80dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0b5300 con 0x7f5e7c161340 2023-11-28T13:11:00.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.514+0000 7f5e73fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e7c161340 0x7f5e7c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:00.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.514+0000 7f5e7ad9d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5e7c0d2320 0x7f5e7c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:00.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.514+0000 7f5e7a59c700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f5e7c15c9b0 0x7f5e7c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:56364/0 (socket says 172.21.15.156:56364) 2023-11-28T13:11:00.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.514+0000 7f5e7a59c700 1 -- 172.21.15.156:0/2826729296 learned_addr learned my addr 172.21.15.156:0/2826729296 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:11:00.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.514+0000 7f5e7ad9d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5e7c0d2320 0x7f5e7c0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:46232/0 (socket says 172.21.15.156:46232) 2023-11-28T13:11:00.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.514+0000 7f5e73fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e7c161340 0x7f5e7c0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:45814/0 (socket says 172.21.15.156:45814) 2023-11-28T13:11:00.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.515+0000 7f5e7ad9d700 1 -- 172.21.15.156:0/2826729296 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e7c161340 msgr2=0x7f5e7c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:00.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.515+0000 7f5e7ad9d700 1 --2- 172.21.15.156:0/2826729296 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e7c161340 0x7f5e7c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.515+0000 7f5e7ad9d700 1 -- 172.21.15.156:0/2826729296 >> v1:172.21.15.186:6789/0 conn(0x7f5e7c15c9b0 legacy=0x7f5e7c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:00.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.515+0000 7f5e7ad9d700 1 -- 172.21.15.156:0/2826729296 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7c0b5520 con 0x7f5e7c0d2320 2023-11-28T13:11:00.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.515+0000 7f5e73fff700 1 --2- 172.21.15.156:0/2826729296 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e7c161340 0x7f5e7c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:11:00.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.515+0000 7f5e7ad9d700 1 --2- 172.21.15.156:0/2826729296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e7c0d2320 0x7f5e7c0cd7c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009dc0 tx=0x7f5e7400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90e6fb88157ffc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:00.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.515+0000 7f5e79d9b700 1 -- 172.21.15.156:0/2826729296 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7400e070 con 0x7f5e7c0d2320 2023-11-28T13:11:00.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.515+0000 7f5e79d9b700 1 -- 172.21.15.156:0/2826729296 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7400bae0 con 0x7f5e7c0d2320 2023-11-28T13:11:00.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.515+0000 7f5e79d9b700 1 -- 172.21.15.156:0/2826729296 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7400be90 con 0x7f5e7c0d2320 2023-11-28T13:11:00.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.516+0000 7f5e80dff700 1 -- 172.21.15.156:0/2826729296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e7c0d2320 msgr2=0x7f5e7c0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:00.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.516+0000 7f5e80dff700 1 --2- 172.21.15.156:0/2826729296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e7c0d2320 0x7f5e7c0cd7c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009dc0 tx=0x7f5e7400b330 comp rx=0 tx=0).stop 2023-11-28T13:11:00.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.516+0000 7f5e80dff700 1 -- 172.21.15.156:0/2826729296 shutdown_connections 2023-11-28T13:11:00.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.516+0000 7f5e80dff700 1 --2- 172.21.15.156:0/2826729296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e7c0d2320 0x7f5e7c0cd7c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.516+0000 7f5e80dff700 1 --2- 172.21.15.156:0/2826729296 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e7c161340 0x7f5e7c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.516+0000 7f5e80dff700 1 -- 172.21.15.156:0/2826729296 >> 172.21.15.156:0/2826729296 conn(0x7f5e7c072600 msgr2=0x7f5e7c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:00.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.516+0000 7f5e80dff700 1 -- 172.21.15.156:0/2826729296 shutdown_connections 2023-11-28T13:11:00.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.516+0000 7f5e80dff700 1 -- 172.21.15.156:0/2826729296 wait complete. 2023-11-28T13:11:00.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.518+0000 7f5e80dff700 1 Processor -- start 2023-11-28T13:11:00.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.518+0000 7f5e80dff700 1 -- start start 2023-11-28T13:11:00.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.519+0000 7f5e80dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e7c0d2320 0x7f5e7c151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:00.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.519+0000 7f5e80dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e7c15c9b0 0x7f5e7c0c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:00.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.519+0000 7f5e80dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e7c161340 0x7f5e7c0c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:00.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.519+0000 7f5e80dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c1531e0 con 0x7f5e7c15c9b0 2023-11-28T13:11:00.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.519+0000 7f5e80dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c152560 con 0x7f5e7c0d2320 2023-11-28T13:11:00.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.519+0000 7f5e80dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c152840 con 0x7f5e7c161340 2023-11-28T13:11:00.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.519+0000 7f5e73fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e7c15c9b0 0x7f5e7c0c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:00.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.519+0000 7f5e73fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e7c15c9b0 0x7f5e7c0c8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53116/0 (socket says 172.21.15.156:53116) 2023-11-28T13:11:00.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.519+0000 7f5e73fff700 1 -- 172.21.15.156:0/896310015 learned_addr learned my addr 172.21.15.156:0/896310015 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:00.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.519+0000 7f5e7a59c700 1 --2- 172.21.15.156:0/896310015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e7c0d2320 0x7f5e7c151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:00.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.519+0000 7f5e73fff700 1 -- 172.21.15.156:0/896310015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e7c161340 msgr2=0x7f5e7c0c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:00.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.519+0000 7f5e7ad9d700 1 --2- 172.21.15.156:0/896310015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e7c161340 0x7f5e7c0c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:00.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.520+0000 7f5e73fff700 1 --2- 172.21.15.156:0/896310015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e7c161340 0x7f5e7c0c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.520+0000 7f5e73fff700 1 -- 172.21.15.156:0/896310015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e7c0d2320 msgr2=0x7f5e7c151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:00.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.520+0000 7f5e73fff700 1 --2- 172.21.15.156:0/896310015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e7c0d2320 0x7f5e7c151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.520+0000 7f5e73fff700 1 -- 172.21.15.156:0/896310015 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e640094a0 con 0x7f5e7c15c9b0 2023-11-28T13:11:00.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.520+0000 7f5e7a59c700 1 --2- 172.21.15.156:0/896310015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e7c0d2320 0x7f5e7c151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:11:00.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.520+0000 7f5e7ad9d700 1 --2- 172.21.15.156:0/896310015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e7c161340 0x7f5e7c0c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:11:00.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.520+0000 7f5e73fff700 1 --2- 172.21.15.156:0/896310015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e7c15c9b0 0x7f5e7c0c8e30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400ebe0 tx=0x7f5e64009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:00.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.520+0000 7f5e72ffd700 1 -- 172.21.15.156:0/896310015 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6400c910 con 0x7f5e7c15c9b0 2023-11-28T13:11:00.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.520+0000 7f5e72ffd700 1 -- 172.21.15.156:0/896310015 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6400ca70 con 0x7f5e7c15c9b0 2023-11-28T13:11:00.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.520+0000 7f5e72ffd700 1 -- 172.21.15.156:0/896310015 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6400cde0 con 0x7f5e7c15c9b0 2023-11-28T13:11:00.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.520+0000 7f5e80dff700 1 -- 172.21.15.156:0/896310015 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e74009a70 con 0x7f5e7c15c9b0 2023-11-28T13:11:00.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.520+0000 7f5e80dff700 1 -- 172.21.15.156:0/896310015 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7c0d15e0 con 0x7f5e7c15c9b0 2023-11-28T13:11:00.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.521+0000 7f5e80dff700 1 -- 172.21.15.156:0/896310015 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7c0d1300 con 0x7f5e7c15c9b0 2023-11-28T13:11:00.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.522+0000 7f5e72ffd700 1 -- 172.21.15.156:0/896310015 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5e64019030 con 0x7f5e7c15c9b0 2023-11-28T13:11:00.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.523+0000 7f5e72ffd700 1 --2- 172.21.15.156:0/896310015 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5e5c0682f0 0x7f5e5c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:00.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.523+0000 7f5e72ffd700 1 -- 172.21.15.156:0/896310015 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5e64085cf0 con 0x7f5e7c15c9b0 2023-11-28T13:11:00.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.524+0000 7f5e7a59c700 1 --2- 172.21.15.156:0/896310015 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5e5c0682f0 0x7f5e5c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:00.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.524+0000 7f5e7a59c700 1 --2- 172.21.15.156:0/896310015 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5e5c0682f0 0x7f5e5c06a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c005910 tx=0x7f5e6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:00.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.528+0000 7f5e72ffd700 1 -- 172.21.15.156:0/896310015 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5e64052550 con 0x7f5e7c15c9b0 2023-11-28T13:11:00.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.761+0000 7f5e80dff700 1 -- 172.21.15.156:0/896310015 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e7c15cd80 con 0x7f5e7c15c9b0 2023-11-28T13:11:00.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.762+0000 7f5e72ffd700 1 -- 172.21.15.156:0/896310015 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4658 (secure 0 0 0) 0x7f5e64017090 con 0x7f5e7c15c9b0 2023-11-28T13:11:00.763 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:11:00.763 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3942362276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3942362276},{"type":"v1","addr":"172.21.15.186:6835","nonce":3942362276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/722198303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":722198303},{"type":"v1","addr":"172.21.15.186:6837","nonce":722198303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5916,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3601975324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3601975324},{"type":"v1","addr":"172.21.15.156:6835","nonce":3601975324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:10:54.928601+0000","modified":"2023-11-28T13:10:57.942034+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5919":{"gid":5919,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6834/3314379816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3314379816},{"type":"v1","addr":"172.21.15.156:6834","nonce":3314379816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:11:00.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.764+0000 7f5e80dff700 1 -- 172.21.15.156:0/896310015 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5e5c0682f0 msgr2=0x7f5e5c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:00.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.764+0000 7f5e80dff700 1 --2- 172.21.15.156:0/896310015 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5e5c0682f0 0x7f5e5c06a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c005910 tx=0x7f5e6c009040 comp rx=0 tx=0).stop 2023-11-28T13:11:00.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.764+0000 7f5e80dff700 1 -- 172.21.15.156:0/896310015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e7c15c9b0 msgr2=0x7f5e7c0c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:00.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.764+0000 7f5e80dff700 1 --2- 172.21.15.156:0/896310015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e7c15c9b0 0x7f5e7c0c8e30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400ebe0 tx=0x7f5e64009ec0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.764+0000 7f5e80dff700 1 -- 172.21.15.156:0/896310015 shutdown_connections 2023-11-28T13:11:00.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.764+0000 7f5e80dff700 1 --2- 172.21.15.156:0/896310015 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5e5c0682f0 0x7f5e5c06a7a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.764+0000 7f5e80dff700 1 --2- 172.21.15.156:0/896310015 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e7c161340 0x7f5e7c0c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.764+0000 7f5e80dff700 1 --2- 172.21.15.156:0/896310015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e7c0d2320 0x7f5e7c151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.764+0000 7f5e80dff700 1 --2- 172.21.15.156:0/896310015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e7c15c9b0 0x7f5e7c0c8e30 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:00.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.764+0000 7f5e80dff700 1 -- 172.21.15.156:0/896310015 >> 172.21.15.156:0/896310015 conn(0x7f5e7c072600 msgr2=0x7f5e7c15fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:00.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.764+0000 7f5e80dff700 1 -- 172.21.15.156:0/896310015 shutdown_connections 2023-11-28T13:11:00.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:00.764+0000 7f5e80dff700 1 -- 172.21.15.156:0/896310015 wait complete. 2023-11-28T13:11:00.767 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 31 2023-11-28T13:11:01.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T13:10:54.928601+0000', 'modified': '2023-11-28T13:10:57.942034+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5919}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5919': {'gid': 5919, 'name': 'c', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.156:6834/3314379816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6832', 'nonce': 3314379816}, {'type': 'v1', 'addr': '172.21.15.156:6834', 'nonce': 3314379816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T13:11:01.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T13:11:01.187 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T13:11:01.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.385+0000 7f2586bff700 1 Processor -- start 2023-11-28T13:11:01.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.385+0000 7f2586bff700 1 -- start start 2023-11-28T13:11:01.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.386+0000 7f2586bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f258015c9b0 0x7f25800b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.386+0000 7f2586bff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f25800cd7c0 0x7f25800cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.386+0000 7f2586bff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25800d1950 con 0x7f2580161340 2023-11-28T13:11:01.388 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.386+0000 7f2586bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25800b5800 con 0x7f258015c9b0 2023-11-28T13:11:01.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.386+0000 7f2586bff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25800ce140 con 0x7f25800cd7c0 2023-11-28T13:11:01.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.386+0000 7f258499b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f258015c9b0 0x7f25800b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.386+0000 7f258499b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f258015c9b0 0x7f25800b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53132/0 (socket says 172.21.15.156:53132) 2023-11-28T13:11:01.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.386+0000 7f258499b700 1 -- 172.21.15.156:0/1151157664 learned_addr learned my addr 172.21.15.156:0/1151157664 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:01.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.387+0000 7f258519c700 1 --2- 172.21.15.156:0/1151157664 >> v2:172.21.15.186:3300/0 conn(0x7f25800cd7c0 0x7f25800cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.387+0000 7f258499b700 1 -- 172.21.15.156:0/1151157664 >> v2:172.21.15.186:3300/0 conn(0x7f25800cd7c0 msgr2=0x7f25800cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:01.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.387+0000 7f258499b700 1 --2- 172.21.15.156:0/1151157664 >> v2:172.21.15.186:3300/0 conn(0x7f25800cd7c0 0x7f25800cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.387+0000 7f258499b700 1 -- 172.21.15.156:0/1151157664 >> v1:172.21.15.156:6789/0 conn(0x7f2580161340 legacy=0x7f25800d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:01.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.387+0000 7f258499b700 1 -- 172.21.15.156:0/1151157664 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25800d24f0 con 0x7f258015c9b0 2023-11-28T13:11:01.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.387+0000 7f258499b700 1 --2- 172.21.15.156:0/1151157664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f258015c9b0 0x7f25800b4bf0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f258006c170 tx=0x7f2574009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d78cd9eb660e32 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:01.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.387+0000 7f257f7fe700 1 -- 172.21.15.156:0/1151157664 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257400b870 con 0x7f258015c9b0 2023-11-28T13:11:01.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.387+0000 7f257f7fe700 1 -- 172.21.15.156:0/1151157664 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257400b9d0 con 0x7f258015c9b0 2023-11-28T13:11:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.387+0000 7f257f7fe700 1 -- 172.21.15.156:0/1151157664 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257400bd60 con 0x7f258015c9b0 2023-11-28T13:11:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.388+0000 7f2586bff700 1 -- 172.21.15.156:0/1151157664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f258015c9b0 msgr2=0x7f25800b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.388+0000 7f2586bff700 1 --2- 172.21.15.156:0/1151157664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f258015c9b0 0x7f25800b4bf0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f258006c170 tx=0x7f2574009bd0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.388+0000 7f2586bff700 1 -- 172.21.15.156:0/1151157664 shutdown_connections 2023-11-28T13:11:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.388+0000 7f2586bff700 1 --2- 172.21.15.156:0/1151157664 >> v2:172.21.15.186:3300/0 conn(0x7f25800cd7c0 0x7f25800cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.388+0000 7f2586bff700 1 --2- 172.21.15.156:0/1151157664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f258015c9b0 0x7f25800b4bf0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.388+0000 7f2586bff700 1 -- 172.21.15.156:0/1151157664 >> 172.21.15.156:0/1151157664 conn(0x7f2580072600 msgr2=0x7f25800cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.388+0000 7f2586bff700 1 -- 172.21.15.156:0/1151157664 shutdown_connections 2023-11-28T13:11:01.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.388+0000 7f2586bff700 1 -- 172.21.15.156:0/1151157664 wait complete. 2023-11-28T13:11:01.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.390+0000 7f2586bff700 1 Processor -- start 2023-11-28T13:11:01.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.390+0000 7f2586bff700 1 -- start start 2023-11-28T13:11:01.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.391+0000 7f2586bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25800cd7c0 0x7f25800ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.391+0000 7f2586bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f258015c9b0 0x7f2580157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.391+0000 7f2586bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2580161340 0x7f258014a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.391+0000 7f2586bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25800ca780 con 0x7f2580161340 2023-11-28T13:11:01.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.391+0000 7f2586bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2580156280 con 0x7f25800cd7c0 2023-11-28T13:11:01.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.391+0000 7f2586bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2580156560 con 0x7f258015c9b0 2023-11-28T13:11:01.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.391+0000 7f258519c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2580161340 0x7f258014a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.391+0000 7f258519c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2580161340 0x7f258014a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53144/0 (socket says 172.21.15.156:53144) 2023-11-28T13:11:01.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.391+0000 7f258519c700 1 -- 172.21.15.156:0/1677469516 learned_addr learned my addr 172.21.15.156:0/1677469516 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:01.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.391+0000 7f258499b700 1 --2- 172.21.15.156:0/1677469516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25800cd7c0 0x7f25800ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.391+0000 7f257ffff700 1 --2- 172.21.15.156:0/1677469516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f258015c9b0 0x7f2580157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.391+0000 7f258519c700 1 -- 172.21.15.156:0/1677469516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f258015c9b0 msgr2=0x7f2580157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:01.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.391+0000 7f258519c700 1 --2- 172.21.15.156:0/1677469516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f258015c9b0 0x7f2580157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.392+0000 7f258519c700 1 -- 172.21.15.156:0/1677469516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25800cd7c0 msgr2=0x7f25800ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:01.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.392+0000 7f258519c700 1 --2- 172.21.15.156:0/1677469516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25800cd7c0 0x7f25800ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.392+0000 7f258519c700 1 -- 172.21.15.156:0/1677469516 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2574009550 con 0x7f2580161340 2023-11-28T13:11:01.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.392+0000 7f257ffff700 1 --2- 172.21.15.156:0/1677469516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f258015c9b0 0x7f2580157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:11:01.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.392+0000 7f258519c700 1 --2- 172.21.15.156:0/1677469516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2580161340 0x7f258014a480 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2578007e60 tx=0x7f257800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:01.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.392+0000 7f257d7fa700 1 -- 172.21.15.156:0/1677469516 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257800ede0 con 0x7f2580161340 2023-11-28T13:11:01.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.392+0000 7f257d7fa700 1 -- 172.21.15.156:0/1677469516 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2578014420 con 0x7f2580161340 2023-11-28T13:11:01.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.392+0000 7f257d7fa700 1 -- 172.21.15.156:0/1677469516 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25780147d0 con 0x7f2580161340 2023-11-28T13:11:01.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.392+0000 7f2586bff700 1 -- 172.21.15.156:0/1677469516 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25800d1300 con 0x7f2580161340 2023-11-28T13:11:01.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.392+0000 7f2586bff700 1 -- 172.21.15.156:0/1677469516 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25800b5520 con 0x7f2580161340 2023-11-28T13:11:01.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.395+0000 7f257d7fa700 1 -- 172.21.15.156:0/1677469516 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2578012050 con 0x7f2580161340 2023-11-28T13:11:01.397 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.396+0000 7f257d7fa700 1 --2- 172.21.15.156:0/1677469516 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25640682f0 0x7f256406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.396+0000 7f257d7fa700 1 -- 172.21.15.156:0/1677469516 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2578086320 con 0x7f2580161340 2023-11-28T13:11:01.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.396+0000 7f258499b700 1 --2- 172.21.15.156:0/1677469516 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25640682f0 0x7f256406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.397+0000 7f258499b700 1 --2- 172.21.15.156:0/1677469516 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25640682f0 0x7f256406a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2574014040 tx=0x7f2574008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:01.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.398+0000 7f259a54bb80 1 Processor -- start 2023-11-28T13:11:01.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.398+0000 7f259a54bb80 1 -- start start 2023-11-28T13:11:01.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f259a54bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad957aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f259a54bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a2130 0x560ad95a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f259a54bb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a5120 0x560ad95c0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f259a54bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad9464bc0 con 0x560ad95a4810 2023-11-28T13:11:01.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f259a54bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad94d3350 con 0x560ad95a2130 2023-11-28T13:11:01.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f259a54bb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ad94cc880 con 0x560ad95a5120 2023-11-28T13:11:01.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f258499b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad957aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f258499b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad957aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53146/0 (socket says 172.21.15.156:53146) 2023-11-28T13:11:01.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f258499b700 1 -- 172.21.15.156:0/2323605 learned_addr learned my addr 172.21.15.156:0/2323605 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:01.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f257ffff700 1 --2- 172.21.15.156:0/2323605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a2130 0x560ad95a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f258519c700 1 --2- 172.21.15.156:0/2323605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a5120 0x560ad95c0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f258499b700 1 -- 172.21.15.156:0/2323605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a5120 msgr2=0x560ad95c0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:01.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f258499b700 1 --2- 172.21.15.156:0/2323605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a5120 0x560ad95c0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f258499b700 1 -- 172.21.15.156:0/2323605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a2130 msgr2=0x560ad95a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:01.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f258499b700 1 --2- 172.21.15.156:0/2323605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a2130 0x560ad95a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f258499b700 1 -- 172.21.15.156:0/2323605 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ad934e5e0 con 0x560ad95a4810 2023-11-28T13:11:01.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f258519c700 1 --2- 172.21.15.156:0/2323605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a5120 0x560ad95c0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:11:01.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.399+0000 7f257ffff700 1 --2- 172.21.15.156:0/2323605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a2130 0x560ad95a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:11:01.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.400+0000 7f258499b700 1 --2- 172.21.15.156:0/2323605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad957aa20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2574009fd0 tx=0x7f2574010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab260c8bf189b34 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:01.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.400+0000 7f256a7fc700 1 -- 172.21.15.156:0/2323605 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257401f070 con 0x560ad95a4810 2023-11-28T13:11:01.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.400+0000 7f256a7fc700 1 -- 172.21.15.156:0/2323605 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257400bd00 con 0x560ad95a4810 2023-11-28T13:11:01.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.400+0000 7f256a7fc700 1 -- 172.21.15.156:0/2323605 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574025680 con 0x560ad95a4810 2023-11-28T13:11:01.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.400+0000 7f259a54bb80 1 -- 172.21.15.156:0/2323605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 msgr2=0x560ad957aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:01.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.400+0000 7f259a54bb80 1 --2- 172.21.15.156:0/2323605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad957aa20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2574009fd0 tx=0x7f2574010000 comp rx=0 tx=0).stop 2023-11-28T13:11:01.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.401+0000 7f259a54bb80 1 -- 172.21.15.156:0/2323605 shutdown_connections 2023-11-28T13:11:01.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.401+0000 7f259a54bb80 1 --2- 172.21.15.156:0/2323605 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a5120 0x560ad95c0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.401+0000 7f259a54bb80 1 --2- 172.21.15.156:0/2323605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad957aa20 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.401+0000 7f259a54bb80 1 --2- 172.21.15.156:0/2323605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a2130 0x560ad95a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.401+0000 7f259a54bb80 1 -- 172.21.15.156:0/2323605 >> 172.21.15.156:0/2323605 conn(0x560ad9577d60 msgr2=0x560ad95786f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:01.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.401+0000 7f259a54bb80 1 -- 172.21.15.156:0/2323605 shutdown_connections 2023-11-28T13:11:01.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.401+0000 7f259a54bb80 1 -- 172.21.15.156:0/2323605 wait complete. 2023-11-28T13:11:01.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.401+0000 7f259a54bb80 1 -- start start 2023-11-28T13:11:01.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.402+0000 7f259a54bb80 1 Processor -- start 2023-11-28T13:11:01.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f259a54bb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a2130 0x560ad95b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f259a54bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad95949a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f259a54bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a5120 0x560ad9594ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f259a54bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad93508d0 con 0x560ad95a4810 2023-11-28T13:11:01.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f259a54bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad95a3c70 con 0x560ad95a5120 2023-11-28T13:11:01.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f259a54bb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ad95a6d10 con 0x560ad95a2130 2023-11-28T13:11:01.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f259a54bb80 10 client.0 _ll_register_callbacks cb 0x560ad9372240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:11:01.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f257ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad95949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f257ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad95949a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53156/0 (socket says 172.21.15.156:53156) 2023-11-28T13:11:01.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f257ffff700 1 -- 172.21.15.156:0/3182162319 learned_addr learned my addr 172.21.15.156:0/3182162319 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:01.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f258499b700 1 --2- 172.21.15.156:0/3182162319 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a2130 0x560ad95b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f258519c700 1 --2- 172.21.15.156:0/3182162319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a5120 0x560ad9594ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f257ffff700 1 -- 172.21.15.156:0/3182162319 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a2130 msgr2=0x560ad95b74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:01.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f257ffff700 1 --2- 172.21.15.156:0/3182162319 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a2130 0x560ad95b74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f257ffff700 1 -- 172.21.15.156:0/3182162319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a5120 msgr2=0x560ad9594ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:01.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f257ffff700 1 --2- 172.21.15.156:0/3182162319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a5120 0x560ad9594ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f257ffff700 1 -- 172.21.15.156:0/3182162319 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25800d24f0 con 0x560ad95a4810 2023-11-28T13:11:01.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f258519c700 1 --2- 172.21.15.156:0/3182162319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a5120 0x560ad9594ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:11:01.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.403+0000 7f258499b700 1 --2- 172.21.15.156:0/3182162319 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a2130 0x560ad95b74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:11:01.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.404+0000 7f257ffff700 1 --2- 172.21.15.156:0/3182162319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad95949a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f256c000f60 tx=0x7f256c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940673ef17333377 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:01.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.404+0000 7f255f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:11:01.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.404+0000 7f255f7fe700 1 -- 172.21.15.156:0/3182162319 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c001a30 con 0x560ad95a4810 2023-11-28T13:11:01.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.404+0000 7f255f7fe700 1 -- 172.21.15.156:0/3182162319 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f256c001b90 con 0x560ad95a4810 2023-11-28T13:11:01.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.404+0000 7f259a54bb80 1 -- 172.21.15.156:0/3182162319 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ad949a140 con 0x560ad95a4810 2023-11-28T13:11:01.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.404+0000 7f255f7fe700 1 -- 172.21.15.156:0/3182162319 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f256c001d50 con 0x560ad95a4810 2023-11-28T13:11:01.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.404+0000 7f259a54bb80 10 client.6039 fetch_fsmap learned FSMap version 31 2023-11-28T13:11:01.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.404+0000 7f259a54bb80 1 -- 172.21.15.156:0/3182162319 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x560ad92fad90 con 0x560ad95a4810 2023-11-28T13:11:01.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.404+0000 7f255f7fe700 1 -- 172.21.15.156:0/3182162319 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560ad949a140 con 0x560ad95a4810 2023-11-28T13:11:01.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.404+0000 7f255f7fe700 1 -- 172.21.15.156:0/3182162319 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f256c014400 con 0x560ad95a4810 2023-11-28T13:11:01.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.404+0000 7f259a54bb80 10 client.6039 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T13:11:01.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.404+0000 7f259a54bb80 10 client.6039 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:11:01.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.404+0000 7f259a54bb80 20 client.6039 populate_metadata read hostname 'smithi156' 2023-11-28T13:11:01.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.404+0000 7f259a54bb80 1 --2- 172.21.15.156:0/3182162319 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x560ad95b6be0 0x560ad95a6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.404+0000 7f259a54bb80 4 client.6039 mds_command: new command op to 5919 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:11:01.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.404+0000 7f259a54bb80 1 -- 172.21.15.156:0/3182162319 --> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ad9350f10 con 0x560ad95b6be0 2023-11-28T13:11:01.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.405+0000 7f258499b700 1 --2- 172.21.15.156:0/3182162319 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x560ad95b6be0 0x560ad95a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.405+0000 7f258499b700 1 --2- 172.21.15.156:0/3182162319 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x560ad95b6be0 0x560ad95a6440 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b86b1fc9203e171b server_cookie=9a0d741aea4fca5e in_seq=0 out_seq=0 2023-11-28T13:11:01.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.405+0000 7f255f7fe700 10 client.6039 ms_handle_connect on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:01.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.406+0000 7f255f7fe700 1 -- 172.21.15.156:0/3182162319 <== mds.0 v2:172.21.15.156:6832/3314379816 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x560ad9350f10 con 0x560ad95b6be0 2023-11-28T13:11:01.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.406+0000 7f255f7fe700 10 client.6039 handle_command_reply: tid=0 2023-11-28T13:11:01.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.406+0000 7f259a54bb80 1 client.6039 shutdown 2023-11-28T13:11:01.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.406+0000 7f259a54bb80 1 -- 172.21.15.156:0/3182162319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 msgr2=0x560ad95949a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:01.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.406+0000 7f259a54bb80 1 --2- 172.21.15.156:0/3182162319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad95949a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f256c000f60 tx=0x7f256c001120 comp rx=0 tx=0).stop 2023-11-28T13:11:01.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.406+0000 7f259a54bb80 1 -- 172.21.15.156:0/3182162319 shutdown_connections 2023-11-28T13:11:01.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.406+0000 7f259a54bb80 1 --2- 172.21.15.156:0/3182162319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a5120 0x560ad9594ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.406+0000 7f259a54bb80 1 --2- 172.21.15.156:0/3182162319 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a2130 0x560ad95b74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.406+0000 7f259a54bb80 1 --2- 172.21.15.156:0/3182162319 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x560ad95b6be0 0x560ad95a6440 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.406+0000 7f259a54bb80 1 --2- 172.21.15.156:0/3182162319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad95949a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.406+0000 7f259a54bb80 1 -- 172.21.15.156:0/3182162319 >> 172.21.15.156:0/3182162319 conn(0x560ad9577d60 msgr2=0x560ad957a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:01.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.406+0000 7f255f7fe700 0 client.6039 ms_handle_reset on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:01.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.406+0000 7f259a54bb80 1 -- 172.21.15.156:0/3182162319 shutdown_connections 2023-11-28T13:11:01.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.406+0000 7f259a54bb80 1 -- 172.21.15.156:0/3182162319 wait complete. 2023-11-28T13:11:01.417 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.407+0000 7f259a54bb80 20 client.6039 trim_cache size 0 max 16384 2023-11-28T13:11:01.428 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.425+0000 7f259a54bb80 1 Processor -- start 2023-11-28T13:11:01.428 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.425+0000 7f259a54bb80 1 -- start start 2023-11-28T13:11:01.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.426+0000 7f259a54bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a2130 0x560ad95b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.426+0000 7f259a54bb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a4810 0x560ad9573640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.426+0000 7f259a54bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a5120 0x560ad95783f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.426+0000 7f259a54bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad959ac00 con 0x560ad95a5120 2023-11-28T13:11:01.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.426+0000 7f259a54bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad95b8fe0 con 0x560ad95a2130 2023-11-28T13:11:01.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.426+0000 7f259a54bb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ad956e180 con 0x560ad95a4810 2023-11-28T13:11:01.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.426+0000 7f258519c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a5120 0x560ad95783f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.426+0000 7f258519c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a5120 0x560ad95783f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53158/0 (socket says 172.21.15.156:53158) 2023-11-28T13:11:01.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.426+0000 7f258519c700 1 -- 172.21.15.156:0/3894844846 learned_addr learned my addr 172.21.15.156:0/3894844846 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:01.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.426+0000 7f257ffff700 1 --2- 172.21.15.156:0/3894844846 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a4810 0x560ad9573640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.426+0000 7f258499b700 1 --2- 172.21.15.156:0/3894844846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a2130 0x560ad95b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.426+0000 7f258519c700 1 -- 172.21.15.156:0/3894844846 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a4810 msgr2=0x560ad9573640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:01.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.426+0000 7f258519c700 1 --2- 172.21.15.156:0/3894844846 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a4810 0x560ad9573640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.426+0000 7f258519c700 1 -- 172.21.15.156:0/3894844846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a2130 msgr2=0x560ad95b80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:01.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.426+0000 7f258519c700 1 --2- 172.21.15.156:0/3894844846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a2130 0x560ad95b80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.426+0000 7f258519c700 1 -- 172.21.15.156:0/3894844846 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ad9493140 con 0x560ad95a5120 2023-11-28T13:11:01.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.427+0000 7f258519c700 1 --2- 172.21.15.156:0/3894844846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a5120 0x560ad95783f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2578056680 tx=0x7f2578086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6f171c1af3133e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:01.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.427+0000 7f256affd700 1 -- 172.21.15.156:0/3894844846 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257805e7e0 con 0x560ad95a5120 2023-11-28T13:11:01.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.427+0000 7f256affd700 1 -- 172.21.15.156:0/3894844846 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2578069e90 con 0x560ad95a5120 2023-11-28T13:11:01.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.427+0000 7f256affd700 1 -- 172.21.15.156:0/3894844846 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578055be0 con 0x560ad95a5120 2023-11-28T13:11:01.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.427+0000 7f259a54bb80 1 -- 172.21.15.156:0/3894844846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a5120 msgr2=0x560ad95783f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:01.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.427+0000 7f259a54bb80 1 --2- 172.21.15.156:0/3894844846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a5120 0x560ad95783f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2578056680 tx=0x7f2578086040 comp rx=0 tx=0).stop 2023-11-28T13:11:01.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.427+0000 7f259a54bb80 1 -- 172.21.15.156:0/3894844846 shutdown_connections 2023-11-28T13:11:01.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.427+0000 7f259a54bb80 1 --2- 172.21.15.156:0/3894844846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a5120 0x560ad95783f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.427+0000 7f259a54bb80 1 --2- 172.21.15.156:0/3894844846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a2130 0x560ad95b80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.427+0000 7f259a54bb80 1 --2- 172.21.15.156:0/3894844846 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a4810 0x560ad9573640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.427+0000 7f259a54bb80 1 -- 172.21.15.156:0/3894844846 >> 172.21.15.156:0/3894844846 conn(0x560ad9577d60 msgr2=0x560ad95a2500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:01.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.428+0000 7f259a54bb80 1 -- 172.21.15.156:0/3894844846 shutdown_connections 2023-11-28T13:11:01.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.428+0000 7f259a54bb80 1 -- 172.21.15.156:0/3894844846 wait complete. 2023-11-28T13:11:01.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.428+0000 7f259a54bb80 1 -- start start 2023-11-28T13:11:01.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.428+0000 7f259a54bb80 1 Processor -- start 2023-11-28T13:11:01.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.429+0000 7f259a54bb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a2130 0x560ad95d8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.429+0000 7f259a54bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad95781c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.429+0000 7f259a54bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a5120 0x560ad9578700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.429+0000 7f259a54bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad9626090 con 0x560ad95a4810 2023-11-28T13:11:01.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.429+0000 7f259a54bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad959bcf0 con 0x560ad95a5120 2023-11-28T13:11:01.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.429+0000 7f259a54bb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ad95d86b0 con 0x560ad95a2130 2023-11-28T13:11:01.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.429+0000 7f259a54bb80 10 client.0 _ll_register_callbacks cb 0x560ad9372240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:11:01.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.430+0000 7f257ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad95781c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.430+0000 7f257ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad95781c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53168/0 (socket says 172.21.15.156:53168) 2023-11-28T13:11:01.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.430+0000 7f257ffff700 1 -- 172.21.15.156:0/3638332745 learned_addr learned my addr 172.21.15.156:0/3638332745 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:01.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.430+0000 7f258499b700 1 --2- 172.21.15.156:0/3638332745 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a2130 0x560ad95d8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.430+0000 7f258519c700 1 --2- 172.21.15.156:0/3638332745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a5120 0x560ad9578700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.430+0000 7f257ffff700 1 -- 172.21.15.156:0/3638332745 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a2130 msgr2=0x560ad95d8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:01.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.430+0000 7f257ffff700 1 --2- 172.21.15.156:0/3638332745 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a2130 0x560ad95d8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.430+0000 7f257ffff700 1 -- 172.21.15.156:0/3638332745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a5120 msgr2=0x560ad9578700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:01.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.430+0000 7f257ffff700 1 --2- 172.21.15.156:0/3638332745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a5120 0x560ad9578700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.430+0000 7f257ffff700 1 -- 172.21.15.156:0/3638332745 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25800b5520 con 0x560ad95a4810 2023-11-28T13:11:01.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.430+0000 7f257ffff700 1 --2- 172.21.15.156:0/3638332745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad95781c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x560ad9574c90 tx=0x7f256c001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af656f779b78001 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:01.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.430+0000 7f255f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:11:01.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.431+0000 7f255f7fe700 1 -- 172.21.15.156:0/3638332745 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c001520 con 0x560ad95a4810 2023-11-28T13:11:01.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.431+0000 7f255f7fe700 1 -- 172.21.15.156:0/3638332745 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f256c014960 con 0x560ad95a4810 2023-11-28T13:11:01.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.431+0000 7f259a54bb80 1 -- 172.21.15.156:0/3638332745 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ad948a160 con 0x560ad95a4810 2023-11-28T13:11:01.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.431+0000 7f255f7fe700 1 -- 172.21.15.156:0/3638332745 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f256c005130 con 0x560ad95a4810 2023-11-28T13:11:01.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.431+0000 7f259a54bb80 10 client.6045 fetch_fsmap learned FSMap version 31 2023-11-28T13:11:01.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.431+0000 7f259a54bb80 1 -- 172.21.15.156:0/3638332745 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x560ad9321200 con 0x560ad95a4810 2023-11-28T13:11:01.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.431+0000 7f255f7fe700 1 -- 172.21.15.156:0/3638332745 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560ad948a160 con 0x560ad95a4810 2023-11-28T13:11:01.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.431+0000 7f255f7fe700 1 -- 172.21.15.156:0/3638332745 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f256c001a30 con 0x560ad95a4810 2023-11-28T13:11:01.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.431+0000 7f259a54bb80 10 client.6045 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T13:11:01.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.431+0000 7f259a54bb80 10 client.6045 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:11:01.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.431+0000 7f259a54bb80 20 client.6045 populate_metadata read hostname 'smithi156' 2023-11-28T13:11:01.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.431+0000 7f259a54bb80 1 --2- 172.21.15.156:0/3638332745 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x560ad96251c0 0x560ad9625590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.431+0000 7f259a54bb80 4 client.6045 mds_command: new command op to 5919 tid=0[{"prefix": "status"}] 2023-11-28T13:11:01.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.431+0000 7f259a54bb80 1 -- 172.21.15.156:0/3638332745 --> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560ad9474170 con 0x560ad96251c0 2023-11-28T13:11:01.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.431+0000 7f258499b700 1 --2- 172.21.15.156:0/3638332745 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x560ad96251c0 0x560ad9625590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.441 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.432+0000 7f258499b700 1 --2- 172.21.15.156:0/3638332745 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x560ad96251c0 0x560ad9625590 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2df826bc458e18cb server_cookie=85a092314b3ba9c6 in_seq=0 out_seq=0 2023-11-28T13:11:01.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.432+0000 7f255f7fe700 10 client.6045 ms_handle_connect on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:01.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.432+0000 7f255f7fe700 1 -- 172.21.15.156:0/3638332745 <== mds.0 v2:172.21.15.156:6832/3314379816 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x560ad9474170 con 0x560ad96251c0 2023-11-28T13:11:01.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.432+0000 7f255f7fe700 10 client.6045 handle_command_reply: tid=0 2023-11-28T13:11:01.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.432+0000 7f259a54bb80 1 client.6045 shutdown 2023-11-28T13:11:01.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.432+0000 7f259a54bb80 1 -- 172.21.15.156:0/3638332745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 msgr2=0x560ad95781c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:01.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.432+0000 7f259a54bb80 1 --2- 172.21.15.156:0/3638332745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad95781c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x560ad9574c90 tx=0x7f256c001080 comp rx=0 tx=0).stop 2023-11-28T13:11:01.442 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.432+0000 7f259a54bb80 1 -- 172.21.15.156:0/3638332745 shutdown_connections 2023-11-28T13:11:01.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.432+0000 7f259a54bb80 1 --2- 172.21.15.156:0/3638332745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560ad95a5120 0x560ad9578700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.432+0000 7f259a54bb80 1 --2- 172.21.15.156:0/3638332745 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560ad95a2130 0x560ad95d8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.432+0000 7f259a54bb80 1 --2- 172.21.15.156:0/3638332745 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x560ad96251c0 0x560ad9625590 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.432+0000 7f259a54bb80 1 --2- 172.21.15.156:0/3638332745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ad95a4810 0x560ad95781c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.432+0000 7f255f7fe700 0 client.6045 ms_handle_reset on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:01.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.432+0000 7f259a54bb80 1 -- 172.21.15.156:0/3638332745 >> 172.21.15.156:0/3638332745 conn(0x560ad9577d60 msgr2=0x560ad95a63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:01.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.433+0000 7f259a54bb80 1 -- 172.21.15.156:0/3638332745 shutdown_connections 2023-11-28T13:11:01.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.433+0000 7f259a54bb80 1 -- 172.21.15.156:0/3638332745 wait complete. 2023-11-28T13:11:01.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.433+0000 7f259a54bb80 20 client.6045 trim_cache size 0 max 16384 2023-11-28T13:11:01.444 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:11:01.444 INFO:teuthology.orchestra.run.smithi156.stdout: "cluster_fsid": "12d9e849-dc60-46f0-8e8e-9e291633f61b", 2023-11-28T13:11:01.444 INFO:teuthology.orchestra.run.smithi156.stdout: "whoami": 0, 2023-11-28T13:11:01.444 INFO:teuthology.orchestra.run.smithi156.stdout: "id": 5919, 2023-11-28T13:11:01.444 INFO:teuthology.orchestra.run.smithi156.stdout: "want_state": "up:active", 2023-11-28T13:11:01.444 INFO:teuthology.orchestra.run.smithi156.stdout: "state": "up:active", 2023-11-28T13:11:01.444 INFO:teuthology.orchestra.run.smithi156.stdout: "fs_name": "cephfs", 2023-11-28T13:11:01.444 INFO:teuthology.orchestra.run.smithi156.stdout: "rank_uptime": 6.5023587999999997, 2023-11-28T13:11:01.444 INFO:teuthology.orchestra.run.smithi156.stdout: "mdsmap_epoch": 31, 2023-11-28T13:11:01.444 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_epoch": 59, 2023-11-28T13:11:01.445 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_epoch_barrier": 59, 2023-11-28T13:11:01.445 INFO:teuthology.orchestra.run.smithi156.stdout: "uptime": 12.425369331000001 2023-11-28T13:11:01.445 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:11:01.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.433+0000 7f2586bff700 1 -- 172.21.15.156:0/1677469516 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25640682f0 msgr2=0x7f256406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:01.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.433+0000 7f2586bff700 1 --2- 172.21.15.156:0/1677469516 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25640682f0 0x7f256406a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2574014040 tx=0x7f2574008030 comp rx=0 tx=0).stop 2023-11-28T13:11:01.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.433+0000 7f2586bff700 1 -- 172.21.15.156:0/1677469516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2580161340 msgr2=0x7f258014a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:01.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.433+0000 7f2586bff700 1 --2- 172.21.15.156:0/1677469516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2580161340 0x7f258014a480 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2578007e60 tx=0x7f257800e3c0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.433+0000 7f2586bff700 1 -- 172.21.15.156:0/1677469516 shutdown_connections 2023-11-28T13:11:01.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.433+0000 7f2586bff700 1 --2- 172.21.15.156:0/1677469516 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25640682f0 0x7f256406a7a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.433+0000 7f2586bff700 1 --2- 172.21.15.156:0/1677469516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2580161340 0x7f258014a480 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.433+0000 7f2586bff700 1 --2- 172.21.15.156:0/1677469516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25800cd7c0 0x7f25800ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.433+0000 7f2586bff700 1 --2- 172.21.15.156:0/1677469516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f258015c9b0 0x7f2580157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.433+0000 7f2586bff700 1 -- 172.21.15.156:0/1677469516 >> 172.21.15.156:0/1677469516 conn(0x7f2580072600 msgr2=0x7f2580159de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:01.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.433+0000 7f2586bff700 1 -- 172.21.15.156:0/1677469516 shutdown_connections 2023-11-28T13:11:01.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.433+0000 7f2586bff700 1 -- 172.21.15.156:0/1677469516 wait complete. 2023-11-28T13:11:01.453 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:11:01.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.648+0000 7fdac3dff700 1 Processor -- start 2023-11-28T13:11:01.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.648+0000 7fdac3dff700 1 -- start start 2023-11-28T13:11:01.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.649+0000 7fdac3dff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdabc0b0670 0x7fdabc15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.649+0000 7fdac3dff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fdabc0cc730 0x7fdabc0b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.649+0000 7fdac3dff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdabc0ce100 con 0x7fdabc0ad0a0 2023-11-28T13:11:01.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.649+0000 7fdac3dff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdabc0acc10 con 0x7fdabc0b0670 2023-11-28T13:11:01.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.649+0000 7fdac3dff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdabc15dd30 con 0x7fdabc0cc730 2023-11-28T13:11:01.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.650+0000 7fdac1b9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdabc0b0670 0x7fdabc15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.650+0000 7fdac1b9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdabc0b0670 0x7fdabc15d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53180/0 (socket says 172.21.15.156:53180) 2023-11-28T13:11:01.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.650+0000 7fdac239c700 1 --1- 172.21.15.156:0/889685960 >> v1:172.21.15.156:6789/0 conn(0x7fdabc0ad0a0 0x7fdabc0c91b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37794/0 (socket says 172.21.15.156:37794) 2023-11-28T13:11:01.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.650+0000 7fdac1b9b700 1 -- 172.21.15.156:0/889685960 learned_addr learned my addr 172.21.15.156:0/889685960 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:01.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.650+0000 7fdac139a700 1 --2- 172.21.15.156:0/889685960 >> v2:172.21.15.186:3300/0 conn(0x7fdabc0cc730 0x7fdabc0b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.650+0000 7fdac1b9b700 1 -- 172.21.15.156:0/889685960 >> v2:172.21.15.186:3300/0 conn(0x7fdabc0cc730 msgr2=0x7fdabc0b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:01.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.650+0000 7fdac1b9b700 1 --2- 172.21.15.156:0/889685960 >> v2:172.21.15.186:3300/0 conn(0x7fdabc0cc730 0x7fdabc0b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.650+0000 7fdac1b9b700 1 -- 172.21.15.156:0/889685960 >> v1:172.21.15.156:6789/0 conn(0x7fdabc0ad0a0 legacy=0x7fdabc0c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:01.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.650+0000 7fdac1b9b700 1 -- 172.21.15.156:0/889685960 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc0ac8c0 con 0x7fdabc0b0670 2023-11-28T13:11:01.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.650+0000 7fdac139a700 1 --2- 172.21.15.156:0/889685960 >> v2:172.21.15.186:3300/0 conn(0x7fdabc0cc730 0x7fdabc0b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:11:01.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.650+0000 7fdac1b9b700 1 --2- 172.21.15.156:0/889685960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdabc0b0670 0x7fdabc15d7f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fdabc06e100 tx=0x7fdab8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d23fadaf5c545e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:01.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.650+0000 7fdac0b99700 1 -- 172.21.15.156:0/889685960 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab800b870 con 0x7fdabc0b0670 2023-11-28T13:11:01.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.651+0000 7fdac0b99700 1 -- 172.21.15.156:0/889685960 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab800b9d0 con 0x7fdabc0b0670 2023-11-28T13:11:01.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.651+0000 7fdac0b99700 1 -- 172.21.15.156:0/889685960 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab800bd60 con 0x7fdabc0b0670 2023-11-28T13:11:01.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.651+0000 7fdac3dff700 1 -- 172.21.15.156:0/889685960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdabc0b0670 msgr2=0x7fdabc15d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:01.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.651+0000 7fdac3dff700 1 --2- 172.21.15.156:0/889685960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdabc0b0670 0x7fdabc15d7f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fdabc06e100 tx=0x7fdab8009bd0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.651+0000 7fdac3dff700 1 -- 172.21.15.156:0/889685960 shutdown_connections 2023-11-28T13:11:01.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.651+0000 7fdac3dff700 1 --2- 172.21.15.156:0/889685960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdabc0b0670 0x7fdabc15d7f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.651+0000 7fdac3dff700 1 --2- 172.21.15.156:0/889685960 >> v2:172.21.15.186:3300/0 conn(0x7fdabc0cc730 0x7fdabc0b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.651+0000 7fdac3dff700 1 -- 172.21.15.156:0/889685960 >> 172.21.15.156:0/889685960 conn(0x7fdabc0cb670 msgr2=0x7fdabc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:01.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.651+0000 7fdac3dff700 1 -- 172.21.15.156:0/889685960 shutdown_connections 2023-11-28T13:11:01.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.651+0000 7fdac3dff700 1 -- 172.21.15.156:0/889685960 wait complete. 2023-11-28T13:11:01.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.653+0000 7fdac3dff700 1 Processor -- start 2023-11-28T13:11:01.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.653+0000 7fdac3dff700 1 -- start start 2023-11-28T13:11:01.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.654+0000 7fdac3dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdabc0ad0a0 0x7fdabc15b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.654+0000 7fdac3dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdabc0b0670 0x7fdabc153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.654+0000 7fdac3dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdabc0cc730 0x7fdabc153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.654+0000 7fdac3dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc1541e0 con 0x7fdabc0cc730 2023-11-28T13:11:01.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.654+0000 7fdac3dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0c33a0 con 0x7fdabc0b0670 2023-11-28T13:11:01.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.654+0000 7fdac3dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0c3680 con 0x7fdabc0ad0a0 2023-11-28T13:11:01.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.654+0000 7fdac239c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdabc0cc730 0x7fdabc153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.654+0000 7fdac239c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdabc0cc730 0x7fdabc153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53186/0 (socket says 172.21.15.156:53186) 2023-11-28T13:11:01.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.654+0000 7fdac239c700 1 -- 172.21.15.156:0/1161415965 learned_addr learned my addr 172.21.15.156:0/1161415965 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:01.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.654+0000 7fdac139a700 1 --2- 172.21.15.156:0/1161415965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdabc0b0670 0x7fdabc153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.654+0000 7fdac1b9b700 1 --2- 172.21.15.156:0/1161415965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdabc0ad0a0 0x7fdabc15b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.655+0000 7fdac239c700 1 -- 172.21.15.156:0/1161415965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdabc0ad0a0 msgr2=0x7fdabc15b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:01.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.655+0000 7fdac239c700 1 --2- 172.21.15.156:0/1161415965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdabc0ad0a0 0x7fdabc15b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.655+0000 7fdac239c700 1 -- 172.21.15.156:0/1161415965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdabc0b0670 msgr2=0x7fdabc153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:01.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.655+0000 7fdac239c700 1 --2- 172.21.15.156:0/1161415965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdabc0b0670 0x7fdabc153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.655+0000 7fdac239c700 1 -- 172.21.15.156:0/1161415965 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab8009550 con 0x7fdabc0cc730 2023-11-28T13:11:01.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.655+0000 7fdac239c700 1 --2- 172.21.15.156:0/1161415965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdabc0cc730 0x7fdabc153ca0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdab400cbc0 tx=0x7fdab400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:01.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.655+0000 7fdab27fc700 1 -- 172.21.15.156:0/1161415965 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab400a9e0 con 0x7fdabc0cc730 2023-11-28T13:11:01.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.655+0000 7fdab27fc700 1 -- 172.21.15.156:0/1161415965 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab400ab40 con 0x7fdabc0cc730 2023-11-28T13:11:01.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.655+0000 7fdac3dff700 1 -- 172.21.15.156:0/1161415965 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdabc0ce330 con 0x7fdabc0cc730 2023-11-28T13:11:01.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.655+0000 7fdab27fc700 1 -- 172.21.15.156:0/1161415965 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab4007070 con 0x7fdabc0cc730 2023-11-28T13:11:01.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.655+0000 7fdac3dff700 1 -- 172.21.15.156:0/1161415965 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdabc0b0f80 con 0x7fdabc0cc730 2023-11-28T13:11:01.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.656+0000 7fdac3dff700 1 -- 172.21.15.156:0/1161415965 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdabc0ce610 con 0x7fdabc0cc730 2023-11-28T13:11:01.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.657+0000 7fdab27fc700 1 -- 172.21.15.156:0/1161415965 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdab4019030 con 0x7fdabc0cc730 2023-11-28T13:11:01.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.658+0000 7fdab27fc700 1 --2- 172.21.15.156:0/1161415965 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdaa00682f0 0x7fdaa006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:01.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.658+0000 7fdab27fc700 1 -- 172.21.15.156:0/1161415965 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdab4085820 con 0x7fdabc0cc730 2023-11-28T13:11:01.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.658+0000 7fdac1b9b700 1 --2- 172.21.15.156:0/1161415965 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdaa00682f0 0x7fdaa006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:01.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.659+0000 7fdac1b9b700 1 --2- 172.21.15.156:0/1161415965 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdaa00682f0 0x7fdaa006a7a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdab8014040 tx=0x7fdab8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:01.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.662+0000 7fdab27fc700 1 -- 172.21.15.156:0/1161415965 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdab404b5e0 con 0x7fdabc0cc730 2023-11-28T13:11:01.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.895+0000 7fdac3dff700 1 -- 172.21.15.156:0/1161415965 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdabc0745c0 con 0x7fdabc0cc730 2023-11-28T13:11:01.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.895+0000 7fdab27fc700 1 -- 172.21.15.156:0/1161415965 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdab4017070 con 0x7fdabc0cc730 2023-11-28T13:11:01.897 INFO:teuthology.orchestra.run.smithi156.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T13:11:01.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.898+0000 7fdac3dff700 1 -- 172.21.15.156:0/1161415965 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdaa00682f0 msgr2=0x7fdaa006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:01.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.898+0000 7fdac3dff700 1 --2- 172.21.15.156:0/1161415965 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdaa00682f0 0x7fdaa006a7a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdab8014040 tx=0x7fdab8008030 comp rx=0 tx=0).stop 2023-11-28T13:11:01.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.898+0000 7fdac3dff700 1 -- 172.21.15.156:0/1161415965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdabc0cc730 msgr2=0x7fdabc153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:01.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.898+0000 7fdac3dff700 1 --2- 172.21.15.156:0/1161415965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdabc0cc730 0x7fdabc153ca0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdab400cbc0 tx=0x7fdab400ced0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.898+0000 7fdac3dff700 1 -- 172.21.15.156:0/1161415965 shutdown_connections 2023-11-28T13:11:01.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.898+0000 7fdac3dff700 1 --2- 172.21.15.156:0/1161415965 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdaa00682f0 0x7fdaa006a7a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.898+0000 7fdac3dff700 1 --2- 172.21.15.156:0/1161415965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdabc0cc730 0x7fdabc153ca0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.898+0000 7fdac3dff700 1 --2- 172.21.15.156:0/1161415965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdabc0ad0a0 0x7fdabc15b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.898+0000 7fdac3dff700 1 --2- 172.21.15.156:0/1161415965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdabc0b0670 0x7fdabc153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:01.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.898+0000 7fdac3dff700 1 -- 172.21.15.156:0/1161415965 >> 172.21.15.156:0/1161415965 conn(0x7fdabc0cb670 msgr2=0x7fdabc0ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:01.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.898+0000 7fdac3dff700 1 -- 172.21.15.156:0/1161415965 shutdown_connections 2023-11-28T13:11:01.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:01.898+0000 7fdac3dff700 1 -- 172.21.15.156:0/1161415965 wait complete. 2023-11-28T13:11:01.918 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T13:11:01.918 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:11:01.919 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-11-28T13:11:01.919 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T13:11:01.919 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T13:11:01.919 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T13:11:01.919 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T13:11:01.919 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:11:01.919 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:11:01.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:11:01.920 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-11-28T13:11:01.966 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:11:01.966 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:11:01.966 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:11:01.966 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:11:01.966 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-28T13:11:01.966 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:11:01.966 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:11:01.966 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:11:01.967 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f0 2023-11-28T13:11:01.967 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:11:01.967 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:11:01.967 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f1 2023-11-28T13:11:01.967 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:11:01.967 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-11-28T13:11:01.967 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:11:01.967 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-11-28T13:11:01.967 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T13:11:01.968 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42002sec preferred_lft 42002sec 2023-11-28T13:11:01.968 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::d27c:fc8f:2674:dd68/64 scope link noprefixroute 2023-11-28T13:11:01.968 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:11:01.968 INFO:teuthology.orchestra.run.smithi156.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:11:01.968 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 3e:ff:33:55:aa:40 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:11:01.968 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T13:11:01.968 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:11:01.968 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::4497:b0ff:fe27:d641/64 scope link 2023-11-28T13:11:01.969 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:11:01.969 INFO:teuthology.orchestra.run.smithi156.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:11:01.969 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 96:cd:43:a7:29:10 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:11:01.969 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::94cd:43ff:fea7:2910/64 scope link 2023-11-28T13:11:01.969 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:11:01.969 INFO:teuthology.orchestra.run.smithi156.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:11:01.969 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 3e:ff:33:55:aa:40 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T13:11:01.969 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::3cff:33ff:fe55:aa40/64 scope link 2023-11-28T13:11:01.969 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:11:01.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:11:01.970 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:11:02.074 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-28T13:11:02.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:11:02.077 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-11-28T13:11:02.123 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-11-28T13:11:02.123 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:11:02.123 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T13:11:02.123 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T13:11:02.124 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:11:02.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:11:02.125 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-28T13:11:02.125 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-28T13:11:02.125 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T13:11:02.125 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T13:11:02.125 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T13:11:02.125 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-28T13:11:02.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:11:02.337 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T13:11:02.381 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T13:11:02.382 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T13:11:02.382 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:11:02.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:11:02.383 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.0 up 2023-11-28T13:11:02.493 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:11:02.494 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T13:11:02.541 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:11:02.541 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:11:02.542 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:11:02.626 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T13:11:02.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:11:02.627 DEBUG:teuthology.orchestra.run.smithi156:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:11:02.771 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:11:02.772 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:11:02.772 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T13:11:02.790 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:11:02.791 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:11:02.791 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T13:11:02.791 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T13:11:02.791 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:11:02.813 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:11:02.814 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:11:02.814 INFO:teuthology.orchestra.run.smithi156.stdout:invalid new device string format 2023-11-28T13:11:02.814 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:3300,172.21.15.186:3300,172.21.15.186:3301" 2023-11-28T13:11:02.814 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:11:02.814 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790" 2023-11-28T13:11:02.814 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.156:6789,172.21.15.186:6789,172.21.15.186:6790:/ 2023-11-28T13:11:02.814 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=12d9e849-dc60-46f0-8e8e-9e291633f61b" will pass to kernel 2023-11-28T13:11:02.819 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T13:11:02.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:11:02.820 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:11:02.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:11:02.889 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:11:02.989 INFO:teuthology.orchestra.run.smithi156.stdout:client5836 2023-11-28T13:11:02.990 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:11:02.990 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:11:03.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:11:03.034 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2023-11-28T13:11:03.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:11:03.125 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2023-11-28T13:11:03.171 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:11:03.172 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-11-28T13:11:03.172 DEBUG:teuthology.orchestra.run.smithi156:> import stat 2023-11-28T13:11:03.172 DEBUG:teuthology.orchestra.run.smithi156:> 2023-11-28T13:11:03.172 DEBUG:teuthology.orchestra.run.smithi156:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2023-11-28T13:11:03.172 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-11-28T13:11:03.325 INFO:teuthology.orchestra.run.smithi156.stdout:1099511627777 2023-11-28T13:11:03.363 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:11:03.368 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:11:03.509 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:11:03.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.704+0000 7feb19dff700 1 Processor -- start 2023-11-28T13:11:03.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.705+0000 7feb19dff700 1 -- start start 2023-11-28T13:11:03.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.706+0000 7feb19dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb140cd7c0 0x7feb140cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:03.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.706+0000 7feb19dff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb140d1950 con 0x7feb14161340 2023-11-28T13:11:03.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.706+0000 7feb19dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb140ce140 con 0x7feb1415c9b0 2023-11-28T13:11:03.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.706+0000 7feb19dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb140b5800 con 0x7feb140cd7c0 2023-11-28T13:11:03.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.706+0000 7feb0bfff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7feb14161340 0x7feb140d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37800/0 (socket says 172.21.15.156:37800) 2023-11-28T13:11:03.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.706+0000 7feb0bfff700 1 -- 172.21.15.156:0/1250752586 learned_addr learned my addr 172.21.15.156:0/1250752586 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:11:03.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.706+0000 7feb13fff700 1 --2- 172.21.15.156:0/1250752586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb140cd7c0 0x7feb140cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:03.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.706+0000 7feb12ffd700 1 -- 172.21.15.156:0/1250752586 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2485623719 0 0) 0x7feb140d1950 con 0x7feb14161340 2023-11-28T13:11:03.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.706+0000 7feb12ffd700 1 -- 172.21.15.156:0/1250752586 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb000035b0 con 0x7feb14161340 2023-11-28T13:11:03.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.707+0000 7feb12ffd700 1 -- 172.21.15.156:0/1250752586 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069133887 0 0) 0x7feb140ce140 con 0x7feb1415c9b0 2023-11-28T13:11:03.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.707+0000 7feb12ffd700 1 -- 172.21.15.156:0/1250752586 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb140d1950 con 0x7feb1415c9b0 2023-11-28T13:11:03.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.707+0000 7feb12ffd700 1 -- 172.21.15.156:0/1250752586 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1055822531 0 0) 0x7feb000035b0 con 0x7feb14161340 2023-11-28T13:11:03.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.707+0000 7feb13fff700 1 -- 172.21.15.156:0/1250752586 >> v1:172.21.15.186:6789/0 conn(0x7feb1415c9b0 legacy=0x7feb140b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:03.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.707+0000 7feb13fff700 1 -- 172.21.15.156:0/1250752586 >> v1:172.21.15.156:6789/0 conn(0x7feb14161340 legacy=0x7feb140d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:03.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.707+0000 7feb13fff700 1 -- 172.21.15.156:0/1250752586 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb140d24f0 con 0x7feb140cd7c0 2023-11-28T13:11:03.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.707+0000 7feb13fff700 1 --2- 172.21.15.156:0/1250752586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb140cd7c0 0x7feb140cdc00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009dc0 tx=0x7feb0c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f04d8e4d264051e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:03.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.707+0000 7feb12ffd700 1 -- 172.21.15.156:0/1250752586 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c00e070 con 0x7feb140cd7c0 2023-11-28T13:11:03.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.707+0000 7feb12ffd700 1 -- 172.21.15.156:0/1250752586 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0c003a20 con 0x7feb140cd7c0 2023-11-28T13:11:03.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.707+0000 7feb12ffd700 1 -- 172.21.15.156:0/1250752586 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc001670 con 0x7feb140cd7c0 2023-11-28T13:11:03.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.708+0000 7feb19dff700 1 -- 172.21.15.156:0/1250752586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb140cd7c0 msgr2=0x7feb140cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:03.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.708+0000 7feb19dff700 1 --2- 172.21.15.156:0/1250752586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb140cd7c0 0x7feb140cdc00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009dc0 tx=0x7feb0c00b330 comp rx=0 tx=0).stop 2023-11-28T13:11:03.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.708+0000 7feb19dff700 1 -- 172.21.15.156:0/1250752586 shutdown_connections 2023-11-28T13:11:03.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.708+0000 7feb19dff700 1 --2- 172.21.15.156:0/1250752586 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb140cd7c0 0x7feb140cdc00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:03.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.708+0000 7feb19dff700 1 -- 172.21.15.156:0/1250752586 >> 172.21.15.156:0/1250752586 conn(0x7feb14072600 msgr2=0x7feb140cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:03.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.708+0000 7feb19dff700 1 -- 172.21.15.156:0/1250752586 shutdown_connections 2023-11-28T13:11:03.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.708+0000 7feb19dff700 1 -- 172.21.15.156:0/1250752586 wait complete. 2023-11-28T13:11:03.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.710+0000 7feb19dff700 1 Processor -- start 2023-11-28T13:11:03.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.710+0000 7feb19dff700 1 -- start start 2023-11-28T13:11:03.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.711+0000 7feb19dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb140cd7c0 0x7feb140c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:03.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.711+0000 7feb19dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb1415c9b0 0x7feb140b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:03.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.711+0000 7feb19dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb14161340 0x7feb140b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:03.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.711+0000 7feb19dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb141e6480 con 0x7feb140cd7c0 2023-11-28T13:11:03.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.711+0000 7feb19dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb14157270 con 0x7feb1415c9b0 2023-11-28T13:11:03.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.711+0000 7feb19dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb141e66d0 con 0x7feb14161340 2023-11-28T13:11:03.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.711+0000 7feb137fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb140cd7c0 0x7feb140c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:03.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.711+0000 7feb137fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb140cd7c0 0x7feb140c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53192/0 (socket says 172.21.15.156:53192) 2023-11-28T13:11:03.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.711+0000 7feb137fe700 1 -- 172.21.15.156:0/327118661 learned_addr learned my addr 172.21.15.156:0/327118661 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:03.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.711+0000 7feb0bfff700 1 --2- 172.21.15.156:0/327118661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb1415c9b0 0x7feb140b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:03.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.711+0000 7feb13fff700 1 --2- 172.21.15.156:0/327118661 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb14161340 0x7feb140b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:03.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.711+0000 7feb137fe700 1 -- 172.21.15.156:0/327118661 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb14161340 msgr2=0x7feb140b8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:03.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.711+0000 7feb137fe700 1 --2- 172.21.15.156:0/327118661 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb14161340 0x7feb140b8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:03.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.711+0000 7feb137fe700 1 -- 172.21.15.156:0/327118661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb1415c9b0 msgr2=0x7feb140b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:03.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.711+0000 7feb137fe700 1 --2- 172.21.15.156:0/327118661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb1415c9b0 0x7feb140b84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:03.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.711+0000 7feb137fe700 1 -- 172.21.15.156:0/327118661 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c009a70 con 0x7feb140cd7c0 2023-11-28T13:11:03.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.711+0000 7feb13fff700 1 --2- 172.21.15.156:0/327118661 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb14161340 0x7feb140b8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:11:03.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.712+0000 7feb137fe700 1 --2- 172.21.15.156:0/327118661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb140cd7c0 0x7feb140c87b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7feb0400c450 tx=0x7feb0400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:03.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.712+0000 7feb10ff9700 1 -- 172.21.15.156:0/327118661 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb040052d0 con 0x7feb140cd7c0 2023-11-28T13:11:03.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.712+0000 7feb10ff9700 1 -- 172.21.15.156:0/327118661 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb04005430 con 0x7feb140cd7c0 2023-11-28T13:11:03.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.712+0000 7feb19dff700 1 -- 172.21.15.156:0/327118661 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb140d1300 con 0x7feb140cd7c0 2023-11-28T13:11:03.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.712+0000 7feb10ff9700 1 -- 172.21.15.156:0/327118661 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb040038c0 con 0x7feb140cd7c0 2023-11-28T13:11:03.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.712+0000 7feb19dff700 1 -- 172.21.15.156:0/327118661 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb140b5520 con 0x7feb140cd7c0 2023-11-28T13:11:03.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.713+0000 7feb19dff700 1 -- 172.21.15.156:0/327118661 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb140d15e0 con 0x7feb140cd7c0 2023-11-28T13:11:03.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.714+0000 7feb10ff9700 1 -- 172.21.15.156:0/327118661 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feb04019030 con 0x7feb140cd7c0 2023-11-28T13:11:03.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.715+0000 7feb10ff9700 1 --2- 172.21.15.156:0/327118661 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feaf40682f0 0x7feaf406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:03.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.715+0000 7feb10ff9700 1 -- 172.21.15.156:0/327118661 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feb04084ff0 con 0x7feb140cd7c0 2023-11-28T13:11:03.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.715+0000 7feb0bfff700 1 --2- 172.21.15.156:0/327118661 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feaf40682f0 0x7feaf406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:03.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.716+0000 7feb0bfff700 1 --2- 172.21.15.156:0/327118661 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feaf40682f0 0x7feaf406a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7feafc001380 tx=0x7feafc0015f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:03.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.719+0000 7feb10ff9700 1 -- 172.21.15.156:0/327118661 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feb04051850 con 0x7feb140cd7c0 2023-11-28T13:11:03.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.952+0000 7feb19dff700 1 -- 172.21.15.156:0/327118661 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb140745c0 con 0x7feb140cd7c0 2023-11-28T13:11:03.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.953+0000 7feb10ff9700 1 -- 172.21.15.156:0/327118661 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4658 (secure 0 0 0) 0x7feb04054e70 con 0x7feb140cd7c0 2023-11-28T13:11:03.955 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:11:03.955 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3942362276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3942362276},{"type":"v1","addr":"172.21.15.186:6835","nonce":3942362276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/722198303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":722198303},{"type":"v1","addr":"172.21.15.186:6837","nonce":722198303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5916,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3601975324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3601975324},{"type":"v1","addr":"172.21.15.156:6835","nonce":3601975324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:10:54.928601+0000","modified":"2023-11-28T13:10:57.942034+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5919":{"gid":5919,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6834/3314379816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3314379816},{"type":"v1","addr":"172.21.15.156:6834","nonce":3314379816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:11:03.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.955+0000 7feb19dff700 1 -- 172.21.15.156:0/327118661 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feaf40682f0 msgr2=0x7feaf406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:03.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.955+0000 7feb19dff700 1 --2- 172.21.15.156:0/327118661 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feaf40682f0 0x7feaf406a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7feafc001380 tx=0x7feafc0015f0 comp rx=0 tx=0).stop 2023-11-28T13:11:03.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.955+0000 7feb19dff700 1 -- 172.21.15.156:0/327118661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb140cd7c0 msgr2=0x7feb140c87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:03.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.955+0000 7feb19dff700 1 --2- 172.21.15.156:0/327118661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb140cd7c0 0x7feb140c87b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7feb0400c450 tx=0x7feb0400c810 comp rx=0 tx=0).stop 2023-11-28T13:11:03.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.955+0000 7feb19dff700 1 -- 172.21.15.156:0/327118661 shutdown_connections 2023-11-28T13:11:03.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.955+0000 7feb19dff700 1 --2- 172.21.15.156:0/327118661 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7feaf40682f0 0x7feaf406a7a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:03.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.955+0000 7feb19dff700 1 --2- 172.21.15.156:0/327118661 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feb14161340 0x7feb140b8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:03.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.955+0000 7feb19dff700 1 --2- 172.21.15.156:0/327118661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb140cd7c0 0x7feb140c87b0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:03.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.955+0000 7feb19dff700 1 --2- 172.21.15.156:0/327118661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb1415c9b0 0x7feb140b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:03.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.955+0000 7feb19dff700 1 -- 172.21.15.156:0/327118661 >> 172.21.15.156:0/327118661 conn(0x7feb14072600 msgr2=0x7feb1406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:03.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.955+0000 7feb19dff700 1 -- 172.21.15.156:0/327118661 shutdown_connections 2023-11-28T13:11:03.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:03.956+0000 7feb19dff700 1 -- 172.21.15.156:0/327118661 wait complete. 2023-11-28T13:11:03.960 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 31 2023-11-28T13:11:03.975 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-11-28T13:11:04.201 INFO:teuthology.orchestra.run.smithi156.stdout:{"message":"","return_code":0} 2023-11-28T13:11:04.202 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T13:11:04.202 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T13:11:04.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.287+0000 7fa302e48080 1 Processor -- start 2023-11-28T13:11:04.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.287+0000 7fa302e48080 1 -- start start 2023-11-28T13:11:04.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.288+0000 7fa302e48080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2dc003b80 0x5631c81acd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.288+0000 7fa302e48080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5631c81b4d70 0x5631c81b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.288+0000 7fa302e48080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631c81b0a10 0x5631c81b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.288+0000 7fa302e48080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c8082000 con 0x7fa2dc003b80 2023-11-28T13:11:04.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.288+0000 7fa302e48080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c8014fe0 con 0x5631c81b0a10 2023-11-28T13:11:04.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.288+0000 7fa302e48080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631c811c070 con 0x5631c81b4d70 2023-11-28T13:11:04.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.288+0000 7fa2f459b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2dc003b80 0x5631c81acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.288+0000 7fa2f459b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2dc003b80 0x5631c81acd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53198/0 (socket says 172.21.15.156:53198) 2023-11-28T13:11:04.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.288+0000 7fa2f459b700 1 -- 172.21.15.156:0/160077813 learned_addr learned my addr 172.21.15.156:0/160077813 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:04.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.288+0000 7fa2f4d9c700 1 --2- 172.21.15.156:0/160077813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631c81b0a10 0x5631c81b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.288+0000 7fa2f3d9a700 1 --2- 172.21.15.156:0/160077813 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5631c81b4d70 0x5631c81b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.288+0000 7fa2f459b700 1 -- 172.21.15.156:0/160077813 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5631c81b4d70 msgr2=0x5631c81b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:04.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.288+0000 7fa2f459b700 1 --2- 172.21.15.156:0/160077813 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5631c81b4d70 0x5631c81b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.288+0000 7fa2f459b700 1 -- 172.21.15.156:0/160077813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631c81b0a10 msgr2=0x5631c81b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:04.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.288+0000 7fa2f459b700 1 --2- 172.21.15.156:0/160077813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631c81b0a10 0x5631c81b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.288+0000 7fa2f459b700 1 -- 172.21.15.156:0/160077813 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631c802d360 con 0x7fa2dc003b80 2023-11-28T13:11:04.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.289+0000 7fa2f459b700 1 --2- 172.21.15.156:0/160077813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2dc003b80 0x5631c81acd90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40097e0 tx=0x7fa2e4005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e080918b2fa7eb99 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.289+0000 7fa2f2d98700 1 -- 172.21.15.156:0/160077813 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e400f050 con 0x7fa2dc003b80 2023-11-28T13:11:04.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.289+0000 7fa2f2d98700 1 -- 172.21.15.156:0/160077813 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e4010040 con 0x7fa2dc003b80 2023-11-28T13:11:04.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.289+0000 7fa2f2d98700 1 -- 172.21.15.156:0/160077813 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4008b90 con 0x7fa2dc003b80 2023-11-28T13:11:04.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.289+0000 7fa302e48080 1 -- 172.21.15.156:0/160077813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2dc003b80 msgr2=0x5631c81acd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.289+0000 7fa302e48080 1 --2- 172.21.15.156:0/160077813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2dc003b80 0x5631c81acd90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40097e0 tx=0x7fa2e4005a00 comp rx=0 tx=0).stop 2023-11-28T13:11:04.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.289+0000 7fa302e48080 1 -- 172.21.15.156:0/160077813 shutdown_connections 2023-11-28T13:11:04.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.289+0000 7fa302e48080 1 --2- 172.21.15.156:0/160077813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631c81b0a10 0x5631c81b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.289+0000 7fa302e48080 1 --2- 172.21.15.156:0/160077813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2dc003b80 0x5631c81acd90 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.289+0000 7fa302e48080 1 --2- 172.21.15.156:0/160077813 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5631c81b4d70 0x5631c81b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.289+0000 7fa302e48080 1 -- 172.21.15.156:0/160077813 >> 172.21.15.156:0/160077813 conn(0x5631c81269d0 msgr2=0x5631c81b5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:04.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.289+0000 7fa302e48080 1 -- 172.21.15.156:0/160077813 shutdown_connections 2023-11-28T13:11:04.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.289+0000 7fa302e48080 1 -- 172.21.15.156:0/160077813 wait complete. 2023-11-28T13:11:04.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.290+0000 7fa302e48080 1 Processor -- start 2023-11-28T13:11:04.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.290+0000 7fa302e48080 1 -- start start 2023-11-28T13:11:04.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.291+0000 7fa302e48080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631c81b0a10 0x5631c81a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.291+0000 7fa302e48080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631c81b4d70 0x5631c81a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.291+0000 7fa302e48080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2dc003b80 0x5631c81a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.291+0000 7fa302e48080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c811dcc0 con 0x5631c81b0a10 2023-11-28T13:11:04.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.291+0000 7fa302e48080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c81b5bd0 con 0x5631c81b4d70 2023-11-28T13:11:04.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.291+0000 7fa302e48080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631c81a46e0 con 0x7fa2dc003b80 2023-11-28T13:11:04.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.291+0000 7fa2f459b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631c81b0a10 0x5631c81a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.291+0000 7fa2f459b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631c81b0a10 0x5631c81a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53204/0 (socket says 172.21.15.156:53204) 2023-11-28T13:11:04.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.291+0000 7fa2f459b700 1 -- 172.21.15.156:0/4283739356 learned_addr learned my addr 172.21.15.156:0/4283739356 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:04.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.292+0000 7fa2f4d9c700 1 --2- 172.21.15.156:0/4283739356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2dc003b80 0x5631c81a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.292+0000 7fa2f3d9a700 1 --2- 172.21.15.156:0/4283739356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631c81b4d70 0x5631c81a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.292+0000 7fa2f459b700 1 -- 172.21.15.156:0/4283739356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2dc003b80 msgr2=0x5631c81a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.292+0000 7fa2f459b700 1 --2- 172.21.15.156:0/4283739356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa2dc003b80 0x5631c81a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.292+0000 7fa2f459b700 1 -- 172.21.15.156:0/4283739356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631c81b4d70 msgr2=0x5631c81a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.292+0000 7fa2f459b700 1 --2- 172.21.15.156:0/4283739356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631c81b4d70 0x5631c81a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.292+0000 7fa2f459b700 1 -- 172.21.15.156:0/4283739356 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e4009410 con 0x5631c81b0a10 2023-11-28T13:11:04.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.292+0000 7fa2f3d9a700 1 --2- 172.21.15.156:0/4283739356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5631c81b4d70 0x5631c81a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:11:04.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.292+0000 7fa2f459b700 1 --2- 172.21.15.156:0/4283739356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631c81b0a10 0x5631c81a0930 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4005a40 tx=0x7fa2e4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.292+0000 7fa2f1d96700 1 -- 172.21.15.156:0/4283739356 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4015870 con 0x5631c81b0a10 2023-11-28T13:11:04.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.292+0000 7fa2f1d96700 1 -- 172.21.15.156:0/4283739356 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e4023070 con 0x5631c81b0a10 2023-11-28T13:11:04.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.292+0000 7fa2f1d96700 1 -- 172.21.15.156:0/4283739356 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4015c30 con 0x5631c81b0a10 2023-11-28T13:11:04.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.292+0000 7fa302e48080 1 -- 172.21.15.156:0/4283739356 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5631c8122440 con 0x5631c81b0a10 2023-11-28T13:11:04.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.293+0000 7fa302e48080 1 -- 172.21.15.156:0/4283739356 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631c8020830 con 0x5631c81b0a10 2023-11-28T13:11:04.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.294+0000 7fa2f1d96700 1 -- 172.21.15.156:0/4283739356 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa2e4031070 con 0x5631c81b0a10 2023-11-28T13:11:04.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.295+0000 7fa2f1d96700 1 --2- 172.21.15.156:0/4283739356 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa2e0066c10 0x7fa2e00690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.295+0000 7fa2f1d96700 1 -- 172.21.15.156:0/4283739356 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa2e4025070 con 0x5631c81b0a10 2023-11-28T13:11:04.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.295+0000 7fa302e48080 1 --2- 172.21.15.156:0/4283739356 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x5631c8109120 0x5631c82496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.295+0000 7fa302e48080 1 -- 172.21.15.156:0/4283739356 --> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59) v8 -- 0x5631c811f080 con 0x5631c8109120 2023-11-28T13:11:04.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.295+0000 7fa2f3d9a700 1 --2- 172.21.15.156:0/4283739356 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa2e0066c10 0x7fa2e00690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.296+0000 7fa2f4d9c700 1 --2- 172.21.15.156:0/4283739356 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x5631c8109120 0x5631c82496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.296+0000 7fa2f3d9a700 1 --2- 172.21.15.156:0/4283739356 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa2e0066c10 0x7fa2e00690c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x5631c81b09e0 tx=0x7fa2e8001730 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.296+0000 7fa2f4d9c700 1 --2- 172.21.15.156:0/4283739356 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x5631c8109120 0x5631c82496f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.297+0000 7fa2f4d9c700 1 -- 172.21.15.156:0/4283739356 <== osd.6 v2:172.21.15.186:6824/132946 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7fa2ec001250 con 0x5631c8109120 2023-11-28T13:11:04.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.297+0000 7fa302e48080 1 -- 172.21.15.156:0/4283739356 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x5631c8109120 msgr2=0x5631c82496f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.297+0000 7fa302e48080 1 --2- 172.21.15.156:0/4283739356 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x5631c8109120 0x5631c82496f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.297+0000 7fa302e48080 1 -- 172.21.15.156:0/4283739356 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa2e0066c10 msgr2=0x7fa2e00690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.297+0000 7fa302e48080 1 --2- 172.21.15.156:0/4283739356 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa2e0066c10 0x7fa2e00690c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x5631c81b09e0 tx=0x7fa2e8001730 comp rx=0 tx=0).stop 2023-11-28T13:11:04.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.297+0000 7fa302e48080 1 -- 172.21.15.156:0/4283739356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631c81b0a10 msgr2=0x5631c81a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.297+0000 7fa302e48080 1 --2- 172.21.15.156:0/4283739356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631c81b0a10 0x5631c81a0930 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4005a40 tx=0x7fa2e4002fb0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.297+0000 7fa2f4d9c700 1 -- 172.21.15.156:0/4283739356 reap_dead start 2023-11-28T13:11:04.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.297+0000 7fa302e48080 1 -- 172.21.15.156:0/4283739356 shutdown_connections 2023-11-28T13:11:04.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.297+0000 7fa302e48080 1 -- 172.21.15.156:0/4283739356 >> 172.21.15.156:0/4283739356 conn(0x5631c81269d0 msgr2=0x5631c81b5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:04.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.297+0000 7fa302e48080 1 -- 172.21.15.156:0/4283739356 shutdown_connections 2023-11-28T13:11:04.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.297+0000 7fa302e48080 1 -- 172.21.15.156:0/4283739356 wait complete. 2023-11-28T13:11:04.304 INFO:teuthology.orchestra.run.smithi156.stdout:Xfiledir_test 2023-11-28T13:11:04.304 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:11:04.304 DEBUG:teuthology.orchestra.run.smithi156:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T13:11:04.402 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:11:04.402 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 1099511627777, 2023-11-28T13:11:04.402 INFO:teuthology.orchestra.run.smithi156.stdout: "ancestors": [ 2023-11-28T13:11:04.402 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:11:04.402 INFO:teuthology.orchestra.run.smithi156.stdout: "dirino": 1099511627776, 2023-11-28T13:11:04.402 INFO:teuthology.orchestra.run.smithi156.stdout: "dname": "file", 2023-11-28T13:11:04.402 INFO:teuthology.orchestra.run.smithi156.stdout: "version": 2 2023-11-28T13:11:04.402 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-11-28T13:11:04.402 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:11:04.403 INFO:teuthology.orchestra.run.smithi156.stdout: "dirino": 1, 2023-11-28T13:11:04.403 INFO:teuthology.orchestra.run.smithi156.stdout: "dname": "dir_test", 2023-11-28T13:11:04.403 INFO:teuthology.orchestra.run.smithi156.stdout: "version": 4 2023-11-28T13:11:04.403 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:11:04.403 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2023-11-28T13:11:04.403 INFO:teuthology.orchestra.run.smithi156.stdout: "pool": 10, 2023-11-28T13:11:04.403 INFO:teuthology.orchestra.run.smithi156.stdout: "old_pools": [] 2023-11-28T13:11:04.403 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:11:04.403 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:11:04.406 DEBUG:teuthology.orchestra.run.smithi156:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2023-11-28T13:11:04.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.490+0000 7f3405435080 1 Processor -- start 2023-11-28T13:11:04.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.490+0000 7f3405435080 1 -- start start 2023-11-28T13:11:04.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.491+0000 7f3405435080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5615aac9e930 0x5615aad248c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.491+0000 7f3405435080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615aad2e260 0x5615aad321e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.491+0000 7f3405435080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5615aad2e630 0x5615aac9aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.491+0000 7f3405435080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615aabfe000 con 0x5615aad2e260 2023-11-28T13:11:04.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.491+0000 7f3405435080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615aab90fe0 con 0x5615aad2e630 2023-11-28T13:11:04.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.491+0000 7f3405435080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5615aac87990 con 0x5615aac9e930 2023-11-28T13:11:04.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.492+0000 7f33f639a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615aad2e260 0x5615aad321e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.492+0000 7f33f639a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615aad2e260 0x5615aad321e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53220/0 (socket says 172.21.15.156:53220) 2023-11-28T13:11:04.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.492+0000 7f33f639a700 1 -- 172.21.15.156:0/2851201572 learned_addr learned my addr 172.21.15.156:0/2851201572 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:04.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.492+0000 7f33f739c700 1 --2- 172.21.15.156:0/2851201572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5615aad2e630 0x5615aac9aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.492+0000 7f33f6b9b700 1 --2- 172.21.15.156:0/2851201572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5615aac9e930 0x5615aad248c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.492+0000 7f33f639a700 1 -- 172.21.15.156:0/2851201572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5615aac9e930 msgr2=0x5615aad248c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:04.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.492+0000 7f33f639a700 1 --2- 172.21.15.156:0/2851201572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5615aac9e930 0x5615aad248c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.492+0000 7f33f639a700 1 -- 172.21.15.156:0/2851201572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5615aad2e630 msgr2=0x5615aac9aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:04.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.492+0000 7f33f639a700 1 --2- 172.21.15.156:0/2851201572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5615aad2e630 0x5615aac9aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.492+0000 7f33f639a700 1 -- 172.21.15.156:0/2851201572 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f000a100 con 0x5615aad2e260 2023-11-28T13:11:04.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.492+0000 7f33f639a700 1 --2- 172.21.15.156:0/2851201572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615aad2e260 0x5615aad321e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f33ec006dc0 tx=0x7f33ec00b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedb1ba5ea7f54a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.492+0000 7f33f5398700 1 -- 172.21.15.156:0/2851201572 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec00bdb0 con 0x5615aad2e260 2023-11-28T13:11:04.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.492+0000 7f33f5398700 1 -- 172.21.15.156:0/2851201572 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ec00e470 con 0x5615aad2e260 2023-11-28T13:11:04.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.492+0000 7f33f5398700 1 -- 172.21.15.156:0/2851201572 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec00e820 con 0x5615aad2e260 2023-11-28T13:11:04.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.493+0000 7f3405435080 1 -- 172.21.15.156:0/2851201572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615aad2e260 msgr2=0x5615aad321e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.493+0000 7f3405435080 1 --2- 172.21.15.156:0/2851201572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615aad2e260 0x5615aad321e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f33ec006dc0 tx=0x7f33ec00b5d0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.493+0000 7f3405435080 1 -- 172.21.15.156:0/2851201572 shutdown_connections 2023-11-28T13:11:04.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.493+0000 7f3405435080 1 --2- 172.21.15.156:0/2851201572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5615aad2e630 0x5615aac9aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.493+0000 7f3405435080 1 --2- 172.21.15.156:0/2851201572 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5615aac9e930 0x5615aad248c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.493+0000 7f3405435080 1 --2- 172.21.15.156:0/2851201572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615aad2e260 0x5615aad321e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.493+0000 7f3405435080 1 -- 172.21.15.156:0/2851201572 >> 172.21.15.156:0/2851201572 conn(0x5615aaca29e0 msgr2=0x5615aac99ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:04.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.493+0000 7f3405435080 1 -- 172.21.15.156:0/2851201572 shutdown_connections 2023-11-28T13:11:04.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.493+0000 7f3405435080 1 -- 172.21.15.156:0/2851201572 wait complete. 2023-11-28T13:11:04.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.494+0000 7f3405435080 1 Processor -- start 2023-11-28T13:11:04.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.494+0000 7f3405435080 1 -- start start 2023-11-28T13:11:04.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.495+0000 7f3405435080 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615aac9e930 0x5615aad27d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.495+0000 7f3405435080 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5615aad2e260 0x5615aad240a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.495+0000 7f3405435080 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5615aad2e630 0x5615aac846f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.495+0000 7f3405435080 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615aad25540 con 0x5615aac9e930 2023-11-28T13:11:04.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.495+0000 7f3405435080 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615aac9a2f0 con 0x5615aad2e260 2023-11-28T13:11:04.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.495+0000 7f3405435080 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x5615aac9a010 con 0x5615aad2e630 2023-11-28T13:11:04.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.495+0000 7f33f6b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615aac9e930 0x5615aad27d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.495+0000 7f33f6b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615aac9e930 0x5615aad27d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53226/0 (socket says 172.21.15.156:53226) 2023-11-28T13:11:04.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.495+0000 7f33f6b9b700 1 -- 172.21.15.156:0/2549555435 learned_addr learned my addr 172.21.15.156:0/2549555435 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:04.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.495+0000 7f33f639a700 1 --2- 172.21.15.156:0/2549555435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5615aad2e260 0x5615aad240a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.495+0000 7f33f739c700 1 --2- 172.21.15.156:0/2549555435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5615aad2e630 0x5615aac846f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.495+0000 7f33f6b9b700 1 -- 172.21.15.156:0/2549555435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5615aad2e630 msgr2=0x5615aac846f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.495+0000 7f33f6b9b700 1 --2- 172.21.15.156:0/2549555435 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x5615aad2e630 0x5615aac846f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.495+0000 7f33f6b9b700 1 -- 172.21.15.156:0/2549555435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5615aad2e260 msgr2=0x5615aad240a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.495+0000 7f33f6b9b700 1 --2- 172.21.15.156:0/2549555435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5615aad2e260 0x5615aad240a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.495+0000 7f33f6b9b700 1 -- 172.21.15.156:0/2549555435 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615aab9c4a0 con 0x5615aac9e930 2023-11-28T13:11:04.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.495+0000 7f33f639a700 1 --2- 172.21.15.156:0/2549555435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5615aad2e260 0x5615aad240a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:11:04.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.496+0000 7f33f6b9b700 1 --2- 172.21.15.156:0/2549555435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615aac9e930 0x5615aad27d50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f33e800df70 tx=0x7f33e800b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.496+0000 7f33dffff700 1 -- 172.21.15.156:0/2549555435 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e800bc70 con 0x5615aac9e930 2023-11-28T13:11:04.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.496+0000 7f33dffff700 1 -- 172.21.15.156:0/2549555435 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e8008040 con 0x5615aac9e930 2023-11-28T13:11:04.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.496+0000 7f3405435080 1 -- 172.21.15.156:0/2549555435 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5615aab9c830 con 0x5615aac9e930 2023-11-28T13:11:04.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.496+0000 7f33dffff700 1 -- 172.21.15.156:0/2549555435 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e80032e0 con 0x5615aac9e930 2023-11-28T13:11:04.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.496+0000 7f3405435080 1 -- 172.21.15.156:0/2549555435 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615aaba9360 con 0x5615aac9e930 2023-11-28T13:11:04.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.498+0000 7f33dffff700 1 -- 172.21.15.156:0/2549555435 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f33e800e040 con 0x5615aac9e930 2023-11-28T13:11:04.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.499+0000 7f33dffff700 1 --2- 172.21.15.156:0/2549555435 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f33e4066c10 0x7f33e40690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.499+0000 7f33dffff700 1 -- 172.21.15.156:0/2549555435 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f33e80846c0 con 0x5615aac9e930 2023-11-28T13:11:04.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.499+0000 7f3405435080 1 --2- 172.21.15.156:0/2549555435 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x5615aad286f0 0x5615aadc60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.503 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.499+0000 7f3405435080 1 -- 172.21.15.156:0/2549555435 --> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59) v8 -- 0x5615aac9b150 con 0x5615aad286f0 2023-11-28T13:11:04.503 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.500+0000 7f33f639a700 1 --2- 172.21.15.156:0/2549555435 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f33e4066c10 0x7f33e40690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.503 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.500+0000 7f33f739c700 1 --2- 172.21.15.156:0/2549555435 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x5615aad286f0 0x5615aadc60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.503 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.500+0000 7f33f739c700 1 --2- 172.21.15.156:0/2549555435 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x5615aad286f0 0x5615aadc60b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.503 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.500+0000 7f33f639a700 1 --2- 172.21.15.156:0/2549555435 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f33e4066c10 0x7f33e40690c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f33ec006dc0 tx=0x7f33ec01a040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.502+0000 7f33f739c700 1 -- 172.21.15.156:0/2549555435 <== osd.6 v2:172.21.15.186:6824/132946 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v59'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f33f000fba0 con 0x5615aad286f0 2023-11-28T13:11:04.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.502+0000 7f3405435080 1 -- 172.21.15.156:0/2549555435 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x5615aad286f0 msgr2=0x5615aadc60b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.502+0000 7f3405435080 1 --2- 172.21.15.156:0/2549555435 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x5615aad286f0 0x5615aadc60b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.502+0000 7f3405435080 1 -- 172.21.15.156:0/2549555435 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f33e4066c10 msgr2=0x7f33e40690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.502+0000 7f3405435080 1 --2- 172.21.15.156:0/2549555435 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f33e4066c10 0x7f33e40690c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f33ec006dc0 tx=0x7f33ec01a040 comp rx=0 tx=0).stop 2023-11-28T13:11:04.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.502+0000 7f3405435080 1 -- 172.21.15.156:0/2549555435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615aac9e930 msgr2=0x5615aad27d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.502+0000 7f3405435080 1 --2- 172.21.15.156:0/2549555435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615aac9e930 0x5615aad27d50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f33e800df70 tx=0x7f33e800b4c0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.502+0000 7f33f739c700 1 -- 172.21.15.156:0/2549555435 reap_dead start 2023-11-28T13:11:04.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.502+0000 7f3405435080 1 -- 172.21.15.156:0/2549555435 shutdown_connections 2023-11-28T13:11:04.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.502+0000 7f3405435080 1 -- 172.21.15.156:0/2549555435 >> 172.21.15.156:0/2549555435 conn(0x5615aaca29e0 msgr2=0x5615aad321e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:04.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.502+0000 7f3405435080 1 -- 172.21.15.156:0/2549555435 shutdown_connections 2023-11-28T13:11:04.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.502+0000 7f3405435080 1 -- 172.21.15.156:0/2549555435 wait complete. 2023-11-28T13:11:04.508 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2023-11-28T13:11:04.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.708+0000 7fb8b1fff700 1 Processor -- start 2023-11-28T13:11:04.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.708+0000 7fb8b1fff700 1 -- start start 2023-11-28T13:11:04.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.710+0000 7fb8b1fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb8ac15a760 0x7fb8ac161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.710+0000 7fb8b1fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb8ac0cded0 0x7fb8ac0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.710+0000 7fb8b1fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ac15e0a0 con 0x7fb8ac15dc30 2023-11-28T13:11:04.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.710+0000 7fb8b1fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ac15ab30 con 0x7fb8ac15a760 2023-11-28T13:11:04.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.710+0000 7fb8b1fff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ac160d70 con 0x7fb8ac0cded0 2023-11-28T13:11:04.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.710+0000 7fb8ab7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb8ac15a760 0x7fb8ac161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.710+0000 7fb8ab7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb8ac15a760 0x7fb8ac161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53242/0 (socket says 172.21.15.156:53242) 2023-11-28T13:11:04.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.710+0000 7fb8ab7fe700 1 -- 172.21.15.156:0/1231160533 learned_addr learned my addr 172.21.15.156:0/1231160533 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:04.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.710+0000 7fb8abfff700 1 --2- 172.21.15.156:0/1231160533 >> v2:172.21.15.186:3300/0 conn(0x7fb8ac0cded0 0x7fb8ac0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.710+0000 7fb8ab7fe700 1 -- 172.21.15.156:0/1231160533 >> v2:172.21.15.186:3300/0 conn(0x7fb8ac0cded0 msgr2=0x7fb8ac0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:04.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.710+0000 7fb8ab7fe700 1 --2- 172.21.15.156:0/1231160533 >> v2:172.21.15.186:3300/0 conn(0x7fb8ac0cded0 0x7fb8ac0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.710+0000 7fb8ab7fe700 1 -- 172.21.15.156:0/1231160533 >> v1:172.21.15.186:6789/0 conn(0x7fb8ac15dc30 legacy=0x7fb8ac0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.710+0000 7fb8ab7fe700 1 -- 172.21.15.156:0/1231160533 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ac0b5130 con 0x7fb8ac15a760 2023-11-28T13:11:04.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.711+0000 7fb8ab7fe700 1 --2- 172.21.15.156:0/1231160533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ac15a760 0x7fb8ac161200 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac067500 tx=0x7fb89c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db050d6ae577e62 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.711+0000 7fb8aaffd700 1 -- 172.21.15.156:0/1231160533 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c011830 con 0x7fb8ac15a760 2023-11-28T13:11:04.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.711+0000 7fb8aaffd700 1 -- 172.21.15.156:0/1231160533 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89c011990 con 0x7fb8ac15a760 2023-11-28T13:11:04.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.711+0000 7fb8aaffd700 1 -- 172.21.15.156:0/1231160533 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c011d20 con 0x7fb8ac15a760 2023-11-28T13:11:04.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.711+0000 7fb8b1fff700 1 -- 172.21.15.156:0/1231160533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ac15a760 msgr2=0x7fb8ac161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.711+0000 7fb8b1fff700 1 --2- 172.21.15.156:0/1231160533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ac15a760 0x7fb8ac161200 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac067500 tx=0x7fb89c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.712+0000 7fb8b1fff700 1 -- 172.21.15.156:0/1231160533 shutdown_connections 2023-11-28T13:11:04.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.712+0000 7fb8b1fff700 1 --2- 172.21.15.156:0/1231160533 >> v2:172.21.15.186:3300/0 conn(0x7fb8ac0cded0 0x7fb8ac0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.712+0000 7fb8b1fff700 1 --2- 172.21.15.156:0/1231160533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ac15a760 0x7fb8ac161200 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.712+0000 7fb8b1fff700 1 -- 172.21.15.156:0/1231160533 >> 172.21.15.156:0/1231160533 conn(0x7fb8ac072620 msgr2=0x7fb8ac0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:04.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.712+0000 7fb8b1fff700 1 -- 172.21.15.156:0/1231160533 shutdown_connections 2023-11-28T13:11:04.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.712+0000 7fb8b1fff700 1 -- 172.21.15.156:0/1231160533 wait complete. 2023-11-28T13:11:04.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.713+0000 7fb8b1fff700 1 Processor -- start 2023-11-28T13:11:04.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.714+0000 7fb8b1fff700 1 -- start start 2023-11-28T13:11:04.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.714+0000 7fb8b1fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8ac0cded0 0x7fb8ac0b8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.714+0000 7fb8b1fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ac15a760 0x7fb8ac153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.714+0000 7fb8b1fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8ac15dc30 0x7fb8ac0c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.714+0000 7fb8b1fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac0c7bd0 con 0x7fb8ac15a760 2023-11-28T13:11:04.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.714+0000 7fb8b1fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac0c7e00 con 0x7fb8ac0cded0 2023-11-28T13:11:04.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.714+0000 7fb8b1fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac0c80e0 con 0x7fb8ac15dc30 2023-11-28T13:11:04.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.715+0000 7fb8a3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ac15a760 0x7fb8ac153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.715+0000 7fb8a3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ac15a760 0x7fb8ac153b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53246/0 (socket says 172.21.15.156:53246) 2023-11-28T13:11:04.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.715+0000 7fb8a3fff700 1 -- 172.21.15.156:0/3602708562 learned_addr learned my addr 172.21.15.156:0/3602708562 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:04.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.715+0000 7fb8ab7fe700 1 --2- 172.21.15.156:0/3602708562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8ac0cded0 0x7fb8ac0b8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.715+0000 7fb8abfff700 1 --2- 172.21.15.156:0/3602708562 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8ac15dc30 0x7fb8ac0c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.715+0000 7fb8a3fff700 1 -- 172.21.15.156:0/3602708562 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8ac15dc30 msgr2=0x7fb8ac0c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.718 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.715+0000 7fb8a3fff700 1 --2- 172.21.15.156:0/3602708562 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8ac15dc30 0x7fb8ac0c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.715+0000 7fb8a3fff700 1 -- 172.21.15.156:0/3602708562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8ac0cded0 msgr2=0x7fb8ac0b8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.715+0000 7fb8a3fff700 1 --2- 172.21.15.156:0/3602708562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8ac0cded0 0x7fb8ac0b8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.715+0000 7fb8a3fff700 1 -- 172.21.15.156:0/3602708562 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89c009550 con 0x7fb8ac15a760 2023-11-28T13:11:04.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.715+0000 7fb8ab7fe700 1 --2- 172.21.15.156:0/3602708562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8ac0cded0 0x7fb8ac0b8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:11:04.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.715+0000 7fb8a3fff700 1 --2- 172.21.15.156:0/3602708562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ac15a760 0x7fb8ac153b60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb89400c450 tx=0x7fb89400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.716+0000 7fb8a8ff9700 1 -- 172.21.15.156:0/3602708562 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8940052d0 con 0x7fb8ac15a760 2023-11-28T13:11:04.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.716+0000 7fb8a8ff9700 1 -- 172.21.15.156:0/3602708562 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb894005430 con 0x7fb8ac15a760 2023-11-28T13:11:04.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.716+0000 7fb8a8ff9700 1 -- 172.21.15.156:0/3602708562 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8940038c0 con 0x7fb8ac15a760 2023-11-28T13:11:04.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.716+0000 7fb8b1fff700 1 -- 172.21.15.156:0/3602708562 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ac0b5670 con 0x7fb8ac15a760 2023-11-28T13:11:04.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.716+0000 7fb8b1fff700 1 -- 172.21.15.156:0/3602708562 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ac0b5390 con 0x7fb8ac15a760 2023-11-28T13:11:04.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.719+0000 7fb8a8ff9700 1 -- 172.21.15.156:0/3602708562 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb894019030 con 0x7fb8ac15a760 2023-11-28T13:11:04.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.719+0000 7fb8a8ff9700 1 --2- 172.21.15.156:0/3602708562 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb88c0682f0 0x7fb88c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.720+0000 7fb8a8ff9700 1 -- 172.21.15.156:0/3602708562 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb894084e00 con 0x7fb8ac15a760 2023-11-28T13:11:04.721 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.720+0000 7fb8ab7fe700 1 --2- 172.21.15.156:0/3602708562 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb88c0682f0 0x7fb88c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.722 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.720+0000 7fb8ab7fe700 1 --2- 172.21.15.156:0/3602708562 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb88c0682f0 0x7fb88c06a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009c10 tx=0x7fb89c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.721+0000 7fb8c5a21b80 1 Processor -- start 2023-11-28T13:11:04.723 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.722+0000 7fb8c5a21b80 1 -- start start 2023-11-28T13:11:04.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.722+0000 7fb8c5a21b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f507ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.722+0000 7fb8c5a21b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 0x55bb2f531c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.722+0000 7fb8c5a21b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 0x55bb2f54d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.722+0000 7fb8c5a21b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f3f1c70 con 0x55bb2f5318c0 2023-11-28T13:11:04.727 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.722+0000 7fb8c5a21b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f460400 con 0x55bb2f52f1e0 2023-11-28T13:11:04.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.722+0000 7fb8c5a21b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f459930 con 0x55bb2f5321d0 2023-11-28T13:11:04.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.723+0000 7fb8ab7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f507ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.723+0000 7fb8ab7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f507ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53258/0 (socket says 172.21.15.156:53258) 2023-11-28T13:11:04.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.723+0000 7fb8ab7fe700 1 -- 172.21.15.156:0/1555363944 learned_addr learned my addr 172.21.15.156:0/1555363944 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:04.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.723+0000 7fb8a3fff700 1 --2- 172.21.15.156:0/1555363944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 0x55bb2f531c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.723+0000 7fb8abfff700 1 --2- 172.21.15.156:0/1555363944 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 0x55bb2f54d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.723+0000 7fb8ab7fe700 1 -- 172.21.15.156:0/1555363944 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 msgr2=0x55bb2f54d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:04.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.723+0000 7fb8ab7fe700 1 --2- 172.21.15.156:0/1555363944 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 0x55bb2f54d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.728 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.723+0000 7fb8ab7fe700 1 -- 172.21.15.156:0/1555363944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 msgr2=0x55bb2f531c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:04.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.723+0000 7fb8ab7fe700 1 --2- 172.21.15.156:0/1555363944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 0x55bb2f531c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.723+0000 7fb8ab7fe700 1 -- 172.21.15.156:0/1555363944 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb2f2db690 con 0x55bb2f5318c0 2023-11-28T13:11:04.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.723+0000 7fb8ab7fe700 1 --2- 172.21.15.156:0/1555363944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f507ad0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb89c011da0 tx=0x7fb89c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2baf669d62743c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.724+0000 7fb8a17fa700 1 -- 172.21.15.156:0/1555363944 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c0038a0 con 0x55bb2f5318c0 2023-11-28T13:11:04.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.724+0000 7fb8a17fa700 1 -- 172.21.15.156:0/1555363944 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89c003a00 con 0x55bb2f5318c0 2023-11-28T13:11:04.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.724+0000 7fb8a17fa700 1 -- 172.21.15.156:0/1555363944 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c003d70 con 0x55bb2f5318c0 2023-11-28T13:11:04.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.724+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/1555363944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 msgr2=0x55bb2f507ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.724+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/1555363944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f507ad0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb89c011da0 tx=0x7fb89c010000 comp rx=0 tx=0).stop 2023-11-28T13:11:04.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.724+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/1555363944 shutdown_connections 2023-11-28T13:11:04.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.724+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/1555363944 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 0x55bb2f54d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.724+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/1555363944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f507ad0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.724+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/1555363944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 0x55bb2f531c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.724+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/1555363944 >> 172.21.15.156:0/1555363944 conn(0x55bb2f504e10 msgr2=0x55bb2f5057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:04.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.724+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/1555363944 shutdown_connections 2023-11-28T13:11:04.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.724+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/1555363944 wait complete. 2023-11-28T13:11:04.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.725+0000 7fb8c5a21b80 1 -- start start 2023-11-28T13:11:04.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.725+0000 7fb8c5a21b80 1 Processor -- start 2023-11-28T13:11:04.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.726+0000 7fb8c5a21b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 0x55bb2f532a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.726+0000 7fb8c5a21b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f524490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb8c5a21b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 0x55bb2f544750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb8c5a21b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f2dd980 con 0x55bb2f5318c0 2023-11-28T13:11:04.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb8c5a21b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f533500 con 0x55bb2f52f1e0 2023-11-28T13:11:04.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb8c5a21b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f5231a0 con 0x55bb2f5321d0 2023-11-28T13:11:04.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb8c5a21b80 10 client.0 _ll_register_callbacks cb 0x55bb2f2ff2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:11:04.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb8a3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f524490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb8a3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f524490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53262/0 (socket says 172.21.15.156:53262) 2023-11-28T13:11:04.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb8a3fff700 1 -- 172.21.15.156:0/1653222359 learned_addr learned my addr 172.21.15.156:0/1653222359 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:04.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb8ab7fe700 1 --2- 172.21.15.156:0/1653222359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 0x55bb2f532a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb8abfff700 1 --2- 172.21.15.156:0/1653222359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 0x55bb2f544750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb8a3fff700 1 -- 172.21.15.156:0/1653222359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 msgr2=0x55bb2f544750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:04.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb8a3fff700 1 --2- 172.21.15.156:0/1653222359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 0x55bb2f544750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb8a3fff700 1 -- 172.21.15.156:0/1653222359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 msgr2=0x55bb2f532a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:04.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb8a3fff700 1 --2- 172.21.15.156:0/1653222359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 0x55bb2f532a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb8a3fff700 1 -- 172.21.15.156:0/1653222359 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ac0b5130 con 0x55bb2f5318c0 2023-11-28T13:11:04.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb8abfff700 1 --2- 172.21.15.156:0/1653222359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 0x55bb2f544750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:11:04.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb8a3fff700 1 --2- 172.21.15.156:0/1653222359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f524490 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb894051820 tx=0x7fb89405d4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7520894901a9ca7d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.727+0000 7fb88a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:11:04.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.728+0000 7fb88a7fc700 1 -- 172.21.15.156:0/1653222359 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb894051310 con 0x55bb2f5318c0 2023-11-28T13:11:04.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.728+0000 7fb88a7fc700 1 -- 172.21.15.156:0/1653222359 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb894051470 con 0x55bb2f5318c0 2023-11-28T13:11:04.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.728+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/1653222359 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb2f4271f0 con 0x55bb2f5318c0 2023-11-28T13:11:04.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.728+0000 7fb88a7fc700 1 -- 172.21.15.156:0/1653222359 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb894083170 con 0x55bb2f5318c0 2023-11-28T13:11:04.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.728+0000 7fb8c5a21b80 10 client.6093 fetch_fsmap learned FSMap version 31 2023-11-28T13:11:04.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.728+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/1653222359 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55bb2f287e40 con 0x55bb2f5318c0 2023-11-28T13:11:04.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.728+0000 7fb88a7fc700 1 -- 172.21.15.156:0/1653222359 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bb2f4271f0 con 0x55bb2f5318c0 2023-11-28T13:11:04.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.728+0000 7fb88a7fc700 1 -- 172.21.15.156:0/1653222359 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb894054280 con 0x55bb2f5318c0 2023-11-28T13:11:04.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.728+0000 7fb8c5a21b80 10 client.6093 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T13:11:04.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.728+0000 7fb8c5a21b80 10 client.6093 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T13:11:04.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.728+0000 7fb8c5a21b80 20 client.6093 populate_metadata read hostname 'smithi156' 2023-11-28T13:11:04.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.728+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/1653222359 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x55bb2f544c90 0x55bb2f543b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.728+0000 7fb8c5a21b80 4 client.6093 mds_command: new command op to 5919 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:11:04.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.728+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/1653222359 --> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb2f2ddfc0 con 0x55bb2f544c90 2023-11-28T13:11:04.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.728+0000 7fb8ab7fe700 1 --2- 172.21.15.156:0/1653222359 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x55bb2f544c90 0x55bb2f543b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.729+0000 7fb8ab7fe700 1 --2- 172.21.15.156:0/1653222359 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x55bb2f544c90 0x55bb2f543b90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c3f399bc4cf0ea server_cookie=bb419bd1b3b64df9 in_seq=0 out_seq=0 2023-11-28T13:11:04.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.729+0000 7fb88a7fc700 10 client.6093 ms_handle_connect on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:04.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.729+0000 7fb88a7fc700 1 -- 172.21.15.156:0/1653222359 <== mds.0 v2:172.21.15.156:6832/3314379816 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55bb2f2ddfc0 con 0x55bb2f544c90 2023-11-28T13:11:04.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.730+0000 7fb88a7fc700 10 client.6093 handle_command_reply: tid=0 2023-11-28T13:11:04.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.730+0000 7fb8c5a21b80 1 client.6093 shutdown 2023-11-28T13:11:04.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.730+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/1653222359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 msgr2=0x55bb2f524490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.730+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/1653222359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f524490 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb894051820 tx=0x7fb89405d4e0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.730+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/1653222359 shutdown_connections 2023-11-28T13:11:04.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.730+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/1653222359 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 0x55bb2f544750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.730+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/1653222359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 0x55bb2f532a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.730+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/1653222359 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x55bb2f544c90 0x55bb2f543b90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.730+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/1653222359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f524490 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.730+0000 7fb88a7fc700 0 client.6093 ms_handle_reset on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:04.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.730+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/1653222359 >> 172.21.15.156:0/1653222359 conn(0x55bb2f504e10 msgr2=0x55bb2f507a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:04.742 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.730+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/1653222359 shutdown_connections 2023-11-28T13:11:04.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.730+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/1653222359 wait complete. 2023-11-28T13:11:04.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.730+0000 7fb8c5a21b80 20 client.6093 trim_cache size 0 max 16384 2023-11-28T13:11:04.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.747+0000 7fb8c5a21b80 1 Processor -- start 2023-11-28T13:11:04.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.747+0000 7fb8c5a21b80 1 -- start start 2023-11-28T13:11:04.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.749+0000 7fb8c5a21b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 0x55bb2f530a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.749+0000 7fb8c5a21b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f520ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.749+0000 7fb8c5a21b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 0x55bb2f521010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.749+0000 7fb8c5a21b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f532af0 con 0x55bb2f5318c0 2023-11-28T13:11:04.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.749+0000 7fb8c5a21b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f56e8c0 con 0x55bb2f52f1e0 2023-11-28T13:11:04.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.749+0000 7fb8c5a21b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f521780 con 0x55bb2f5321d0 2023-11-28T13:11:04.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.749+0000 7fb8a3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f520ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.749+0000 7fb8a3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f520ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53278/0 (socket says 172.21.15.156:53278) 2023-11-28T13:11:04.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.749+0000 7fb8a3fff700 1 -- 172.21.15.156:0/2073206324 learned_addr learned my addr 172.21.15.156:0/2073206324 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:04.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.749+0000 7fb8ab7fe700 1 --2- 172.21.15.156:0/2073206324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 0x55bb2f530a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.749+0000 7fb8a3fff700 1 -- 172.21.15.156:0/2073206324 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 msgr2=0x55bb2f521010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:04.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.749+0000 7fb8a3fff700 1 --2- 172.21.15.156:0/2073206324 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 0x55bb2f521010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.749+0000 7fb8a3fff700 1 -- 172.21.15.156:0/2073206324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 msgr2=0x55bb2f530a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:04.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.749+0000 7fb8a3fff700 1 --2- 172.21.15.156:0/2073206324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 0x55bb2f530a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.749+0000 7fb8a3fff700 1 -- 172.21.15.156:0/2073206324 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb2f4201f0 con 0x55bb2f5318c0 2023-11-28T13:11:04.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.749+0000 7fb8a3fff700 1 --2- 172.21.15.156:0/2073206324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f520ad0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb8940012f0 tx=0x7fb894059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bff1bb0a2fe34d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.750+0000 7fb8a1ffb700 1 -- 172.21.15.156:0/2073206324 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb894054600 con 0x55bb2f5318c0 2023-11-28T13:11:04.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.750+0000 7fb8a1ffb700 1 -- 172.21.15.156:0/2073206324 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb894083170 con 0x55bb2f5318c0 2023-11-28T13:11:04.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.750+0000 7fb8a1ffb700 1 -- 172.21.15.156:0/2073206324 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb894051350 con 0x55bb2f5318c0 2023-11-28T13:11:04.755 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.750+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/2073206324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 msgr2=0x55bb2f520ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.750+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/2073206324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f520ad0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb8940012f0 tx=0x7fb894059000 comp rx=0 tx=0).stop 2023-11-28T13:11:04.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.750+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/2073206324 shutdown_connections 2023-11-28T13:11:04.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.750+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/2073206324 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 0x55bb2f521010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.750+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/2073206324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 0x55bb2f530a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.750+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/2073206324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f520ad0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.750+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/2073206324 >> 172.21.15.156:0/2073206324 conn(0x55bb2f504e10 msgr2=0x55bb2f5453d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:04.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.750+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/2073206324 shutdown_connections 2023-11-28T13:11:04.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.750+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/2073206324 wait complete. 2023-11-28T13:11:04.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.751+0000 7fb8c5a21b80 1 -- start start 2023-11-28T13:11:04.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.751+0000 7fb8c5a21b80 1 Processor -- start 2023-11-28T13:11:04.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.752+0000 7fb8c5a21b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 0x55bb2f5500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.752+0000 7fb8c5a21b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f550630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.752+0000 7fb8c5a21b80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 0x55bb2f565280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.752+0000 7fb8c5a21b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f52bf90 con 0x55bb2f5318c0 2023-11-28T13:11:04.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.752+0000 7fb8c5a21b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f52a170 con 0x55bb2f52f1e0 2023-11-28T13:11:04.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.752+0000 7fb8c5a21b80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f564df0 con 0x55bb2f5321d0 2023-11-28T13:11:04.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.752+0000 7fb8c5a21b80 10 client.0 _ll_register_callbacks cb 0x55bb2f2ff2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:11:04.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.752+0000 7fb8a3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f550630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.753+0000 7fb8a3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f550630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53294/0 (socket says 172.21.15.156:53294) 2023-11-28T13:11:04.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.753+0000 7fb8a3fff700 1 -- 172.21.15.156:0/4056958508 learned_addr learned my addr 172.21.15.156:0/4056958508 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:04.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.753+0000 7fb8ab7fe700 1 --2- 172.21.15.156:0/4056958508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 0x55bb2f5500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.753+0000 7fb8abfff700 1 --2- 172.21.15.156:0/4056958508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 0x55bb2f565280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.753+0000 7fb8a3fff700 1 -- 172.21.15.156:0/4056958508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 msgr2=0x55bb2f565280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:04.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.753+0000 7fb8a3fff700 1 --2- 172.21.15.156:0/4056958508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 0x55bb2f565280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.753+0000 7fb8a3fff700 1 -- 172.21.15.156:0/4056958508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 msgr2=0x55bb2f5500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:04.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.753+0000 7fb8a3fff700 1 --2- 172.21.15.156:0/4056958508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 0x55bb2f5500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.753+0000 7fb8a3fff700 1 -- 172.21.15.156:0/4056958508 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb2f287e40 con 0x55bb2f5318c0 2023-11-28T13:11:04.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.753+0000 7fb8a3fff700 1 --2- 172.21.15.156:0/4056958508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f550630 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb8940012f0 tx=0x7fb894001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3863bd9556d90adb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.753+0000 7fb88a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:11:04.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.753+0000 7fb88a7fc700 1 -- 172.21.15.156:0/4056958508 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb894068c60 con 0x55bb2f5318c0 2023-11-28T13:11:04.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.753+0000 7fb88a7fc700 1 -- 172.21.15.156:0/4056958508 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb894054600 con 0x55bb2f5318c0 2023-11-28T13:11:04.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.753+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/4056958508 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb2f417210 con 0x55bb2f5318c0 2023-11-28T13:11:04.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.754+0000 7fb88a7fc700 1 -- 172.21.15.156:0/4056958508 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb894068dc0 con 0x55bb2f5318c0 2023-11-28T13:11:04.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.754+0000 7fb8c5a21b80 10 client.6099 fetch_fsmap learned FSMap version 31 2023-11-28T13:11:04.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.754+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/4056958508 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55bb2f2ae2b0 con 0x55bb2f5318c0 2023-11-28T13:11:04.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.754+0000 7fb88a7fc700 1 -- 172.21.15.156:0/4056958508 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bb2f417210 con 0x55bb2f5318c0 2023-11-28T13:11:04.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.754+0000 7fb88a7fc700 1 -- 172.21.15.156:0/4056958508 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb894003a20 con 0x55bb2f5318c0 2023-11-28T13:11:04.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.754+0000 7fb8c5a21b80 10 client.6099 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T13:11:04.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.754+0000 7fb8c5a21b80 10 client.6099 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T13:11:04.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.754+0000 7fb8c5a21b80 20 client.6099 populate_metadata read hostname 'smithi156' 2023-11-28T13:11:04.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.754+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/4056958508 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x55bb2f52e680 0x55bb2f5657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.754+0000 7fb8c5a21b80 4 client.6099 mds_command: new command op to 5919 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2023-11-28T13:11:04.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.754+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/4056958508 --> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x55bb2f401220 con 0x55bb2f52e680 2023-11-28T13:11:04.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.754+0000 7fb8ab7fe700 1 --2- 172.21.15.156:0/4056958508 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x55bb2f52e680 0x55bb2f5657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.755+0000 7fb88a7fc700 10 client.6099 ms_handle_connect on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:04.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.755+0000 7fb8ab7fe700 1 --2- 172.21.15.156:0/4056958508 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x55bb2f52e680 0x55bb2f5657c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ec76f11f37cb26d server_cookie=82c8c9ed8d6138d9 in_seq=0 out_seq=0 2023-11-28T13:11:04.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.755+0000 7fb88a7fc700 1 -- 172.21.15.156:0/4056958508 <== mds.0 v2:172.21.15.156:6832/3314379816 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bb2f401220 con 0x55bb2f52e680 2023-11-28T13:11:04.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.755+0000 7fb88a7fc700 10 client.6099 handle_command_reply: tid=0 2023-11-28T13:11:04.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.755+0000 7fb8c5a21b80 1 client.6099 shutdown 2023-11-28T13:11:04.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.755+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/4056958508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 msgr2=0x55bb2f550630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.755+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/4056958508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f550630 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb8940012f0 tx=0x7fb894001080 comp rx=0 tx=0).stop 2023-11-28T13:11:04.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.756+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/4056958508 shutdown_connections 2023-11-28T13:11:04.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.756+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/4056958508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55bb2f5321d0 0x55bb2f565280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.756+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/4056958508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb2f52f1e0 0x55bb2f5500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.756+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/4056958508 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x55bb2f52e680 0x55bb2f5657c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.756+0000 7fb8c5a21b80 1 --2- 172.21.15.156:0/4056958508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb2f5318c0 0x55bb2f550630 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.756+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/4056958508 >> 172.21.15.156:0/4056958508 conn(0x55bb2f504e10 msgr2=0x55bb2f544580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:04.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.756+0000 7fb88a7fc700 0 client.6099 ms_handle_reset on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:04.762 INFO:tasks.ceph.mds.c.smithi156.stderr:2023-11-28T13:11:04.756+0000 7f6a2e1ea700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5836=0-4194304@1} caps={5836=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5836 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x557bd7d45600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T13:11:04.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.756+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/4056958508 shutdown_connections 2023-11-28T13:11:04.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.756+0000 7fb8c5a21b80 1 -- 172.21.15.156:0/4056958508 wait complete. 2023-11-28T13:11:04.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.756+0000 7fb8c5a21b80 20 client.6099 trim_cache size 0 max 16384 2023-11-28T13:11:04.763 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:11:04.763 INFO:teuthology.orchestra.run.smithi156.stdout: "return_code": 0, 2023-11-28T13:11:04.763 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_tag": "b01a359d-08c0-436d-bef6-427ca2e8b2b3", 2023-11-28T13:11:04.763 INFO:teuthology.orchestra.run.smithi156.stdout: "mode": "asynchronous" 2023-11-28T13:11:04.763 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:11:04.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.757+0000 7fb8b1fff700 1 -- 172.21.15.156:0/3602708562 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb88c0682f0 msgr2=0x7fb88c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.757+0000 7fb8b1fff700 1 --2- 172.21.15.156:0/3602708562 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb88c0682f0 0x7fb88c06a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009c10 tx=0x7fb89c008030 comp rx=0 tx=0).stop 2023-11-28T13:11:04.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.757+0000 7fb8b1fff700 1 -- 172.21.15.156:0/3602708562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ac15a760 msgr2=0x7fb8ac153b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.757+0000 7fb8b1fff700 1 --2- 172.21.15.156:0/3602708562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ac15a760 0x7fb8ac153b60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb89400c450 tx=0x7fb89400c810 comp rx=0 tx=0).stop 2023-11-28T13:11:04.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.757+0000 7fb8b1fff700 1 -- 172.21.15.156:0/3602708562 shutdown_connections 2023-11-28T13:11:04.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.757+0000 7fb8b1fff700 1 --2- 172.21.15.156:0/3602708562 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb88c0682f0 0x7fb88c06a7a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.757+0000 7fb8b1fff700 1 --2- 172.21.15.156:0/3602708562 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8ac15dc30 0x7fb8ac0c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.757+0000 7fb8b1fff700 1 --2- 172.21.15.156:0/3602708562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8ac0cded0 0x7fb8ac0b8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.757+0000 7fb8b1fff700 1 --2- 172.21.15.156:0/3602708562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ac15a760 0x7fb8ac153b60 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.757+0000 7fb8b1fff700 1 -- 172.21.15.156:0/3602708562 >> 172.21.15.156:0/3602708562 conn(0x7fb8ac072620 msgr2=0x7fb8ac0b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:04.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.757+0000 7fb8b1fff700 1 -- 172.21.15.156:0/3602708562 shutdown_connections 2023-11-28T13:11:04.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.757+0000 7fb8b1fff700 1 -- 172.21.15.156:0/3602708562 wait complete. 2023-11-28T13:11:04.776 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T13:11:04.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.975+0000 7fa93edff700 1 Processor -- start 2023-11-28T13:11:04.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.975+0000 7fa93edff700 1 -- start start 2023-11-28T13:11:04.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.976+0000 7fa93edff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa93815c9b0 0x7fa9380b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.976+0000 7fa93edff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa9380cd7c0 0x7fa9380cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.976+0000 7fa93edff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9380ce160 con 0x7fa938161340 2023-11-28T13:11:04.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.976+0000 7fa93edff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9380b5800 con 0x7fa9380cd7c0 2023-11-28T13:11:04.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.976+0000 7fa93edff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9380d1950 con 0x7fa93815c9b0 2023-11-28T13:11:04.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.977+0000 7fa93cb9b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa93815c9b0 0x7fa9380b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.977+0000 7fa93d39c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa9380cd7c0 0x7fa9380cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.977+0000 7fa937fff700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fa938161340 0x7fa9380d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:56412/0 (socket says 172.21.15.156:56412) 2023-11-28T13:11:04.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.977+0000 7fa937fff700 1 -- 172.21.15.156:0/3992316488 learned_addr learned my addr 172.21.15.156:0/3992316488 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:11:04.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.977+0000 7fa93d39c700 1 -- 172.21.15.156:0/3992316488 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa93815c9b0 msgr2=0x7fa9380b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:04.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.977+0000 7fa93d39c700 1 --2- 172.21.15.156:0/3992316488 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa93815c9b0 0x7fa9380b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.977+0000 7fa9377fe700 1 -- 172.21.15.156:0/3992316488 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1135904896 0 0) 0x7fa9380ce160 con 0x7fa938161340 2023-11-28T13:11:04.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.977+0000 7fa93d39c700 1 -- 172.21.15.156:0/3992316488 >> v1:172.21.15.186:6789/0 conn(0x7fa938161340 legacy=0x7fa9380d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.977+0000 7fa93d39c700 1 -- 172.21.15.156:0/3992316488 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9380d24f0 con 0x7fa9380cd7c0 2023-11-28T13:11:04.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.978+0000 7fa93d39c700 1 --2- 172.21.15.156:0/3992316488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9380cd7c0 0x7fa9380cdc20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa930009d90 tx=0x7fa93000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1796055a967d2c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.978+0000 7fa9377fe700 1 -- 172.21.15.156:0/3992316488 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93000e070 con 0x7fa9380cd7c0 2023-11-28T13:11:04.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.978+0000 7fa9377fe700 1 -- 172.21.15.156:0/3992316488 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93000bb20 con 0x7fa9380cd7c0 2023-11-28T13:11:04.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.978+0000 7fa9377fe700 1 -- 172.21.15.156:0/3992316488 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa930012430 con 0x7fa9380cd7c0 2023-11-28T13:11:04.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.978+0000 7fa93edff700 1 -- 172.21.15.156:0/3992316488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9380cd7c0 msgr2=0x7fa9380cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.978+0000 7fa93edff700 1 --2- 172.21.15.156:0/3992316488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9380cd7c0 0x7fa9380cdc20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa930009d90 tx=0x7fa93000b330 comp rx=0 tx=0).stop 2023-11-28T13:11:04.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.978+0000 7fa93edff700 1 -- 172.21.15.156:0/3992316488 shutdown_connections 2023-11-28T13:11:04.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.978+0000 7fa93edff700 1 --2- 172.21.15.156:0/3992316488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9380cd7c0 0x7fa9380cdc20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.978+0000 7fa93edff700 1 --2- 172.21.15.156:0/3992316488 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa93815c9b0 0x7fa9380b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.978+0000 7fa93edff700 1 -- 172.21.15.156:0/3992316488 >> 172.21.15.156:0/3992316488 conn(0x7fa938072600 msgr2=0x7fa9380cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:04.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.978+0000 7fa93edff700 1 -- 172.21.15.156:0/3992316488 shutdown_connections 2023-11-28T13:11:04.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.979+0000 7fa93edff700 1 -- 172.21.15.156:0/3992316488 wait complete. 2023-11-28T13:11:04.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.980+0000 7fa93edff700 1 Processor -- start 2023-11-28T13:11:04.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.980+0000 7fa93edff700 1 -- start start 2023-11-28T13:11:04.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.981+0000 7fa93edff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9380cd7c0 0x7fa9381f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.981+0000 7fa93edff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa93815c9b0 0x7fa93815e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.981+0000 7fa93edff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa938161340 0x7fa93815a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.981+0000 7fa93edff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93815f440 con 0x7fa9380cd7c0 2023-11-28T13:11:04.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.981+0000 7fa93edff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93815eba0 con 0x7fa938161340 2023-11-28T13:11:04.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.981+0000 7fa93edff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93815ee80 con 0x7fa93815c9b0 2023-11-28T13:11:04.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.982+0000 7fa93cb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9380cd7c0 0x7fa9381f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.982+0000 7fa93cb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9380cd7c0 0x7fa9381f52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53310/0 (socket says 172.21.15.156:53310) 2023-11-28T13:11:04.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.982+0000 7fa93cb9b700 1 -- 172.21.15.156:0/2475017090 learned_addr learned my addr 172.21.15.156:0/2475017090 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:04.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.982+0000 7fa937fff700 1 --2- 172.21.15.156:0/2475017090 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa93815c9b0 0x7fa93815e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.982+0000 7fa93d39c700 1 --2- 172.21.15.156:0/2475017090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa938161340 0x7fa93815a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.982+0000 7fa93cb9b700 1 -- 172.21.15.156:0/2475017090 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa93815c9b0 msgr2=0x7fa93815e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.982+0000 7fa93cb9b700 1 --2- 172.21.15.156:0/2475017090 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa93815c9b0 0x7fa93815e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.982+0000 7fa93cb9b700 1 -- 172.21.15.156:0/2475017090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa938161340 msgr2=0x7fa93815a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.982+0000 7fa93cb9b700 1 --2- 172.21.15.156:0/2475017090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa938161340 0x7fa93815a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.982+0000 7fa93cb9b700 1 -- 172.21.15.156:0/2475017090 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa930009a70 con 0x7fa9380cd7c0 2023-11-28T13:11:04.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.982+0000 7fa93cb9b700 1 --2- 172.21.15.156:0/2475017090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9380cd7c0 0x7fa9381f52f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00e870 tx=0x7fa92c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.983+0000 7fa9357fa700 1 -- 172.21.15.156:0/2475017090 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c009590 con 0x7fa9380cd7c0 2023-11-28T13:11:04.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.983+0000 7fa9357fa700 1 -- 172.21.15.156:0/2475017090 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92c013680 con 0x7fa9380cd7c0 2023-11-28T13:11:04.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.983+0000 7fa9357fa700 1 -- 172.21.15.156:0/2475017090 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c0139f0 con 0x7fa9380cd7c0 2023-11-28T13:11:04.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.983+0000 7fa93edff700 1 -- 172.21.15.156:0/2475017090 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9380d1300 con 0x7fa9380cd7c0 2023-11-28T13:11:04.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.983+0000 7fa93edff700 1 -- 172.21.15.156:0/2475017090 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9380b5520 con 0x7fa9380cd7c0 2023-11-28T13:11:04.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.985+0000 7fa9357fa700 1 -- 172.21.15.156:0/2475017090 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa92c019030 con 0x7fa9380cd7c0 2023-11-28T13:11:04.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.986+0000 7fa9357fa700 1 --2- 172.21.15.156:0/2475017090 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa91c0682f0 0x7fa91c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.986+0000 7fa9357fa700 1 -- 172.21.15.156:0/2475017090 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa92c085950 con 0x7fa9380cd7c0 2023-11-28T13:11:04.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.986+0000 7fa937fff700 1 --2- 172.21.15.156:0/2475017090 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa91c0682f0 0x7fa91c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.987+0000 7fa937fff700 1 --2- 172.21.15.156:0/2475017090 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa91c0682f0 0x7fa91c06a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa9380b90e0 tx=0x7fa924007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.987+0000 7fa95268ab80 1 Processor -- start 2023-11-28T13:11:04.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.988+0000 7fa95268ab80 1 -- start start 2023-11-28T13:11:04.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.988+0000 7fa95268ab80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff33860 0x56291ff09a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.988+0000 7fa95268ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff31180 0x56291ff33c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.988+0000 7fa95268ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff34170 0x56291ff4f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:04.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.988+0000 7fa95268ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291fdf3c10 con 0x56291ff31180 2023-11-28T13:11:04.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.988+0000 7fa95268ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291fe623a0 con 0x56291ff34170 2023-11-28T13:11:04.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.988+0000 7fa95268ab80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56291fe5b8d0 con 0x56291ff33860 2023-11-28T13:11:04.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.989+0000 7fa937fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff31180 0x56291ff33c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.989+0000 7fa937fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff31180 0x56291ff33c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53316/0 (socket says 172.21.15.156:53316) 2023-11-28T13:11:04.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.989+0000 7fa937fff700 1 -- 172.21.15.156:0/3039344152 learned_addr learned my addr 172.21.15.156:0/3039344152 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:04.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.989+0000 7fa93d39c700 1 --2- 172.21.15.156:0/3039344152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff34170 0x56291ff4f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.989+0000 7fa93cb9b700 1 --2- 172.21.15.156:0/3039344152 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff33860 0x56291ff09a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:04.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.989+0000 7fa937fff700 1 -- 172.21.15.156:0/3039344152 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff33860 msgr2=0x56291ff09a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:04.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.989+0000 7fa937fff700 1 --2- 172.21.15.156:0/3039344152 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff33860 0x56291ff09a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.989+0000 7fa937fff700 1 -- 172.21.15.156:0/3039344152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff34170 msgr2=0x56291ff4f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:04.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.989+0000 7fa937fff700 1 --2- 172.21.15.156:0/3039344152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff34170 0x56291ff4f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.989+0000 7fa937fff700 1 -- 172.21.15.156:0/3039344152 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56291fcdd630 con 0x56291ff31180 2023-11-28T13:11:04.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.989+0000 7fa937fff700 1 --2- 172.21.15.156:0/3039344152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff31180 0x56291ff33c30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa9240137f0 tx=0x7fa924013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8746d6a25a88ece server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:04.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.990+0000 7fa9227fc700 1 -- 172.21.15.156:0/3039344152 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa924001290 con 0x56291ff31180 2023-11-28T13:11:04.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.990+0000 7fa9227fc700 1 -- 172.21.15.156:0/3039344152 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9240013f0 con 0x56291ff31180 2023-11-28T13:11:04.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.990+0000 7fa9227fc700 1 -- 172.21.15.156:0/3039344152 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa924001760 con 0x56291ff31180 2023-11-28T13:11:04.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.990+0000 7fa95268ab80 1 -- 172.21.15.156:0/3039344152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff31180 msgr2=0x56291ff33c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:04.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.990+0000 7fa95268ab80 1 --2- 172.21.15.156:0/3039344152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff31180 0x56291ff33c30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa9240137f0 tx=0x7fa924013000 comp rx=0 tx=0).stop 2023-11-28T13:11:04.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.990+0000 7fa95268ab80 1 -- 172.21.15.156:0/3039344152 shutdown_connections 2023-11-28T13:11:04.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.990+0000 7fa95268ab80 1 --2- 172.21.15.156:0/3039344152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff34170 0x56291ff4f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.990+0000 7fa95268ab80 1 --2- 172.21.15.156:0/3039344152 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff33860 0x56291ff09a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.990+0000 7fa95268ab80 1 --2- 172.21.15.156:0/3039344152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff31180 0x56291ff33c30 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:04.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.990+0000 7fa95268ab80 1 -- 172.21.15.156:0/3039344152 >> 172.21.15.156:0/3039344152 conn(0x56291ff06db0 msgr2=0x56291ff07740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:04.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.990+0000 7fa95268ab80 1 -- 172.21.15.156:0/3039344152 shutdown_connections 2023-11-28T13:11:04.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.991+0000 7fa95268ab80 1 -- 172.21.15.156:0/3039344152 wait complete. 2023-11-28T13:11:04.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.991+0000 7fa95268ab80 1 -- start start 2023-11-28T13:11:04.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.991+0000 7fa95268ab80 1 Processor -- start 2023-11-28T13:11:05.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.992+0000 7fa95268ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff31180 0x56291ff468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:05.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.992+0000 7fa95268ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff33860 0x56291ff26160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:05.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.992+0000 7fa95268ab80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff34170 0x56291ff266a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:05.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.992+0000 7fa95268ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291fcdf920 con 0x56291ff33860 2023-11-28T13:11:05.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.992+0000 7fa95268ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291ff31c20 con 0x56291ff31180 2023-11-28T13:11:05.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.992+0000 7fa95268ab80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56291ff24040 con 0x56291ff34170 2023-11-28T13:11:05.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.993+0000 7fa95268ab80 10 client.0 _ll_register_callbacks cb 0x56291fd01290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:11:05.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.993+0000 7fa937fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff33860 0x56291ff26160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:05.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.993+0000 7fa93cb9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff31180 0x56291ff468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:05.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.993+0000 7fa93cb9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff31180 0x56291ff468f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:46472/0 (socket says 172.21.15.156:46472) 2023-11-28T13:11:05.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.993+0000 7fa93cb9b700 1 -- 172.21.15.156:0/2076045621 learned_addr learned my addr 172.21.15.156:0/2076045621 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:05.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.993+0000 7fa93d39c700 1 --2- 172.21.15.156:0/2076045621 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff34170 0x56291ff266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:05.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.993+0000 7fa93cb9b700 1 -- 172.21.15.156:0/2076045621 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff34170 msgr2=0x56291ff266a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:05.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.993+0000 7fa93cb9b700 1 --2- 172.21.15.156:0/2076045621 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff34170 0x56291ff266a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.993+0000 7fa93cb9b700 1 -- 172.21.15.156:0/2076045621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff33860 msgr2=0x56291ff26160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:05.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.993+0000 7fa93cb9b700 1 --2- 172.21.15.156:0/2076045621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff33860 0x56291ff26160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.993+0000 7fa93cb9b700 1 -- 172.21.15.156:0/2076045621 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9240134d0 con 0x56291ff31180 2023-11-28T13:11:05.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.993+0000 7fa937fff700 1 --2- 172.21.15.156:0/2076045621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff33860 0x56291ff26160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:11:05.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.993+0000 7fa93cb9b700 1 --2- 172.21.15.156:0/2076045621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff31180 0x56291ff468f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa92c052370 tx=0x7fa92c0553b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef2f578bdd7fb9f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:05.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.993+0000 7fa9177fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-28T13:11:05.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.994+0000 7fa9177fe700 1 -- 172.21.15.156:0/2076045621 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c051c60 con 0x56291ff31180 2023-11-28T13:11:05.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.994+0000 7fa9177fe700 1 -- 172.21.15.156:0/2076045621 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa92c051dc0 con 0x56291ff31180 2023-11-28T13:11:05.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.994+0000 7fa95268ab80 1 -- 172.21.15.156:0/2076045621 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56291fe29190 con 0x56291ff31180 2023-11-28T13:11:05.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.994+0000 7fa9177fe700 1 -- 172.21.15.156:0/2076045621 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa92c051f80 con 0x56291ff31180 2023-11-28T13:11:05.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.994+0000 7fa95268ab80 10 client.5896 fetch_fsmap learned FSMap version 31 2023-11-28T13:11:05.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.994+0000 7fa95268ab80 1 -- 172.21.15.156:0/2076045621 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x56291fc89de0 con 0x56291ff31180 2023-11-28T13:11:05.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.994+0000 7fa9177fe700 1 -- 172.21.15.156:0/2076045621 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56291fe29190 con 0x56291ff31180 2023-11-28T13:11:05.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.994+0000 7fa9177fe700 1 -- 172.21.15.156:0/2076045621 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa92c054dd0 con 0x56291ff31180 2023-11-28T13:11:05.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.994+0000 7fa95268ab80 10 client.5896 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T13:11:05.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.994+0000 7fa95268ab80 10 client.5896 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T13:11:05.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.994+0000 7fa95268ab80 20 client.5896 populate_metadata read hostname 'smithi156' 2023-11-28T13:11:05.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.994+0000 7fa95268ab80 1 --2- 172.21.15.156:0/2076045621 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x56291ff45fe0 0x56291ff61b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:05.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.994+0000 7fa95268ab80 4 client.5896 mds_command: new command op to 5919 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:11:05.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.994+0000 7fa95268ab80 1 -- 172.21.15.156:0/2076045621 --> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56291fcdff60 con 0x56291ff45fe0 2023-11-28T13:11:05.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.995+0000 7fa937fff700 1 --2- 172.21.15.156:0/2076045621 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x56291ff45fe0 0x56291ff61b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:05.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.995+0000 7fa9177fe700 10 client.5896 ms_handle_connect on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:05.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.995+0000 7fa937fff700 1 --2- 172.21.15.156:0/2076045621 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x56291ff45fe0 0x56291ff61b80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6fa42b3d47ce172 server_cookie=61fa5c67efa95929 in_seq=0 out_seq=0 2023-11-28T13:11:05.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.996+0000 7fa9177fe700 1 -- 172.21.15.156:0/2076045621 <== mds.0 v2:172.21.15.156:6832/3314379816 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x56291fcdff60 con 0x56291ff45fe0 2023-11-28T13:11:05.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.996+0000 7fa9177fe700 10 client.5896 handle_command_reply: tid=0 2023-11-28T13:11:05.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.996+0000 7fa95268ab80 1 client.5896 shutdown 2023-11-28T13:11:05.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.996+0000 7fa95268ab80 1 -- 172.21.15.156:0/2076045621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff31180 msgr2=0x56291ff468f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:05.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.996+0000 7fa95268ab80 1 --2- 172.21.15.156:0/2076045621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff31180 0x56291ff468f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa92c052370 tx=0x7fa92c0553b0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.996+0000 7fa95268ab80 1 -- 172.21.15.156:0/2076045621 shutdown_connections 2023-11-28T13:11:05.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.996+0000 7fa95268ab80 1 --2- 172.21.15.156:0/2076045621 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff34170 0x56291ff266a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.996+0000 7fa95268ab80 1 --2- 172.21.15.156:0/2076045621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff31180 0x56291ff468f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.996+0000 7fa95268ab80 1 --2- 172.21.15.156:0/2076045621 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x56291ff45fe0 0x56291ff61b80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.996+0000 7fa95268ab80 1 --2- 172.21.15.156:0/2076045621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff33860 0x56291ff26160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.996+0000 7fa95268ab80 1 -- 172.21.15.156:0/2076045621 >> 172.21.15.156:0/2076045621 conn(0x56291ff06db0 msgr2=0x56291ff08280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:05.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.996+0000 7fa9177fe700 0 client.5896 ms_handle_reset on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:05.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.996+0000 7fa95268ab80 1 -- 172.21.15.156:0/2076045621 shutdown_connections 2023-11-28T13:11:05.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.996+0000 7fa95268ab80 1 -- 172.21.15.156:0/2076045621 wait complete. 2023-11-28T13:11:05.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:04.996+0000 7fa95268ab80 20 client.5896 trim_cache size 0 max 16384 2023-11-28T13:11:05.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.013+0000 7fa95268ab80 1 Processor -- start 2023-11-28T13:11:05.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.014+0000 7fa95268ab80 1 -- start start 2023-11-28T13:11:05.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.014+0000 7fa95268ab80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff31180 0x56291ff26140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:05.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.014+0000 7fa95268ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff33860 0x56291ff26680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:05.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.014+0000 7fa95268ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff34170 0x56291ff67bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:05.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.014+0000 7fa95268ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291ff07f20 con 0x56291ff34170 2023-11-28T13:11:05.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.014+0000 7fa95268ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291ff0a7f0 con 0x56291ff33860 2023-11-28T13:11:05.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.014+0000 7fa95268ab80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56291ff05bd0 con 0x56291ff31180 2023-11-28T13:11:05.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.015+0000 7fa93d39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff34170 0x56291ff67bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:05.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.015+0000 7fa93d39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff34170 0x56291ff67bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53332/0 (socket says 172.21.15.156:53332) 2023-11-28T13:11:05.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.015+0000 7fa93d39c700 1 -- 172.21.15.156:0/3942956500 learned_addr learned my addr 172.21.15.156:0/3942956500 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:05.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.015+0000 7fa937fff700 1 --2- 172.21.15.156:0/3942956500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff33860 0x56291ff26680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:05.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.015+0000 7fa93cb9b700 1 --2- 172.21.15.156:0/3942956500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff31180 0x56291ff26140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:05.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.015+0000 7fa93d39c700 1 -- 172.21.15.156:0/3942956500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff31180 msgr2=0x56291ff26140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:05.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.015+0000 7fa93d39c700 1 --2- 172.21.15.156:0/3942956500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff31180 0x56291ff26140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.015+0000 7fa93d39c700 1 -- 172.21.15.156:0/3942956500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff33860 msgr2=0x56291ff26680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:05.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.015+0000 7fa93d39c700 1 --2- 172.21.15.156:0/3942956500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff33860 0x56291ff26680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.015+0000 7fa93d39c700 1 -- 172.21.15.156:0/3942956500 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56291fe22190 con 0x56291ff34170 2023-11-28T13:11:05.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.015+0000 7fa93d39c700 1 --2- 172.21.15.156:0/3942956500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff34170 0x56291ff67bb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa938067550 tx=0x7fa930011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca60dc8e90e00cc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:05.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.016+0000 7fa922ffd700 1 -- 172.21.15.156:0/3942956500 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9300125e0 con 0x56291ff34170 2023-11-28T13:11:05.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.016+0000 7fa922ffd700 1 -- 172.21.15.156:0/3942956500 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa930023040 con 0x56291ff34170 2023-11-28T13:11:05.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.016+0000 7fa922ffd700 1 -- 172.21.15.156:0/3942956500 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9300129a0 con 0x56291ff34170 2023-11-28T13:11:05.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.016+0000 7fa95268ab80 1 -- 172.21.15.156:0/3942956500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff34170 msgr2=0x56291ff67bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:05.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.016+0000 7fa95268ab80 1 --2- 172.21.15.156:0/3942956500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff34170 0x56291ff67bb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa938067550 tx=0x7fa930011000 comp rx=0 tx=0).stop 2023-11-28T13:11:05.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.016+0000 7fa95268ab80 1 -- 172.21.15.156:0/3942956500 shutdown_connections 2023-11-28T13:11:05.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.016+0000 7fa95268ab80 1 --2- 172.21.15.156:0/3942956500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff34170 0x56291ff67bb0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.016+0000 7fa95268ab80 1 --2- 172.21.15.156:0/3942956500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff31180 0x56291ff26140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.016+0000 7fa95268ab80 1 --2- 172.21.15.156:0/3942956500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff33860 0x56291ff26680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.016+0000 7fa95268ab80 1 -- 172.21.15.156:0/3942956500 >> 172.21.15.156:0/3942956500 conn(0x56291ff06db0 msgr2=0x56291ff24520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:05.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.016+0000 7fa95268ab80 1 -- 172.21.15.156:0/3942956500 shutdown_connections 2023-11-28T13:11:05.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.016+0000 7fa95268ab80 1 -- 172.21.15.156:0/3942956500 wait complete. 2023-11-28T13:11:05.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.017+0000 7fa95268ab80 1 -- start start 2023-11-28T13:11:05.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.017+0000 7fa95268ab80 1 Processor -- start 2023-11-28T13:11:05.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.018+0000 7fa95268ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff31180 0x56291ff67bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:05.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.018+0000 7fa95268ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff33860 0x56291ff680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:05.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.018+0000 7fa95268ab80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff34170 0x56291ffb3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:05.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.018+0000 7fa95268ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291ffb42e0 con 0x56291ff31180 2023-11-28T13:11:05.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.018+0000 7fa95268ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291ff08b70 con 0x56291ff33860 2023-11-28T13:11:05.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.018+0000 7fa95268ab80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56291ff090a0 con 0x56291ff34170 2023-11-28T13:11:05.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.018+0000 7fa95268ab80 10 client.0 _ll_register_callbacks cb 0x56291fd01290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:11:05.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.018+0000 7fa93cb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff31180 0x56291ff67bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:05.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.018+0000 7fa937fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff33860 0x56291ff680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:05.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.018+0000 7fa937fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff33860 0x56291ff680f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:46484/0 (socket says 172.21.15.156:46484) 2023-11-28T13:11:05.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.018+0000 7fa937fff700 1 -- 172.21.15.156:0/3256093449 learned_addr learned my addr 172.21.15.156:0/3256093449 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:05.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.018+0000 7fa93d39c700 1 --2- 172.21.15.156:0/3256093449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff34170 0x56291ffb3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:05.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.019+0000 7fa937fff700 1 -- 172.21.15.156:0/3256093449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff34170 msgr2=0x56291ffb3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:05.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.019+0000 7fa937fff700 1 --2- 172.21.15.156:0/3256093449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff34170 0x56291ffb3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.019+0000 7fa937fff700 1 -- 172.21.15.156:0/3256093449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff31180 msgr2=0x56291ff67bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:05.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.019+0000 7fa937fff700 1 --2- 172.21.15.156:0/3256093449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff31180 0x56291ff67bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.019+0000 7fa937fff700 1 -- 172.21.15.156:0/3256093449 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9380d24f0 con 0x56291ff33860 2023-11-28T13:11:05.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.019+0000 7fa937fff700 1 --2- 172.21.15.156:0/3256093449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff33860 0x56291ff680f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa924013b00 tx=0x7fa924012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f3ec7a488503aef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:05.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.019+0000 7fa9177fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-28T13:11:05.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.019+0000 7fa9177fe700 1 -- 172.21.15.156:0/3256093449 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa924033070 con 0x56291ff33860 2023-11-28T13:11:05.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.019+0000 7fa9177fe700 1 -- 172.21.15.156:0/3256093449 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa92401b050 con 0x56291ff33860 2023-11-28T13:11:05.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.019+0000 7fa95268ab80 1 -- 172.21.15.156:0/3256093449 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56291fe191b0 con 0x56291ff33860 2023-11-28T13:11:05.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.020+0000 7fa9177fe700 1 -- 172.21.15.156:0/3256093449 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9240076d0 con 0x56291ff33860 2023-11-28T13:11:05.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.020+0000 7fa95268ab80 10 client.5902 fetch_fsmap learned FSMap version 31 2023-11-28T13:11:05.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.020+0000 7fa95268ab80 1 -- 172.21.15.156:0/3256093449 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x56291fcb0250 con 0x56291ff33860 2023-11-28T13:11:05.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.020+0000 7fa9177fe700 1 -- 172.21.15.156:0/3256093449 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56291fe191b0 con 0x56291ff33860 2023-11-28T13:11:05.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.020+0000 7fa9177fe700 1 -- 172.21.15.156:0/3256093449 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa924021a50 con 0x56291ff33860 2023-11-28T13:11:05.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.020+0000 7fa95268ab80 10 client.5902 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T13:11:05.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.020+0000 7fa95268ab80 10 client.5902 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T13:11:05.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.020+0000 7fa95268ab80 20 client.5902 populate_metadata read hostname 'smithi156' 2023-11-28T13:11:05.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.020+0000 7fa95268ab80 1 --2- 172.21.15.156:0/3256093449 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x56291ff474b0 0x56291ffb39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:05.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.020+0000 7fa95268ab80 4 client.5902 mds_command: new command op to 5919 tid=0[{"prefix": "scrub status"}] 2023-11-28T13:11:05.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.020+0000 7fa95268ab80 1 -- 172.21.15.156:0/3256093449 --> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56291fe031c0 con 0x56291ff474b0 2023-11-28T13:11:05.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.020+0000 7fa93cb9b700 1 --2- 172.21.15.156:0/3256093449 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x56291ff474b0 0x56291ffb39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.021+0000 7fa93cb9b700 1 --2- 172.21.15.156:0/3256093449 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x56291ff474b0 0x56291ffb39d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e21cd9902302f1 server_cookie=9d1deb63ba1ca39d in_seq=0 out_seq=0 2023-11-28T13:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.021+0000 7fa9177fe700 10 client.5902 ms_handle_connect on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.021+0000 7fa9177fe700 1 -- 172.21.15.156:0/3256093449 <== mds.0 v2:172.21.15.156:6832/3314379816 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x56291fe031c0 con 0x56291ff474b0 2023-11-28T13:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.021+0000 7fa9177fe700 10 client.5902 handle_command_reply: tid=0 2023-11-28T13:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.021+0000 7fa95268ab80 1 client.5902 shutdown 2023-11-28T13:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.021+0000 7fa95268ab80 1 -- 172.21.15.156:0/3256093449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff33860 msgr2=0x56291ff680f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.021+0000 7fa95268ab80 1 --2- 172.21.15.156:0/3256093449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff33860 0x56291ff680f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa924013b00 tx=0x7fa924012000 comp rx=0 tx=0).stop 2023-11-28T13:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.021+0000 7fa95268ab80 1 -- 172.21.15.156:0/3256093449 shutdown_connections 2023-11-28T13:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.021+0000 7fa95268ab80 1 --2- 172.21.15.156:0/3256093449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56291ff34170 0x56291ffb3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.021+0000 7fa95268ab80 1 --2- 172.21.15.156:0/3256093449 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x56291ff474b0 0x56291ffb39d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.021+0000 7fa95268ab80 1 --2- 172.21.15.156:0/3256093449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291ff31180 0x56291ff67bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.021+0000 7fa95268ab80 1 --2- 172.21.15.156:0/3256093449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56291ff33860 0x56291ff680f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.021+0000 7fa9177fe700 0 client.5902 ms_handle_reset on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:05.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.021+0000 7fa95268ab80 1 -- 172.21.15.156:0/3256093449 >> 172.21.15.156:0/3256093449 conn(0x56291ff06db0 msgr2=0x56291ff36df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:05.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.021+0000 7fa95268ab80 1 -- 172.21.15.156:0/3256093449 shutdown_connections 2023-11-28T13:11:05.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.021+0000 7fa95268ab80 1 -- 172.21.15.156:0/3256093449 wait complete. 2023-11-28T13:11:05.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.022+0000 7fa95268ab80 20 client.5902 trim_cache size 0 max 16384 2023-11-28T13:11:05.030 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:11:05.030 INFO:teuthology.orchestra.run.smithi156.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T13:11:05.030 INFO:teuthology.orchestra.run.smithi156.stdout: "scrubs": { 2023-11-28T13:11:05.030 INFO:teuthology.orchestra.run.smithi156.stdout: "b01a359d-08c0-436d-bef6-427ca2e8b2b3": { 2023-11-28T13:11:05.031 INFO:teuthology.orchestra.run.smithi156.stdout: "path": "/", 2023-11-28T13:11:05.031 INFO:teuthology.orchestra.run.smithi156.stdout: "tag": "b01a359d-08c0-436d-bef6-427ca2e8b2b3", 2023-11-28T13:11:05.031 INFO:teuthology.orchestra.run.smithi156.stdout: "options": "recursive" 2023-11-28T13:11:05.031 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:11:05.031 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:11:05.031 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:11:05.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.022+0000 7fa93edff700 1 -- 172.21.15.156:0/2475017090 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa91c0682f0 msgr2=0x7fa91c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:05.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.022+0000 7fa93edff700 1 --2- 172.21.15.156:0/2475017090 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa91c0682f0 0x7fa91c06a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa9380b90e0 tx=0x7fa924007e70 comp rx=0 tx=0).stop 2023-11-28T13:11:05.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.022+0000 7fa93edff700 1 -- 172.21.15.156:0/2475017090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9380cd7c0 msgr2=0x7fa9381f52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:05.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.022+0000 7fa93edff700 1 --2- 172.21.15.156:0/2475017090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9380cd7c0 0x7fa9381f52f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00e870 tx=0x7fa92c00eb80 comp rx=0 tx=0).stop 2023-11-28T13:11:05.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.022+0000 7fa93edff700 1 -- 172.21.15.156:0/2475017090 shutdown_connections 2023-11-28T13:11:05.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.022+0000 7fa93edff700 1 --2- 172.21.15.156:0/2475017090 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa91c0682f0 0x7fa91c06a7a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.022+0000 7fa93edff700 1 --2- 172.21.15.156:0/2475017090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa938161340 0x7fa93815a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.022+0000 7fa93edff700 1 --2- 172.21.15.156:0/2475017090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9380cd7c0 0x7fa9381f52f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.022+0000 7fa93edff700 1 --2- 172.21.15.156:0/2475017090 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa93815c9b0 0x7fa93815e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:05.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.022+0000 7fa93edff700 1 -- 172.21.15.156:0/2475017090 >> 172.21.15.156:0/2475017090 conn(0x7fa938072600 msgr2=0x7fa9380d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:05.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.022+0000 7fa93edff700 1 -- 172.21.15.156:0/2475017090 shutdown_connections 2023-11-28T13:11:05.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:05.022+0000 7fa93edff700 1 -- 172.21.15.156:0/2475017090 wait complete. 2023-11-28T13:11:05.041 INFO:tasks.cephfs.filesystem:scrub status for tag:b01a359d-08c0-436d-bef6-427ca2e8b2b3 - {'path': '/', 'tag': 'b01a359d-08c0-436d-bef6-427ca2e8b2b3', 'options': 'recursive'} 2023-11-28T13:11:11.455 INFO:tasks.ceph.mon.a.smithi156.stderr:2023-11-28T13:11:11.454+0000 7f61ad1f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-28T13:11:33.517 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:11:33.523 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:11:35.042 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T13:11:35.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.276+0000 7fb41ebff700 1 Processor -- start 2023-11-28T13:11:35.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.276+0000 7fb41ebff700 1 -- start start 2023-11-28T13:11:35.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.277+0000 7fb41ebff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb418161340 0x7fb4180d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.277+0000 7fb41ebff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb4180d2320 0x7fb4180cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.277+0000 7fb41ebff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4180d1950 con 0x7fb41815c9b0 2023-11-28T13:11:35.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.277+0000 7fb41ebff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4180b5800 con 0x7fb4180d2320 2023-11-28T13:11:35.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.277+0000 7fb41ebff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4180b5300 con 0x7fb418161340 2023-11-28T13:11:35.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.278+0000 7fb41d19c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb4180d2320 0x7fb4180cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.278+0000 7fb41d19c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb4180d2320 0x7fb4180cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35500/0 (socket says 172.21.15.156:35500) 2023-11-28T13:11:35.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.278+0000 7fb41d19c700 1 -- 172.21.15.156:0/4284363219 learned_addr learned my addr 172.21.15.156:0/4284363219 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:35.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.278+0000 7fb417fff700 1 --2- 172.21.15.156:0/4284363219 >> v2:172.21.15.186:3300/0 conn(0x7fb418161340 0x7fb4180d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.278+0000 7fb41d19c700 1 -- 172.21.15.156:0/4284363219 >> v2:172.21.15.186:3300/0 conn(0x7fb418161340 msgr2=0x7fb4180d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:35.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.278+0000 7fb41d19c700 1 --2- 172.21.15.156:0/4284363219 >> v2:172.21.15.186:3300/0 conn(0x7fb418161340 0x7fb4180d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.278+0000 7fb41d19c700 1 -- 172.21.15.156:0/4284363219 >> v1:172.21.15.186:6789/0 conn(0x7fb41815c9b0 legacy=0x7fb4180b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:35.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.278+0000 7fb41d19c700 1 -- 172.21.15.156:0/4284363219 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4180b5520 con 0x7fb4180d2320 2023-11-28T13:11:35.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.278+0000 7fb41d19c700 1 --2- 172.21.15.156:0/4284363219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180d2320 0x7fb4180cd7c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb4180685c0 tx=0x7fb41000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc9401ac3fb196e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:35.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.279+0000 7fb4177fe700 1 -- 172.21.15.156:0/4284363219 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41000e070 con 0x7fb4180d2320 2023-11-28T13:11:35.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.279+0000 7fb4177fe700 1 -- 172.21.15.156:0/4284363219 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41000b980 con 0x7fb4180d2320 2023-11-28T13:11:35.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.279+0000 7fb4177fe700 1 -- 172.21.15.156:0/4284363219 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41000bd30 con 0x7fb4180d2320 2023-11-28T13:11:35.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.279+0000 7fb41ebff700 1 -- 172.21.15.156:0/4284363219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180d2320 msgr2=0x7fb4180cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:35.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.279+0000 7fb41ebff700 1 --2- 172.21.15.156:0/4284363219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180d2320 0x7fb4180cd7c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb4180685c0 tx=0x7fb41000b330 comp rx=0 tx=0).stop 2023-11-28T13:11:35.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.279+0000 7fb41ebff700 1 -- 172.21.15.156:0/4284363219 shutdown_connections 2023-11-28T13:11:35.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.279+0000 7fb41ebff700 1 --2- 172.21.15.156:0/4284363219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180d2320 0x7fb4180cd7c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.279+0000 7fb41ebff700 1 --2- 172.21.15.156:0/4284363219 >> v2:172.21.15.186:3300/0 conn(0x7fb418161340 0x7fb4180d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.279+0000 7fb41ebff700 1 -- 172.21.15.156:0/4284363219 >> 172.21.15.156:0/4284363219 conn(0x7fb418072600 msgr2=0x7fb4180cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:35.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.279+0000 7fb41ebff700 1 -- 172.21.15.156:0/4284363219 shutdown_connections 2023-11-28T13:11:35.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.279+0000 7fb41ebff700 1 -- 172.21.15.156:0/4284363219 wait complete. 2023-11-28T13:11:35.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.281+0000 7fb41ebff700 1 Processor -- start 2023-11-28T13:11:35.282 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.281+0000 7fb41ebff700 1 -- start start 2023-11-28T13:11:35.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.282+0000 7fb41ebff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4180d2320 0x7fb418153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.282+0000 7fb41ebff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb41815c9b0 0x7fb4181534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.282+0000 7fb41ebff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb418161340 0x7fb4180c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.282+0000 7fb41ebff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb418153a20 con 0x7fb418161340 2023-11-28T13:11:35.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.282+0000 7fb41ebff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4180c8f30 con 0x7fb4180d2320 2023-11-28T13:11:35.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.282+0000 7fb41ebff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4180c7e90 con 0x7fb41815c9b0 2023-11-28T13:11:35.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.282+0000 7fb41d19c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb418161340 0x7fb4180c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.282+0000 7fb41d19c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb418161340 0x7fb4180c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35508/0 (socket says 172.21.15.156:35508) 2023-11-28T13:11:35.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.282+0000 7fb41d19c700 1 -- 172.21.15.156:0/2488858109 learned_addr learned my addr 172.21.15.156:0/2488858109 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:35.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.283+0000 7fb417fff700 1 --2- 172.21.15.156:0/2488858109 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb41815c9b0 0x7fb4181534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.283+0000 7fb41c99b700 1 --2- 172.21.15.156:0/2488858109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4180d2320 0x7fb418153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.283+0000 7fb41d19c700 1 -- 172.21.15.156:0/2488858109 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb41815c9b0 msgr2=0x7fb4181534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:35.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.283+0000 7fb41d19c700 1 --2- 172.21.15.156:0/2488858109 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb41815c9b0 0x7fb4181534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.283+0000 7fb41d19c700 1 -- 172.21.15.156:0/2488858109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4180d2320 msgr2=0x7fb418153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:35.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.283+0000 7fb41d19c700 1 --2- 172.21.15.156:0/2488858109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4180d2320 0x7fb418153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.283+0000 7fb41d19c700 1 -- 172.21.15.156:0/2488858109 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb410009b30 con 0x7fb418161340 2023-11-28T13:11:35.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.283+0000 7fb41d19c700 1 --2- 172.21.15.156:0/2488858109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb418161340 0x7fb4180c7670 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb410012000 tx=0x7fb410004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:35.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.283+0000 7fb4157fa700 1 -- 172.21.15.156:0/2488858109 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4100092e0 con 0x7fb418161340 2023-11-28T13:11:35.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.283+0000 7fb4157fa700 1 -- 172.21.15.156:0/2488858109 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb410004960 con 0x7fb418161340 2023-11-28T13:11:35.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.283+0000 7fb41ebff700 1 -- 172.21.15.156:0/2488858109 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4180d15e0 con 0x7fb418161340 2023-11-28T13:11:35.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.283+0000 7fb4157fa700 1 -- 172.21.15.156:0/2488858109 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41001b670 con 0x7fb418161340 2023-11-28T13:11:35.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.283+0000 7fb41ebff700 1 -- 172.21.15.156:0/2488858109 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4180d1300 con 0x7fb418161340 2023-11-28T13:11:35.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.286+0000 7fb4157fa700 1 -- 172.21.15.156:0/2488858109 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb410021030 con 0x7fb418161340 2023-11-28T13:11:35.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.286+0000 7fb4157fa700 1 --2- 172.21.15.156:0/2488858109 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3fc0682f0 0x7fb3fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.287+0000 7fb4157fa700 1 -- 172.21.15.156:0/2488858109 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4100872a0 con 0x7fb418161340 2023-11-28T13:11:35.287 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.287+0000 7fb41c99b700 1 --2- 172.21.15.156:0/2488858109 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3fc0682f0 0x7fb3fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.288+0000 7fb41c99b700 1 --2- 172.21.15.156:0/2488858109 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3fc0682f0 0x7fb3fc06a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb4181547d0 tx=0x7fb40c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:35.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.288+0000 7fb43256bb80 1 Processor -- start 2023-11-28T13:11:35.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.288+0000 7fb43256bb80 1 -- start start 2023-11-28T13:11:35.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.289+0000 7fb43256bb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e580860 0x56182e556a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.289+0000 7fb43256bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e57e180 0x56182e580c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.289+0000 7fb43256bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e581170 0x56182e59c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.289+0000 7fb43256bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182e440c10 con 0x56182e57e180 2023-11-28T13:11:35.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.289+0000 7fb43256bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182e4af3a0 con 0x56182e581170 2023-11-28T13:11:35.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.289+0000 7fb43256bb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56182e4a88d0 con 0x56182e580860 2023-11-28T13:11:35.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.290+0000 7fb417fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e57e180 0x56182e580c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.290+0000 7fb417fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e57e180 0x56182e580c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35514/0 (socket says 172.21.15.156:35514) 2023-11-28T13:11:35.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.290+0000 7fb417fff700 1 -- 172.21.15.156:0/1278806988 learned_addr learned my addr 172.21.15.156:0/1278806988 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:35.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.290+0000 7fb41d19c700 1 --2- 172.21.15.156:0/1278806988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e581170 0x56182e59c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.290+0000 7fb41c99b700 1 --2- 172.21.15.156:0/1278806988 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e580860 0x56182e556a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.290+0000 7fb417fff700 1 -- 172.21.15.156:0/1278806988 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e580860 msgr2=0x56182e556a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:35.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.290+0000 7fb417fff700 1 --2- 172.21.15.156:0/1278806988 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e580860 0x56182e556a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.290+0000 7fb417fff700 1 -- 172.21.15.156:0/1278806988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e581170 msgr2=0x56182e59c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:35.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.290+0000 7fb417fff700 1 --2- 172.21.15.156:0/1278806988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e581170 0x56182e59c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.290+0000 7fb417fff700 1 -- 172.21.15.156:0/1278806988 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56182e32a630 con 0x56182e57e180 2023-11-28T13:11:35.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.290+0000 7fb41c99b700 1 --2- 172.21.15.156:0/1278806988 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e580860 0x56182e556a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:11:35.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.290+0000 7fb417fff700 1 --2- 172.21.15.156:0/1278806988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e57e180 0x56182e580c30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb404009b20 tx=0x7fb404009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc82d107eeed74c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:35.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.290+0000 7fb4027fc700 1 -- 172.21.15.156:0/1278806988 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404004020 con 0x56182e57e180 2023-11-28T13:11:35.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.290+0000 7fb4027fc700 1 -- 172.21.15.156:0/1278806988 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40400c050 con 0x56182e57e180 2023-11-28T13:11:35.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.290+0000 7fb4027fc700 1 -- 172.21.15.156:0/1278806988 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4040043c0 con 0x56182e57e180 2023-11-28T13:11:35.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.291+0000 7fb43256bb80 1 -- 172.21.15.156:0/1278806988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e57e180 msgr2=0x56182e580c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:35.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.291+0000 7fb43256bb80 1 --2- 172.21.15.156:0/1278806988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e57e180 0x56182e580c30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb404009b20 tx=0x7fb404009e30 comp rx=0 tx=0).stop 2023-11-28T13:11:35.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.291+0000 7fb43256bb80 1 -- 172.21.15.156:0/1278806988 shutdown_connections 2023-11-28T13:11:35.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.291+0000 7fb43256bb80 1 --2- 172.21.15.156:0/1278806988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e581170 0x56182e59c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.291+0000 7fb43256bb80 1 --2- 172.21.15.156:0/1278806988 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e580860 0x56182e556a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.291+0000 7fb43256bb80 1 --2- 172.21.15.156:0/1278806988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e57e180 0x56182e580c30 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.291+0000 7fb43256bb80 1 -- 172.21.15.156:0/1278806988 >> 172.21.15.156:0/1278806988 conn(0x56182e553db0 msgr2=0x56182e554740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:35.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.291+0000 7fb43256bb80 1 -- 172.21.15.156:0/1278806988 shutdown_connections 2023-11-28T13:11:35.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.291+0000 7fb43256bb80 1 -- 172.21.15.156:0/1278806988 wait complete. 2023-11-28T13:11:35.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.292+0000 7fb43256bb80 1 -- start start 2023-11-28T13:11:35.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.292+0000 7fb43256bb80 1 Processor -- start 2023-11-28T13:11:35.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.293+0000 7fb43256bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e57e180 0x56182e5a3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.293+0000 7fb43256bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e577430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.293+0000 7fb43256bb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e581170 0x56182e54ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.293+0000 7fb43256bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182e32c920 con 0x56182e580860 2023-11-28T13:11:35.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.293+0000 7fb43256bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182e5a3d10 con 0x56182e57e180 2023-11-28T13:11:35.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.293+0000 7fb43256bb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56182e577970 con 0x56182e581170 2023-11-28T13:11:35.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.293+0000 7fb43256bb80 10 client.0 _ll_register_callbacks cb 0x56182e34e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:11:35.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.293+0000 7fb417fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e577430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.293+0000 7fb417fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e577430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35526/0 (socket says 172.21.15.156:35526) 2023-11-28T13:11:35.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.293+0000 7fb417fff700 1 -- 172.21.15.156:0/4259605996 learned_addr learned my addr 172.21.15.156:0/4259605996 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:35.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.294+0000 7fb41d19c700 1 --2- 172.21.15.156:0/4259605996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e581170 0x56182e54ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.294+0000 7fb41c99b700 1 --2- 172.21.15.156:0/4259605996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e57e180 0x56182e5a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.294+0000 7fb417fff700 1 -- 172.21.15.156:0/4259605996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e581170 msgr2=0x56182e54ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:35.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.294+0000 7fb417fff700 1 --2- 172.21.15.156:0/4259605996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e581170 0x56182e54ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.294+0000 7fb417fff700 1 -- 172.21.15.156:0/4259605996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e57e180 msgr2=0x56182e5a3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:35.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.294+0000 7fb417fff700 1 --2- 172.21.15.156:0/4259605996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e57e180 0x56182e5a3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.294+0000 7fb417fff700 1 -- 172.21.15.156:0/4259605996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb404009720 con 0x56182e580860 2023-11-28T13:11:35.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.294+0000 7fb417fff700 1 --2- 172.21.15.156:0/4259605996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e577430 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb404018040 tx=0x7fb4040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131e363b7c7c74f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:35.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.294+0000 7fb3f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:11:35.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.294+0000 7fb3f77fe700 1 -- 172.21.15.156:0/4259605996 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4040091e0 con 0x56182e580860 2023-11-28T13:11:35.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.294+0000 7fb3f77fe700 1 -- 172.21.15.156:0/4259605996 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb404025070 con 0x56182e580860 2023-11-28T13:11:35.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.294+0000 7fb43256bb80 1 -- 172.21.15.156:0/4259605996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56182e476190 con 0x56182e580860 2023-11-28T13:11:35.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.295+0000 7fb3f77fe700 1 -- 172.21.15.156:0/4259605996 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb404015680 con 0x56182e580860 2023-11-28T13:11:35.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.295+0000 7fb43256bb80 10 client.6126 fetch_fsmap learned FSMap version 32 2023-11-28T13:11:35.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.295+0000 7fb43256bb80 1 -- 172.21.15.156:0/4259605996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x56182e2d6de0 con 0x56182e580860 2023-11-28T13:11:35.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.295+0000 7fb3f77fe700 1 -- 172.21.15.156:0/4259605996 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56182e476190 con 0x56182e580860 2023-11-28T13:11:35.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.295+0000 7fb3f77fe700 1 -- 172.21.15.156:0/4259605996 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb4040157e0 con 0x56182e580860 2023-11-28T13:11:35.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.295+0000 7fb43256bb80 10 client.6126 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T13:11:35.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.295+0000 7fb43256bb80 10 client.6126 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T13:11:35.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.295+0000 7fb43256bb80 20 client.6126 populate_metadata read hostname 'smithi156' 2023-11-28T13:11:35.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.295+0000 7fb43256bb80 1 --2- 172.21.15.156:0/4259605996 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x56182e579090 0x56182e59fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.295+0000 7fb43256bb80 4 client.6126 mds_command: new command op to 5919 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:11:35.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.295+0000 7fb43256bb80 1 -- 172.21.15.156:0/4259605996 --> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56182e32cf60 con 0x56182e579090 2023-11-28T13:11:35.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.295+0000 7fb41c99b700 1 --2- 172.21.15.156:0/4259605996 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x56182e579090 0x56182e59fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.296+0000 7fb41c99b700 1 --2- 172.21.15.156:0/4259605996 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x56182e579090 0x56182e59fd20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=942bc1d938dab8e7 server_cookie=163a7a3a59739cdd in_seq=0 out_seq=0 2023-11-28T13:11:35.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.296+0000 7fb3f77fe700 10 client.6126 ms_handle_connect on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:35.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.296+0000 7fb3f77fe700 1 -- 172.21.15.156:0/4259605996 <== mds.0 v2:172.21.15.156:6832/3314379816 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x56182e32cf60 con 0x56182e579090 2023-11-28T13:11:35.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.296+0000 7fb3f77fe700 10 client.6126 handle_command_reply: tid=0 2023-11-28T13:11:35.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.296+0000 7fb43256bb80 1 client.6126 shutdown 2023-11-28T13:11:35.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.297+0000 7fb43256bb80 1 -- 172.21.15.156:0/4259605996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 msgr2=0x56182e577430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:35.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.297+0000 7fb43256bb80 1 --2- 172.21.15.156:0/4259605996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e577430 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb404018040 tx=0x7fb4040047c0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.297+0000 7fb43256bb80 1 -- 172.21.15.156:0/4259605996 shutdown_connections 2023-11-28T13:11:35.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.297+0000 7fb43256bb80 1 --2- 172.21.15.156:0/4259605996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e581170 0x56182e54ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.297+0000 7fb43256bb80 1 --2- 172.21.15.156:0/4259605996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e57e180 0x56182e5a3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.297+0000 7fb43256bb80 1 --2- 172.21.15.156:0/4259605996 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x56182e579090 0x56182e59fd20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.297+0000 7fb43256bb80 1 --2- 172.21.15.156:0/4259605996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e577430 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.297+0000 7fb43256bb80 1 -- 172.21.15.156:0/4259605996 >> 172.21.15.156:0/4259605996 conn(0x56182e553db0 msgr2=0x56182e59eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:35.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.297+0000 7fb3f77fe700 0 client.6126 ms_handle_reset on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:35.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.297+0000 7fb43256bb80 1 -- 172.21.15.156:0/4259605996 shutdown_connections 2023-11-28T13:11:35.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.297+0000 7fb43256bb80 1 -- 172.21.15.156:0/4259605996 wait complete. 2023-11-28T13:11:35.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.297+0000 7fb43256bb80 20 client.6126 trim_cache size 0 max 16384 2023-11-28T13:11:35.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.314+0000 7fb43256bb80 1 Processor -- start 2023-11-28T13:11:35.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.314+0000 7fb43256bb80 1 -- start start 2023-11-28T13:11:35.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.315+0000 7fb43256bb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e57e180 0x56182e575070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.315+0000 7fb43256bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e5a1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.315+0000 7fb43256bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e581170 0x56182e5a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.315+0000 7fb43256bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182e554be0 con 0x56182e580860 2023-11-28T13:11:35.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.315+0000 7fb43256bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182e5a1d20 con 0x56182e581170 2023-11-28T13:11:35.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.315+0000 7fb43256bb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56182e550f80 con 0x56182e57e180 2023-11-28T13:11:35.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.315+0000 7fb417fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e5a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.315+0000 7fb417fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e5a1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35536/0 (socket says 172.21.15.156:35536) 2023-11-28T13:11:35.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.315+0000 7fb417fff700 1 -- 172.21.15.156:0/2503128591 learned_addr learned my addr 172.21.15.156:0/2503128591 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:35.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.316+0000 7fb41d19c700 1 --2- 172.21.15.156:0/2503128591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e581170 0x56182e5a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.316+0000 7fb41c99b700 1 --2- 172.21.15.156:0/2503128591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e57e180 0x56182e575070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.316+0000 7fb417fff700 1 -- 172.21.15.156:0/2503128591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e57e180 msgr2=0x56182e575070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:35.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.316+0000 7fb417fff700 1 --2- 172.21.15.156:0/2503128591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e57e180 0x56182e575070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.316+0000 7fb417fff700 1 -- 172.21.15.156:0/2503128591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e581170 msgr2=0x56182e5a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:35.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.316+0000 7fb417fff700 1 --2- 172.21.15.156:0/2503128591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e581170 0x56182e5a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.316+0000 7fb417fff700 1 -- 172.21.15.156:0/2503128591 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56182e46f190 con 0x56182e580860 2023-11-28T13:11:35.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.316+0000 7fb41c99b700 1 --2- 172.21.15.156:0/2503128591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e57e180 0x56182e575070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:11:35.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.316+0000 7fb417fff700 1 --2- 172.21.15.156:0/2503128591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e5a1070 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb4040096f0 tx=0x7fb404021070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36118cea9c63017 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:35.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.316+0000 7fb402ffd700 1 -- 172.21.15.156:0/2503128591 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4040104d0 con 0x56182e580860 2023-11-28T13:11:35.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.316+0000 7fb402ffd700 1 -- 172.21.15.156:0/2503128591 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40400c040 con 0x56182e580860 2023-11-28T13:11:35.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.316+0000 7fb402ffd700 1 -- 172.21.15.156:0/2503128591 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4040158c0 con 0x56182e580860 2023-11-28T13:11:35.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.317+0000 7fb43256bb80 1 -- 172.21.15.156:0/2503128591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 msgr2=0x56182e5a1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:35.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.317+0000 7fb43256bb80 1 --2- 172.21.15.156:0/2503128591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e5a1070 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb4040096f0 tx=0x7fb404021070 comp rx=0 tx=0).stop 2023-11-28T13:11:35.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.317+0000 7fb43256bb80 1 -- 172.21.15.156:0/2503128591 shutdown_connections 2023-11-28T13:11:35.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.317+0000 7fb43256bb80 1 --2- 172.21.15.156:0/2503128591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e581170 0x56182e5a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.317+0000 7fb43256bb80 1 --2- 172.21.15.156:0/2503128591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e57e180 0x56182e575070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.317+0000 7fb43256bb80 1 --2- 172.21.15.156:0/2503128591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e5a1070 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.317+0000 7fb43256bb80 1 -- 172.21.15.156:0/2503128591 >> 172.21.15.156:0/2503128591 conn(0x56182e553db0 msgr2=0x56182e577240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:35.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.317+0000 7fb43256bb80 1 -- 172.21.15.156:0/2503128591 shutdown_connections 2023-11-28T13:11:35.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.317+0000 7fb43256bb80 1 -- 172.21.15.156:0/2503128591 wait complete. 2023-11-28T13:11:35.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.318+0000 7fb43256bb80 1 -- start start 2023-11-28T13:11:35.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.318+0000 7fb43256bb80 1 Processor -- start 2023-11-28T13:11:35.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.319+0000 7fb43256bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e57e180 0x56182e600190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.319+0000 7fb43256bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e5510c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.319+0000 7fb43256bb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e581170 0x56182e551600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.319+0000 7fb43256bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182e575770 con 0x56182e580860 2023-11-28T13:11:35.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.319+0000 7fb43256bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182e573ba0 con 0x56182e57e180 2023-11-28T13:11:35.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.319+0000 7fb43256bb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56182e573e80 con 0x56182e581170 2023-11-28T13:11:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.319+0000 7fb43256bb80 10 client.0 _ll_register_callbacks cb 0x56182e34e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:11:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.319+0000 7fb417fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e5510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.319+0000 7fb417fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e5510c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35542/0 (socket says 172.21.15.156:35542) 2023-11-28T13:11:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.319+0000 7fb417fff700 1 -- 172.21.15.156:0/2841071621 learned_addr learned my addr 172.21.15.156:0/2841071621 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.319+0000 7fb41d19c700 1 --2- 172.21.15.156:0/2841071621 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e581170 0x56182e551600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.319+0000 7fb41c99b700 1 --2- 172.21.15.156:0/2841071621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e57e180 0x56182e600190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.319+0000 7fb417fff700 1 -- 172.21.15.156:0/2841071621 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e581170 msgr2=0x56182e551600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.319+0000 7fb417fff700 1 --2- 172.21.15.156:0/2841071621 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e581170 0x56182e551600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.319+0000 7fb417fff700 1 -- 172.21.15.156:0/2841071621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e57e180 msgr2=0x56182e600190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.319+0000 7fb417fff700 1 --2- 172.21.15.156:0/2841071621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e57e180 0x56182e600190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.320+0000 7fb417fff700 1 -- 172.21.15.156:0/2841071621 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56182e2d6de0 con 0x56182e580860 2023-11-28T13:11:35.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.320+0000 7fb41c99b700 1 --2- 172.21.15.156:0/2841071621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e57e180 0x56182e600190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:11:35.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.320+0000 7fb417fff700 1 --2- 172.21.15.156:0/2841071621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e5510c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb404015fd0 tx=0x7fb404008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0a68f884bce217 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:35.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.320+0000 7fb3f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:11:35.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.320+0000 7fb3f77fe700 1 -- 172.21.15.156:0/2841071621 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40402ada0 con 0x56182e580860 2023-11-28T13:11:35.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.320+0000 7fb3f77fe700 1 -- 172.21.15.156:0/2841071621 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb404025040 con 0x56182e580860 2023-11-28T13:11:35.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.320+0000 7fb43256bb80 1 -- 172.21.15.156:0/2841071621 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56182e4661b0 con 0x56182e580860 2023-11-28T13:11:35.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.320+0000 7fb3f77fe700 1 -- 172.21.15.156:0/2841071621 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb40402a480 con 0x56182e580860 2023-11-28T13:11:35.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.320+0000 7fb43256bb80 10 client.6132 fetch_fsmap learned FSMap version 32 2023-11-28T13:11:35.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.320+0000 7fb43256bb80 1 -- 172.21.15.156:0/2841071621 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x56182e2fd250 con 0x56182e580860 2023-11-28T13:11:35.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.320+0000 7fb3f77fe700 1 -- 172.21.15.156:0/2841071621 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56182e4661b0 con 0x56182e580860 2023-11-28T13:11:35.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.321+0000 7fb3f77fe700 1 -- 172.21.15.156:0/2841071621 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb40402a5e0 con 0x56182e580860 2023-11-28T13:11:35.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.321+0000 7fb43256bb80 10 client.6132 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T13:11:35.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.321+0000 7fb43256bb80 10 client.6132 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T13:11:35.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.321+0000 7fb43256bb80 20 client.6132 populate_metadata read hostname 'smithi156' 2023-11-28T13:11:35.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.321+0000 7fb43256bb80 1 --2- 172.21.15.156:0/2841071621 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x56182e575070 0x56182e54f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.321+0000 7fb43256bb80 4 client.6132 mds_command: new command op to 5919 tid=0[{"prefix": "scrub status"}] 2023-11-28T13:11:35.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.321+0000 7fb43256bb80 1 -- 172.21.15.156:0/2841071621 --> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56182e4501c0 con 0x56182e575070 2023-11-28T13:11:35.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.321+0000 7fb41c99b700 1 --2- 172.21.15.156:0/2841071621 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x56182e575070 0x56182e54f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.321+0000 7fb3f77fe700 10 client.6132 ms_handle_connect on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:35.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.321+0000 7fb41c99b700 1 --2- 172.21.15.156:0/2841071621 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x56182e575070 0x56182e54f810 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6448405d75d422a3 server_cookie=72aa014b23cf7e8a in_seq=0 out_seq=0 2023-11-28T13:11:35.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.321+0000 7fb3f77fe700 1 -- 172.21.15.156:0/2841071621 <== mds.0 v2:172.21.15.156:6832/3314379816 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56182e4501c0 con 0x56182e575070 2023-11-28T13:11:35.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.322+0000 7fb3f77fe700 10 client.6132 handle_command_reply: tid=0 2023-11-28T13:11:35.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.322+0000 7fb43256bb80 1 client.6132 shutdown 2023-11-28T13:11:35.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.322+0000 7fb43256bb80 1 -- 172.21.15.156:0/2841071621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 msgr2=0x56182e5510c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:35.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.322+0000 7fb43256bb80 1 --2- 172.21.15.156:0/2841071621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e5510c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb404015fd0 tx=0x7fb404008040 comp rx=0 tx=0).stop 2023-11-28T13:11:35.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.322+0000 7fb43256bb80 1 -- 172.21.15.156:0/2841071621 shutdown_connections 2023-11-28T13:11:35.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.322+0000 7fb43256bb80 1 --2- 172.21.15.156:0/2841071621 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56182e581170 0x56182e551600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.322+0000 7fb43256bb80 1 --2- 172.21.15.156:0/2841071621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56182e57e180 0x56182e600190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.322+0000 7fb43256bb80 1 --2- 172.21.15.156:0/2841071621 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x56182e575070 0x56182e54f810 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.322+0000 7fb43256bb80 1 --2- 172.21.15.156:0/2841071621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56182e580860 0x56182e5510c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.322+0000 7fb3f77fe700 0 client.6132 ms_handle_reset on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:35.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.322+0000 7fb43256bb80 1 -- 172.21.15.156:0/2841071621 >> 172.21.15.156:0/2841071621 conn(0x56182e553db0 msgr2=0x56182e5a1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:35.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.322+0000 7fb43256bb80 1 -- 172.21.15.156:0/2841071621 shutdown_connections 2023-11-28T13:11:35.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.322+0000 7fb43256bb80 1 -- 172.21.15.156:0/2841071621 wait complete. 2023-11-28T13:11:35.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.322+0000 7fb43256bb80 20 client.6132 trim_cache size 0 max 16384 2023-11-28T13:11:35.329 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:11:35.329 INFO:teuthology.orchestra.run.smithi156.stdout: "status": "no active scrubs running", 2023-11-28T13:11:35.329 INFO:teuthology.orchestra.run.smithi156.stdout: "scrubs": {} 2023-11-28T13:11:35.329 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:11:35.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.323+0000 7fb41ebff700 1 -- 172.21.15.156:0/2488858109 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3fc0682f0 msgr2=0x7fb3fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:35.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.323+0000 7fb41ebff700 1 --2- 172.21.15.156:0/2488858109 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3fc0682f0 0x7fb3fc06a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb4181547d0 tx=0x7fb40c009040 comp rx=0 tx=0).stop 2023-11-28T13:11:35.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.323+0000 7fb41ebff700 1 -- 172.21.15.156:0/2488858109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb418161340 msgr2=0x7fb4180c7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:35.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.323+0000 7fb41ebff700 1 --2- 172.21.15.156:0/2488858109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb418161340 0x7fb4180c7670 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb410012000 tx=0x7fb410004ad0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.323+0000 7fb41ebff700 1 -- 172.21.15.156:0/2488858109 shutdown_connections 2023-11-28T13:11:35.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.323+0000 7fb41ebff700 1 --2- 172.21.15.156:0/2488858109 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb3fc0682f0 0x7fb3fc06a7a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.323+0000 7fb41ebff700 1 --2- 172.21.15.156:0/2488858109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb418161340 0x7fb4180c7670 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.323+0000 7fb41ebff700 1 --2- 172.21.15.156:0/2488858109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4180d2320 0x7fb418153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.323+0000 7fb41ebff700 1 --2- 172.21.15.156:0/2488858109 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb41815c9b0 0x7fb4181534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.323+0000 7fb41ebff700 1 -- 172.21.15.156:0/2488858109 >> 172.21.15.156:0/2488858109 conn(0x7fb418072600 msgr2=0x7fb41815fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:35.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.323+0000 7fb41ebff700 1 -- 172.21.15.156:0/2488858109 shutdown_connections 2023-11-28T13:11:35.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.323+0000 7fb41ebff700 1 -- 172.21.15.156:0/2488858109 wait complete. 2023-11-28T13:11:35.341 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T13:11:35.341 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:11:35.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.537+0000 7f23c5bff700 1 Processor -- start 2023-11-28T13:11:35.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.538+0000 7f23c5bff700 1 -- start start 2023-11-28T13:11:35.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.539+0000 7f23c5bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f23c015c9b0 0x7f23c00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.539+0000 7f23c5bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23c0161340 0x7f23c00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.539+0000 7f23c5bff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f23c00d2320 0x7f23c00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.539+0000 7f23c5bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c00b5800 con 0x7f23c015c9b0 2023-11-28T13:11:35.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.539+0000 7f23c5bff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c00d1950 con 0x7f23c00d2320 2023-11-28T13:11:35.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.539+0000 7f23c5bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c00b5520 con 0x7f23c0161340 2023-11-28T13:11:35.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.539+0000 7f23bf7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f23c015c9b0 0x7f23c00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.539+0000 7f23bf7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f23c015c9b0 0x7f23c00b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35552/0 (socket says 172.21.15.156:35552) 2023-11-28T13:11:35.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.539+0000 7f23bf7fe700 1 -- 172.21.15.156:0/1870682130 learned_addr learned my addr 172.21.15.156:0/1870682130 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:35.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.539+0000 7f23b7fff700 1 --2- 172.21.15.156:0/1870682130 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23c0161340 0x7f23c00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.539+0000 7f23bffff700 1 --2- 172.21.15.156:0/1870682130 >> v2:172.21.15.186:3300/0 conn(0x7f23c00d2320 0x7f23c00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.539+0000 7f23bf7fe700 1 -- 172.21.15.156:0/1870682130 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23c0161340 msgr2=0x7f23c00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:35.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.539+0000 7f23bf7fe700 1 --2- 172.21.15.156:0/1870682130 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23c0161340 0x7f23c00d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.540+0000 7f23bf7fe700 1 -- 172.21.15.156:0/1870682130 >> v2:172.21.15.186:3300/0 conn(0x7f23c00d2320 msgr2=0x7f23c00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:35.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.540+0000 7f23bf7fe700 1 --2- 172.21.15.156:0/1870682130 >> v2:172.21.15.186:3300/0 conn(0x7f23c00d2320 0x7f23c00cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.540+0000 7f23bf7fe700 1 -- 172.21.15.156:0/1870682130 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c00d1300 con 0x7f23c015c9b0 2023-11-28T13:11:35.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.540+0000 7f23bf7fe700 1 --2- 172.21.15.156:0/1870682130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c015c9b0 0x7f23c00b4bf0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f23c008c4d0 tx=0x7f23b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7691129a862d96 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:35.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.540+0000 7f23beffd700 1 -- 172.21.15.156:0/1870682130 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b000b870 con 0x7f23c015c9b0 2023-11-28T13:11:35.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.540+0000 7f23beffd700 1 -- 172.21.15.156:0/1870682130 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b000b9d0 con 0x7f23c015c9b0 2023-11-28T13:11:35.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.540+0000 7f23beffd700 1 -- 172.21.15.156:0/1870682130 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b000bd60 con 0x7f23c015c9b0 2023-11-28T13:11:35.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.540+0000 7f23c5bff700 1 -- 172.21.15.156:0/1870682130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c015c9b0 msgr2=0x7f23c00b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:35.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.540+0000 7f23c5bff700 1 --2- 172.21.15.156:0/1870682130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c015c9b0 0x7f23c00b4bf0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f23c008c4d0 tx=0x7f23b0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.541+0000 7f23c5bff700 1 -- 172.21.15.156:0/1870682130 shutdown_connections 2023-11-28T13:11:35.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.541+0000 7f23c5bff700 1 --2- 172.21.15.156:0/1870682130 >> v2:172.21.15.186:3300/0 conn(0x7f23c00d2320 0x7f23c00cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.541+0000 7f23c5bff700 1 --2- 172.21.15.156:0/1870682130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c015c9b0 0x7f23c00b4bf0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.541+0000 7f23c5bff700 1 --2- 172.21.15.156:0/1870682130 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23c0161340 0x7f23c00d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.541+0000 7f23c5bff700 1 -- 172.21.15.156:0/1870682130 >> 172.21.15.156:0/1870682130 conn(0x7f23c0072600 msgr2=0x7f23c00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:35.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.541+0000 7f23c5bff700 1 -- 172.21.15.156:0/1870682130 shutdown_connections 2023-11-28T13:11:35.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.541+0000 7f23c5bff700 1 -- 172.21.15.156:0/1870682130 wait complete. 2023-11-28T13:11:35.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.542+0000 7f23c5bff700 1 Processor -- start 2023-11-28T13:11:35.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.543+0000 7f23c5bff700 1 -- start start 2023-11-28T13:11:35.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.543+0000 7f23c5bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00d2320 0x7f23c015dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.544+0000 7f23c5bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23c015c9b0 0x7f23c014dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.544+0000 7f23c5bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c0161340 0x7f23c0155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.544+0000 7f23c5bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c01e68a0 con 0x7f23c00d2320 2023-11-28T13:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.544+0000 7f23c5bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c015e150 con 0x7f23c0161340 2023-11-28T13:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.544+0000 7f23c5bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c015e3f0 con 0x7f23c015c9b0 2023-11-28T13:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.544+0000 7f23bf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00d2320 0x7f23c015dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.544+0000 7f23bf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00d2320 0x7f23c015dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35556/0 (socket says 172.21.15.156:35556) 2023-11-28T13:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.544+0000 7f23bf7fe700 1 -- 172.21.15.156:0/15220967 learned_addr learned my addr 172.21.15.156:0/15220967 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.544+0000 7f23bffff700 1 --2- 172.21.15.156:0/15220967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c0161340 0x7f23c0155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.544+0000 7f23b7fff700 1 --2- 172.21.15.156:0/15220967 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23c015c9b0 0x7f23c014dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.544+0000 7f23bf7fe700 1 -- 172.21.15.156:0/15220967 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23c015c9b0 msgr2=0x7f23c014dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:35.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.544+0000 7f23bf7fe700 1 --2- 172.21.15.156:0/15220967 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23c015c9b0 0x7f23c014dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.544+0000 7f23bf7fe700 1 -- 172.21.15.156:0/15220967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c0161340 msgr2=0x7f23c0155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:35.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.544+0000 7f23bf7fe700 1 --2- 172.21.15.156:0/15220967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c0161340 0x7f23c0155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.544+0000 7f23bf7fe700 1 -- 172.21.15.156:0/15220967 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b0009550 con 0x7f23c00d2320 2023-11-28T13:11:35.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.544+0000 7f23bf7fe700 1 --2- 172.21.15.156:0/15220967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00d2320 0x7f23c015dc10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f23b0000c00 tx=0x7f23b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:35.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.545+0000 7f23bcff9700 1 -- 172.21.15.156:0/15220967 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0004310 con 0x7f23c00d2320 2023-11-28T13:11:35.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.545+0000 7f23bcff9700 1 -- 172.21.15.156:0/15220967 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b001e070 con 0x7f23c00d2320 2023-11-28T13:11:35.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.545+0000 7f23bcff9700 1 -- 172.21.15.156:0/15220967 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b00036a0 con 0x7f23c00d2320 2023-11-28T13:11:35.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.545+0000 7f23c5bff700 1 -- 172.21.15.156:0/15220967 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c00b5130 con 0x7f23c00d2320 2023-11-28T13:11:35.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.545+0000 7f23c5bff700 1 -- 172.21.15.156:0/15220967 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c00d15e0 con 0x7f23c00d2320 2023-11-28T13:11:35.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.546+0000 7f23c5bff700 1 -- 172.21.15.156:0/15220967 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c00b52c0 con 0x7f23c00d2320 2023-11-28T13:11:35.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.547+0000 7f23bcff9700 1 -- 172.21.15.156:0/15220967 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f23b0029020 con 0x7f23c00d2320 2023-11-28T13:11:35.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.548+0000 7f23bcff9700 1 --2- 172.21.15.156:0/15220967 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f23a00682f0 0x7f23a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:35.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.548+0000 7f23bcff9700 1 -- 172.21.15.156:0/15220967 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f23b0086e30 con 0x7f23c00d2320 2023-11-28T13:11:35.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.548+0000 7f23b7fff700 1 --2- 172.21.15.156:0/15220967 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f23a00682f0 0x7f23a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:35.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.549+0000 7f23b7fff700 1 --2- 172.21.15.156:0/15220967 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f23a00682f0 0x7f23a006a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f23c01f6210 tx=0x7f23a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:35.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.552+0000 7f23bcff9700 1 -- 172.21.15.156:0/15220967 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f23b00535a0 con 0x7f23c00d2320 2023-11-28T13:11:35.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.786+0000 7f23c5bff700 1 -- 172.21.15.156:0/15220967 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23c00ce4b0 con 0x7f23c00d2320 2023-11-28T13:11:35.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.786+0000 7f23bcff9700 1 -- 172.21.15.156:0/15220967 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4658 (secure 0 0 0) 0x7f23b0056bc0 con 0x7f23c00d2320 2023-11-28T13:11:35.787 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:11:35.787 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3942362276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3942362276},{"type":"v1","addr":"172.21.15.186:6835","nonce":3942362276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/722198303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":722198303},{"type":"v1","addr":"172.21.15.186:6837","nonce":722198303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5916,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3601975324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3601975324},{"type":"v1","addr":"172.21.15.156:6835","nonce":3601975324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:10:54.928601+0000","modified":"2023-11-28T13:11:11.455488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5919":{"gid":5919,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6834/3314379816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3314379816},{"type":"v1","addr":"172.21.15.156:6834","nonce":3314379816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:11:35.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.788+0000 7f23c5bff700 1 -- 172.21.15.156:0/15220967 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f23a00682f0 msgr2=0x7f23a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:35.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.788+0000 7f23c5bff700 1 --2- 172.21.15.156:0/15220967 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f23a00682f0 0x7f23a006a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f23c01f6210 tx=0x7f23a8008040 comp rx=0 tx=0).stop 2023-11-28T13:11:35.789 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.788+0000 7f23c5bff700 1 -- 172.21.15.156:0/15220967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00d2320 msgr2=0x7f23c015dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:35.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.788+0000 7f23c5bff700 1 --2- 172.21.15.156:0/15220967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00d2320 0x7f23c015dc10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f23b0000c00 tx=0x7f23b0004750 comp rx=0 tx=0).stop 2023-11-28T13:11:35.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.789+0000 7f23c5bff700 1 -- 172.21.15.156:0/15220967 shutdown_connections 2023-11-28T13:11:35.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.789+0000 7f23c5bff700 1 --2- 172.21.15.156:0/15220967 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f23a00682f0 0x7f23a006a7a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.789+0000 7f23c5bff700 1 --2- 172.21.15.156:0/15220967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c0161340 0x7f23c0155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.789+0000 7f23c5bff700 1 --2- 172.21.15.156:0/15220967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00d2320 0x7f23c015dc10 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.789+0000 7f23c5bff700 1 --2- 172.21.15.156:0/15220967 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23c015c9b0 0x7f23c014dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:35.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.789+0000 7f23c5bff700 1 -- 172.21.15.156:0/15220967 >> 172.21.15.156:0/15220967 conn(0x7f23c0072600 msgr2=0x7f23c015ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:35.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.789+0000 7f23c5bff700 1 -- 172.21.15.156:0/15220967 shutdown_connections 2023-11-28T13:11:35.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:35.789+0000 7f23c5bff700 1 -- 172.21.15.156:0/15220967 wait complete. 2023-11-28T13:11:35.791 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 32 2023-11-28T13:11:35.808 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2023-11-28T13:11:36.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.007+0000 7f4c585ff700 1 Processor -- start 2023-11-28T13:11:36.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.008+0000 7f4c585ff700 1 -- start start 2023-11-28T13:11:36.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.009+0000 7f4c585ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4c5015c9b0 0x7f4c500b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.009+0000 7f4c585ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c50161340 0x7f4c500d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.009+0000 7f4c585ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c500d1950 con 0x7f4c500d2320 2023-11-28T13:11:36.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.009+0000 7f4c585ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c500b5800 con 0x7f4c5015c9b0 2023-11-28T13:11:36.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.009+0000 7f4c585ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c500b5520 con 0x7f4c50161340 2023-11-28T13:11:36.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.009+0000 7f4c5639b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4c5015c9b0 0x7f4c500b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.009+0000 7f4c5639b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4c5015c9b0 0x7f4c500b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37450/0 (socket says 172.21.15.156:37450) 2023-11-28T13:11:36.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.009+0000 7f4c5639b700 1 -- 172.21.15.156:0/644733012 learned_addr learned my addr 172.21.15.156:0/644733012 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:36.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.009+0000 7f4c55b9a700 1 --2- 172.21.15.156:0/644733012 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c50161340 0x7f4c500d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.010+0000 7f4c5639b700 1 -- 172.21.15.156:0/644733012 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c50161340 msgr2=0x7f4c500d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:36.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.010+0000 7f4c5639b700 1 --2- 172.21.15.156:0/644733012 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c50161340 0x7f4c500d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.010+0000 7f4c5639b700 1 -- 172.21.15.156:0/644733012 >> v1:172.21.15.156:6789/0 conn(0x7f4c500d2320 legacy=0x7f4c500cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:36.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.010+0000 7f4c5639b700 1 -- 172.21.15.156:0/644733012 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c500d1300 con 0x7f4c5015c9b0 2023-11-28T13:11:36.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.010+0000 7f4c5639b700 1 --2- 172.21.15.156:0/644733012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c5015c9b0 0x7f4c500b4bf0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0098f0 tx=0x7f4c4c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8601648f753350 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:36.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.010+0000 7f4c55399700 1 -- 172.21.15.156:0/644733012 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c011830 con 0x7f4c5015c9b0 2023-11-28T13:11:36.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.010+0000 7f4c55399700 1 -- 172.21.15.156:0/644733012 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4c011990 con 0x7f4c5015c9b0 2023-11-28T13:11:36.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.010+0000 7f4c55399700 1 -- 172.21.15.156:0/644733012 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c011d20 con 0x7f4c5015c9b0 2023-11-28T13:11:36.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.011+0000 7f4c585ff700 1 -- 172.21.15.156:0/644733012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c5015c9b0 msgr2=0x7f4c500b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:36.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.011+0000 7f4c585ff700 1 --2- 172.21.15.156:0/644733012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c5015c9b0 0x7f4c500b4bf0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0098f0 tx=0x7f4c4c009c00 comp rx=0 tx=0).stop 2023-11-28T13:11:36.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.011+0000 7f4c585ff700 1 -- 172.21.15.156:0/644733012 shutdown_connections 2023-11-28T13:11:36.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.011+0000 7f4c585ff700 1 --2- 172.21.15.156:0/644733012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c5015c9b0 0x7f4c500b4bf0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.011+0000 7f4c585ff700 1 --2- 172.21.15.156:0/644733012 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c50161340 0x7f4c500d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.011+0000 7f4c585ff700 1 -- 172.21.15.156:0/644733012 >> 172.21.15.156:0/644733012 conn(0x7f4c50072600 msgr2=0x7f4c500cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:36.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.011+0000 7f4c585ff700 1 -- 172.21.15.156:0/644733012 shutdown_connections 2023-11-28T13:11:36.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.011+0000 7f4c585ff700 1 -- 172.21.15.156:0/644733012 wait complete. 2023-11-28T13:11:36.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.013+0000 7f4c585ff700 1 Processor -- start 2023-11-28T13:11:36.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.013+0000 7f4c585ff700 1 -- start start 2023-11-28T13:11:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.014+0000 7f4c585ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c500d2320 0x7f4c50151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.014+0000 7f4c585ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c5015c9b0 0x7f4c50150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.014+0000 7f4c585ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c50161340 0x7f4c501e8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.014+0000 7f4c585ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c50150760 con 0x7f4c50161340 2023-11-28T13:11:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.014+0000 7f4c585ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c50154af0 con 0x7f4c5015c9b0 2023-11-28T13:11:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.014+0000 7f4c585ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c50154dd0 con 0x7f4c500d2320 2023-11-28T13:11:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.014+0000 7f4c56b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c50161340 0x7f4c501e8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.014+0000 7f4c56b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c50161340 0x7f4c501e8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37462/0 (socket says 172.21.15.156:37462) 2023-11-28T13:11:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.014+0000 7f4c55b9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c5015c9b0 0x7f4c50150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.014+0000 7f4c56b9c700 1 -- 172.21.15.156:0/3615416964 learned_addr learned my addr 172.21.15.156:0/3615416964 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:36.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.014+0000 7f4c5639b700 1 --2- 172.21.15.156:0/3615416964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c500d2320 0x7f4c50151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.014+0000 7f4c56b9c700 1 -- 172.21.15.156:0/3615416964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c500d2320 msgr2=0x7f4c50151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:36.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.014+0000 7f4c56b9c700 1 --2- 172.21.15.156:0/3615416964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c500d2320 0x7f4c50151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.014+0000 7f4c56b9c700 1 -- 172.21.15.156:0/3615416964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c5015c9b0 msgr2=0x7f4c50150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:36.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.014+0000 7f4c56b9c700 1 --2- 172.21.15.156:0/3615416964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c5015c9b0 0x7f4c50150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.014+0000 7f4c56b9c700 1 -- 172.21.15.156:0/3615416964 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c009550 con 0x7f4c50161340 2023-11-28T13:11:36.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.014+0000 7f4c55b9a700 1 --2- 172.21.15.156:0/3615416964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c5015c9b0 0x7f4c50150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:11:36.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.015+0000 7f4c56b9c700 1 --2- 172.21.15.156:0/3615416964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c50161340 0x7f4c501e8430 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800cb50 tx=0x7f4c4800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:36.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.015+0000 7f4c46ffd700 1 -- 172.21.15.156:0/3615416964 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48003c00 con 0x7f4c50161340 2023-11-28T13:11:36.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.015+0000 7f4c46ffd700 1 -- 172.21.15.156:0/3615416964 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c48003d60 con 0x7f4c50161340 2023-11-28T13:11:36.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.015+0000 7f4c46ffd700 1 -- 172.21.15.156:0/3615416964 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4800aa20 con 0x7f4c50161340 2023-11-28T13:11:36.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.015+0000 7f4c585ff700 1 -- 172.21.15.156:0/3615416964 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c500b5130 con 0x7f4c50161340 2023-11-28T13:11:36.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.015+0000 7f4c585ff700 1 -- 172.21.15.156:0/3615416964 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c500d15e0 con 0x7f4c50161340 2023-11-28T13:11:36.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.017+0000 7f4c46ffd700 1 -- 172.21.15.156:0/3615416964 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4c48010050 con 0x7f4c50161340 2023-11-28T13:11:36.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.018+0000 7f4c46ffd700 1 --2- 172.21.15.156:0/3615416964 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4c340682f0 0x7f4c3406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.018+0000 7f4c46ffd700 1 -- 172.21.15.156:0/3615416964 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c48085bf0 con 0x7f4c50161340 2023-11-28T13:11:36.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.018+0000 7f4c5639b700 1 --2- 172.21.15.156:0/3615416964 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4c340682f0 0x7f4c3406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.019+0000 7f4c5639b700 1 --2- 172.21.15.156:0/3615416964 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4c340682f0 0x7f4c3406a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009c40 tx=0x7f4c4c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:36.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.020+0000 7f4c6bfbbb80 1 Processor -- start 2023-11-28T13:11:36.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.020+0000 7f4c6bfbbb80 1 -- start start 2023-11-28T13:11:36.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.021+0000 7f4c6bfbbb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc63a8b0 0x555cdc610ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.021+0000 7f4c6bfbbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc6381d0 0x555cdc63ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.021+0000 7f4c6bfbbb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc63b1c0 0x555cdc656810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.022+0000 7f4c6bfbbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdc4fac60 con 0x555cdc6381d0 2023-11-28T13:11:36.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.022+0000 7f4c6bfbbb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdc5693f0 con 0x555cdc63a8b0 2023-11-28T13:11:36.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.022+0000 7f4c6bfbbb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555cdc562920 con 0x555cdc63b1c0 2023-11-28T13:11:36.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.022+0000 7f4c55b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc6381d0 0x555cdc63ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.022+0000 7f4c55b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc6381d0 0x555cdc63ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37466/0 (socket says 172.21.15.156:37466) 2023-11-28T13:11:36.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.022+0000 7f4c55b9a700 1 -- 172.21.15.156:0/313012728 learned_addr learned my addr 172.21.15.156:0/313012728 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:36.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.022+0000 7f4c5639b700 1 --2- 172.21.15.156:0/313012728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc63a8b0 0x555cdc610ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.022+0000 7f4c56b9c700 1 --2- 172.21.15.156:0/313012728 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc63b1c0 0x555cdc656810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.022+0000 7f4c55b9a700 1 -- 172.21.15.156:0/313012728 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc63b1c0 msgr2=0x555cdc656810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:36.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.022+0000 7f4c55b9a700 1 --2- 172.21.15.156:0/313012728 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc63b1c0 0x555cdc656810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.022+0000 7f4c55b9a700 1 -- 172.21.15.156:0/313012728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc63a8b0 msgr2=0x555cdc610ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:36.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.022+0000 7f4c55b9a700 1 --2- 172.21.15.156:0/313012728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc63a8b0 0x555cdc610ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.022+0000 7f4c55b9a700 1 -- 172.21.15.156:0/313012728 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cdc3e4680 con 0x555cdc6381d0 2023-11-28T13:11:36.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.022+0000 7f4c55b9a700 1 --2- 172.21.15.156:0/313012728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc6381d0 0x555cdc63ac80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4c40011320 tx=0x7f4c40011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410ad8d2da17ada9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:36.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.023+0000 7f4c33fff700 1 -- 172.21.15.156:0/313012728 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40004020 con 0x555cdc6381d0 2023-11-28T13:11:36.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.023+0000 7f4c33fff700 1 -- 172.21.15.156:0/313012728 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c40004180 con 0x555cdc6381d0 2023-11-28T13:11:36.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.023+0000 7f4c33fff700 1 -- 172.21.15.156:0/313012728 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c400044f0 con 0x555cdc6381d0 2023-11-28T13:11:36.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.023+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/313012728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc6381d0 msgr2=0x555cdc63ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:36.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.023+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/313012728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc6381d0 0x555cdc63ac80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4c40011320 tx=0x7f4c40011630 comp rx=0 tx=0).stop 2023-11-28T13:11:36.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.023+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/313012728 shutdown_connections 2023-11-28T13:11:36.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.023+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/313012728 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc63b1c0 0x555cdc656810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.023+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/313012728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc63a8b0 0x555cdc610ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.023+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/313012728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc6381d0 0x555cdc63ac80 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.023+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/313012728 >> 172.21.15.156:0/313012728 conn(0x555cdc60de00 msgr2=0x555cdc60e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:36.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.023+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/313012728 shutdown_connections 2023-11-28T13:11:36.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.023+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/313012728 wait complete. 2023-11-28T13:11:36.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.024+0000 7f4c6bfbbb80 1 -- start start 2023-11-28T13:11:36.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.024+0000 7f4c6bfbbb80 1 Processor -- start 2023-11-28T13:11:36.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.025+0000 7f4c6bfbbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc6381d0 0x555cdc659d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.025+0000 7f4c6bfbbb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc63a8b0 0x555cdc63dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.025+0000 7f4c6bfbbb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc63b1c0 0x555cdc63cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.025+0000 7f4c6bfbbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdc3e6970 con 0x555cdc6381d0 2023-11-28T13:11:36.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.025+0000 7f4c6bfbbb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdc65a760 con 0x555cdc63a8b0 2023-11-28T13:11:36.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.025+0000 7f4c6bfbbb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555cdc65a2d0 con 0x555cdc63b1c0 2023-11-28T13:11:36.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.025+0000 7f4c6bfbbb80 10 client.0 _ll_register_callbacks cb 0x555cdc4082e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:11:36.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.026+0000 7f4c5639b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc6381d0 0x555cdc659d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.026+0000 7f4c5639b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc6381d0 0x555cdc659d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37468/0 (socket says 172.21.15.156:37468) 2023-11-28T13:11:36.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.026+0000 7f4c5639b700 1 -- 172.21.15.156:0/3083745668 learned_addr learned my addr 172.21.15.156:0/3083745668 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:36.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.026+0000 7f4c56b9c700 1 --2- 172.21.15.156:0/3083745668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc63b1c0 0x555cdc63cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.026+0000 7f4c55b9a700 1 --2- 172.21.15.156:0/3083745668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc63a8b0 0x555cdc63dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.026+0000 7f4c5639b700 1 -- 172.21.15.156:0/3083745668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc63b1c0 msgr2=0x555cdc63cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:36.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.026+0000 7f4c5639b700 1 --2- 172.21.15.156:0/3083745668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc63b1c0 0x555cdc63cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.026+0000 7f4c5639b700 1 -- 172.21.15.156:0/3083745668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc63a8b0 msgr2=0x555cdc63dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:36.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.026+0000 7f4c5639b700 1 --2- 172.21.15.156:0/3083745668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc63a8b0 0x555cdc63dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.026+0000 7f4c5639b700 1 -- 172.21.15.156:0/3083745668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c40009c50 con 0x555cdc6381d0 2023-11-28T13:11:36.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.026+0000 7f4c5639b700 1 --2- 172.21.15.156:0/3083745668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc6381d0 0x555cdc659d90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009f50 tx=0x7f4c4c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661386b97f999c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:36.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.026+0000 7f4c317fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:11:36.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.027+0000 7f4c317fa700 1 -- 172.21.15.156:0/3083745668 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c02e070 con 0x555cdc6381d0 2023-11-28T13:11:36.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.027+0000 7f4c317fa700 1 -- 172.21.15.156:0/3083745668 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c4c0039f0 con 0x555cdc6381d0 2023-11-28T13:11:36.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.027+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/3083745668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555cdc5301e0 con 0x555cdc6381d0 2023-11-28T13:11:36.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.027+0000 7f4c317fa700 1 -- 172.21.15.156:0/3083745668 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c4c020070 con 0x555cdc6381d0 2023-11-28T13:11:36.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.027+0000 7f4c6bfbbb80 10 client.6150 fetch_fsmap learned FSMap version 32 2023-11-28T13:11:36.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.027+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/3083745668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x555cdc390e30 con 0x555cdc6381d0 2023-11-28T13:11:36.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.027+0000 7f4c317fa700 1 -- 172.21.15.156:0/3083745668 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555cdc5301e0 con 0x555cdc6381d0 2023-11-28T13:11:36.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.027+0000 7f4c317fa700 1 -- 172.21.15.156:0/3083745668 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4c4c003bb0 con 0x555cdc6381d0 2023-11-28T13:11:36.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.027+0000 7f4c6bfbbb80 10 client.6150 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T13:11:36.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.027+0000 7f4c6bfbbb80 10 client.6150 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:11:36.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.027+0000 7f4c6bfbbb80 20 client.6150 populate_metadata read hostname 'smithi156' 2023-11-28T13:11:36.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.027+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/3083745668 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x555cdc631e70 0x555cdc632240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.027+0000 7f4c6bfbbb80 4 client.6150 mds_command: new command op to 5919 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:11:36.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.027+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/3083745668 --> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555cdc3e6fb0 con 0x555cdc631e70 2023-11-28T13:11:36.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.028+0000 7f4c55b9a700 1 --2- 172.21.15.156:0/3083745668 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x555cdc631e70 0x555cdc632240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.028+0000 7f4c55b9a700 1 --2- 172.21.15.156:0/3083745668 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x555cdc631e70 0x555cdc632240 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae57361b85e87678 server_cookie=e3d91c66f64ea6a0 in_seq=0 out_seq=0 2023-11-28T13:11:36.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.028+0000 7f4c317fa700 10 client.6150 ms_handle_connect on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:36.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.029+0000 7f4c317fa700 1 -- 172.21.15.156:0/3083745668 <== mds.0 v2:172.21.15.156:6832/3314379816 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x555cdc3e6fb0 con 0x555cdc631e70 2023-11-28T13:11:36.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.029+0000 7f4c317fa700 10 client.6150 handle_command_reply: tid=0 2023-11-28T13:11:36.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.029+0000 7f4c6bfbbb80 1 client.6150 shutdown 2023-11-28T13:11:36.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.029+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/3083745668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc6381d0 msgr2=0x555cdc659d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:36.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.029+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/3083745668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc6381d0 0x555cdc659d90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009f50 tx=0x7f4c4c0047c0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.029+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/3083745668 shutdown_connections 2023-11-28T13:11:36.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.029+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/3083745668 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc63b1c0 0x555cdc63cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.029+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/3083745668 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x555cdc631e70 0x555cdc632240 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.029+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/3083745668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc6381d0 0x555cdc659d90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.029+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/3083745668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc63a8b0 0x555cdc63dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.029+0000 7f4c317fa700 0 client.6150 ms_handle_reset on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:36.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.029+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/3083745668 >> 172.21.15.156:0/3083745668 conn(0x555cdc60de00 msgr2=0x555cdc60f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:36.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.029+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/3083745668 shutdown_connections 2023-11-28T13:11:36.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.029+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/3083745668 wait complete. 2023-11-28T13:11:36.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.029+0000 7f4c6bfbbb80 20 client.6150 trim_cache size 0 max 16384 2023-11-28T13:11:36.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.042+0000 7f4c6bfbbb80 1 Processor -- start 2023-11-28T13:11:36.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.042+0000 7f4c6bfbbb80 1 -- start start 2023-11-28T13:11:36.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.043+0000 7f4c6bfbbb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc6381d0 0x555cdc60e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.043+0000 7f4c6bfbbb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc63a8b0 0x555cdc631420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.043+0000 7f4c6bfbbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc63b1c0 0x555cdc65c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.043+0000 7f4c6bfbbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdc633890 con 0x555cdc63b1c0 2023-11-28T13:11:36.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.043+0000 7f4c6bfbbb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdc63d360 con 0x555cdc6381d0 2023-11-28T13:11:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.043+0000 7f4c6bfbbb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555cdc60d940 con 0x555cdc63a8b0 2023-11-28T13:11:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.044+0000 7f4c56b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc63b1c0 0x555cdc65c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.044+0000 7f4c5639b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc6381d0 0x555cdc60e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.044+0000 7f4c56b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc63b1c0 0x555cdc65c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37472/0 (socket says 172.21.15.156:37472) 2023-11-28T13:11:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.044+0000 7f4c56b9c700 1 -- 172.21.15.156:0/30615986 learned_addr learned my addr 172.21.15.156:0/30615986 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.044+0000 7f4c55b9a700 1 --2- 172.21.15.156:0/30615986 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc63a8b0 0x555cdc631420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.044+0000 7f4c56b9c700 1 -- 172.21.15.156:0/30615986 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc63a8b0 msgr2=0x555cdc631420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.044+0000 7f4c56b9c700 1 --2- 172.21.15.156:0/30615986 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc63a8b0 0x555cdc631420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.044+0000 7f4c56b9c700 1 -- 172.21.15.156:0/30615986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc6381d0 msgr2=0x555cdc60e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.044+0000 7f4c56b9c700 1 --2- 172.21.15.156:0/30615986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc6381d0 0x555cdc60e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.044+0000 7f4c56b9c700 1 -- 172.21.15.156:0/30615986 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cdc5291e0 con 0x555cdc63b1c0 2023-11-28T13:11:36.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.044+0000 7f4c56b9c700 1 --2- 172.21.15.156:0/30615986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc63b1c0 0x555cdc65c250 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f4c48055f00 tx=0x7f4c4805e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181ac4212a1a3090 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:36.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.044+0000 7f4c44bff700 1 -- 172.21.15.156:0/30615986 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48055710 con 0x555cdc63b1c0 2023-11-28T13:11:36.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.044+0000 7f4c44bff700 1 -- 172.21.15.156:0/30615986 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c48054480 con 0x555cdc63b1c0 2023-11-28T13:11:36.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.044+0000 7f4c44bff700 1 -- 172.21.15.156:0/30615986 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4805de20 con 0x555cdc63b1c0 2023-11-28T13:11:36.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.045+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/30615986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc63b1c0 msgr2=0x555cdc65c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:36.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.045+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/30615986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc63b1c0 0x555cdc65c250 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f4c48055f00 tx=0x7f4c4805e160 comp rx=0 tx=0).stop 2023-11-28T13:11:36.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.045+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/30615986 shutdown_connections 2023-11-28T13:11:36.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.045+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/30615986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc63b1c0 0x555cdc65c250 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.045+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/30615986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc6381d0 0x555cdc60e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.045+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/30615986 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc63a8b0 0x555cdc631420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.045+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/30615986 >> 172.21.15.156:0/30615986 conn(0x555cdc60de00 msgr2=0x555cdc657f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:36.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.045+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/30615986 shutdown_connections 2023-11-28T13:11:36.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.045+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/30615986 wait complete. 2023-11-28T13:11:36.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.046+0000 7f4c6bfbbb80 1 -- start start 2023-11-28T13:11:36.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.046+0000 7f4c6bfbbb80 1 Processor -- start 2023-11-28T13:11:36.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.047+0000 7f4c6bfbbb80 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc6381d0 0x555cdc60a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.047+0000 7f4c6bfbbb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc63a8b0 0x555cdc658fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.047+0000 7f4c6bfbbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc63b1c0 0x555cdc69fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.052 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.047+0000 7f4c6bfbbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdc64bd70 con 0x555cdc63b1c0 2023-11-28T13:11:36.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.047+0000 7f4c6bfbbb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cdc6a1040 con 0x555cdc63a8b0 2023-11-28T13:11:36.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.047+0000 7f4c6bfbbb80 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x555cdc6599e0 con 0x555cdc6381d0 2023-11-28T13:11:36.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.047+0000 7f4c6bfbbb80 10 client.0 _ll_register_callbacks cb 0x555cdc4082e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:11:36.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.047+0000 7f4c56b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc63b1c0 0x555cdc69fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.047+0000 7f4c56b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc63b1c0 0x555cdc69fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37474/0 (socket says 172.21.15.156:37474) 2023-11-28T13:11:36.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.047+0000 7f4c56b9c700 1 -- 172.21.15.156:0/183107837 learned_addr learned my addr 172.21.15.156:0/183107837 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:36.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.047+0000 7f4c5639b700 1 --2- 172.21.15.156:0/183107837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc6381d0 0x555cdc60a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.047+0000 7f4c55b9a700 1 --2- 172.21.15.156:0/183107837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc63a8b0 0x555cdc658fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.047+0000 7f4c56b9c700 1 -- 172.21.15.156:0/183107837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc6381d0 msgr2=0x555cdc60a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:36.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.047+0000 7f4c56b9c700 1 --2- 172.21.15.156:0/183107837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc6381d0 0x555cdc60a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.047+0000 7f4c56b9c700 1 -- 172.21.15.156:0/183107837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc63a8b0 msgr2=0x555cdc658fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:36.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.047+0000 7f4c56b9c700 1 --2- 172.21.15.156:0/183107837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc63a8b0 0x555cdc658fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.047+0000 7f4c56b9c700 1 -- 172.21.15.156:0/183107837 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c500d15e0 con 0x555cdc63b1c0 2023-11-28T13:11:36.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.048+0000 7f4c56b9c700 1 --2- 172.21.15.156:0/183107837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc63b1c0 0x555cdc69fe40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4c48015000 tx=0x7f4c4805b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3ae9451a670faf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:36.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.048+0000 7f4c317fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-11-28T13:11:36.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.048+0000 7f4c317fa700 1 -- 172.21.15.156:0/183107837 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48055510 con 0x555cdc63b1c0 2023-11-28T13:11:36.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.048+0000 7f4c317fa700 1 -- 172.21.15.156:0/183107837 <== mon.0 v2:172.21.15.156:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c48055670 con 0x555cdc63b1c0 2023-11-28T13:11:36.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.048+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/183107837 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555cdc520200 con 0x555cdc63b1c0 2023-11-28T13:11:36.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.048+0000 7f4c317fa700 1 -- 172.21.15.156:0/183107837 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c4800ab80 con 0x555cdc63b1c0 2023-11-28T13:11:36.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.048+0000 7f4c6bfbbb80 10 client.6156 fetch_fsmap learned FSMap version 32 2023-11-28T13:11:36.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.048+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/183107837 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x555cdc3b72a0 con 0x555cdc63b1c0 2023-11-28T13:11:36.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.048+0000 7f4c317fa700 1 -- 172.21.15.156:0/183107837 <== mon.0 v2:172.21.15.156:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555cdc520200 con 0x555cdc63b1c0 2023-11-28T13:11:36.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.048+0000 7f4c317fa700 1 -- 172.21.15.156:0/183107837 <== mon.0 v2:172.21.15.156:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4c4800ace0 con 0x555cdc63b1c0 2023-11-28T13:11:36.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.048+0000 7f4c6bfbbb80 10 client.6156 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T13:11:36.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.048+0000 7f4c6bfbbb80 10 client.6156 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:11:36.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.048+0000 7f4c6bfbbb80 20 client.6156 populate_metadata read hostname 'smithi156' 2023-11-28T13:11:36.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.048+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/183107837 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x555cdc6386d0 0x555cdc638aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.048+0000 7f4c6bfbbb80 4 client.6156 mds_command: new command op to 5919 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-11-28T13:11:36.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.048+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/183107837 --> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x555cdc50a210 con 0x555cdc6386d0 2023-11-28T13:11:36.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.049+0000 7f4c5639b700 1 --2- 172.21.15.156:0/183107837 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x555cdc6386d0 0x555cdc638aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.049+0000 7f4c5639b700 1 --2- 172.21.15.156:0/183107837 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x555cdc6386d0 0x555cdc638aa0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fab99565712eed3 server_cookie=6fdb475c8ae8b138 in_seq=0 out_seq=0 2023-11-28T13:11:36.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.049+0000 7f4c317fa700 10 client.6156 ms_handle_connect on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:36.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.049+0000 7f4c317fa700 1 -- 172.21.15.156:0/183107837 <== mds.0 v2:172.21.15.156:6832/3314379816 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x555cdc50a210 con 0x555cdc6386d0 2023-11-28T13:11:36.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.049+0000 7f4c317fa700 10 client.6156 handle_command_reply: tid=0 2023-11-28T13:11:36.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.049+0000 7f4c6bfbbb80 1 client.6156 shutdown 2023-11-28T13:11:36.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.049+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/183107837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc63b1c0 msgr2=0x555cdc69fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:36.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.049+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/183107837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc63b1c0 0x555cdc69fe40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4c48015000 tx=0x7f4c4805b040 comp rx=0 tx=0).stop 2023-11-28T13:11:36.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.050+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/183107837 shutdown_connections 2023-11-28T13:11:36.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.050+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/183107837 >> [v2:172.21.15.156:6832/3314379816,v1:172.21.15.156:6834/3314379816] conn(0x555cdc6386d0 0x555cdc638aa0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.050+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/183107837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cdc63b1c0 0x555cdc69fe40 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.050+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/183107837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x555cdc6381d0 0x555cdc60a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.050+0000 7f4c317fa700 0 client.6156 ms_handle_reset on v2:172.21.15.156:6832/3314379816 2023-11-28T13:11:36.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.050+0000 7f4c6bfbbb80 1 --2- 172.21.15.156:0/183107837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555cdc63a8b0 0x555cdc658fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.050+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/183107837 >> 172.21.15.156:0/183107837 conn(0x555cdc60de00 msgr2=0x555cdc657ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:36.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.050+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/183107837 shutdown_connections 2023-11-28T13:11:36.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.050+0000 7f4c6bfbbb80 1 -- 172.21.15.156:0/183107837 wait complete. 2023-11-28T13:11:36.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.050+0000 7f4c6bfbbb80 20 client.6156 trim_cache size 0 max 16384 2023-11-28T13:11:36.059 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:11:36.059 INFO:teuthology.orchestra.run.smithi156.stdout:[ 2023-11-28T13:11:36.059 INFO:teuthology.orchestra.run.smithi156.stdout: { 2023-11-28T13:11:36.059 INFO:teuthology.orchestra.run.smithi156.stdout: "damage_type": "backtrace", 2023-11-28T13:11:36.060 INFO:teuthology.orchestra.run.smithi156.stdout: "id": 3996227696, 2023-11-28T13:11:36.060 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 1099511627777, 2023-11-28T13:11:36.060 INFO:teuthology.orchestra.run.smithi156.stdout: "path": "/dir_test/file" 2023-11-28T13:11:36.060 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-11-28T13:11:36.060 INFO:teuthology.orchestra.run.smithi156.stdout:] 2023-11-28T13:11:36.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.050+0000 7f4c585ff700 1 -- 172.21.15.156:0/3615416964 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4c340682f0 msgr2=0x7f4c3406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:36.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.050+0000 7f4c585ff700 1 --2- 172.21.15.156:0/3615416964 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4c340682f0 0x7f4c3406a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009c40 tx=0x7f4c4c008030 comp rx=0 tx=0).stop 2023-11-28T13:11:36.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.050+0000 7f4c585ff700 1 -- 172.21.15.156:0/3615416964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c50161340 msgr2=0x7f4c501e8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:36.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.050+0000 7f4c585ff700 1 --2- 172.21.15.156:0/3615416964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c50161340 0x7f4c501e8430 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800cb50 tx=0x7f4c4800ce60 comp rx=0 tx=0).stop 2023-11-28T13:11:36.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.051+0000 7f4c585ff700 1 -- 172.21.15.156:0/3615416964 shutdown_connections 2023-11-28T13:11:36.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.051+0000 7f4c585ff700 1 --2- 172.21.15.156:0/3615416964 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4c340682f0 0x7f4c3406a7a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.051+0000 7f4c585ff700 1 --2- 172.21.15.156:0/3615416964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c50161340 0x7f4c501e8430 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.051+0000 7f4c585ff700 1 --2- 172.21.15.156:0/3615416964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4c500d2320 0x7f4c50151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.051+0000 7f4c585ff700 1 --2- 172.21.15.156:0/3615416964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c5015c9b0 0x7f4c50150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.051+0000 7f4c585ff700 1 -- 172.21.15.156:0/3615416964 >> 172.21.15.156:0/3615416964 conn(0x7f4c50072600 msgr2=0x7f4c50160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:36.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.051+0000 7f4c585ff700 1 -- 172.21.15.156:0/3615416964 shutdown_connections 2023-11-28T13:11:36.062 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.051+0000 7f4c585ff700 1 -- 172.21.15.156:0/3615416964 wait complete. 2023-11-28T13:11:36.069 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:11:36.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.264+0000 7fbd145ff700 1 Processor -- start 2023-11-28T13:11:36.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.264+0000 7fbd145ff700 1 -- start start 2023-11-28T13:11:36.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.265+0000 7fbd145ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbd0c161340 0x7fbd0c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.265+0000 7fbd145ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd0c0d2320 0x7fbd0c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.265+0000 7fbd145ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd0c0d1950 con 0x7fbd0c15c9b0 2023-11-28T13:11:36.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.265+0000 7fbd145ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd0c0b5800 con 0x7fbd0c161340 2023-11-28T13:11:36.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.265+0000 7fbd145ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c0b5300 con 0x7fbd0c0d2320 2023-11-28T13:11:36.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.265+0000 7fbd1239b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fbd0c15c9b0 0x7fbd0c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:32990/0 (socket says 172.21.15.156:32990) 2023-11-28T13:11:36.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.266+0000 7fbd1239b700 1 -- 172.21.15.156:0/3178652654 learned_addr learned my addr 172.21.15.156:0/3178652654 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:11:36.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.266+0000 7fbd11b9a700 1 --2- 172.21.15.156:0/3178652654 >> v2:172.21.15.156:3300/0 conn(0x7fbd0c161340 0x7fbd0c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.266+0000 7fbd12b9c700 1 --2- 172.21.15.156:0/3178652654 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd0c0d2320 0x7fbd0c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.266+0000 7fbd11b9a700 1 -- 172.21.15.156:0/3178652654 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd0c0d2320 msgr2=0x7fbd0c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:36.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.266+0000 7fbd11b9a700 1 --2- 172.21.15.156:0/3178652654 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd0c0d2320 0x7fbd0c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.266+0000 7fbd11b9a700 1 -- 172.21.15.156:0/3178652654 >> v1:172.21.15.156:6789/0 conn(0x7fbd0c15c9b0 legacy=0x7fbd0c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:36.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.266+0000 7fbd11b9a700 1 -- 172.21.15.156:0/3178652654 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c0b5520 con 0x7fbd0c161340 2023-11-28T13:11:36.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.266+0000 7fbd11b9a700 1 --2- 172.21.15.156:0/3178652654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd0c161340 0x7fbd0c0d1de0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009840 tx=0x7fbcfc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1cac2678633c21 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:36.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.266+0000 7fbd11399700 1 -- 172.21.15.156:0/3178652654 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc011830 con 0x7fbd0c161340 2023-11-28T13:11:36.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.267+0000 7fbd11399700 1 -- 172.21.15.156:0/3178652654 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc011990 con 0x7fbd0c161340 2023-11-28T13:11:36.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.267+0000 7fbd11399700 1 -- 172.21.15.156:0/3178652654 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc011d20 con 0x7fbd0c161340 2023-11-28T13:11:36.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.267+0000 7fbd145ff700 1 -- 172.21.15.156:0/3178652654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd0c161340 msgr2=0x7fbd0c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:36.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.267+0000 7fbd145ff700 1 --2- 172.21.15.156:0/3178652654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd0c161340 0x7fbd0c0d1de0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009840 tx=0x7fbcfc009b50 comp rx=0 tx=0).stop 2023-11-28T13:11:36.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.267+0000 7fbd145ff700 1 -- 172.21.15.156:0/3178652654 shutdown_connections 2023-11-28T13:11:36.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.267+0000 7fbd145ff700 1 --2- 172.21.15.156:0/3178652654 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd0c0d2320 0x7fbd0c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.267+0000 7fbd145ff700 1 --2- 172.21.15.156:0/3178652654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd0c161340 0x7fbd0c0d1de0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.267+0000 7fbd145ff700 1 -- 172.21.15.156:0/3178652654 >> 172.21.15.156:0/3178652654 conn(0x7fbd0c072600 msgr2=0x7fbd0c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:36.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.267+0000 7fbd145ff700 1 -- 172.21.15.156:0/3178652654 shutdown_connections 2023-11-28T13:11:36.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.267+0000 7fbd145ff700 1 -- 172.21.15.156:0/3178652654 wait complete. 2023-11-28T13:11:36.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.269+0000 7fbd145ff700 1 Processor -- start 2023-11-28T13:11:36.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.269+0000 7fbd145ff700 1 -- start start 2023-11-28T13:11:36.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.270+0000 7fbd145ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd0c0d2320 0x7fbd0c150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.270+0000 7fbd145ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd0c15c9b0 0x7fbd0c1578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.270+0000 7fbd145ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd0c161340 0x7fbd0c1550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.270+0000 7fbd145ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c1509d0 con 0x7fbd0c161340 2023-11-28T13:11:36.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.270+0000 7fbd145ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c1e8920 con 0x7fbd0c15c9b0 2023-11-28T13:11:36.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.270+0000 7fbd145ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c1e79c0 con 0x7fbd0c0d2320 2023-11-28T13:11:36.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.270+0000 7fbd12b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd0c161340 0x7fbd0c1550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.270+0000 7fbd12b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd0c161340 0x7fbd0c1550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37490/0 (socket says 172.21.15.156:37490) 2023-11-28T13:11:36.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.270+0000 7fbd12b9c700 1 -- 172.21.15.156:0/3183709402 learned_addr learned my addr 172.21.15.156:0/3183709402 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:36.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.270+0000 7fbd1239b700 1 --2- 172.21.15.156:0/3183709402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd0c0d2320 0x7fbd0c150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.271+0000 7fbd11b9a700 1 --2- 172.21.15.156:0/3183709402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd0c15c9b0 0x7fbd0c1578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.271+0000 7fbd12b9c700 1 -- 172.21.15.156:0/3183709402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd0c0d2320 msgr2=0x7fbd0c150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:36.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.271+0000 7fbd12b9c700 1 --2- 172.21.15.156:0/3183709402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd0c0d2320 0x7fbd0c150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.271+0000 7fbd12b9c700 1 -- 172.21.15.156:0/3183709402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd0c15c9b0 msgr2=0x7fbd0c1578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:36.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.271+0000 7fbd12b9c700 1 --2- 172.21.15.156:0/3183709402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd0c15c9b0 0x7fbd0c1578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.271+0000 7fbd12b9c700 1 -- 172.21.15.156:0/3183709402 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc0094a0 con 0x7fbd0c161340 2023-11-28T13:11:36.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.271+0000 7fbd1239b700 1 --2- 172.21.15.156:0/3183709402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd0c0d2320 0x7fbd0c150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:11:36.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.271+0000 7fbd11b9a700 1 --2- 172.21.15.156:0/3183709402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd0c15c9b0 0x7fbd0c1578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:11:36.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.271+0000 7fbd12b9c700 1 --2- 172.21.15.156:0/3183709402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd0c161340 0x7fbd0c1550b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009fd0 tx=0x7fbd0400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:36.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.271+0000 7fbd02ffd700 1 -- 172.21.15.156:0/3183709402 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0400ede0 con 0x7fbd0c161340 2023-11-28T13:11:36.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.271+0000 7fbd02ffd700 1 -- 172.21.15.156:0/3183709402 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd04014420 con 0x7fbd0c161340 2023-11-28T13:11:36.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.271+0000 7fbd145ff700 1 -- 172.21.15.156:0/3183709402 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0c0d15e0 con 0x7fbd0c161340 2023-11-28T13:11:36.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.271+0000 7fbd02ffd700 1 -- 172.21.15.156:0/3183709402 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd040147d0 con 0x7fbd0c161340 2023-11-28T13:11:36.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.271+0000 7fbd145ff700 1 -- 172.21.15.156:0/3183709402 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0c0d1300 con 0x7fbd0c161340 2023-11-28T13:11:36.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.272+0000 7fbd145ff700 1 -- 172.21.15.156:0/3183709402 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd0c15cd80 con 0x7fbd0c161340 2023-11-28T13:11:36.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.273+0000 7fbd02ffd700 1 -- 172.21.15.156:0/3183709402 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbd04012050 con 0x7fbd0c161340 2023-11-28T13:11:36.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.274+0000 7fbd02ffd700 1 --2- 172.21.15.156:0/3183709402 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcf00682f0 0x7fbcf006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:36.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.275+0000 7fbd02ffd700 1 -- 172.21.15.156:0/3183709402 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbd040863d0 con 0x7fbd0c161340 2023-11-28T13:11:36.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.275+0000 7fbd1239b700 1 --2- 172.21.15.156:0/3183709402 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcf00682f0 0x7fbcf006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:36.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.276+0000 7fbd1239b700 1 --2- 172.21.15.156:0/3183709402 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcf00682f0 0x7fbcf006a7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbd08007fd0 tx=0x7fbd08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:36.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.279+0000 7fbd02ffd700 1 -- 172.21.15.156:0/3183709402 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbd04052b40 con 0x7fbd0c161340 2023-11-28T13:11:36.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.513+0000 7fbd145ff700 1 -- 172.21.15.156:0/3183709402 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbd0c0745c0 con 0x7fbd0c161340 2023-11-28T13:11:36.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.514+0000 7fbd02ffd700 1 -- 172.21.15.156:0/3183709402 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fbd04056160 con 0x7fbd0c161340 2023-11-28T13:11:36.514 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:11:36.515 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:11:36.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.517+0000 7fbd145ff700 1 -- 172.21.15.156:0/3183709402 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcf00682f0 msgr2=0x7fbcf006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:36.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.517+0000 7fbd145ff700 1 --2- 172.21.15.156:0/3183709402 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcf00682f0 0x7fbcf006a7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbd08007fd0 tx=0x7fbd08009040 comp rx=0 tx=0).stop 2023-11-28T13:11:36.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.517+0000 7fbd145ff700 1 -- 172.21.15.156:0/3183709402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd0c161340 msgr2=0x7fbd0c1550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:36.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.517+0000 7fbd145ff700 1 --2- 172.21.15.156:0/3183709402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd0c161340 0x7fbd0c1550b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009fd0 tx=0x7fbd0400e3c0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.517+0000 7fbd145ff700 1 -- 172.21.15.156:0/3183709402 shutdown_connections 2023-11-28T13:11:36.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.517+0000 7fbd145ff700 1 --2- 172.21.15.156:0/3183709402 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcf00682f0 0x7fbcf006a7a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.517+0000 7fbd145ff700 1 --2- 172.21.15.156:0/3183709402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd0c161340 0x7fbd0c1550b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.517+0000 7fbd145ff700 1 --2- 172.21.15.156:0/3183709402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd0c0d2320 0x7fbd0c150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.517+0000 7fbd145ff700 1 --2- 172.21.15.156:0/3183709402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd0c15c9b0 0x7fbd0c1578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:36.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.517+0000 7fbd145ff700 1 -- 172.21.15.156:0/3183709402 >> 172.21.15.156:0/3183709402 conn(0x7fbd0c072600 msgr2=0x7fbd0c0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:36.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.517+0000 7fbd145ff700 1 -- 172.21.15.156:0/3183709402 shutdown_connections 2023-11-28T13:11:36.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:36.517+0000 7fbd145ff700 1 -- 172.21.15.156:0/3183709402 wait complete. 2023-11-28T13:11:36.536 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:11:41.536 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:11:41.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.739+0000 7f6fd73ff700 1 Processor -- start 2023-11-28T13:11:41.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.739+0000 7f6fd73ff700 1 -- start start 2023-11-28T13:11:41.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.741+0000 7f6fd73ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fd0161340 0x7f6fd00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:41.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.741+0000 7f6fd73ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6fd00d2320 0x7f6fd00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:41.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.741+0000 7f6fd73ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd00d1950 con 0x7f6fd015c9b0 2023-11-28T13:11:41.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.741+0000 7f6fd73ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd00b5800 con 0x7f6fd00d2320 2023-11-28T13:11:41.743 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.741+0000 7f6fd73ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00b5300 con 0x7f6fd0161340 2023-11-28T13:11:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.741+0000 7f6fd599c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6fd00d2320 0x7f6fd00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.741+0000 7f6fd599c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6fd00d2320 0x7f6fd00cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37492/0 (socket says 172.21.15.156:37492) 2023-11-28T13:11:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.741+0000 7f6fd599c700 1 -- 172.21.15.156:0/762389050 learned_addr learned my addr 172.21.15.156:0/762389050 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.741+0000 7f6fd519b700 1 --1- 172.21.15.156:0/762389050 >> v1:172.21.15.156:6789/0 conn(0x7f6fd015c9b0 0x7f6fd00b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:32992/0 (socket says 172.21.15.156:32992) 2023-11-28T13:11:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.741+0000 7f6fd499a700 1 --2- 172.21.15.156:0/762389050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fd0161340 0x7f6fd00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.741+0000 7f6fd599c700 1 -- 172.21.15.156:0/762389050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fd0161340 msgr2=0x7f6fd00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.741+0000 7f6fd599c700 1 --2- 172.21.15.156:0/762389050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fd0161340 0x7f6fd00d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.741+0000 7f6fd599c700 1 -- 172.21.15.156:0/762389050 >> v1:172.21.15.156:6789/0 conn(0x7f6fd015c9b0 legacy=0x7f6fd00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.741+0000 7f6fd599c700 1 -- 172.21.15.156:0/762389050 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd00b5520 con 0x7f6fd00d2320 2023-11-28T13:11:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.742+0000 7f6fd599c700 1 --2- 172.21.15.156:0/762389050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd00d2320 0x7f6fd00cd7c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009e80 tx=0x7f6fc800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a8b28c4a25003a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.742+0000 7f6fc7fff700 1 -- 172.21.15.156:0/762389050 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc800e070 con 0x7f6fd00d2320 2023-11-28T13:11:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.742+0000 7f6fc7fff700 1 -- 172.21.15.156:0/762389050 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc800b980 con 0x7f6fd00d2320 2023-11-28T13:11:41.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.742+0000 7f6fc7fff700 1 -- 172.21.15.156:0/762389050 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc800bd30 con 0x7f6fd00d2320 2023-11-28T13:11:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.742+0000 7f6fd73ff700 1 -- 172.21.15.156:0/762389050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd00d2320 msgr2=0x7f6fd00cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.742+0000 7f6fd73ff700 1 --2- 172.21.15.156:0/762389050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd00d2320 0x7f6fd00cd7c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009e80 tx=0x7f6fc800b330 comp rx=0 tx=0).stop 2023-11-28T13:11:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.742+0000 7f6fd73ff700 1 -- 172.21.15.156:0/762389050 shutdown_connections 2023-11-28T13:11:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.742+0000 7f6fd73ff700 1 --2- 172.21.15.156:0/762389050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd00d2320 0x7f6fd00cd7c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.742+0000 7f6fd73ff700 1 --2- 172.21.15.156:0/762389050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fd0161340 0x7f6fd00d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.742+0000 7f6fd73ff700 1 -- 172.21.15.156:0/762389050 >> 172.21.15.156:0/762389050 conn(0x7f6fd0072600 msgr2=0x7f6fd00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.743+0000 7f6fd73ff700 1 -- 172.21.15.156:0/762389050 shutdown_connections 2023-11-28T13:11:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.743+0000 7f6fd73ff700 1 -- 172.21.15.156:0/762389050 wait complete. 2023-11-28T13:11:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.744+0000 7f6fd73ff700 1 Processor -- start 2023-11-28T13:11:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.744+0000 7f6fd73ff700 1 -- start start 2023-11-28T13:11:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.745+0000 7f6fd73ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fd00d2320 0x7f6fd0150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.745+0000 7f6fd73ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fd015c9b0 0x7f6fd014fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.745+0000 7f6fd73ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd0161340 0x7f6fd01e7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.745+0000 7f6fd73ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd01e7e80 con 0x7f6fd0161340 2023-11-28T13:11:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.745+0000 7f6fd73ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00c85b0 con 0x7f6fd015c9b0 2023-11-28T13:11:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.745+0000 7f6fd73ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00c8890 con 0x7f6fd00d2320 2023-11-28T13:11:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.745+0000 7f6fd599c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd0161340 0x7f6fd01e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.745+0000 7f6fd599c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd0161340 0x7f6fd01e7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37496/0 (socket says 172.21.15.156:37496) 2023-11-28T13:11:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.745+0000 7f6fd599c700 1 -- 172.21.15.156:0/444877177 learned_addr learned my addr 172.21.15.156:0/444877177 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.746+0000 7f6fd499a700 1 --2- 172.21.15.156:0/444877177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fd015c9b0 0x7f6fd014fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.746+0000 7f6fd519b700 1 --2- 172.21.15.156:0/444877177 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fd00d2320 0x7f6fd0150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.746+0000 7f6fd599c700 1 -- 172.21.15.156:0/444877177 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fd00d2320 msgr2=0x7f6fd0150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.746+0000 7f6fd599c700 1 --2- 172.21.15.156:0/444877177 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fd00d2320 0x7f6fd0150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.746+0000 7f6fd599c700 1 -- 172.21.15.156:0/444877177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fd015c9b0 msgr2=0x7f6fd014fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.746+0000 7f6fd599c700 1 --2- 172.21.15.156:0/444877177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fd015c9b0 0x7f6fd014fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:41.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.746+0000 7f6fd599c700 1 -- 172.21.15.156:0/444877177 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc8009b30 con 0x7f6fd0161340 2023-11-28T13:11:41.750 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.746+0000 7f6fd599c700 1 --2- 172.21.15.156:0/444877177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd0161340 0x7f6fd01e7630 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00675d0 tx=0x7f6fc8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:41.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.746+0000 7f6fc5ffb700 1 -- 172.21.15.156:0/444877177 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8004960 con 0x7f6fd0161340 2023-11-28T13:11:41.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.746+0000 7f6fc5ffb700 1 -- 172.21.15.156:0/444877177 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc80099b0 con 0x7f6fd0161340 2023-11-28T13:11:41.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.746+0000 7f6fc5ffb700 1 -- 172.21.15.156:0/444877177 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc801b670 con 0x7f6fd0161340 2023-11-28T13:11:41.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.746+0000 7f6fd73ff700 1 -- 172.21.15.156:0/444877177 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd00d15e0 con 0x7f6fd0161340 2023-11-28T13:11:41.751 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.746+0000 7f6fd73ff700 1 -- 172.21.15.156:0/444877177 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd00d1300 con 0x7f6fd0161340 2023-11-28T13:11:41.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.748+0000 7f6fd73ff700 1 -- 172.21.15.156:0/444877177 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd015cd80 con 0x7f6fd0161340 2023-11-28T13:11:41.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.748+0000 7f6fc5ffb700 1 -- 172.21.15.156:0/444877177 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6fc8021030 con 0x7f6fd0161340 2023-11-28T13:11:41.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.749+0000 7f6fc5ffb700 1 --2- 172.21.15.156:0/444877177 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6fb40682f0 0x7f6fb406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:41.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.749+0000 7f6fc5ffb700 1 -- 172.21.15.156:0/444877177 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6fc8087320 con 0x7f6fd0161340 2023-11-28T13:11:41.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.750+0000 7f6fd519b700 1 --2- 172.21.15.156:0/444877177 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6fb40682f0 0x7f6fb406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:41.752 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.751+0000 7f6fd519b700 1 --2- 172.21.15.156:0/444877177 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6fb40682f0 0x7f6fb406a7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc000f60 tx=0x7f6fcc007d10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:41.755 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.754+0000 7f6fc5ffb700 1 -- 172.21.15.156:0/444877177 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6fc8053a90 con 0x7f6fd0161340 2023-11-28T13:11:41.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.984+0000 7f6fd73ff700 1 -- 172.21.15.156:0/444877177 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6fd00c8d50 con 0x7f6fd0161340 2023-11-28T13:11:41.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.985+0000 7f6fc5ffb700 1 -- 172.21.15.156:0/444877177 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f6fc80570b0 con 0x7f6fd0161340 2023-11-28T13:11:41.987 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:11:41.987 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:11:41.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.987+0000 7f6fd73ff700 1 -- 172.21.15.156:0/444877177 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6fb40682f0 msgr2=0x7f6fb406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.988+0000 7f6fd73ff700 1 --2- 172.21.15.156:0/444877177 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6fb40682f0 0x7f6fb406a7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc000f60 tx=0x7f6fcc007d10 comp rx=0 tx=0).stop 2023-11-28T13:11:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.988+0000 7f6fd73ff700 1 -- 172.21.15.156:0/444877177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd0161340 msgr2=0x7f6fd01e7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.988+0000 7f6fd73ff700 1 --2- 172.21.15.156:0/444877177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd0161340 0x7f6fd01e7630 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00675d0 tx=0x7f6fc8004ad0 comp rx=0 tx=0).stop 2023-11-28T13:11:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.988+0000 7f6fd73ff700 1 -- 172.21.15.156:0/444877177 shutdown_connections 2023-11-28T13:11:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.988+0000 7f6fd73ff700 1 --2- 172.21.15.156:0/444877177 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6fb40682f0 0x7f6fb406a7a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.988+0000 7f6fd73ff700 1 --2- 172.21.15.156:0/444877177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fd0161340 0x7f6fd01e7630 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.988+0000 7f6fd73ff700 1 --2- 172.21.15.156:0/444877177 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6fd00d2320 0x7f6fd0150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.988+0000 7f6fd73ff700 1 --2- 172.21.15.156:0/444877177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6fd015c9b0 0x7f6fd014fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.988+0000 7f6fd73ff700 1 -- 172.21.15.156:0/444877177 >> 172.21.15.156:0/444877177 conn(0x7f6fd0072600 msgr2=0x7f6fd00b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.988+0000 7f6fd73ff700 1 -- 172.21.15.156:0/444877177 shutdown_connections 2023-11-28T13:11:41.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:41.988+0000 7f6fd73ff700 1 -- 172.21.15.156:0/444877177 wait complete. 2023-11-28T13:11:42.007 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:11:47.009 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:11:47.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.214+0000 7f46d05ff700 1 Processor -- start 2023-11-28T13:11:47.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.214+0000 7f46d05ff700 1 -- start start 2023-11-28T13:11:47.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.215+0000 7f46d05ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f46c815c9b0 0x7f46c80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:47.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.216+0000 7f46d05ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f46c80cd7c0 0x7f46c80cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:47.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.216+0000 7f46d05ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c80d1950 con 0x7f46c8161340 2023-11-28T13:11:47.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.216+0000 7f46d05ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c80b5800 con 0x7f46c80cd7c0 2023-11-28T13:11:47.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.216+0000 7f46d05ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c80ce140 con 0x7f46c815c9b0 2023-11-28T13:11:47.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.216+0000 7f46ceb9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f46c80cd7c0 0x7f46c80cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:47.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.216+0000 7f46ceb9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f46c80cd7c0 0x7f46c80cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50094/0 (socket says 172.21.15.156:50094) 2023-11-28T13:11:47.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.216+0000 7f46ceb9c700 1 -- 172.21.15.156:0/4031769722 learned_addr learned my addr 172.21.15.156:0/4031769722 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:47.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.216+0000 7f46ce39b700 1 --2- 172.21.15.156:0/4031769722 >> v2:172.21.15.186:3300/0 conn(0x7f46c815c9b0 0x7f46c80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:47.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.216+0000 7f46ceb9c700 1 -- 172.21.15.156:0/4031769722 >> v2:172.21.15.186:3300/0 conn(0x7f46c815c9b0 msgr2=0x7f46c80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:47.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.216+0000 7f46ceb9c700 1 --2- 172.21.15.156:0/4031769722 >> v2:172.21.15.186:3300/0 conn(0x7f46c815c9b0 0x7f46c80b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:47.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.216+0000 7f46ceb9c700 1 -- 172.21.15.156:0/4031769722 >> v1:172.21.15.156:6789/0 conn(0x7f46c8161340 legacy=0x7f46c80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:47.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.216+0000 7f46ceb9c700 1 -- 172.21.15.156:0/4031769722 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c80d24f0 con 0x7f46c80cd7c0 2023-11-28T13:11:47.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.217+0000 7f46ceb9c700 1 --2- 172.21.15.156:0/4031769722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c80cd7c0 0x7f46c80cdc00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f46c8067460 tx=0x7f46c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4694a676784ec9b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:47.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.217+0000 7f46cd399700 1 -- 172.21.15.156:0/4031769722 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c000e070 con 0x7f46c80cd7c0 2023-11-28T13:11:47.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.217+0000 7f46cd399700 1 -- 172.21.15.156:0/4031769722 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c000b9c0 con 0x7f46c80cd7c0 2023-11-28T13:11:47.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.217+0000 7f46cd399700 1 -- 172.21.15.156:0/4031769722 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c000bd70 con 0x7f46c80cd7c0 2023-11-28T13:11:47.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.217+0000 7f46d05ff700 1 -- 172.21.15.156:0/4031769722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c80cd7c0 msgr2=0x7f46c80cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:47.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.217+0000 7f46d05ff700 1 --2- 172.21.15.156:0/4031769722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c80cd7c0 0x7f46c80cdc00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f46c8067460 tx=0x7f46c000b330 comp rx=0 tx=0).stop 2023-11-28T13:11:47.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.217+0000 7f46d05ff700 1 -- 172.21.15.156:0/4031769722 shutdown_connections 2023-11-28T13:11:47.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.217+0000 7f46d05ff700 1 --2- 172.21.15.156:0/4031769722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c80cd7c0 0x7f46c80cdc00 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:47.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.217+0000 7f46d05ff700 1 --2- 172.21.15.156:0/4031769722 >> v2:172.21.15.186:3300/0 conn(0x7f46c815c9b0 0x7f46c80b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:47.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.217+0000 7f46d05ff700 1 -- 172.21.15.156:0/4031769722 >> 172.21.15.156:0/4031769722 conn(0x7f46c8072600 msgr2=0x7f46c80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:47.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.217+0000 7f46d05ff700 1 -- 172.21.15.156:0/4031769722 shutdown_connections 2023-11-28T13:11:47.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.218+0000 7f46d05ff700 1 -- 172.21.15.156:0/4031769722 wait complete. 2023-11-28T13:11:47.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.219+0000 7f46d05ff700 1 Processor -- start 2023-11-28T13:11:47.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.219+0000 7f46d05ff700 1 -- start start 2023-11-28T13:11:47.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.220+0000 7f46d05ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c80cd7c0 0x7f46c8156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:47.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.220+0000 7f46d05ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c815c9b0 0x7f46c815a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:47.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.220+0000 7f46d05ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46c8161340 0x7f46c815acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:47.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.220+0000 7f46d05ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c8156fa0 con 0x7f46c815c9b0 2023-11-28T13:11:47.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.220+0000 7f46d05ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c80c7960 con 0x7f46c80cd7c0 2023-11-28T13:11:47.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.220+0000 7f46d05ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c80c7c40 con 0x7f46c8161340 2023-11-28T13:11:47.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.220+0000 7f46cdb9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c815c9b0 0x7f46c815a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:47.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.221+0000 7f46cdb9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c815c9b0 0x7f46c815a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50100/0 (socket says 172.21.15.156:50100) 2023-11-28T13:11:47.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.221+0000 7f46cdb9a700 1 -- 172.21.15.156:0/3213346360 learned_addr learned my addr 172.21.15.156:0/3213346360 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:47.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.221+0000 7f46cdb9a700 1 -- 172.21.15.156:0/3213346360 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46c8161340 msgr2=0x7f46c815acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:47.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.221+0000 7f46cdb9a700 1 --2- 172.21.15.156:0/3213346360 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46c8161340 0x7f46c815acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:47.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.221+0000 7f46cdb9a700 1 -- 172.21.15.156:0/3213346360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c80cd7c0 msgr2=0x7f46c8156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:47.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.221+0000 7f46ce39b700 1 --2- 172.21.15.156:0/3213346360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c80cd7c0 0x7f46c8156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:47.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.221+0000 7f46cdb9a700 1 --2- 172.21.15.156:0/3213346360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c80cd7c0 0x7f46c8156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:47.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.221+0000 7f46cdb9a700 1 -- 172.21.15.156:0/3213346360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c0009b30 con 0x7f46c815c9b0 2023-11-28T13:11:47.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.221+0000 7f46ce39b700 1 --2- 172.21.15.156:0/3213346360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c80cd7c0 0x7f46c8156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:11:47.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.221+0000 7f46cdb9a700 1 --2- 172.21.15.156:0/3213346360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c815c9b0 0x7f46c815a790 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f46b800c450 tx=0x7f46b800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:47.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.221+0000 7f46beffd700 1 -- 172.21.15.156:0/3213346360 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b80052d0 con 0x7f46c815c9b0 2023-11-28T13:11:47.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.221+0000 7f46beffd700 1 -- 172.21.15.156:0/3213346360 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b8005430 con 0x7f46c815c9b0 2023-11-28T13:11:47.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.221+0000 7f46d05ff700 1 -- 172.21.15.156:0/3213346360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c80d1300 con 0x7f46c815c9b0 2023-11-28T13:11:47.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.221+0000 7f46beffd700 1 -- 172.21.15.156:0/3213346360 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b80038c0 con 0x7f46c815c9b0 2023-11-28T13:11:47.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.222+0000 7f46d05ff700 1 -- 172.21.15.156:0/3213346360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c80b5520 con 0x7f46c815c9b0 2023-11-28T13:11:47.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.223+0000 7f46d05ff700 1 -- 172.21.15.156:0/3213346360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c80d15e0 con 0x7f46c815c9b0 2023-11-28T13:11:47.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.223+0000 7f46beffd700 1 -- 172.21.15.156:0/3213346360 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f46b8019030 con 0x7f46c815c9b0 2023-11-28T13:11:47.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.224+0000 7f46beffd700 1 --2- 172.21.15.156:0/3213346360 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f46ac068320 0x7f46ac06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:47.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.224+0000 7f46beffd700 1 -- 172.21.15.156:0/3213346360 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f46b8084ff0 con 0x7f46c815c9b0 2023-11-28T13:11:47.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.225+0000 7f46ce39b700 1 --2- 172.21.15.156:0/3213346360 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f46ac068320 0x7f46ac06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:47.236 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.226+0000 7f46ce39b700 1 --2- 172.21.15.156:0/3213346360 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f46ac068320 0x7f46ac06a7d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009820 tx=0x7f46c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:47.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.228+0000 7f46beffd700 1 -- 172.21.15.156:0/3213346360 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f46b80517c0 con 0x7f46c815c9b0 2023-11-28T13:11:47.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.462+0000 7f46d05ff700 1 -- 172.21.15.156:0/3213346360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f46c80745c0 con 0x7f46c815c9b0 2023-11-28T13:11:47.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.462+0000 7f46beffd700 1 -- 172.21.15.156:0/3213346360 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f46b8054de0 con 0x7f46c815c9b0 2023-11-28T13:11:47.464 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:11:47.464 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:11:47.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.465+0000 7f46d05ff700 1 -- 172.21.15.156:0/3213346360 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f46ac068320 msgr2=0x7f46ac06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:47.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.465+0000 7f46d05ff700 1 --2- 172.21.15.156:0/3213346360 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f46ac068320 0x7f46ac06a7d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009820 tx=0x7f46c4008040 comp rx=0 tx=0).stop 2023-11-28T13:11:47.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.465+0000 7f46d05ff700 1 -- 172.21.15.156:0/3213346360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c815c9b0 msgr2=0x7f46c815a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:47.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.465+0000 7f46d05ff700 1 --2- 172.21.15.156:0/3213346360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c815c9b0 0x7f46c815a790 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f46b800c450 tx=0x7f46b800c810 comp rx=0 tx=0).stop 2023-11-28T13:11:47.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.465+0000 7f46d05ff700 1 -- 172.21.15.156:0/3213346360 shutdown_connections 2023-11-28T13:11:47.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.465+0000 7f46d05ff700 1 --2- 172.21.15.156:0/3213346360 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f46ac068320 0x7f46ac06a7d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:47.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.465+0000 7f46d05ff700 1 --2- 172.21.15.156:0/3213346360 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46c8161340 0x7f46c815acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:47.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.465+0000 7f46d05ff700 1 --2- 172.21.15.156:0/3213346360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46c80cd7c0 0x7f46c8156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:47.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.465+0000 7f46d05ff700 1 --2- 172.21.15.156:0/3213346360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46c815c9b0 0x7f46c815a790 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:47.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.465+0000 7f46d05ff700 1 -- 172.21.15.156:0/3213346360 >> 172.21.15.156:0/3213346360 conn(0x7f46c8072600 msgr2=0x7f46c80b8490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:47.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.465+0000 7f46d05ff700 1 -- 172.21.15.156:0/3213346360 shutdown_connections 2023-11-28T13:11:47.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:47.465+0000 7f46d05ff700 1 -- 172.21.15.156:0/3213346360 wait complete. 2023-11-28T13:11:47.485 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:11:52.486 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:11:52.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.696+0000 7fc6d35ff700 1 Processor -- start 2023-11-28T13:11:52.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.696+0000 7fc6d35ff700 1 -- start start 2023-11-28T13:11:52.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.697+0000 7fc6d35ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc6cc158ce0 0x7fc6cc15d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:52.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.697+0000 7fc6d35ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6cc06fb30 0x7fc6cc0cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:52.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.697+0000 7fc6d35ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6cc06bf10 con 0x7fc6cc0ce2e0 2023-11-28T13:11:52.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.697+0000 7fc6d35ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6cc0ce720 con 0x7fc6cc158ce0 2023-11-28T13:11:52.699 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.697+0000 7fc6d35ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc15d730 con 0x7fc6cc06fb30 2023-11-28T13:11:52.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.698+0000 7fc6d0b9a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6cc06fb30 0x7fc6cc0cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:52.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.698+0000 7fc6d0b9a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6cc06fb30 0x7fc6cc0cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:54588/0 (socket says 172.21.15.156:54588) 2023-11-28T13:11:52.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.698+0000 7fc6d0b9a700 1 -- 172.21.15.156:0/3225502063 learned_addr learned my addr 172.21.15.156:0/3225502063 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:52.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.698+0000 7fc6d139b700 1 --2- 172.21.15.156:0/3225502063 >> v2:172.21.15.186:3300/0 conn(0x7fc6cc158ce0 0x7fc6cc15d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:52.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.698+0000 7fc6d0b9a700 1 -- 172.21.15.156:0/3225502063 >> v2:172.21.15.186:3300/0 conn(0x7fc6cc158ce0 msgr2=0x7fc6cc15d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:52.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.698+0000 7fc6d0b9a700 1 --2- 172.21.15.156:0/3225502063 >> v2:172.21.15.186:3300/0 conn(0x7fc6cc158ce0 0x7fc6cc15d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:52.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.698+0000 7fc6d0b9a700 1 -- 172.21.15.156:0/3225502063 >> v1:172.21.15.186:6789/0 conn(0x7fc6cc0ce2e0 legacy=0x7fc6cc06c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:52.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.698+0000 7fc6d0b9a700 1 -- 172.21.15.156:0/3225502063 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc1590b0 con 0x7fc6cc06fb30 2023-11-28T13:11:52.700 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.699+0000 7fc6d0b9a700 1 --2- 172.21.15.156:0/3225502063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6cc06fb30 0x7fc6cc0cdda0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc00aa90 tx=0x7fc6bc00ada0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fa5fcef766b0407 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:52.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.699+0000 7fc6c3fff700 1 -- 172.21.15.156:0/3225502063 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00cb70 con 0x7fc6cc06fb30 2023-11-28T13:11:52.701 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.699+0000 7fc6c3fff700 1 -- 172.21.15.156:0/3225502063 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6bc00ccd0 con 0x7fc6cc06fb30 2023-11-28T13:11:52.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.699+0000 7fc6c3fff700 1 -- 172.21.15.156:0/3225502063 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc018620 con 0x7fc6cc06fb30 2023-11-28T13:11:52.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.699+0000 7fc6d35ff700 1 -- 172.21.15.156:0/3225502063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6cc06fb30 msgr2=0x7fc6cc0cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:52.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.699+0000 7fc6d35ff700 1 --2- 172.21.15.156:0/3225502063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6cc06fb30 0x7fc6cc0cdda0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc00aa90 tx=0x7fc6bc00ada0 comp rx=0 tx=0).stop 2023-11-28T13:11:52.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.700+0000 7fc6d35ff700 1 -- 172.21.15.156:0/3225502063 shutdown_connections 2023-11-28T13:11:52.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.700+0000 7fc6d35ff700 1 --2- 172.21.15.156:0/3225502063 >> v2:172.21.15.186:3300/0 conn(0x7fc6cc158ce0 0x7fc6cc15d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:52.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.700+0000 7fc6d35ff700 1 --2- 172.21.15.156:0/3225502063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6cc06fb30 0x7fc6cc0cdda0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:52.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.700+0000 7fc6d35ff700 1 -- 172.21.15.156:0/3225502063 >> 172.21.15.156:0/3225502063 conn(0x7fc6cc0cb670 msgr2=0x7fc6cc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:52.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.700+0000 7fc6d35ff700 1 -- 172.21.15.156:0/3225502063 shutdown_connections 2023-11-28T13:11:52.702 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.700+0000 7fc6d35ff700 1 -- 172.21.15.156:0/3225502063 wait complete. 2023-11-28T13:11:52.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.701+0000 7fc6d35ff700 1 Processor -- start 2023-11-28T13:11:52.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.702+0000 7fc6d35ff700 1 -- start start 2023-11-28T13:11:52.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.702+0000 7fc6d35ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6cc06fb30 0x7fc6cc147080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:52.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.703+0000 7fc6d35ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6cc0ce2e0 0x7fc6cc0b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:52.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.703+0000 7fc6d35ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6cc158ce0 0x7fc6cc0c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:52.705 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.703+0000 7fc6d35ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc1475c0 con 0x7fc6cc06fb30 2023-11-28T13:11:52.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.703+0000 7fc6d35ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc0c5d50 con 0x7fc6cc0ce2e0 2023-11-28T13:11:52.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.703+0000 7fc6d35ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc1592a0 con 0x7fc6cc158ce0 2023-11-28T13:11:52.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.703+0000 7fc6d139b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6cc06fb30 0x7fc6cc147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:52.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.703+0000 7fc6d139b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6cc06fb30 0x7fc6cc147080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50110/0 (socket says 172.21.15.156:50110) 2023-11-28T13:11:52.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.703+0000 7fc6d139b700 1 -- 172.21.15.156:0/1379526197 learned_addr learned my addr 172.21.15.156:0/1379526197 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:52.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.703+0000 7fc6d0b9a700 1 --2- 172.21.15.156:0/1379526197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6cc0ce2e0 0x7fc6cc0b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:52.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.703+0000 7fc6d1b9c700 1 --2- 172.21.15.156:0/1379526197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6cc158ce0 0x7fc6cc0c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:52.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.703+0000 7fc6d139b700 1 -- 172.21.15.156:0/1379526197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6cc158ce0 msgr2=0x7fc6cc0c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:52.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.703+0000 7fc6d139b700 1 --2- 172.21.15.156:0/1379526197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6cc158ce0 0x7fc6cc0c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:52.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.703+0000 7fc6d139b700 1 -- 172.21.15.156:0/1379526197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6cc0ce2e0 msgr2=0x7fc6cc0b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:52.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.703+0000 7fc6d139b700 1 --2- 172.21.15.156:0/1379526197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6cc0ce2e0 0x7fc6cc0b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:52.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.703+0000 7fc6d139b700 1 -- 172.21.15.156:0/1379526197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc00a6f0 con 0x7fc6cc06fb30 2023-11-28T13:11:52.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.703+0000 7fc6d0b9a700 1 --2- 172.21.15.156:0/1379526197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6cc0ce2e0 0x7fc6cc0b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:11:52.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.704+0000 7fc6d139b700 1 --2- 172.21.15.156:0/1379526197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6cc06fb30 0x7fc6cc147080 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800e870 tx=0x7fc6c800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:52.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.704+0000 7fc6c1ffb700 1 -- 172.21.15.156:0/1379526197 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8009590 con 0x7fc6cc06fb30 2023-11-28T13:11:52.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.704+0000 7fc6c1ffb700 1 -- 172.21.15.156:0/1379526197 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c800ee70 con 0x7fc6cc06fb30 2023-11-28T13:11:52.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.704+0000 7fc6d35ff700 1 -- 172.21.15.156:0/1379526197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6cc0cb3b0 con 0x7fc6cc06fb30 2023-11-28T13:11:52.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.704+0000 7fc6c1ffb700 1 -- 172.21.15.156:0/1379526197 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c800ca50 con 0x7fc6cc06fb30 2023-11-28T13:11:52.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.704+0000 7fc6d35ff700 1 -- 172.21.15.156:0/1379526197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6cc0cb0d0 con 0x7fc6cc06fb30 2023-11-28T13:11:52.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.705+0000 7fc6d35ff700 1 -- 172.21.15.156:0/1379526197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6cc06b8c0 con 0x7fc6cc06fb30 2023-11-28T13:11:52.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.706+0000 7fc6c1ffb700 1 -- 172.21.15.156:0/1379526197 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc6c8019030 con 0x7fc6cc06fb30 2023-11-28T13:11:52.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.707+0000 7fc6c1ffb700 1 --2- 172.21.15.156:0/1379526197 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6b00682f0 0x7fc6b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:52.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.707+0000 7fc6c1ffb700 1 -- 172.21.15.156:0/1379526197 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fc6c8085ca0 con 0x7fc6cc06fb30 2023-11-28T13:11:52.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.707+0000 7fc6d0b9a700 1 --2- 172.21.15.156:0/1379526197 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6b00682f0 0x7fc6b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:52.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.708+0000 7fc6d0b9a700 1 --2- 172.21.15.156:0/1379526197 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6b00682f0 0x7fc6b006a7a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc00ade0 tx=0x7fc6bc01a040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:52.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.711+0000 7fc6c1ffb700 1 -- 172.21.15.156:0/1379526197 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc6c8052380 con 0x7fc6cc06fb30 2023-11-28T13:11:52.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.943+0000 7fc6d35ff700 1 -- 172.21.15.156:0/1379526197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc6cc0c68d0 con 0x7fc6cc06fb30 2023-11-28T13:11:52.945 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.943+0000 7fc6c1ffb700 1 -- 172.21.15.156:0/1379526197 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fc6c80559a0 con 0x7fc6cc06fb30 2023-11-28T13:11:52.945 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:11:52.945 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:11:52.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.946+0000 7fc6d35ff700 1 -- 172.21.15.156:0/1379526197 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6b00682f0 msgr2=0x7fc6b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:52.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.946+0000 7fc6d35ff700 1 --2- 172.21.15.156:0/1379526197 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6b00682f0 0x7fc6b006a7a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc00ade0 tx=0x7fc6bc01a040 comp rx=0 tx=0).stop 2023-11-28T13:11:52.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.946+0000 7fc6d35ff700 1 -- 172.21.15.156:0/1379526197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6cc06fb30 msgr2=0x7fc6cc147080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:52.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.946+0000 7fc6d35ff700 1 --2- 172.21.15.156:0/1379526197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6cc06fb30 0x7fc6cc147080 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800e870 tx=0x7fc6c800ec30 comp rx=0 tx=0).stop 2023-11-28T13:11:52.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.946+0000 7fc6d35ff700 1 -- 172.21.15.156:0/1379526197 shutdown_connections 2023-11-28T13:11:52.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.946+0000 7fc6d35ff700 1 --2- 172.21.15.156:0/1379526197 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6b00682f0 0x7fc6b006a7a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:52.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.946+0000 7fc6d35ff700 1 --2- 172.21.15.156:0/1379526197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc6cc158ce0 0x7fc6cc0c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:52.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.946+0000 7fc6d35ff700 1 --2- 172.21.15.156:0/1379526197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6cc06fb30 0x7fc6cc147080 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:52.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.946+0000 7fc6d35ff700 1 --2- 172.21.15.156:0/1379526197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6cc0ce2e0 0x7fc6cc0b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:52.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.946+0000 7fc6d35ff700 1 -- 172.21.15.156:0/1379526197 >> 172.21.15.156:0/1379526197 conn(0x7fc6cc0cb670 msgr2=0x7fc6cc155bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:52.951 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.946+0000 7fc6d35ff700 1 -- 172.21.15.156:0/1379526197 shutdown_connections 2023-11-28T13:11:52.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:52.946+0000 7fc6d35ff700 1 -- 172.21.15.156:0/1379526197 wait complete. 2023-11-28T13:11:52.966 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:11:57.967 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:11:58.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.172+0000 7f9109bff700 1 Processor -- start 2023-11-28T13:11:58.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.172+0000 7f9109bff700 1 -- start start 2023-11-28T13:11:58.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.173+0000 7f9109bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9104161340 0x7f91040d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:58.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.173+0000 7f9109bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f91040d2320 0x7f91040cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:58.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.173+0000 7f9109bff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91040d1950 con 0x7f910415c9b0 2023-11-28T13:11:58.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.173+0000 7f9109bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91040b5800 con 0x7f91040d2320 2023-11-28T13:11:58.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.173+0000 7f9109bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91040b5300 con 0x7f9104161340 2023-11-28T13:11:58.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.174+0000 7f9103fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f91040d2320 0x7f91040cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:58.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.174+0000 7f9103fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f91040d2320 0x7f91040cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:40558/0 (socket says 172.21.15.156:40558) 2023-11-28T13:11:58.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.174+0000 7f9103fff700 1 -- 172.21.15.156:0/3576431418 learned_addr learned my addr 172.21.15.156:0/3576431418 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:58.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.174+0000 7f9102ffd700 1 --2- 172.21.15.156:0/3576431418 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9104161340 0x7f91040d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:58.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.174+0000 7f9103fff700 1 -- 172.21.15.156:0/3576431418 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9104161340 msgr2=0x7f91040d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:11:58.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.174+0000 7f9103fff700 1 --2- 172.21.15.156:0/3576431418 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9104161340 0x7f91040d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:58.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.174+0000 7f9103fff700 1 -- 172.21.15.156:0/3576431418 >> v1:172.21.15.186:6789/0 conn(0x7f910415c9b0 legacy=0x7f91040b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:58.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.174+0000 7f9103fff700 1 -- 172.21.15.156:0/3576431418 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91040b5520 con 0x7f91040d2320 2023-11-28T13:11:58.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.174+0000 7f9103fff700 1 --2- 172.21.15.156:0/3576431418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040d2320 0x7f91040cd7c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009e80 tx=0x7f90fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e916bd4403e782 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:58.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.175+0000 7f91027fc700 1 -- 172.21.15.156:0/3576431418 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc00e070 con 0x7f91040d2320 2023-11-28T13:11:58.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.175+0000 7f91027fc700 1 -- 172.21.15.156:0/3576431418 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90fc00b8d0 con 0x7f91040d2320 2023-11-28T13:11:58.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.175+0000 7f91027fc700 1 -- 172.21.15.156:0/3576431418 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc00bc80 con 0x7f91040d2320 2023-11-28T13:11:58.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.175+0000 7f9109bff700 1 -- 172.21.15.156:0/3576431418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040d2320 msgr2=0x7f91040cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:58.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.175+0000 7f9109bff700 1 --2- 172.21.15.156:0/3576431418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040d2320 0x7f91040cd7c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009e80 tx=0x7f90fc00b330 comp rx=0 tx=0).stop 2023-11-28T13:11:58.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.175+0000 7f9109bff700 1 -- 172.21.15.156:0/3576431418 shutdown_connections 2023-11-28T13:11:58.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.175+0000 7f9109bff700 1 --2- 172.21.15.156:0/3576431418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040d2320 0x7f91040cd7c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:58.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.175+0000 7f9109bff700 1 --2- 172.21.15.156:0/3576431418 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9104161340 0x7f91040d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:58.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.175+0000 7f9109bff700 1 -- 172.21.15.156:0/3576431418 >> 172.21.15.156:0/3576431418 conn(0x7f9104072600 msgr2=0x7f91040cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:58.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.175+0000 7f9109bff700 1 -- 172.21.15.156:0/3576431418 shutdown_connections 2023-11-28T13:11:58.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.175+0000 7f9109bff700 1 -- 172.21.15.156:0/3576431418 wait complete. 2023-11-28T13:11:58.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.177+0000 7f9109bff700 1 Processor -- start 2023-11-28T13:11:58.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.177+0000 7f9109bff700 1 -- start start 2023-11-28T13:11:58.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.178+0000 7f9109bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040d2320 0x7f9104150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:58.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.178+0000 7f9109bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f910415c9b0 0x7f910414fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:58.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.178+0000 7f9109bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9104161340 0x7f91041e7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:58.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.178+0000 7f9109bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91041e7e80 con 0x7f91040d2320 2023-11-28T13:11:58.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.178+0000 7f9109bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91040c85b0 con 0x7f9104161340 2023-11-28T13:11:58.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.178+0000 7f9109bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91040c8890 con 0x7f910415c9b0 2023-11-28T13:11:58.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.178+0000 7f91037fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040d2320 0x7f9104150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:58.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.178+0000 7f91037fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040d2320 0x7f9104150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:40568/0 (socket says 172.21.15.156:40568) 2023-11-28T13:11:58.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.178+0000 7f91037fe700 1 -- 172.21.15.156:0/2061046641 learned_addr learned my addr 172.21.15.156:0/2061046641 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:11:58.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.178+0000 7f9102ffd700 1 --2- 172.21.15.156:0/2061046641 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f910415c9b0 0x7f910414fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:58.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.178+0000 7f9103fff700 1 --2- 172.21.15.156:0/2061046641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9104161340 0x7f91041e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:58.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.179+0000 7f91037fe700 1 -- 172.21.15.156:0/2061046641 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f910415c9b0 msgr2=0x7f910414fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:58.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.179+0000 7f91037fe700 1 --2- 172.21.15.156:0/2061046641 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f910415c9b0 0x7f910414fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:58.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.179+0000 7f91037fe700 1 -- 172.21.15.156:0/2061046641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9104161340 msgr2=0x7f91041e7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:58.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.179+0000 7f91037fe700 1 --2- 172.21.15.156:0/2061046641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9104161340 0x7f91041e7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:58.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.179+0000 7f91037fe700 1 -- 172.21.15.156:0/2061046641 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90fc009b30 con 0x7f91040d2320 2023-11-28T13:11:58.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.179+0000 7f91037fe700 1 --2- 172.21.15.156:0/2061046641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040d2320 0x7f9104150690 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f90f400c450 tx=0x7f90f400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:58.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.179+0000 7f90ebfff700 1 -- 172.21.15.156:0/2061046641 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f40052d0 con 0x7f91040d2320 2023-11-28T13:11:58.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.179+0000 7f90ebfff700 1 -- 172.21.15.156:0/2061046641 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f4005430 con 0x7f91040d2320 2023-11-28T13:11:58.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.179+0000 7f9109bff700 1 -- 172.21.15.156:0/2061046641 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91040d15e0 con 0x7f91040d2320 2023-11-28T13:11:58.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.179+0000 7f90ebfff700 1 -- 172.21.15.156:0/2061046641 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f40038c0 con 0x7f91040d2320 2023-11-28T13:11:58.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.179+0000 7f9109bff700 1 -- 172.21.15.156:0/2061046641 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91040d1300 con 0x7f91040d2320 2023-11-28T13:11:58.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.180+0000 7f9109bff700 1 -- 172.21.15.156:0/2061046641 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910415cd80 con 0x7f91040d2320 2023-11-28T13:11:58.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.181+0000 7f90ebfff700 1 -- 172.21.15.156:0/2061046641 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f90f4019030 con 0x7f91040d2320 2023-11-28T13:11:58.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.182+0000 7f90ebfff700 1 --2- 172.21.15.156:0/2061046641 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f90f00682f0 0x7f90f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:11:58.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.183+0000 7f90ebfff700 1 -- 172.21.15.156:0/2061046641 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f90f4084fe0 con 0x7f91040d2320 2023-11-28T13:11:58.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.183+0000 7f9102ffd700 1 --2- 172.21.15.156:0/2061046641 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f90f00682f0 0x7f90f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:11:58.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.184+0000 7f9102ffd700 1 --2- 172.21.15.156:0/2061046641 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f90f00682f0 0x7f90f006a7a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f91041f5ed0 tx=0x7f90f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:11:58.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.187+0000 7f90ebfff700 1 -- 172.21.15.156:0/2061046641 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f90f40516e0 con 0x7f91040d2320 2023-11-28T13:11:58.419 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.418+0000 7f9109bff700 1 -- 172.21.15.156:0/2061046641 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f91040745c0 con 0x7f91040d2320 2023-11-28T13:11:58.420 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.418+0000 7f90ebfff700 1 -- 172.21.15.156:0/2061046641 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f90f4054d00 con 0x7f91040d2320 2023-11-28T13:11:58.420 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:11:58.420 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:11:58.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.421+0000 7f9109bff700 1 -- 172.21.15.156:0/2061046641 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f90f00682f0 msgr2=0x7f90f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:58.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.421+0000 7f9109bff700 1 --2- 172.21.15.156:0/2061046641 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f90f00682f0 0x7f90f006a7a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f91041f5ed0 tx=0x7f90f8008040 comp rx=0 tx=0).stop 2023-11-28T13:11:58.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.421+0000 7f9109bff700 1 -- 172.21.15.156:0/2061046641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040d2320 msgr2=0x7f9104150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:11:58.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.421+0000 7f9109bff700 1 --2- 172.21.15.156:0/2061046641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040d2320 0x7f9104150690 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f90f400c450 tx=0x7f90f400c810 comp rx=0 tx=0).stop 2023-11-28T13:11:58.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.421+0000 7f9109bff700 1 -- 172.21.15.156:0/2061046641 shutdown_connections 2023-11-28T13:11:58.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.421+0000 7f9109bff700 1 --2- 172.21.15.156:0/2061046641 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f90f00682f0 0x7f90f006a7a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:58.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.421+0000 7f9109bff700 1 --2- 172.21.15.156:0/2061046641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9104161340 0x7f91041e7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:58.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.421+0000 7f9109bff700 1 --2- 172.21.15.156:0/2061046641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040d2320 0x7f9104150690 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:58.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.421+0000 7f9109bff700 1 --2- 172.21.15.156:0/2061046641 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f910415c9b0 0x7f910414fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:11:58.423 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.421+0000 7f9109bff700 1 -- 172.21.15.156:0/2061046641 >> 172.21.15.156:0/2061046641 conn(0x7f9104072600 msgr2=0x7f91040b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:11:58.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.421+0000 7f9109bff700 1 -- 172.21.15.156:0/2061046641 shutdown_connections 2023-11-28T13:11:58.424 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:11:58.421+0000 7f9109bff700 1 -- 172.21.15.156:0/2061046641 wait complete. 2023-11-28T13:11:58.441 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:12:03.442 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:12:03.633 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:12:03.638 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:12:03.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.670+0000 7ff19d3ff700 1 Processor -- start 2023-11-28T13:12:03.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.670+0000 7ff19d3ff700 1 -- start start 2023-11-28T13:12:03.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.671+0000 7ff19d3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff19815c9b0 0x7ff1980b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:03.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.671+0000 7ff19d3ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff1980cd7c0 0x7ff1980cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:03.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.671+0000 7ff19d3ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1980ce160 con 0x7ff198161340 2023-11-28T13:12:03.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.671+0000 7ff19d3ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1980b5800 con 0x7ff1980cd7c0 2023-11-28T13:12:03.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.671+0000 7ff19d3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1980d1950 con 0x7ff19815c9b0 2023-11-28T13:12:03.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.671+0000 7ff1967fc700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff198161340 0x7ff1980d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:43140/0 (socket says 172.21.15.156:43140) 2023-11-28T13:12:03.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.672+0000 7ff1967fc700 1 -- 172.21.15.156:0/1958143563 learned_addr learned my addr 172.21.15.156:0/1958143563 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:12:03.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.672+0000 7ff1977fe700 1 --2- 172.21.15.156:0/1958143563 >> v2:172.21.15.186:3300/0 conn(0x7ff1980cd7c0 0x7ff1980cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:03.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.672+0000 7ff196ffd700 1 --2- 172.21.15.156:0/1958143563 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff19815c9b0 0x7ff1980b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:03.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.672+0000 7ff195ffb700 1 -- 172.21.15.156:0/1958143563 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275794981 0 0) 0x7ff1980ce160 con 0x7ff198161340 2023-11-28T13:12:03.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.672+0000 7ff195ffb700 1 -- 172.21.15.156:0/1958143563 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1800035b0 con 0x7ff198161340 2023-11-28T13:12:03.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.672+0000 7ff196ffd700 1 -- 172.21.15.156:0/1958143563 >> v2:172.21.15.186:3300/0 conn(0x7ff1980cd7c0 msgr2=0x7ff1980cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.672+0000 7ff196ffd700 1 --2- 172.21.15.156:0/1958143563 >> v2:172.21.15.186:3300/0 conn(0x7ff1980cd7c0 0x7ff1980cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.672+0000 7ff196ffd700 1 -- 172.21.15.156:0/1958143563 >> v1:172.21.15.156:6789/0 conn(0x7ff198161340 legacy=0x7ff1980d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.672+0000 7ff196ffd700 1 -- 172.21.15.156:0/1958143563 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1980d24f0 con 0x7ff19815c9b0 2023-11-28T13:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.672+0000 7ff1977fe700 1 --2- 172.21.15.156:0/1958143563 >> v2:172.21.15.186:3300/0 conn(0x7ff1980cd7c0 0x7ff1980cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.672+0000 7ff196ffd700 1 --2- 172.21.15.156:0/1958143563 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff19815c9b0 0x7ff1980b4bf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff188009840 tx=0x7ff188009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c5ce2203a91643c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.673+0000 7ff195ffb700 1 -- 172.21.15.156:0/1958143563 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188011830 con 0x7ff19815c9b0 2023-11-28T13:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.673+0000 7ff195ffb700 1 -- 172.21.15.156:0/1958143563 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff188011990 con 0x7ff19815c9b0 2023-11-28T13:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.673+0000 7ff195ffb700 1 -- 172.21.15.156:0/1958143563 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188011d20 con 0x7ff19815c9b0 2023-11-28T13:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.673+0000 7ff19d3ff700 1 -- 172.21.15.156:0/1958143563 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff19815c9b0 msgr2=0x7ff1980b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.673+0000 7ff19d3ff700 1 --2- 172.21.15.156:0/1958143563 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff19815c9b0 0x7ff1980b4bf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff188009840 tx=0x7ff188009b50 comp rx=0 tx=0).stop 2023-11-28T13:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.673+0000 7ff19d3ff700 1 -- 172.21.15.156:0/1958143563 shutdown_connections 2023-11-28T13:12:03.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.673+0000 7ff19d3ff700 1 --2- 172.21.15.156:0/1958143563 >> v2:172.21.15.186:3300/0 conn(0x7ff1980cd7c0 0x7ff1980cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:03.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.673+0000 7ff19d3ff700 1 --2- 172.21.15.156:0/1958143563 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff19815c9b0 0x7ff1980b4bf0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:03.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.673+0000 7ff19d3ff700 1 -- 172.21.15.156:0/1958143563 >> 172.21.15.156:0/1958143563 conn(0x7ff198072600 msgr2=0x7ff1980cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:03.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.674+0000 7ff19d3ff700 1 -- 172.21.15.156:0/1958143563 shutdown_connections 2023-11-28T13:12:03.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.674+0000 7ff19d3ff700 1 -- 172.21.15.156:0/1958143563 wait complete. 2023-11-28T13:12:03.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.675+0000 7ff19d3ff700 1 Processor -- start 2023-11-28T13:12:03.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.675+0000 7ff19d3ff700 1 -- start start 2023-11-28T13:12:03.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.676+0000 7ff19d3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1980cd7c0 0x7ff1980c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:03.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.676+0000 7ff19d3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff19815c9b0 0x7ff1980cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:03.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.676+0000 7ff19d3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff198161340 0x7ff198157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.676+0000 7ff19d3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1980b8bc0 con 0x7ff198161340 2023-11-28T13:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.676+0000 7ff19d3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1980b7d10 con 0x7ff19815c9b0 2023-11-28T13:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.676+0000 7ff19d3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1980b7fb0 con 0x7ff1980cd7c0 2023-11-28T13:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.677+0000 7ff1977fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff198161340 0x7ff198157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.677+0000 7ff196ffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1980cd7c0 0x7ff1980c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.677+0000 7ff1977fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff198161340 0x7ff198157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:40584/0 (socket says 172.21.15.156:40584) 2023-11-28T13:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.677+0000 7ff1977fe700 1 -- 172.21.15.156:0/1198144230 learned_addr learned my addr 172.21.15.156:0/1198144230 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.677+0000 7ff1967fc700 1 --2- 172.21.15.156:0/1198144230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff19815c9b0 0x7ff1980cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:03.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.677+0000 7ff196ffd700 1 -- 172.21.15.156:0/1198144230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff19815c9b0 msgr2=0x7ff1980cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.677+0000 7ff196ffd700 1 --2- 172.21.15.156:0/1198144230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff19815c9b0 0x7ff1980cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.677+0000 7ff196ffd700 1 -- 172.21.15.156:0/1198144230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff198161340 msgr2=0x7ff198157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.677+0000 7ff196ffd700 1 --2- 172.21.15.156:0/1198144230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff198161340 0x7ff198157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.677+0000 7ff196ffd700 1 -- 172.21.15.156:0/1198144230 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff190009a70 con 0x7ff1980cd7c0 2023-11-28T13:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.677+0000 7ff1977fe700 1 --2- 172.21.15.156:0/1198144230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff198161340 0x7ff198157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.677+0000 7ff196ffd700 1 --2- 172.21.15.156:0/1198144230 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1980cd7c0 0x7ff1980c7d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff188016000 tx=0x7ff188004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.678+0000 7ff17f7fe700 1 -- 172.21.15.156:0/1198144230 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188004590 con 0x7ff1980cd7c0 2023-11-28T13:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.678+0000 7ff17f7fe700 1 -- 172.21.15.156:0/1198144230 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18801e070 con 0x7ff1980cd7c0 2023-11-28T13:12:03.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.678+0000 7ff19d3ff700 1 -- 172.21.15.156:0/1198144230 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1880094a0 con 0x7ff1980cd7c0 2023-11-28T13:12:03.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.678+0000 7ff17f7fe700 1 -- 172.21.15.156:0/1198144230 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1880038e0 con 0x7ff1980cd7c0 2023-11-28T13:12:03.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.678+0000 7ff19d3ff700 1 -- 172.21.15.156:0/1198144230 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1980d1300 con 0x7ff1980cd7c0 2023-11-28T13:12:03.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.679+0000 7ff19d3ff700 1 -- 172.21.15.156:0/1198144230 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1980b5520 con 0x7ff1980cd7c0 2023-11-28T13:12:03.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.680+0000 7ff17f7fe700 1 -- 172.21.15.156:0/1198144230 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff18800f050 con 0x7ff1980cd7c0 2023-11-28T13:12:03.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.681+0000 7ff17f7fe700 1 --2- 172.21.15.156:0/1198144230 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff1840682f0 0x7ff18406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:03.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.681+0000 7ff17f7fe700 1 -- 172.21.15.156:0/1198144230 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ff188086cf0 con 0x7ff1980cd7c0 2023-11-28T13:12:03.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.681+0000 7ff1967fc700 1 --2- 172.21.15.156:0/1198144230 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff1840682f0 0x7ff18406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:03.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.682+0000 7ff1967fc700 1 --2- 172.21.15.156:0/1198144230 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff1840682f0 0x7ff18406a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff1980b5450 tx=0x7ff18c007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:03.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.691+0000 7ff17f7fe700 1 -- 172.21.15.156:0/1198144230 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff188053440 con 0x7ff1980cd7c0 2023-11-28T13:12:03.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.931+0000 7ff19d3ff700 1 -- 172.21.15.156:0/1198144230 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff1980d15e0 con 0x7ff1980cd7c0 2023-11-28T13:12:03.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.932+0000 7ff17f7fe700 1 -- 172.21.15.156:0/1198144230 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7ff1880150c0 con 0x7ff1980cd7c0 2023-11-28T13:12:03.934 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:12:03.934 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:12:03.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.934+0000 7ff19d3ff700 1 -- 172.21.15.156:0/1198144230 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff1840682f0 msgr2=0x7ff18406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:03.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.934+0000 7ff19d3ff700 1 --2- 172.21.15.156:0/1198144230 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff1840682f0 0x7ff18406a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff1980b5450 tx=0x7ff18c007dc0 comp rx=0 tx=0).stop 2023-11-28T13:12:03.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.934+0000 7ff19d3ff700 1 -- 172.21.15.156:0/1198144230 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1980cd7c0 msgr2=0x7ff1980c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:03.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.934+0000 7ff19d3ff700 1 --2- 172.21.15.156:0/1198144230 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1980cd7c0 0x7ff1980c7d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff188016000 tx=0x7ff188004750 comp rx=0 tx=0).stop 2023-11-28T13:12:03.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.934+0000 7ff19d3ff700 1 -- 172.21.15.156:0/1198144230 shutdown_connections 2023-11-28T13:12:03.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.934+0000 7ff19d3ff700 1 --2- 172.21.15.156:0/1198144230 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff1840682f0 0x7ff18406a7a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:03.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.934+0000 7ff19d3ff700 1 --2- 172.21.15.156:0/1198144230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff198161340 0x7ff198157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:03.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.934+0000 7ff19d3ff700 1 --2- 172.21.15.156:0/1198144230 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1980cd7c0 0x7ff1980c7d40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:03.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.935+0000 7ff19d3ff700 1 --2- 172.21.15.156:0/1198144230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff19815c9b0 0x7ff1980cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:03.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.935+0000 7ff19d3ff700 1 -- 172.21.15.156:0/1198144230 >> 172.21.15.156:0/1198144230 conn(0x7ff198072600 msgr2=0x7ff19815a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:03.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.935+0000 7ff19d3ff700 1 -- 172.21.15.156:0/1198144230 shutdown_connections 2023-11-28T13:12:03.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:03.935+0000 7ff19d3ff700 1 -- 172.21.15.156:0/1198144230 wait complete. 2023-11-28T13:12:03.956 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:12:08.957 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:12:09.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.161+0000 7f280e9ff700 1 Processor -- start 2023-11-28T13:12:09.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.162+0000 7f280e9ff700 1 -- start start 2023-11-28T13:12:09.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.163+0000 7f280e9ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f280815c9b0 0x7f28080b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:09.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.163+0000 7f280e9ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f28080cd7c0 0x7f28080cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:09.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.163+0000 7f280e9ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28080d1950 con 0x7f2808161340 2023-11-28T13:12:09.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.163+0000 7f280e9ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28080b5800 con 0x7f280815c9b0 2023-11-28T13:12:09.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.163+0000 7f280e9ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28080ce140 con 0x7f28080cd7c0 2023-11-28T13:12:09.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.163+0000 7f2807fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f280815c9b0 0x7f28080b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:09.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.163+0000 7f2807fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f280815c9b0 0x7f28080b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39384/0 (socket says 172.21.15.156:39384) 2023-11-28T13:12:09.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.163+0000 7f2807fff700 1 -- 172.21.15.156:0/2072625912 learned_addr learned my addr 172.21.15.156:0/2072625912 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:12:09.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.163+0000 7f280cf9c700 1 --2- 172.21.15.156:0/2072625912 >> v2:172.21.15.186:3300/0 conn(0x7f28080cd7c0 0x7f28080cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:09.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.164+0000 7f2807fff700 1 -- 172.21.15.156:0/2072625912 >> v2:172.21.15.186:3300/0 conn(0x7f28080cd7c0 msgr2=0x7f28080cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:12:09.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.164+0000 7f2807fff700 1 --2- 172.21.15.156:0/2072625912 >> v2:172.21.15.186:3300/0 conn(0x7f28080cd7c0 0x7f28080cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:09.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.164+0000 7f2807fff700 1 -- 172.21.15.156:0/2072625912 >> v1:172.21.15.186:6789/0 conn(0x7f2808161340 legacy=0x7f28080d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:09.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.164+0000 7f2807fff700 1 -- 172.21.15.156:0/2072625912 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28080d24f0 con 0x7f280815c9b0 2023-11-28T13:12:09.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.164+0000 7f2807fff700 1 --2- 172.21.15.156:0/2072625912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f280815c9b0 0x7f28080b4bf0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f280806c170 tx=0x7f27f8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccd65a26e64fb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:09.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.164+0000 7f2806ffd700 1 -- 172.21.15.156:0/2072625912 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f800b870 con 0x7f280815c9b0 2023-11-28T13:12:09.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.164+0000 7f2806ffd700 1 -- 172.21.15.156:0/2072625912 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f800b9d0 con 0x7f280815c9b0 2023-11-28T13:12:09.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.164+0000 7f2806ffd700 1 -- 172.21.15.156:0/2072625912 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f800bd60 con 0x7f280815c9b0 2023-11-28T13:12:09.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.165+0000 7f280e9ff700 1 -- 172.21.15.156:0/2072625912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f280815c9b0 msgr2=0x7f28080b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:09.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.165+0000 7f280e9ff700 1 --2- 172.21.15.156:0/2072625912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f280815c9b0 0x7f28080b4bf0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f280806c170 tx=0x7f27f8009bd0 comp rx=0 tx=0).stop 2023-11-28T13:12:09.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.165+0000 7f280e9ff700 1 -- 172.21.15.156:0/2072625912 shutdown_connections 2023-11-28T13:12:09.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.165+0000 7f280e9ff700 1 --2- 172.21.15.156:0/2072625912 >> v2:172.21.15.186:3300/0 conn(0x7f28080cd7c0 0x7f28080cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:09.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.165+0000 7f280e9ff700 1 --2- 172.21.15.156:0/2072625912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f280815c9b0 0x7f28080b4bf0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:09.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.165+0000 7f280e9ff700 1 -- 172.21.15.156:0/2072625912 >> 172.21.15.156:0/2072625912 conn(0x7f2808072600 msgr2=0x7f28080cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:09.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.165+0000 7f280e9ff700 1 -- 172.21.15.156:0/2072625912 shutdown_connections 2023-11-28T13:12:09.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.165+0000 7f280e9ff700 1 -- 172.21.15.156:0/2072625912 wait complete. 2023-11-28T13:12:09.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.167+0000 7f280e9ff700 1 Processor -- start 2023-11-28T13:12:09.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.167+0000 7f280e9ff700 1 -- start start 2023-11-28T13:12:09.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.168+0000 7f280e9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28080cd7c0 0x7f28080cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:09.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.168+0000 7f280e9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280815c9b0 0x7f28080c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:09.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.168+0000 7f280e9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2808161340 0x7f2808154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:09.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.168+0000 7f280e9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28080c7bf0 con 0x7f2808161340 2023-11-28T13:12:09.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.168+0000 7f280e9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28080c7e70 con 0x7f280815c9b0 2023-11-28T13:12:09.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.168+0000 7f280e9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28080c8150 con 0x7f28080cd7c0 2023-11-28T13:12:09.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.168+0000 7f280cf9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2808161340 0x7f2808154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:09.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.168+0000 7f280cf9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2808161340 0x7f2808154630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39398/0 (socket says 172.21.15.156:39398) 2023-11-28T13:12:09.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.168+0000 7f280cf9c700 1 -- 172.21.15.156:0/3254133383 learned_addr learned my addr 172.21.15.156:0/3254133383 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:12:09.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.168+0000 7f2807fff700 1 --2- 172.21.15.156:0/3254133383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28080cd7c0 0x7f28080cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:09.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.168+0000 7f28077fe700 1 --2- 172.21.15.156:0/3254133383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280815c9b0 0x7f28080c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:09.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.168+0000 7f280cf9c700 1 -- 172.21.15.156:0/3254133383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28080cd7c0 msgr2=0x7f28080cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:09.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.168+0000 7f280cf9c700 1 --2- 172.21.15.156:0/3254133383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28080cd7c0 0x7f28080cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:09.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.168+0000 7f280cf9c700 1 -- 172.21.15.156:0/3254133383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280815c9b0 msgr2=0x7f28080c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:09.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.168+0000 7f280cf9c700 1 --2- 172.21.15.156:0/3254133383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280815c9b0 0x7f28080c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:09.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.168+0000 7f280cf9c700 1 -- 172.21.15.156:0/3254133383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f8009550 con 0x7f2808161340 2023-11-28T13:12:09.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.168+0000 7f28077fe700 1 --2- 172.21.15.156:0/3254133383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280815c9b0 0x7f28080c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:12:09.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.169+0000 7f280cf9c700 1 --2- 172.21.15.156:0/3254133383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2808161340 0x7f2808154630 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2800007e60 tx=0x7f280000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:09.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.169+0000 7f2804ff9700 1 -- 172.21.15.156:0/3254133383 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280000ede0 con 0x7f2808161340 2023-11-28T13:12:09.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.169+0000 7f2804ff9700 1 -- 172.21.15.156:0/3254133383 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2800014420 con 0x7f2808161340 2023-11-28T13:12:09.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.169+0000 7f280e9ff700 1 -- 172.21.15.156:0/3254133383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28080d1300 con 0x7f2808161340 2023-11-28T13:12:09.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.169+0000 7f2804ff9700 1 -- 172.21.15.156:0/3254133383 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28000147d0 con 0x7f2808161340 2023-11-28T13:12:09.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.169+0000 7f280e9ff700 1 -- 172.21.15.156:0/3254133383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28080b5520 con 0x7f2808161340 2023-11-28T13:12:09.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.170+0000 7f280e9ff700 1 -- 172.21.15.156:0/3254133383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28080d15e0 con 0x7f2808161340 2023-11-28T13:12:09.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.171+0000 7f2804ff9700 1 -- 172.21.15.156:0/3254133383 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2800012050 con 0x7f2808161340 2023-11-28T13:12:09.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.172+0000 7f2804ff9700 1 --2- 172.21.15.156:0/3254133383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f27f40682f0 0x7f27f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:09.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.172+0000 7f2804ff9700 1 -- 172.21.15.156:0/3254133383 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f28000863b0 con 0x7f2808161340 2023-11-28T13:12:09.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.172+0000 7f2807fff700 1 --2- 172.21.15.156:0/3254133383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f27f40682f0 0x7f27f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:09.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.173+0000 7f2807fff700 1 --2- 172.21.15.156:0/3254133383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f27f40682f0 0x7f27f406a7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f27f8014040 tx=0x7f27f8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:09.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.176+0000 7f2804ff9700 1 -- 172.21.15.156:0/3254133383 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2800052a30 con 0x7f2808161340 2023-11-28T13:12:09.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.410+0000 7f280e9ff700 1 -- 172.21.15.156:0/3254133383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2808147d70 con 0x7f2808161340 2023-11-28T13:12:09.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.410+0000 7f2804ff9700 1 -- 172.21.15.156:0/3254133383 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f2808147d70 con 0x7f2808161340 2023-11-28T13:12:09.412 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:12:09.412 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:12:09.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.413+0000 7f280e9ff700 1 -- 172.21.15.156:0/3254133383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f27f40682f0 msgr2=0x7f27f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:09.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.413+0000 7f280e9ff700 1 --2- 172.21.15.156:0/3254133383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f27f40682f0 0x7f27f406a7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f27f8014040 tx=0x7f27f8008030 comp rx=0 tx=0).stop 2023-11-28T13:12:09.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.413+0000 7f280e9ff700 1 -- 172.21.15.156:0/3254133383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2808161340 msgr2=0x7f2808154630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:09.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.413+0000 7f280e9ff700 1 --2- 172.21.15.156:0/3254133383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2808161340 0x7f2808154630 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2800007e60 tx=0x7f280000e3c0 comp rx=0 tx=0).stop 2023-11-28T13:12:09.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.413+0000 7f280e9ff700 1 -- 172.21.15.156:0/3254133383 shutdown_connections 2023-11-28T13:12:09.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.413+0000 7f280e9ff700 1 --2- 172.21.15.156:0/3254133383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f27f40682f0 0x7f27f406a7a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:09.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.413+0000 7f280e9ff700 1 --2- 172.21.15.156:0/3254133383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2808161340 0x7f2808154630 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:09.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.413+0000 7f280e9ff700 1 --2- 172.21.15.156:0/3254133383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28080cd7c0 0x7f28080cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:09.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.413+0000 7f280e9ff700 1 --2- 172.21.15.156:0/3254133383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f280815c9b0 0x7f28080c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:09.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.413+0000 7f280e9ff700 1 -- 172.21.15.156:0/3254133383 >> 172.21.15.156:0/3254133383 conn(0x7f2808072600 msgr2=0x7f2808159e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:09.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.413+0000 7f280e9ff700 1 -- 172.21.15.156:0/3254133383 shutdown_connections 2023-11-28T13:12:09.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:09.413+0000 7f280e9ff700 1 -- 172.21.15.156:0/3254133383 wait complete. 2023-11-28T13:12:09.432 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:12:14.433 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:12:14.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.638+0000 7fb9c7fff700 1 Processor -- start 2023-11-28T13:12:14.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.639+0000 7fb9c7fff700 1 -- start start 2023-11-28T13:12:14.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.640+0000 7fb9c7fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb9c00cd7c0 0x7fb9c00cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:14.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.640+0000 7fb9c7fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c00d1950 con 0x7fb9c015c9b0 2023-11-28T13:12:14.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.640+0000 7fb9c7fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c00ce140 con 0x7fb9c0161340 2023-11-28T13:12:14.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.640+0000 7fb9c7fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c00b5800 con 0x7fb9c00cd7c0 2023-11-28T13:12:14.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.640+0000 7fb9c5d9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb9c015c9b0 0x7fb9c00b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:42104/0 (socket says 172.21.15.156:42104) 2023-11-28T13:12:14.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.640+0000 7fb9c5d9b700 1 -- 172.21.15.156:0/1892859680 learned_addr learned my addr 172.21.15.156:0/1892859680 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:12:14.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.640+0000 7fb9c659c700 1 --2- 172.21.15.156:0/1892859680 >> v2:172.21.15.156:3300/0 conn(0x7fb9c00cd7c0 0x7fb9c00cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:14.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.640+0000 7fb9c659c700 1 -- 172.21.15.156:0/1892859680 >> v1:172.21.15.186:6789/0 conn(0x7fb9c0161340 legacy=0x7fb9c00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:14.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.640+0000 7fb9c659c700 1 -- 172.21.15.156:0/1892859680 >> v1:172.21.15.156:6789/0 conn(0x7fb9c015c9b0 legacy=0x7fb9c00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:14.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.640+0000 7fb9c659c700 1 -- 172.21.15.156:0/1892859680 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c00d24f0 con 0x7fb9c00cd7c0 2023-11-28T13:12:14.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.641+0000 7fb9c659c700 1 --2- 172.21.15.156:0/1892859680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c00cd7c0 0x7fb9c00cdc00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009d90 tx=0x7fb9b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e387d2f0469f61 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:14.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.641+0000 7fb9c4d99700 1 -- 172.21.15.156:0/1892859680 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800e070 con 0x7fb9c00cd7c0 2023-11-28T13:12:14.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.641+0000 7fb9c4d99700 1 -- 172.21.15.156:0/1892859680 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b800bbd0 con 0x7fb9c00cd7c0 2023-11-28T13:12:14.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.641+0000 7fb9c4d99700 1 -- 172.21.15.156:0/1892859680 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8012430 con 0x7fb9c00cd7c0 2023-11-28T13:12:14.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.641+0000 7fb9c7fff700 1 -- 172.21.15.156:0/1892859680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c00cd7c0 msgr2=0x7fb9c00cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:14.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.641+0000 7fb9c7fff700 1 --2- 172.21.15.156:0/1892859680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c00cd7c0 0x7fb9c00cdc00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009d90 tx=0x7fb9b800b330 comp rx=0 tx=0).stop 2023-11-28T13:12:14.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.642+0000 7fb9c7fff700 1 -- 172.21.15.156:0/1892859680 shutdown_connections 2023-11-28T13:12:14.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.642+0000 7fb9c7fff700 1 --2- 172.21.15.156:0/1892859680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c00cd7c0 0x7fb9c00cdc00 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:14.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.642+0000 7fb9c7fff700 1 -- 172.21.15.156:0/1892859680 >> 172.21.15.156:0/1892859680 conn(0x7fb9c0072600 msgr2=0x7fb9c00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:14.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.642+0000 7fb9c7fff700 1 -- 172.21.15.156:0/1892859680 shutdown_connections 2023-11-28T13:12:14.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.642+0000 7fb9c7fff700 1 -- 172.21.15.156:0/1892859680 wait complete. 2023-11-28T13:12:14.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.643+0000 7fb9c7fff700 1 Processor -- start 2023-11-28T13:12:14.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.643+0000 7fb9c7fff700 1 -- start start 2023-11-28T13:12:14.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.644+0000 7fb9c7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9c00cd7c0 0x7fb9c0153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:14.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.644+0000 7fb9c7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c015c9b0 0x7fb9c015ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:14.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.644+0000 7fb9c7fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9c0161340 0x7fb9c015f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:14.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.644+0000 7fb9c7fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00c9fb0 con 0x7fb9c015c9b0 2023-11-28T13:12:14.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.644+0000 7fb9c7fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00cb100 con 0x7fb9c00cd7c0 2023-11-28T13:12:14.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.644+0000 7fb9c7fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00cb3e0 con 0x7fb9c0161340 2023-11-28T13:12:14.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.645+0000 7fb9c559a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c015c9b0 0x7fb9c015ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:14.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.645+0000 7fb9c559a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c015c9b0 0x7fb9c015ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39414/0 (socket says 172.21.15.156:39414) 2023-11-28T13:12:14.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.645+0000 7fb9c559a700 1 -- 172.21.15.156:0/235303065 learned_addr learned my addr 172.21.15.156:0/235303065 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:12:14.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.645+0000 7fb9c5d9b700 1 --2- 172.21.15.156:0/235303065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9c00cd7c0 0x7fb9c0153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:14.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.645+0000 7fb9c659c700 1 --2- 172.21.15.156:0/235303065 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9c0161340 0x7fb9c015f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:14.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.645+0000 7fb9c559a700 1 -- 172.21.15.156:0/235303065 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9c0161340 msgr2=0x7fb9c015f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:14.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.645+0000 7fb9c559a700 1 --2- 172.21.15.156:0/235303065 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9c0161340 0x7fb9c015f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:14.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.645+0000 7fb9c559a700 1 -- 172.21.15.156:0/235303065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9c00cd7c0 msgr2=0x7fb9c0153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:14.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.645+0000 7fb9c559a700 1 --2- 172.21.15.156:0/235303065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9c00cd7c0 0x7fb9c0153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:14.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.645+0000 7fb9c559a700 1 -- 172.21.15.156:0/235303065 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b8009a70 con 0x7fb9c015c9b0 2023-11-28T13:12:14.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.645+0000 7fb9c559a700 1 --2- 172.21.15.156:0/235303065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c015c9b0 0x7fb9c015ef40 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000c4e0 tx=0x7fb9b000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:14.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.645+0000 7fb9b67fc700 1 -- 172.21.15.156:0/235303065 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00052d0 con 0x7fb9c015c9b0 2023-11-28T13:12:14.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.646+0000 7fb9b67fc700 1 -- 172.21.15.156:0/235303065 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b0005430 con 0x7fb9c015c9b0 2023-11-28T13:12:14.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.646+0000 7fb9b67fc700 1 -- 172.21.15.156:0/235303065 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00038c0 con 0x7fb9c015c9b0 2023-11-28T13:12:14.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.646+0000 7fb9c7fff700 1 -- 172.21.15.156:0/235303065 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c00d1300 con 0x7fb9c015c9b0 2023-11-28T13:12:14.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.646+0000 7fb9c7fff700 1 -- 172.21.15.156:0/235303065 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c00b5520 con 0x7fb9c015c9b0 2023-11-28T13:12:14.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.647+0000 7fb9c7fff700 1 -- 172.21.15.156:0/235303065 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c00d15e0 con 0x7fb9c015c9b0 2023-11-28T13:12:14.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.648+0000 7fb9b67fc700 1 -- 172.21.15.156:0/235303065 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb9b0019030 con 0x7fb9c015c9b0 2023-11-28T13:12:14.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.649+0000 7fb9b67fc700 1 --2- 172.21.15.156:0/235303065 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb9a40682f0 0x7fb9a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:14.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.649+0000 7fb9b67fc700 1 -- 172.21.15.156:0/235303065 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fb9b0085070 con 0x7fb9c015c9b0 2023-11-28T13:12:14.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.649+0000 7fb9c5d9b700 1 --2- 172.21.15.156:0/235303065 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb9a40682f0 0x7fb9a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:14.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.650+0000 7fb9c5d9b700 1 --2- 172.21.15.156:0/235303065 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb9a40682f0 0x7fb9a406a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc005910 tx=0x7fb9bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:14.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.653+0000 7fb9b67fc700 1 -- 172.21.15.156:0/235303065 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb9b0051750 con 0x7fb9c015c9b0 2023-11-28T13:12:14.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.886+0000 7fb9c7fff700 1 -- 172.21.15.156:0/235303065 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb9c00745c0 con 0x7fb9c015c9b0 2023-11-28T13:12:14.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.886+0000 7fb9b67fc700 1 -- 172.21.15.156:0/235303065 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fb9b00170c0 con 0x7fb9c015c9b0 2023-11-28T13:12:14.888 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:12:14.888 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:12:14.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.889+0000 7fb9c7fff700 1 -- 172.21.15.156:0/235303065 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb9a40682f0 msgr2=0x7fb9a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:14.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.889+0000 7fb9c7fff700 1 --2- 172.21.15.156:0/235303065 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb9a40682f0 0x7fb9a406a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc005910 tx=0x7fb9bc009040 comp rx=0 tx=0).stop 2023-11-28T13:12:14.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.889+0000 7fb9c7fff700 1 -- 172.21.15.156:0/235303065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c015c9b0 msgr2=0x7fb9c015ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:14.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.889+0000 7fb9c7fff700 1 --2- 172.21.15.156:0/235303065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c015c9b0 0x7fb9c015ef40 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000c4e0 tx=0x7fb9b000c7f0 comp rx=0 tx=0).stop 2023-11-28T13:12:14.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.889+0000 7fb9c7fff700 1 -- 172.21.15.156:0/235303065 shutdown_connections 2023-11-28T13:12:14.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.889+0000 7fb9c7fff700 1 --2- 172.21.15.156:0/235303065 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb9a40682f0 0x7fb9a406a7a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:14.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.889+0000 7fb9c7fff700 1 --2- 172.21.15.156:0/235303065 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb9c0161340 0x7fb9c015f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:14.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.889+0000 7fb9c7fff700 1 --2- 172.21.15.156:0/235303065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9c00cd7c0 0x7fb9c0153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:14.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.889+0000 7fb9c7fff700 1 --2- 172.21.15.156:0/235303065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9c015c9b0 0x7fb9c015ef40 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:14.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.889+0000 7fb9c7fff700 1 -- 172.21.15.156:0/235303065 >> 172.21.15.156:0/235303065 conn(0x7fb9c0072600 msgr2=0x7fb9c0158350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:14.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.889+0000 7fb9c7fff700 1 -- 172.21.15.156:0/235303065 shutdown_connections 2023-11-28T13:12:14.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:14.889+0000 7fb9c7fff700 1 -- 172.21.15.156:0/235303065 wait complete. 2023-11-28T13:12:14.909 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:12:19.910 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:12:20.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.117+0000 7f4aaa7ff700 1 Processor -- start 2023-11-28T13:12:20.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.117+0000 7f4aaa7ff700 1 -- start start 2023-11-28T13:12:20.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.118+0000 7f4aaa7ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4aa415c9b0 0x7f4aa40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:20.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.118+0000 7f4aaa7ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa40d1950 con 0x7f4aa40cd7c0 2023-11-28T13:12:20.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.118+0000 7f4aaa7ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa40d24f0 con 0x7f4aa4161340 2023-11-28T13:12:20.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.118+0000 7f4aaa7ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa40b5800 con 0x7f4aa415c9b0 2023-11-28T13:12:20.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.119+0000 7f4aa3fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4aa415c9b0 0x7f4aa40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:20.120 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.119+0000 7f4aa3fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4aa415c9b0 0x7f4aa40b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53796/0 (socket says 172.21.15.156:53796) 2023-11-28T13:12:20.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.119+0000 7f4aa3fff700 1 -- 172.21.15.156:0/1538346574 learned_addr learned my addr 172.21.15.156:0/1538346574 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:12:20.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.119+0000 7f4aa3fff700 1 -- 172.21.15.156:0/1538346574 >> v1:172.21.15.186:6789/0 conn(0x7f4aa4161340 legacy=0x7f4aa40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:20.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.119+0000 7f4aa3fff700 1 -- 172.21.15.156:0/1538346574 >> v1:172.21.15.156:6789/0 conn(0x7f4aa40cd7c0 legacy=0x7f4aa40b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:20.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.119+0000 7f4aa3fff700 1 -- 172.21.15.156:0/1538346574 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa40b5520 con 0x7f4aa415c9b0 2023-11-28T13:12:20.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.119+0000 7f4aa3fff700 1 --2- 172.21.15.156:0/1538346574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4aa415c9b0 0x7f4aa40b4bf0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4a940098f0 tx=0x7f4a94009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25afd7a3602d6812 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:20.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.120+0000 7f4aa37fe700 1 -- 172.21.15.156:0/1538346574 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9400b8a0 con 0x7f4aa415c9b0 2023-11-28T13:12:20.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.120+0000 7f4aa37fe700 1 -- 172.21.15.156:0/1538346574 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9400ba00 con 0x7f4aa415c9b0 2023-11-28T13:12:20.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.120+0000 7f4aa37fe700 1 -- 172.21.15.156:0/1538346574 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9400bd90 con 0x7f4aa415c9b0 2023-11-28T13:12:20.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.120+0000 7f4aaa7ff700 1 -- 172.21.15.156:0/1538346574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4aa415c9b0 msgr2=0x7f4aa40b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:20.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.120+0000 7f4aaa7ff700 1 --2- 172.21.15.156:0/1538346574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4aa415c9b0 0x7f4aa40b4bf0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4a940098f0 tx=0x7f4a94009c00 comp rx=0 tx=0).stop 2023-11-28T13:12:20.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.120+0000 7f4aaa7ff700 1 -- 172.21.15.156:0/1538346574 shutdown_connections 2023-11-28T13:12:20.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.120+0000 7f4aaa7ff700 1 --2- 172.21.15.156:0/1538346574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4aa415c9b0 0x7f4aa40b4bf0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:20.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.120+0000 7f4aaa7ff700 1 -- 172.21.15.156:0/1538346574 >> 172.21.15.156:0/1538346574 conn(0x7f4aa4072600 msgr2=0x7f4aa40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:20.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.120+0000 7f4aaa7ff700 1 -- 172.21.15.156:0/1538346574 shutdown_connections 2023-11-28T13:12:20.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.120+0000 7f4aaa7ff700 1 -- 172.21.15.156:0/1538346574 wait complete. 2023-11-28T13:12:20.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.122+0000 7f4aaa7ff700 1 Processor -- start 2023-11-28T13:12:20.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.122+0000 7f4aaa7ff700 1 -- start start 2023-11-28T13:12:20.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.123+0000 7f4aaa7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4aa40cd7c0 0x7f4aa415eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:20.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.123+0000 7f4aaa7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aa415c9b0 0x7f4aa4153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:20.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.123+0000 7f4aaa7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4aa4161340 0x7f4aa40c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:20.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.123+0000 7f4aaa7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa4158080 con 0x7f4aa4161340 2023-11-28T13:12:20.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.123+0000 7f4aaa7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41539b0 con 0x7f4aa415c9b0 2023-11-28T13:12:20.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.123+0000 7f4aaa7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa4157da0 con 0x7f4aa40cd7c0 2023-11-28T13:12:20.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.123+0000 7f4aa8d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4aa4161340 0x7f4aa40c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:20.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.123+0000 7f4aa8d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4aa4161340 0x7f4aa40c7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53810/0 (socket says 172.21.15.156:53810) 2023-11-28T13:12:20.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.123+0000 7f4aa8d9c700 1 -- 172.21.15.156:0/2493794584 learned_addr learned my addr 172.21.15.156:0/2493794584 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:12:20.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.124+0000 7f4aa3fff700 1 --2- 172.21.15.156:0/2493794584 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4aa40cd7c0 0x7f4aa415eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:20.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.124+0000 7f4a9bfff700 1 --2- 172.21.15.156:0/2493794584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aa415c9b0 0x7f4aa4153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:20.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.124+0000 7f4aa8d9c700 1 -- 172.21.15.156:0/2493794584 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4aa40cd7c0 msgr2=0x7f4aa415eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:20.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.124+0000 7f4aa8d9c700 1 --2- 172.21.15.156:0/2493794584 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4aa40cd7c0 0x7f4aa415eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:20.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.124+0000 7f4aa8d9c700 1 -- 172.21.15.156:0/2493794584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aa415c9b0 msgr2=0x7f4aa4153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:20.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.124+0000 7f4aa8d9c700 1 --2- 172.21.15.156:0/2493794584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aa415c9b0 0x7f4aa4153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:20.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.124+0000 7f4aa8d9c700 1 -- 172.21.15.156:0/2493794584 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a94009550 con 0x7f4aa4161340 2023-11-28T13:12:20.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.124+0000 7f4aa8d9c700 1 --2- 172.21.15.156:0/2493794584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4aa4161340 0x7f4aa40c7620 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00cb80 tx=0x7f4a9c00ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:20.127 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.124+0000 7f4aa17fa700 1 -- 172.21.15.156:0/2493794584 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c003c00 con 0x7f4aa4161340 2023-11-28T13:12:20.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.124+0000 7f4aa17fa700 1 -- 172.21.15.156:0/2493794584 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9c003d60 con 0x7f4aa4161340 2023-11-28T13:12:20.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.124+0000 7f4aa17fa700 1 -- 172.21.15.156:0/2493794584 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c00aaa0 con 0x7f4aa4161340 2023-11-28T13:12:20.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.124+0000 7f4aaa7ff700 1 -- 172.21.15.156:0/2493794584 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa40d15e0 con 0x7f4aa4161340 2023-11-28T13:12:20.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.125+0000 7f4aaa7ff700 1 -- 172.21.15.156:0/2493794584 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa40d1300 con 0x7f4aa4161340 2023-11-28T13:12:20.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.126+0000 7f4aaa7ff700 1 -- 172.21.15.156:0/2493794584 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa40b5130 con 0x7f4aa4161340 2023-11-28T13:12:20.128 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.127+0000 7f4aa17fa700 1 -- 172.21.15.156:0/2493794584 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4a9c010050 con 0x7f4aa4161340 2023-11-28T13:12:20.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.127+0000 7f4aa17fa700 1 --2- 172.21.15.156:0/2493794584 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4a840682f0 0x7f4a8406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:20.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.127+0000 7f4aa17fa700 1 -- 172.21.15.156:0/2493794584 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f4a9c085bd0 con 0x7f4aa4161340 2023-11-28T13:12:20.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.128+0000 7f4aa3fff700 1 --2- 172.21.15.156:0/2493794584 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4a840682f0 0x7f4a8406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:20.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.129+0000 7f4aa3fff700 1 --2- 172.21.15.156:0/2493794584 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4a840682f0 0x7f4a8406a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4a94014040 tx=0x7f4a94008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:20.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.132+0000 7f4aa17fa700 1 -- 172.21.15.156:0/2493794584 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4a9c0522b0 con 0x7f4aa4161340 2023-11-28T13:12:20.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.367+0000 7f4aaa7ff700 1 -- 172.21.15.156:0/2493794584 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4aa415e370 con 0x7f4aa4161340 2023-11-28T13:12:20.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.367+0000 7f4aa17fa700 1 -- 172.21.15.156:0/2493794584 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f4aa415e370 con 0x7f4aa4161340 2023-11-28T13:12:20.369 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:12:20.369 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:12:20.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.369+0000 7f4aaa7ff700 1 -- 172.21.15.156:0/2493794584 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4a840682f0 msgr2=0x7f4a8406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:20.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.369+0000 7f4aaa7ff700 1 --2- 172.21.15.156:0/2493794584 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4a840682f0 0x7f4a8406a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4a94014040 tx=0x7f4a94008030 comp rx=0 tx=0).stop 2023-11-28T13:12:20.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.369+0000 7f4aaa7ff700 1 -- 172.21.15.156:0/2493794584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4aa4161340 msgr2=0x7f4aa40c7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:20.371 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.369+0000 7f4aaa7ff700 1 --2- 172.21.15.156:0/2493794584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4aa4161340 0x7f4aa40c7620 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00cb80 tx=0x7f4a9c00ce90 comp rx=0 tx=0).stop 2023-11-28T13:12:20.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.369+0000 7f4aaa7ff700 1 -- 172.21.15.156:0/2493794584 shutdown_connections 2023-11-28T13:12:20.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.369+0000 7f4aaa7ff700 1 --2- 172.21.15.156:0/2493794584 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4a840682f0 0x7f4a8406a7a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:20.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.369+0000 7f4aaa7ff700 1 --2- 172.21.15.156:0/2493794584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4aa4161340 0x7f4aa40c7620 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:20.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.369+0000 7f4aaa7ff700 1 --2- 172.21.15.156:0/2493794584 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4aa40cd7c0 0x7f4aa415eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:20.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.369+0000 7f4aaa7ff700 1 --2- 172.21.15.156:0/2493794584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aa415c9b0 0x7f4aa4153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:20.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.369+0000 7f4aaa7ff700 1 -- 172.21.15.156:0/2493794584 >> 172.21.15.156:0/2493794584 conn(0x7f4aa4072600 msgr2=0x7f4aa41544a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:20.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.370+0000 7f4aaa7ff700 1 -- 172.21.15.156:0/2493794584 shutdown_connections 2023-11-28T13:12:20.372 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:20.370+0000 7f4aaa7ff700 1 -- 172.21.15.156:0/2493794584 wait complete. 2023-11-28T13:12:20.390 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:12:25.391 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:12:25.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.596+0000 7ff2ff3ff700 1 Processor -- start 2023-11-28T13:12:25.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.597+0000 7ff2ff3ff700 1 -- start start 2023-11-28T13:12:25.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.598+0000 7ff2ff3ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff2f80cd7c0 0x7ff2f80cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:25.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.598+0000 7ff2ff3ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f80d1950 con 0x7ff2f815c9b0 2023-11-28T13:12:25.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.598+0000 7ff2ff3ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f80ce140 con 0x7ff2f8161340 2023-11-28T13:12:25.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.598+0000 7ff2ff3ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f80b5800 con 0x7ff2f80cd7c0 2023-11-28T13:12:25.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.598+0000 7ff2fd19b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff2f815c9b0 0x7ff2f80b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:49282/0 (socket says 172.21.15.156:49282) 2023-11-28T13:12:25.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.598+0000 7ff2fd19b700 1 -- 172.21.15.156:0/4253796081 learned_addr learned my addr 172.21.15.156:0/4253796081 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:12:25.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.598+0000 7ff2fd99c700 1 --2- 172.21.15.156:0/4253796081 >> v2:172.21.15.186:3300/0 conn(0x7ff2f80cd7c0 0x7ff2f80cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:25.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.598+0000 7ff2effff700 1 -- 172.21.15.156:0/4253796081 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 622909194 0 0) 0x7ff2f80d1950 con 0x7ff2f815c9b0 2023-11-28T13:12:25.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.598+0000 7ff2effff700 1 -- 172.21.15.156:0/4253796081 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e40035b0 con 0x7ff2f815c9b0 2023-11-28T13:12:25.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.599+0000 7ff2effff700 1 -- 172.21.15.156:0/4253796081 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2851903868 0 0) 0x7ff2e40035b0 con 0x7ff2f815c9b0 2023-11-28T13:12:25.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.599+0000 7ff2effff700 1 -- 172.21.15.156:0/4253796081 >> v2:172.21.15.186:3300/0 conn(0x7ff2f80cd7c0 msgr2=0x7ff2f80cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:12:25.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.599+0000 7ff2effff700 1 --2- 172.21.15.156:0/4253796081 >> v2:172.21.15.186:3300/0 conn(0x7ff2f80cd7c0 0x7ff2f80cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:25.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.599+0000 7ff2effff700 1 -- 172.21.15.156:0/4253796081 >> v1:172.21.15.186:6789/0 conn(0x7ff2f8161340 legacy=0x7ff2f80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:25.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.599+0000 7ff2effff700 1 -- 172.21.15.156:0/4253796081 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f80d24f0 con 0x7ff2f815c9b0 2023-11-28T13:12:25.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.599+0000 7ff2effff700 1 -- 172.21.15.156:0/4253796081 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7ff2f4001670 con 0x7ff2f815c9b0 2023-11-28T13:12:25.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.599+0000 7ff2effff700 1 -- 172.21.15.156:0/4253796081 >> v1:172.21.15.156:6789/0 conn(0x7ff2f815c9b0 legacy=0x7ff2f80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:25.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.599+0000 7ff2effff700 1 --2- 172.21.15.156:0/4253796081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2e4003a00 0x7ff2e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:25.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.599+0000 7ff2effff700 1 --2- 172.21.15.156:0/4253796081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2e4004850 0x7ff2e4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:25.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.599+0000 7ff2effff700 1 --2- 172.21.15.156:0/4253796081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2e4005200 0x7ff2e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:25.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.599+0000 7ff2effff700 1 -- 172.21.15.156:0/4253796081 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f80b5800 con 0x7ff2e4003a00 2023-11-28T13:12:25.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.599+0000 7ff2effff700 1 -- 172.21.15.156:0/4253796081 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e400c110 con 0x7ff2e4005200 2023-11-28T13:12:25.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.599+0000 7ff2effff700 1 -- 172.21.15.156:0/4253796081 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e400c2d0 con 0x7ff2e4004850 2023-11-28T13:12:25.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.599+0000 7ff2fd19b700 1 --2- 172.21.15.156:0/4253796081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2e4003a00 0x7ff2e4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:25.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.599+0000 7ff2fd99c700 1 --2- 172.21.15.156:0/4253796081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2e4005200 0x7ff2e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:25.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.599+0000 7ff2fc99a700 1 --2- 172.21.15.156:0/4253796081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2e4004850 0x7ff2e4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:25.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.600+0000 7ff2fd19b700 1 -- 172.21.15.156:0/4253796081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2e4004850 msgr2=0x7ff2e4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:12:25.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.600+0000 7ff2fd19b700 1 --2- 172.21.15.156:0/4253796081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2e4004850 0x7ff2e4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:25.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.600+0000 7ff2fd19b700 1 -- 172.21.15.156:0/4253796081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2e4005200 msgr2=0x7ff2e400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:12:25.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.600+0000 7ff2fd19b700 1 --2- 172.21.15.156:0/4253796081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2e4005200 0x7ff2e400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:25.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.600+0000 7ff2fd19b700 1 -- 172.21.15.156:0/4253796081 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e40035b0 con 0x7ff2e4003a00 2023-11-28T13:12:25.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.600+0000 7ff2fd99c700 1 -- 172.21.15.156:0/4253796081 reap_dead start 2023-11-28T13:12:25.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.600+0000 7ff2fc99a700 1 --2- 172.21.15.156:0/4253796081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2e4004850 0x7ff2e4004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:12:25.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.600+0000 7ff2fd19b700 1 --2- 172.21.15.156:0/4253796081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2e4003a00 0x7ff2e4003e10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400c240 tx=0x7ff2e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ae487b4779e935 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:25.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.600+0000 7ff2effff700 1 -- 172.21.15.156:0/4253796081 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4007c10 con 0x7ff2e4003a00 2023-11-28T13:12:25.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.600+0000 7ff2effff700 1 -- 172.21.15.156:0/4253796081 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f400cdc0 con 0x7ff2e4003a00 2023-11-28T13:12:25.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.600+0000 7ff2effff700 1 -- 172.21.15.156:0/4253796081 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4005ba0 con 0x7ff2e4003a00 2023-11-28T13:12:25.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.601+0000 7ff2ff3ff700 1 -- 172.21.15.156:0/4253796081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2e4003a00 msgr2=0x7ff2e4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:25.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.601+0000 7ff2ff3ff700 1 --2- 172.21.15.156:0/4253796081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2e4003a00 0x7ff2e4003e10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400c240 tx=0x7ff2e4005880 comp rx=0 tx=0).stop 2023-11-28T13:12:25.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.601+0000 7ff2ff3ff700 1 -- 172.21.15.156:0/4253796081 shutdown_connections 2023-11-28T13:12:25.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.601+0000 7ff2ff3ff700 1 --2- 172.21.15.156:0/4253796081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2e4003a00 0x7ff2e4003e10 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:25.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.601+0000 7ff2ff3ff700 1 -- 172.21.15.156:0/4253796081 >> 172.21.15.156:0/4253796081 conn(0x7ff2f8072600 msgr2=0x7ff2f80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:25.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.601+0000 7ff2ff3ff700 1 -- 172.21.15.156:0/4253796081 shutdown_connections 2023-11-28T13:12:25.605 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.601+0000 7ff2ff3ff700 1 -- 172.21.15.156:0/4253796081 wait complete. 2023-11-28T13:12:25.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.603+0000 7ff2ff3ff700 1 Processor -- start 2023-11-28T13:12:25.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.603+0000 7ff2ff3ff700 1 -- start start 2023-11-28T13:12:25.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.604+0000 7ff2ff3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2e4003a00 0x7ff2f80cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:25.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.604+0000 7ff2ff3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2f81f30c0 0x7ff2f80ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:25.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.604+0000 7ff2ff3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f815e5c0 0x7ff2f80c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:25.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.604+0000 7ff2ff3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f81f23d0 con 0x7ff2e4003a00 2023-11-28T13:12:25.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.604+0000 7ff2ff3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f81f2c30 con 0x7ff2f815e5c0 2023-11-28T13:12:25.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.604+0000 7ff2ff3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f80ca8a0 con 0x7ff2f81f30c0 2023-11-28T13:12:25.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.604+0000 7ff2fd19b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2e4003a00 0x7ff2f80cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:25.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.604+0000 7ff2fd19b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2e4003a00 0x7ff2f80cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53838/0 (socket says 172.21.15.156:53838) 2023-11-28T13:12:25.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.604+0000 7ff2fd19b700 1 -- 172.21.15.156:0/2676386344 learned_addr learned my addr 172.21.15.156:0/2676386344 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:12:25.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.604+0000 7ff2fd99c700 1 --2- 172.21.15.156:0/2676386344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f815e5c0 0x7ff2f80c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:25.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.604+0000 7ff2fc99a700 1 --2- 172.21.15.156:0/2676386344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2f81f30c0 0x7ff2f80ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:25.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.604+0000 7ff2fd19b700 1 -- 172.21.15.156:0/2676386344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2f81f30c0 msgr2=0x7ff2f80ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:25.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.604+0000 7ff2fd19b700 1 --2- 172.21.15.156:0/2676386344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2f81f30c0 0x7ff2f80ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:25.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.604+0000 7ff2fd19b700 1 -- 172.21.15.156:0/2676386344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f815e5c0 msgr2=0x7ff2f80c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:25.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.604+0000 7ff2fd19b700 1 --2- 172.21.15.156:0/2676386344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f815e5c0 0x7ff2f80c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:25.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.604+0000 7ff2fd19b700 1 -- 172.21.15.156:0/2676386344 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f80d1300 con 0x7ff2e4003a00 2023-11-28T13:12:25.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.604+0000 7ff2fd99c700 1 --2- 172.21.15.156:0/2676386344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f815e5c0 0x7ff2f80c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:12:25.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.605+0000 7ff2fd19b700 1 --2- 172.21.15.156:0/2676386344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2e4003a00 0x7ff2f80cccf0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400c0e0 tx=0x7ff2f4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:25.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.605+0000 7ff2edffb700 1 -- 172.21.15.156:0/2676386344 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4004c80 con 0x7ff2e4003a00 2023-11-28T13:12:25.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.605+0000 7ff2edffb700 1 -- 172.21.15.156:0/2676386344 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f4004de0 con 0x7ff2e4003a00 2023-11-28T13:12:25.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.605+0000 7ff2edffb700 1 -- 172.21.15.156:0/2676386344 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4007310 con 0x7ff2e4003a00 2023-11-28T13:12:25.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.605+0000 7ff2ff3ff700 1 -- 172.21.15.156:0/2676386344 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f80b5520 con 0x7ff2e4003a00 2023-11-28T13:12:25.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.605+0000 7ff2ff3ff700 1 -- 172.21.15.156:0/2676386344 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f80d15e0 con 0x7ff2e4003a00 2023-11-28T13:12:25.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.606+0000 7ff2ff3ff700 1 -- 172.21.15.156:0/2676386344 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f80745c0 con 0x7ff2e4003a00 2023-11-28T13:12:25.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.607+0000 7ff2edffb700 1 -- 172.21.15.156:0/2676386344 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff2f4010070 con 0x7ff2e4003a00 2023-11-28T13:12:25.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.607+0000 7ff2edffb700 1 --2- 172.21.15.156:0/2676386344 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2dc0682f0 0x7ff2dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:25.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.608+0000 7ff2edffb700 1 -- 172.21.15.156:0/2676386344 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ff2f4086fa0 con 0x7ff2e4003a00 2023-11-28T13:12:25.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.608+0000 7ff2fc99a700 1 --2- 172.21.15.156:0/2676386344 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2dc0682f0 0x7ff2dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:25.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.609+0000 7ff2fc99a700 1 --2- 172.21.15.156:0/2676386344 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2dc0682f0 0x7ff2dc06a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff2f81f34d0 tx=0x7ff2e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:25.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.612+0000 7ff2edffb700 1 -- 172.21.15.156:0/2676386344 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff2f4053600 con 0x7ff2e4003a00 2023-11-28T13:12:25.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.847+0000 7ff2ff3ff700 1 -- 172.21.15.156:0/2676386344 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff2f80d08a0 con 0x7ff2e4003a00 2023-11-28T13:12:25.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.847+0000 7ff2edffb700 1 -- 172.21.15.156:0/2676386344 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7ff2f4056c20 con 0x7ff2e4003a00 2023-11-28T13:12:25.848 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:12:25.848 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:12:25.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.850+0000 7ff2ff3ff700 1 -- 172.21.15.156:0/2676386344 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2dc0682f0 msgr2=0x7ff2dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:25.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.850+0000 7ff2ff3ff700 1 --2- 172.21.15.156:0/2676386344 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2dc0682f0 0x7ff2dc06a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff2f81f34d0 tx=0x7ff2e8009040 comp rx=0 tx=0).stop 2023-11-28T13:12:25.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.850+0000 7ff2ff3ff700 1 -- 172.21.15.156:0/2676386344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2e4003a00 msgr2=0x7ff2f80cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:25.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.850+0000 7ff2ff3ff700 1 --2- 172.21.15.156:0/2676386344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2e4003a00 0x7ff2f80cccf0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400c0e0 tx=0x7ff2f4001a00 comp rx=0 tx=0).stop 2023-11-28T13:12:25.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.850+0000 7ff2ff3ff700 1 -- 172.21.15.156:0/2676386344 shutdown_connections 2023-11-28T13:12:25.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.850+0000 7ff2ff3ff700 1 --2- 172.21.15.156:0/2676386344 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff2dc0682f0 0x7ff2dc06a7a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:25.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.850+0000 7ff2ff3ff700 1 --2- 172.21.15.156:0/2676386344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f815e5c0 0x7ff2f80c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:25.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.850+0000 7ff2ff3ff700 1 --2- 172.21.15.156:0/2676386344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2e4003a00 0x7ff2f80cccf0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:25.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.850+0000 7ff2ff3ff700 1 --2- 172.21.15.156:0/2676386344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2f81f30c0 0x7ff2f80ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:25.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.850+0000 7ff2ff3ff700 1 -- 172.21.15.156:0/2676386344 >> 172.21.15.156:0/2676386344 conn(0x7ff2f8072600 msgr2=0x7ff2f815bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:25.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.850+0000 7ff2ff3ff700 1 -- 172.21.15.156:0/2676386344 shutdown_connections 2023-11-28T13:12:25.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:25.850+0000 7ff2ff3ff700 1 -- 172.21.15.156:0/2676386344 wait complete. 2023-11-28T13:12:25.871 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:12:30.872 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:12:31.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.079+0000 7fcbdc7ff700 1 Processor -- start 2023-11-28T13:12:31.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.079+0000 7fcbdc7ff700 1 -- start start 2023-11-28T13:12:31.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.080+0000 7fcbdc7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbd415ca50 0x7fcbd406b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:31.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.080+0000 7fcbdc7ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fcbd40cce10 0x7fcbd40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:31.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.080+0000 7fcbdc7ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd40cf5e0 con 0x7fcbd406b8c0 2023-11-28T13:12:31.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.080+0000 7fcbdc7ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd40cd290 con 0x7fcbd40cce10 2023-11-28T13:12:31.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.080+0000 7fcbdc7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40cd470 con 0x7fcbd415ca50 2023-11-28T13:12:31.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.081+0000 7fcbd9d9a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fcbd406b8c0 0x7fcbd40cc700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37618/0 (socket says 172.21.15.156:37618) 2023-11-28T13:12:31.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.081+0000 7fcbd9d9a700 1 -- 172.21.15.156:0/1404844556 learned_addr learned my addr 172.21.15.156:0/1404844556 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:12:31.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.081+0000 7fcbdad9c700 1 --2- 172.21.15.156:0/1404844556 >> v2:172.21.15.186:3300/0 conn(0x7fcbd40cce10 0x7fcbd40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:31.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.081+0000 7fcbda59b700 1 --2- 172.21.15.156:0/1404844556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbd415ca50 0x7fcbd406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:31.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.081+0000 7fcbd9599700 1 -- 172.21.15.156:0/1404844556 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628694913 0 0) 0x7fcbd40cf5e0 con 0x7fcbd406b8c0 2023-11-28T13:12:31.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.081+0000 7fcbd9599700 1 -- 172.21.15.156:0/1404844556 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbc00035b0 con 0x7fcbd406b8c0 2023-11-28T13:12:31.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.081+0000 7fcbdad9c700 1 -- 172.21.15.156:0/1404844556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbd415ca50 msgr2=0x7fcbd406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:12:31.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.081+0000 7fcbdad9c700 1 --2- 172.21.15.156:0/1404844556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbd415ca50 0x7fcbd406b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:31.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.081+0000 7fcbdad9c700 1 -- 172.21.15.156:0/1404844556 >> v1:172.21.15.156:6789/0 conn(0x7fcbd406b8c0 legacy=0x7fcbd40cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:31.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.081+0000 7fcbdad9c700 1 -- 172.21.15.156:0/1404844556 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd40cecb0 con 0x7fcbd40cce10 2023-11-28T13:12:31.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.082+0000 7fcbdad9c700 1 --2- 172.21.15.156:0/1404844556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbd40cce10 0x7fcbd40cfa70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009dc0 tx=0x7fcbcc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ba0b6a71a29973 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:31.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.082+0000 7fcbd9599700 1 -- 172.21.15.156:0/1404844556 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc00e070 con 0x7fcbd40cce10 2023-11-28T13:12:31.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.082+0000 7fcbd9599700 1 -- 172.21.15.156:0/1404844556 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbcc00bae0 con 0x7fcbd40cce10 2023-11-28T13:12:31.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.082+0000 7fcbd9599700 1 -- 172.21.15.156:0/1404844556 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc00be50 con 0x7fcbd40cce10 2023-11-28T13:12:31.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.083+0000 7fcbdc7ff700 1 -- 172.21.15.156:0/1404844556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbd40cce10 msgr2=0x7fcbd40cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:31.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.083+0000 7fcbdc7ff700 1 --2- 172.21.15.156:0/1404844556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbd40cce10 0x7fcbd40cfa70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009dc0 tx=0x7fcbcc00b330 comp rx=0 tx=0).stop 2023-11-28T13:12:31.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.083+0000 7fcbdc7ff700 1 -- 172.21.15.156:0/1404844556 shutdown_connections 2023-11-28T13:12:31.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.083+0000 7fcbdc7ff700 1 --2- 172.21.15.156:0/1404844556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbd40cce10 0x7fcbd40cfa70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:31.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.083+0000 7fcbdc7ff700 1 --2- 172.21.15.156:0/1404844556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbd415ca50 0x7fcbd406b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:31.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.083+0000 7fcbdc7ff700 1 -- 172.21.15.156:0/1404844556 >> 172.21.15.156:0/1404844556 conn(0x7fcbd409a230 msgr2=0x7fcbd408afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:31.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.083+0000 7fcbdc7ff700 1 -- 172.21.15.156:0/1404844556 shutdown_connections 2023-11-28T13:12:31.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.083+0000 7fcbdc7ff700 1 -- 172.21.15.156:0/1404844556 wait complete. 2023-11-28T13:12:31.088 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.084+0000 7fcbdc7ff700 1 Processor -- start 2023-11-28T13:12:31.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.085+0000 7fcbdc7ff700 1 -- start start 2023-11-28T13:12:31.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.085+0000 7fcbdc7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbd406b8c0 0x7fcbd40b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:31.089 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.085+0000 7fcbdc7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbd40cce10 0x7fcbd415a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:31.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.085+0000 7fcbdc7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbd415ca50 0x7fcbd40b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:31.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.085+0000 7fcbdc7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd415df40 con 0x7fcbd415ca50 2023-11-28T13:12:31.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.085+0000 7fcbdc7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd415ad40 con 0x7fcbd406b8c0 2023-11-28T13:12:31.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.085+0000 7fcbdc7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd415b010 con 0x7fcbd40cce10 2023-11-28T13:12:31.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.086+0000 7fcbdad9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbd415ca50 0x7fcbd40b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:31.090 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.086+0000 7fcbdad9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbd415ca50 0x7fcbd40b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57826/0 (socket says 172.21.15.156:57826) 2023-11-28T13:12:31.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.086+0000 7fcbdad9c700 1 -- 172.21.15.156:0/1748621155 learned_addr learned my addr 172.21.15.156:0/1748621155 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:12:31.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.086+0000 7fcbd9d9a700 1 --2- 172.21.15.156:0/1748621155 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbd40cce10 0x7fcbd415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:31.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.086+0000 7fcbdad9c700 1 -- 172.21.15.156:0/1748621155 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbd40cce10 msgr2=0x7fcbd415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:31.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.086+0000 7fcbda59b700 1 --2- 172.21.15.156:0/1748621155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbd406b8c0 0x7fcbd40b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:31.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.086+0000 7fcbdad9c700 1 --2- 172.21.15.156:0/1748621155 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbd40cce10 0x7fcbd415a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:31.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.086+0000 7fcbdad9c700 1 -- 172.21.15.156:0/1748621155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbd406b8c0 msgr2=0x7fcbd40b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:31.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.086+0000 7fcbdad9c700 1 --2- 172.21.15.156:0/1748621155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbd406b8c0 0x7fcbd40b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:31.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.086+0000 7fcbdad9c700 1 -- 172.21.15.156:0/1748621155 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc009a70 con 0x7fcbd415ca50 2023-11-28T13:12:31.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.086+0000 7fcbda59b700 1 --2- 172.21.15.156:0/1748621155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbd406b8c0 0x7fcbd40b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:12:31.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.086+0000 7fcbdad9c700 1 --2- 172.21.15.156:0/1748621155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbd415ca50 0x7fcbd40b45f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00d010 tx=0x7fcbcc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:31.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.087+0000 7fcbcaffd700 1 -- 172.21.15.156:0/1748621155 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc004450 con 0x7fcbd415ca50 2023-11-28T13:12:31.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.087+0000 7fcbcaffd700 1 -- 172.21.15.156:0/1748621155 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbcc01f070 con 0x7fcbd415ca50 2023-11-28T13:12:31.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.087+0000 7fcbdc7ff700 1 -- 172.21.15.156:0/1748621155 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd40cf270 con 0x7fcbd415ca50 2023-11-28T13:12:31.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.087+0000 7fcbcaffd700 1 -- 172.21.15.156:0/1748621155 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc0047f0 con 0x7fcbd415ca50 2023-11-28T13:12:31.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.087+0000 7fcbdc7ff700 1 -- 172.21.15.156:0/1748621155 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd40cef90 con 0x7fcbd415ca50 2023-11-28T13:12:31.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.088+0000 7fcbdc7ff700 1 -- 172.21.15.156:0/1748621155 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd415ce20 con 0x7fcbd415ca50 2023-11-28T13:12:31.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.089+0000 7fcbcaffd700 1 -- 172.21.15.156:0/1748621155 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcbcc02b060 con 0x7fcbd415ca50 2023-11-28T13:12:31.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.090+0000 7fcbcaffd700 1 --2- 172.21.15.156:0/1748621155 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcbb80682f0 0x7fcbb806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:31.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.090+0000 7fcbcaffd700 1 -- 172.21.15.156:0/1748621155 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fcbcc086eb0 con 0x7fcbd415ca50 2023-11-28T13:12:31.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.090+0000 7fcbda59b700 1 --2- 172.21.15.156:0/1748621155 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcbb80682f0 0x7fcbb806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:31.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.091+0000 7fcbda59b700 1 --2- 172.21.15.156:0/1748621155 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcbb80682f0 0x7fcbb806a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009ac0 tx=0x7fcbd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:31.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.094+0000 7fcbcaffd700 1 -- 172.21.15.156:0/1748621155 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcbcc053590 con 0x7fcbd415ca50 2023-11-28T13:12:31.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.327+0000 7fcbdc7ff700 1 -- 172.21.15.156:0/1748621155 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcbd40745c0 con 0x7fcbd415ca50 2023-11-28T13:12:31.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.328+0000 7fcbcaffd700 1 -- 172.21.15.156:0/1748621155 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fcbcc021020 con 0x7fcbd415ca50 2023-11-28T13:12:31.328 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:12:31.328 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:12:31.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.330+0000 7fcbdc7ff700 1 -- 172.21.15.156:0/1748621155 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcbb80682f0 msgr2=0x7fcbb806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:31.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.330+0000 7fcbdc7ff700 1 --2- 172.21.15.156:0/1748621155 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcbb80682f0 0x7fcbb806a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009ac0 tx=0x7fcbd0008040 comp rx=0 tx=0).stop 2023-11-28T13:12:31.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.330+0000 7fcbdc7ff700 1 -- 172.21.15.156:0/1748621155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbd415ca50 msgr2=0x7fcbd40b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:31.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.330+0000 7fcbdc7ff700 1 --2- 172.21.15.156:0/1748621155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbd415ca50 0x7fcbd40b45f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00d010 tx=0x7fcbcc004ad0 comp rx=0 tx=0).stop 2023-11-28T13:12:31.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.330+0000 7fcbdc7ff700 1 -- 172.21.15.156:0/1748621155 shutdown_connections 2023-11-28T13:12:31.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.330+0000 7fcbdc7ff700 1 --2- 172.21.15.156:0/1748621155 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcbb80682f0 0x7fcbb806a7a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:31.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.330+0000 7fcbdc7ff700 1 --2- 172.21.15.156:0/1748621155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbd415ca50 0x7fcbd40b45f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:31.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.330+0000 7fcbdc7ff700 1 --2- 172.21.15.156:0/1748621155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbd406b8c0 0x7fcbd40b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:31.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.330+0000 7fcbdc7ff700 1 --2- 172.21.15.156:0/1748621155 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcbd40cce10 0x7fcbd415a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:31.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.330+0000 7fcbdc7ff700 1 -- 172.21.15.156:0/1748621155 >> 172.21.15.156:0/1748621155 conn(0x7fcbd409a230 msgr2=0x7fcbd40cc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:31.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.330+0000 7fcbdc7ff700 1 -- 172.21.15.156:0/1748621155 shutdown_connections 2023-11-28T13:12:31.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:31.330+0000 7fcbdc7ff700 1 -- 172.21.15.156:0/1748621155 wait complete. 2023-11-28T13:12:31.349 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:12:33.774 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:12:33.779 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:12:36.350 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:12:36.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.556+0000 7f63a37ff700 1 Processor -- start 2023-11-28T13:12:36.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.557+0000 7f63a37ff700 1 -- start start 2023-11-28T13:12:36.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.558+0000 7f63a37ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f639c159860 0x7f639c0c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:36.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.558+0000 7f63a37ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f639c0b0d80 0x7f639c0c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:36.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.558+0000 7f63a37ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639c0ccc00 con 0x7f639c159490 2023-11-28T13:12:36.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.558+0000 7f63a37ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639c0ccec0 con 0x7f639c0b0d80 2023-11-28T13:12:36.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.558+0000 7f63a37ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639c0cd1a0 con 0x7f639c159860 2023-11-28T13:12:36.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.558+0000 7f63a159b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f639c159490 0x7f639c0b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:42492/0 (socket says 172.21.15.156:42492) 2023-11-28T13:12:36.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.558+0000 7f63a159b700 1 -- 172.21.15.156:0/2731261552 learned_addr learned my addr 172.21.15.156:0/2731261552 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:12:36.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.558+0000 7f63a1d9c700 1 --2- 172.21.15.156:0/2731261552 >> v2:172.21.15.156:3300/0 conn(0x7f639c0b0d80 0x7f639c0c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:36.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.558+0000 7f63a0d9a700 1 --2- 172.21.15.156:0/2731261552 >> v2:172.21.15.186:3300/0 conn(0x7f639c159860 0x7f639c0c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:36.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.558+0000 7f63a1d9c700 1 -- 172.21.15.156:0/2731261552 >> v2:172.21.15.186:3300/0 conn(0x7f639c159860 msgr2=0x7f639c0c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:12:36.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.558+0000 7f63a1d9c700 1 --2- 172.21.15.156:0/2731261552 >> v2:172.21.15.186:3300/0 conn(0x7f639c159860 0x7f639c0c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:36.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.558+0000 7f63a1d9c700 1 -- 172.21.15.156:0/2731261552 >> v1:172.21.15.156:6789/0 conn(0x7f639c159490 legacy=0x7f639c0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:36.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.558+0000 7f63a1d9c700 1 -- 172.21.15.156:0/2731261552 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c0cd480 con 0x7f639c0b0d80 2023-11-28T13:12:36.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.559+0000 7f63a1d9c700 1 --2- 172.21.15.156:0/2731261552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f639c0b0d80 0x7f639c0c9a00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f639c0cb5d0 tx=0x7f639400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8083861c73cea3b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:36.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.559+0000 7f6393fff700 1 -- 172.21.15.156:0/2731261552 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639400e070 con 0x7f639c0b0d80 2023-11-28T13:12:36.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.559+0000 7f6393fff700 1 -- 172.21.15.156:0/2731261552 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639400bb20 con 0x7f639c0b0d80 2023-11-28T13:12:36.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.559+0000 7f6393fff700 1 -- 172.21.15.156:0/2731261552 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394012430 con 0x7f639c0b0d80 2023-11-28T13:12:36.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.560+0000 7f63a37ff700 1 -- 172.21.15.156:0/2731261552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f639c0b0d80 msgr2=0x7f639c0c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:36.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.560+0000 7f63a37ff700 1 --2- 172.21.15.156:0/2731261552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f639c0b0d80 0x7f639c0c9a00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f639c0cb5d0 tx=0x7f639400b330 comp rx=0 tx=0).stop 2023-11-28T13:12:36.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.560+0000 7f63a37ff700 1 -- 172.21.15.156:0/2731261552 shutdown_connections 2023-11-28T13:12:36.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.560+0000 7f63a37ff700 1 --2- 172.21.15.156:0/2731261552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f639c0b0d80 0x7f639c0c9a00 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:36.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.560+0000 7f63a37ff700 1 --2- 172.21.15.156:0/2731261552 >> v2:172.21.15.186:3300/0 conn(0x7f639c159860 0x7f639c0c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:36.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.560+0000 7f63a37ff700 1 -- 172.21.15.156:0/2731261552 >> 172.21.15.156:0/2731261552 conn(0x7f639c0cb670 msgr2=0x7f639c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:36.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.560+0000 7f63a37ff700 1 -- 172.21.15.156:0/2731261552 shutdown_connections 2023-11-28T13:12:36.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.560+0000 7f63a37ff700 1 -- 172.21.15.156:0/2731261552 wait complete. 2023-11-28T13:12:36.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.562+0000 7f63a37ff700 1 Processor -- start 2023-11-28T13:12:36.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.562+0000 7f63a37ff700 1 -- start start 2023-11-28T13:12:36.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.563+0000 7f63a37ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f639c0b0d80 0x7f639c1559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:36.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.563+0000 7f63a37ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f639c159490 0x7f639c1f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:36.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.563+0000 7f63a37ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f639c071a70 0x7f639c070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:36.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.563+0000 7f63a37ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c0b49d0 con 0x7f639c159490 2023-11-28T13:12:36.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.563+0000 7f63a37ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c155f10 con 0x7f639c0b0d80 2023-11-28T13:12:36.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.563+0000 7f63a37ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639c1f1e20 con 0x7f639c071a70 2023-11-28T13:12:36.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.563+0000 7f63a0d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f639c159490 0x7f639c1f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:36.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.563+0000 7f63a0d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f639c159490 0x7f639c1f18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:40420/0 (socket says 172.21.15.156:40420) 2023-11-28T13:12:36.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.563+0000 7f63a0d9a700 1 -- 172.21.15.156:0/1425471025 learned_addr learned my addr 172.21.15.156:0/1425471025 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:12:36.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.563+0000 7f63a159b700 1 --2- 172.21.15.156:0/1425471025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f639c0b0d80 0x7f639c1559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:36.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.563+0000 7f63a1d9c700 1 --2- 172.21.15.156:0/1425471025 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f639c071a70 0x7f639c070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:36.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.563+0000 7f63a0d9a700 1 -- 172.21.15.156:0/1425471025 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f639c071a70 msgr2=0x7f639c070b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:36.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.563+0000 7f63a0d9a700 1 --2- 172.21.15.156:0/1425471025 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f639c071a70 0x7f639c070b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:36.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.563+0000 7f63a0d9a700 1 -- 172.21.15.156:0/1425471025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f639c0b0d80 msgr2=0x7f639c1559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:36.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.563+0000 7f63a0d9a700 1 --2- 172.21.15.156:0/1425471025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f639c0b0d80 0x7f639c1559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:36.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.563+0000 7f63a0d9a700 1 -- 172.21.15.156:0/1425471025 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6394009a70 con 0x7f639c159490 2023-11-28T13:12:36.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.564+0000 7f63a0d9a700 1 --2- 172.21.15.156:0/1425471025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f639c159490 0x7f639c1f18e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f638c00e8d0 tx=0x7f638c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:36.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.564+0000 7f6391ffb700 1 -- 172.21.15.156:0/1425471025 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c009590 con 0x7f639c159490 2023-11-28T13:12:36.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.564+0000 7f6391ffb700 1 -- 172.21.15.156:0/1425471025 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638c00eed0 con 0x7f639c159490 2023-11-28T13:12:36.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.564+0000 7f63a37ff700 1 -- 172.21.15.156:0/1425471025 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639c0c9f40 con 0x7f639c159490 2023-11-28T13:12:36.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.564+0000 7f6391ffb700 1 -- 172.21.15.156:0/1425471025 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c00ca50 con 0x7f639c159490 2023-11-28T13:12:36.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.564+0000 7f63a37ff700 1 -- 172.21.15.156:0/1425471025 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639c0cd760 con 0x7f639c159490 2023-11-28T13:12:36.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.565+0000 7f63a37ff700 1 -- 172.21.15.156:0/1425471025 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639c0cdc80 con 0x7f639c159490 2023-11-28T13:12:36.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.566+0000 7f6391ffb700 1 -- 172.21.15.156:0/1425471025 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f638c019030 con 0x7f639c159490 2023-11-28T13:12:36.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.567+0000 7f6391ffb700 1 --2- 172.21.15.156:0/1425471025 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6380068320 0x7f638006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:36.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.567+0000 7f6391ffb700 1 -- 172.21.15.156:0/1425471025 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f638c085c00 con 0x7f639c159490 2023-11-28T13:12:36.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.567+0000 7f63a159b700 1 --2- 172.21.15.156:0/1425471025 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6380068320 0x7f638006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:36.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.568+0000 7f63a159b700 1 --2- 172.21.15.156:0/1425471025 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6380068320 0x7f638006a7d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6398005910 tx=0x7f6398009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:36.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.571+0000 7f6391ffb700 1 -- 172.21.15.156:0/1425471025 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f638c0522e0 con 0x7f639c159490 2023-11-28T13:12:36.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.804+0000 7f63a37ff700 1 -- 172.21.15.156:0/1425471025 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f639c0745c0 con 0x7f639c159490 2023-11-28T13:12:36.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.805+0000 7f6391ffb700 1 -- 172.21.15.156:0/1425471025 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f638c055900 con 0x7f639c159490 2023-11-28T13:12:36.806 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:12:36.806 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:12:36.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.807+0000 7f63a37ff700 1 -- 172.21.15.156:0/1425471025 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6380068320 msgr2=0x7f638006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:36.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.807+0000 7f63a37ff700 1 --2- 172.21.15.156:0/1425471025 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6380068320 0x7f638006a7d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6398005910 tx=0x7f6398009040 comp rx=0 tx=0).stop 2023-11-28T13:12:36.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.807+0000 7f63a37ff700 1 -- 172.21.15.156:0/1425471025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f639c159490 msgr2=0x7f639c1f18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:36.808 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.807+0000 7f63a37ff700 1 --2- 172.21.15.156:0/1425471025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f639c159490 0x7f639c1f18e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f638c00e8d0 tx=0x7f638c00ec90 comp rx=0 tx=0).stop 2023-11-28T13:12:36.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.807+0000 7f63a37ff700 1 -- 172.21.15.156:0/1425471025 shutdown_connections 2023-11-28T13:12:36.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.807+0000 7f63a37ff700 1 --2- 172.21.15.156:0/1425471025 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6380068320 0x7f638006a7d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:36.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.807+0000 7f63a37ff700 1 --2- 172.21.15.156:0/1425471025 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f639c071a70 0x7f639c070b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:36.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.807+0000 7f63a37ff700 1 --2- 172.21.15.156:0/1425471025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f639c0b0d80 0x7f639c1559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:36.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.807+0000 7f63a37ff700 1 --2- 172.21.15.156:0/1425471025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f639c159490 0x7f639c1f18e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:36.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.807+0000 7f63a37ff700 1 -- 172.21.15.156:0/1425471025 >> 172.21.15.156:0/1425471025 conn(0x7f639c0cb670 msgr2=0x7f639c153db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:36.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.808+0000 7f63a37ff700 1 -- 172.21.15.156:0/1425471025 shutdown_connections 2023-11-28T13:12:36.809 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:36.808+0000 7f63a37ff700 1 -- 172.21.15.156:0/1425471025 wait complete. 2023-11-28T13:12:36.826 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:12:41.827 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:12:42.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.033+0000 7f1e40bff700 1 Processor -- start 2023-11-28T13:12:42.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.033+0000 7f1e40bff700 1 -- start start 2023-11-28T13:12:42.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.034+0000 7f1e40bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1e3c0d0ff0 0x7f1e3c0cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:42.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.034+0000 7f1e40bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e3c0b37b0 0x7f1e3c0b6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:42.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.034+0000 7f1e40bff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e3c0cf220 con 0x7f1e3c0b4750 2023-11-28T13:12:42.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.034+0000 7f1e40bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3c0b3e50 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.034+0000 7f1e40bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c0cf470 con 0x7f1e3c0b37b0 2023-11-28T13:12:42.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.035+0000 7f1e3a59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1e3c0d0ff0 0x7f1e3c0cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:42.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.035+0000 7f1e3a59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1e3c0d0ff0 0x7f1e3c0cff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:40434/0 (socket says 172.21.15.156:40434) 2023-11-28T13:12:42.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.035+0000 7f1e3a59c700 1 -- 172.21.15.156:0/1691866975 learned_addr learned my addr 172.21.15.156:0/1691866975 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:12:42.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.035+0000 7f1e3ad9d700 1 --2- 172.21.15.156:0/1691866975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e3c0b37b0 0x7f1e3c0b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:42.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.035+0000 7f1e3a59c700 1 -- 172.21.15.156:0/1691866975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e3c0b37b0 msgr2=0x7f1e3c0b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:12:42.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.035+0000 7f1e3a59c700 1 --2- 172.21.15.156:0/1691866975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e3c0b37b0 0x7f1e3c0b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:42.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.035+0000 7f1e3a59c700 1 -- 172.21.15.156:0/1691866975 >> v1:172.21.15.186:6789/0 conn(0x7f1e3c0b4750 legacy=0x7f1e3c0b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:42.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.035+0000 7f1e3a59c700 1 -- 172.21.15.156:0/1691866975 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3c0cf750 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.035+0000 7f1e3a59c700 1 --2- 172.21.15.156:0/1691866975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c0d0ff0 0x7f1e3c0cff50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0098e0 tx=0x7f1e2c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f125ce840daeac4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:42.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.036+0000 7f1e39d9b700 1 -- 172.21.15.156:0/1691866975 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c00b8a0 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.036+0000 7f1e39d9b700 1 -- 172.21.15.156:0/1691866975 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c00ba00 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.036+0000 7f1e39d9b700 1 -- 172.21.15.156:0/1691866975 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c00bdb0 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.036+0000 7f1e40bff700 1 -- 172.21.15.156:0/1691866975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c0d0ff0 msgr2=0x7f1e3c0cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:42.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.036+0000 7f1e40bff700 1 --2- 172.21.15.156:0/1691866975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c0d0ff0 0x7f1e3c0cff50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0098e0 tx=0x7f1e2c009bf0 comp rx=0 tx=0).stop 2023-11-28T13:12:42.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.036+0000 7f1e40bff700 1 -- 172.21.15.156:0/1691866975 shutdown_connections 2023-11-28T13:12:42.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.036+0000 7f1e40bff700 1 --2- 172.21.15.156:0/1691866975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e3c0b37b0 0x7f1e3c0b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:42.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.036+0000 7f1e40bff700 1 --2- 172.21.15.156:0/1691866975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c0d0ff0 0x7f1e3c0cff50 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:42.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.036+0000 7f1e40bff700 1 -- 172.21.15.156:0/1691866975 >> 172.21.15.156:0/1691866975 conn(0x7f1e3c09c3c0 msgr2=0x7f1e3c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:42.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.036+0000 7f1e40bff700 1 -- 172.21.15.156:0/1691866975 shutdown_connections 2023-11-28T13:12:42.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.036+0000 7f1e40bff700 1 -- 172.21.15.156:0/1691866975 wait complete. 2023-11-28T13:12:42.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.038+0000 7f1e40bff700 1 Processor -- start 2023-11-28T13:12:42.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.038+0000 7f1e40bff700 1 -- start start 2023-11-28T13:12:42.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.039+0000 7f1e40bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e3c0b37b0 0x7f1e3c0b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:42.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.039+0000 7f1e40bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e3c0b4750 0x7f1e3c0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:42.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.039+0000 7f1e40bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c0d0ff0 0x7f1e3c0afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:42.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.039+0000 7f1e40bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c0c8090 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.039+0000 7f1e40bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c1e53e0 con 0x7f1e3c0b4750 2023-11-28T13:12:42.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.039+0000 7f1e40bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c1e56c0 con 0x7f1e3c0b37b0 2023-11-28T13:12:42.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.039+0000 7f1e3ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c0d0ff0 0x7f1e3c0afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:42.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.039+0000 7f1e3ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c0d0ff0 0x7f1e3c0afcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:40448/0 (socket says 172.21.15.156:40448) 2023-11-28T13:12:42.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.039+0000 7f1e3ad9d700 1 -- 172.21.15.156:0/477555901 learned_addr learned my addr 172.21.15.156:0/477555901 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:12:42.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.040+0000 7f1e3a59c700 1 --2- 172.21.15.156:0/477555901 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e3c0b37b0 0x7f1e3c0b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:42.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.040+0000 7f1e33fff700 1 --2- 172.21.15.156:0/477555901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e3c0b4750 0x7f1e3c0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:42.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.040+0000 7f1e3ad9d700 1 -- 172.21.15.156:0/477555901 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e3c0b37b0 msgr2=0x7f1e3c0b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:42.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.040+0000 7f1e3ad9d700 1 --2- 172.21.15.156:0/477555901 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e3c0b37b0 0x7f1e3c0b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:42.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.040+0000 7f1e3ad9d700 1 -- 172.21.15.156:0/477555901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e3c0b4750 msgr2=0x7f1e3c0b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:42.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.040+0000 7f1e3ad9d700 1 --2- 172.21.15.156:0/477555901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e3c0b4750 0x7f1e3c0b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:42.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.040+0000 7f1e3ad9d700 1 -- 172.21.15.156:0/477555901 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c0094f0 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.040+0000 7f1e3ad9d700 1 --2- 172.21.15.156:0/477555901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c0d0ff0 0x7f1e3c0afcb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1e34007e40 tx=0x7f1e3400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:42.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.040+0000 7f1e32ffd700 1 -- 172.21.15.156:0/477555901 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3400eea0 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.040+0000 7f1e32ffd700 1 -- 172.21.15.156:0/477555901 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e34014420 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.040+0000 7f1e40bff700 1 -- 172.21.15.156:0/477555901 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e3c0d0a50 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.040+0000 7f1e32ffd700 1 -- 172.21.15.156:0/477555901 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e340147b0 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.040+0000 7f1e40bff700 1 -- 172.21.15.156:0/477555901 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3c0cfa30 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.041+0000 7f1e40bff700 1 -- 172.21.15.156:0/477555901 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e3c0d0490 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.042+0000 7f1e32ffd700 1 -- 172.21.15.156:0/477555901 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1e34012050 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.043+0000 7f1e32ffd700 1 --2- 172.21.15.156:0/477555901 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1e1c0682f0 0x7f1e1c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:42.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.043+0000 7f1e32ffd700 1 -- 172.21.15.156:0/477555901 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f1e34086490 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.044+0000 7f1e3a59c700 1 --2- 172.21.15.156:0/477555901 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1e1c0682f0 0x7f1e1c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:42.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.044+0000 7f1e3a59c700 1 --2- 172.21.15.156:0/477555901 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1e1c0682f0 0x7f1e1c06a7a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009c30 tx=0x7f1e2c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:42.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.047+0000 7f1e32ffd700 1 -- 172.21.15.156:0/477555901 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1e34052b70 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.280+0000 7f1e40bff700 1 -- 172.21.15.156:0/477555901 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1e3c0745c0 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.280+0000 7f1e32ffd700 1 -- 172.21.15.156:0/477555901 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f1e34056190 con 0x7f1e3c0d0ff0 2023-11-28T13:12:42.281 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:12:42.281 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:12:42.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.282+0000 7f1e40bff700 1 -- 172.21.15.156:0/477555901 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1e1c0682f0 msgr2=0x7f1e1c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:42.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.282+0000 7f1e40bff700 1 --2- 172.21.15.156:0/477555901 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1e1c0682f0 0x7f1e1c06a7a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009c30 tx=0x7f1e2c01c040 comp rx=0 tx=0).stop 2023-11-28T13:12:42.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.282+0000 7f1e40bff700 1 -- 172.21.15.156:0/477555901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c0d0ff0 msgr2=0x7f1e3c0afcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:42.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.282+0000 7f1e40bff700 1 --2- 172.21.15.156:0/477555901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c0d0ff0 0x7f1e3c0afcb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1e34007e40 tx=0x7f1e3400e3c0 comp rx=0 tx=0).stop 2023-11-28T13:12:42.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.282+0000 7f1e40bff700 1 -- 172.21.15.156:0/477555901 shutdown_connections 2023-11-28T13:12:42.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.282+0000 7f1e40bff700 1 --2- 172.21.15.156:0/477555901 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1e1c0682f0 0x7f1e1c06a7a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:42.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.282+0000 7f1e40bff700 1 --2- 172.21.15.156:0/477555901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e3c0d0ff0 0x7f1e3c0afcb0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:42.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.282+0000 7f1e40bff700 1 --2- 172.21.15.156:0/477555901 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e3c0b37b0 0x7f1e3c0b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:42.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.282+0000 7f1e40bff700 1 --2- 172.21.15.156:0/477555901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e3c0b4750 0x7f1e3c0b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:42.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.282+0000 7f1e40bff700 1 -- 172.21.15.156:0/477555901 >> 172.21.15.156:0/477555901 conn(0x7f1e3c09c3c0 msgr2=0x7f1e3c0b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:42.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.282+0000 7f1e40bff700 1 -- 172.21.15.156:0/477555901 shutdown_connections 2023-11-28T13:12:42.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:42.283+0000 7f1e40bff700 1 -- 172.21.15.156:0/477555901 wait complete. 2023-11-28T13:12:42.301 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:12:47.302 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:12:47.509 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.508+0000 7f14a1dff700 1 Processor -- start 2023-11-28T13:12:47.509 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.508+0000 7f14a1dff700 1 -- start start 2023-11-28T13:12:47.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.510+0000 7f14a1dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f149c161340 0x7f149c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:47.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.510+0000 7f14a1dff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f149c0d2320 0x7f149c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:47.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.510+0000 7f14a1dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f149c0d1950 con 0x7f149c15c9b0 2023-11-28T13:12:47.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.510+0000 7f14a1dff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f149c0b5800 con 0x7f149c0d2320 2023-11-28T13:12:47.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.510+0000 7f14a1dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149c0b5300 con 0x7f149c161340 2023-11-28T13:12:47.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.510+0000 7f149affd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f149c161340 0x7f149c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:47.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.510+0000 7f149b7fe700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f149c15c9b0 0x7f149c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:55696/0 (socket says 172.21.15.156:55696) 2023-11-28T13:12:47.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.510+0000 7f149b7fe700 1 -- 172.21.15.156:0/2642907892 learned_addr learned my addr 172.21.15.156:0/2642907892 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:12:47.511 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.510+0000 7f149bfff700 1 --2- 172.21.15.156:0/2642907892 >> v2:172.21.15.186:3300/0 conn(0x7f149c0d2320 0x7f149c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:47.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.511+0000 7f149affd700 1 -- 172.21.15.156:0/2642907892 >> v2:172.21.15.186:3300/0 conn(0x7f149c0d2320 msgr2=0x7f149c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:12:47.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.511+0000 7f149affd700 1 --2- 172.21.15.156:0/2642907892 >> v2:172.21.15.186:3300/0 conn(0x7f149c0d2320 0x7f149c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:47.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.511+0000 7f149affd700 1 -- 172.21.15.156:0/2642907892 >> v1:172.21.15.186:6789/0 conn(0x7f149c15c9b0 legacy=0x7f149c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:47.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.511+0000 7f149affd700 1 -- 172.21.15.156:0/2642907892 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c0b5520 con 0x7f149c161340 2023-11-28T13:12:47.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.511+0000 7f149a7fc700 1 -- 172.21.15.156:0/2642907892 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 219768565 0 0) 0x7f149c0d1950 con 0x7f149c15c9b0 2023-11-28T13:12:47.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.511+0000 7f149bfff700 1 --2- 172.21.15.156:0/2642907892 >> v2:172.21.15.186:3300/0 conn(0x7f149c0d2320 0x7f149c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:12:47.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.511+0000 7f149affd700 1 --2- 172.21.15.156:0/2642907892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f149c161340 0x7f149c0d1de0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1490009840 tx=0x7f1490009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9687f601963246e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:47.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.511+0000 7f149a7fc700 1 -- 172.21.15.156:0/2642907892 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1490011830 con 0x7f149c161340 2023-11-28T13:12:47.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.511+0000 7f149a7fc700 1 -- 172.21.15.156:0/2642907892 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1490011990 con 0x7f149c161340 2023-11-28T13:12:47.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.511+0000 7f149a7fc700 1 -- 172.21.15.156:0/2642907892 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1490011d20 con 0x7f149c161340 2023-11-28T13:12:47.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.512+0000 7f14a1dff700 1 -- 172.21.15.156:0/2642907892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f149c161340 msgr2=0x7f149c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:47.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.512+0000 7f14a1dff700 1 --2- 172.21.15.156:0/2642907892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f149c161340 0x7f149c0d1de0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1490009840 tx=0x7f1490009b50 comp rx=0 tx=0).stop 2023-11-28T13:12:47.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.512+0000 7f14a1dff700 1 -- 172.21.15.156:0/2642907892 shutdown_connections 2023-11-28T13:12:47.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.512+0000 7f14a1dff700 1 --2- 172.21.15.156:0/2642907892 >> v2:172.21.15.186:3300/0 conn(0x7f149c0d2320 0x7f149c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:47.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.512+0000 7f14a1dff700 1 --2- 172.21.15.156:0/2642907892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f149c161340 0x7f149c0d1de0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:47.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.512+0000 7f14a1dff700 1 -- 172.21.15.156:0/2642907892 >> 172.21.15.156:0/2642907892 conn(0x7f149c072600 msgr2=0x7f149c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:47.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.512+0000 7f14a1dff700 1 -- 172.21.15.156:0/2642907892 shutdown_connections 2023-11-28T13:12:47.514 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.512+0000 7f14a1dff700 1 -- 172.21.15.156:0/2642907892 wait complete. 2023-11-28T13:12:47.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.514+0000 7f14a1dff700 1 Processor -- start 2023-11-28T13:12:47.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.515+0000 7f14a1dff700 1 -- start start 2023-11-28T13:12:47.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.516+0000 7f14a1dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f149c0d2320 0x7f149c15a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:47.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.516+0000 7f14a1dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f149c15c9b0 0x7f149c1e8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:47.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.516+0000 7f14a1dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f149c161340 0x7f149c157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:47.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.516+0000 7f14a1dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c1fa5f0 con 0x7f149c161340 2023-11-28T13:12:47.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.516+0000 7f14a1dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c1fb170 con 0x7f149c0d2320 2023-11-28T13:12:47.517 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.516+0000 7f14a1dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149c1fa840 con 0x7f149c15c9b0 2023-11-28T13:12:47.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.516+0000 7f149bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f149c161340 0x7f149c157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:47.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.516+0000 7f149bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f149c161340 0x7f149c157020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:56208/0 (socket says 172.21.15.156:56208) 2023-11-28T13:12:47.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.516+0000 7f149bfff700 1 -- 172.21.15.156:0/3721211533 learned_addr learned my addr 172.21.15.156:0/3721211533 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:12:47.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.516+0000 7f149affd700 1 --2- 172.21.15.156:0/3721211533 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f149c15c9b0 0x7f149c1e8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:47.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.516+0000 7f149b7fe700 1 --2- 172.21.15.156:0/3721211533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f149c0d2320 0x7f149c15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:47.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.517+0000 7f149bfff700 1 -- 172.21.15.156:0/3721211533 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f149c15c9b0 msgr2=0x7f149c1e8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:47.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.517+0000 7f149bfff700 1 --2- 172.21.15.156:0/3721211533 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f149c15c9b0 0x7f149c1e8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:47.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.517+0000 7f149bfff700 1 -- 172.21.15.156:0/3721211533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f149c0d2320 msgr2=0x7f149c15a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:47.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.517+0000 7f149bfff700 1 --2- 172.21.15.156:0/3721211533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f149c0d2320 0x7f149c15a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:47.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.517+0000 7f149bfff700 1 -- 172.21.15.156:0/3721211533 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1494009a70 con 0x7f149c161340 2023-11-28T13:12:47.518 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.517+0000 7f149affd700 1 --2- 172.21.15.156:0/3721211533 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f149c15c9b0 0x7f149c1e8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:12:47.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.517+0000 7f149bfff700 1 --2- 172.21.15.156:0/3721211533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f149c161340 0x7f149c157020 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1494009e60 tx=0x7f149400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:47.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.517+0000 7f1483fff700 1 -- 172.21.15.156:0/3721211533 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494009cd0 con 0x7f149c161340 2023-11-28T13:12:47.519 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.517+0000 7f1483fff700 1 -- 172.21.15.156:0/3721211533 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149400f040 con 0x7f149c161340 2023-11-28T13:12:47.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.517+0000 7f14a1dff700 1 -- 172.21.15.156:0/3721211533 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14900094a0 con 0x7f149c161340 2023-11-28T13:12:47.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.517+0000 7f1483fff700 1 -- 172.21.15.156:0/3721211533 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149400ce20 con 0x7f149c161340 2023-11-28T13:12:47.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.517+0000 7f14a1dff700 1 -- 172.21.15.156:0/3721211533 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149c0d15e0 con 0x7f149c161340 2023-11-28T13:12:47.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.518+0000 7f14a1dff700 1 -- 172.21.15.156:0/3721211533 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f149c0d1300 con 0x7f149c161340 2023-11-28T13:12:47.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.519+0000 7f1483fff700 1 -- 172.21.15.156:0/3721211533 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1494014480 con 0x7f149c161340 2023-11-28T13:12:47.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.520+0000 7f1483fff700 1 --2- 172.21.15.156:0/3721211533 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f14880682f0 0x7f148806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:47.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.520+0000 7f1483fff700 1 -- 172.21.15.156:0/3721211533 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f1494086620 con 0x7f149c161340 2023-11-28T13:12:47.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.520+0000 7f149b7fe700 1 --2- 172.21.15.156:0/3721211533 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f14880682f0 0x7f148806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:47.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.521+0000 7f149b7fe700 1 --2- 172.21.15.156:0/3721211533 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f14880682f0 0x7f148806a7a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f148c000f60 tx=0x7f148c007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:47.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.524+0000 7f1483fff700 1 -- 172.21.15.156:0/3721211533 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1494052ca0 con 0x7f149c161340 2023-11-28T13:12:47.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.759+0000 7f14a1dff700 1 -- 172.21.15.156:0/3721211533 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f149c15cd80 con 0x7f149c161340 2023-11-28T13:12:47.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.759+0000 7f1483fff700 1 -- 172.21.15.156:0/3721211533 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f14940562c0 con 0x7f149c161340 2023-11-28T13:12:47.760 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:12:47.760 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:12:47.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.761+0000 7f14a1dff700 1 -- 172.21.15.156:0/3721211533 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f14880682f0 msgr2=0x7f148806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:47.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.761+0000 7f14a1dff700 1 --2- 172.21.15.156:0/3721211533 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f14880682f0 0x7f148806a7a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f148c000f60 tx=0x7f148c007c90 comp rx=0 tx=0).stop 2023-11-28T13:12:47.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.761+0000 7f14a1dff700 1 -- 172.21.15.156:0/3721211533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f149c161340 msgr2=0x7f149c157020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:47.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.761+0000 7f14a1dff700 1 --2- 172.21.15.156:0/3721211533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f149c161340 0x7f149c157020 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1494009e60 tx=0x7f149400c3a0 comp rx=0 tx=0).stop 2023-11-28T13:12:47.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.762+0000 7f14a1dff700 1 -- 172.21.15.156:0/3721211533 shutdown_connections 2023-11-28T13:12:47.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.762+0000 7f14a1dff700 1 --2- 172.21.15.156:0/3721211533 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f14880682f0 0x7f148806a7a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:47.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.762+0000 7f14a1dff700 1 --2- 172.21.15.156:0/3721211533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f149c161340 0x7f149c157020 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:47.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.762+0000 7f14a1dff700 1 --2- 172.21.15.156:0/3721211533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f149c0d2320 0x7f149c15a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:47.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.762+0000 7f14a1dff700 1 --2- 172.21.15.156:0/3721211533 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f149c15c9b0 0x7f149c1e8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:47.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.762+0000 7f14a1dff700 1 -- 172.21.15.156:0/3721211533 >> 172.21.15.156:0/3721211533 conn(0x7f149c072600 msgr2=0x7f149c0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:47.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.762+0000 7f14a1dff700 1 -- 172.21.15.156:0/3721211533 shutdown_connections 2023-11-28T13:12:47.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:47.762+0000 7f14a1dff700 1 -- 172.21.15.156:0/3721211533 wait complete. 2023-11-28T13:12:47.781 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:12:52.783 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:12:52.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.989+0000 7fa374fff700 1 Processor -- start 2023-11-28T13:12:52.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.990+0000 7fa374fff700 1 -- start start 2023-11-28T13:12:53.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.991+0000 7fa374fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa3700d01b0 0x7fa3700d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:53.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.991+0000 7fa374fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3700b20b0 0x7fa3700d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:53.001 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.991+0000 7fa374fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3700b0230 con 0x7fa3700b29b0 2023-11-28T13:12:53.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.991+0000 7fa374fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa37015cec0 con 0x7fa3700d01b0 2023-11-28T13:12:53.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.991+0000 7fa374fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700b4570 con 0x7fa3700b20b0 2023-11-28T13:12:53.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.991+0000 7fa36e59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa3700d01b0 0x7fa3700d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:53.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.991+0000 7fa36e59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa3700d01b0 0x7fa3700d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:56220/0 (socket says 172.21.15.156:56220) 2023-11-28T13:12:53.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.991+0000 7fa36e59c700 1 -- 172.21.15.156:0/848094681 learned_addr learned my addr 172.21.15.156:0/848094681 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:12:53.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.991+0000 7fa36dd9b700 1 --2- 172.21.15.156:0/848094681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3700b20b0 0x7fa3700d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:53.002 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.991+0000 7fa36e59c700 1 -- 172.21.15.156:0/848094681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3700b20b0 msgr2=0x7fa3700d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:12:53.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.991+0000 7fa36e59c700 1 --2- 172.21.15.156:0/848094681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3700b20b0 0x7fa3700d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:53.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.991+0000 7fa36e59c700 1 -- 172.21.15.156:0/848094681 >> v1:172.21.15.186:6789/0 conn(0x7fa3700b29b0 legacy=0x7fa3700b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:12:53.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.991+0000 7fa36e59c700 1 -- 172.21.15.156:0/848094681 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3700b4850 con 0x7fa3700d01b0 2023-11-28T13:12:53.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.992+0000 7fa36e59c700 1 --2- 172.21.15.156:0/848094681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3700d01b0 0x7fa3700d0c20 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa3600098f0 tx=0x7fa360009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a653ce21f6b7b3a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:53.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.992+0000 7fa36d59a700 1 -- 172.21.15.156:0/848094681 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa360004230 con 0x7fa3700d01b0 2023-11-28T13:12:53.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.992+0000 7fa36d59a700 1 -- 172.21.15.156:0/848094681 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa360004390 con 0x7fa3700d01b0 2023-11-28T13:12:53.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.992+0000 7fa36d59a700 1 -- 172.21.15.156:0/848094681 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3600036a0 con 0x7fa3700d01b0 2023-11-28T13:12:53.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.992+0000 7fa374fff700 1 -- 172.21.15.156:0/848094681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3700d01b0 msgr2=0x7fa3700d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:53.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.992+0000 7fa374fff700 1 --2- 172.21.15.156:0/848094681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3700d01b0 0x7fa3700d0c20 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa3600098f0 tx=0x7fa360009c00 comp rx=0 tx=0).stop 2023-11-28T13:12:53.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.992+0000 7fa374fff700 1 -- 172.21.15.156:0/848094681 shutdown_connections 2023-11-28T13:12:53.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.993+0000 7fa374fff700 1 --2- 172.21.15.156:0/848094681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3700d01b0 0x7fa3700d0c20 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:53.003 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.993+0000 7fa374fff700 1 --2- 172.21.15.156:0/848094681 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3700b20b0 0x7fa3700d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:53.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.993+0000 7fa374fff700 1 -- 172.21.15.156:0/848094681 >> 172.21.15.156:0/848094681 conn(0x7fa37009c3f0 msgr2=0x7fa3700d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:53.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.993+0000 7fa374fff700 1 -- 172.21.15.156:0/848094681 shutdown_connections 2023-11-28T13:12:53.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.993+0000 7fa374fff700 1 -- 172.21.15.156:0/848094681 wait complete. 2023-11-28T13:12:53.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.994+0000 7fa374fff700 1 Processor -- start 2023-11-28T13:12:53.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.995+0000 7fa374fff700 1 -- start start 2023-11-28T13:12:53.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.995+0000 7fa374fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3700b20b0 0x7fa3700b7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:53.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.995+0000 7fa374fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3700b29b0 0x7fa3700cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:53.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.995+0000 7fa374fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3700d01b0 0x7fa3700b8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:53.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.996+0000 7fa374fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700cb7c0 con 0x7fa3700b20b0 2023-11-28T13:12:53.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.996+0000 7fa374fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700cc190 con 0x7fa3700b29b0 2023-11-28T13:12:53.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.996+0000 7fa374fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700ccb80 con 0x7fa3700d01b0 2023-11-28T13:12:53.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.996+0000 7fa36e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3700b20b0 0x7fa3700b7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:53.004 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.996+0000 7fa36e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3700b20b0 0x7fa3700b7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:56232/0 (socket says 172.21.15.156:56232) 2023-11-28T13:12:53.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.996+0000 7fa36e59c700 1 -- 172.21.15.156:0/3053899899 learned_addr learned my addr 172.21.15.156:0/3053899899 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:12:53.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.996+0000 7fa36e59c700 1 -- 172.21.15.156:0/3053899899 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3700d01b0 msgr2=0x7fa3700b8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:53.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.996+0000 7fa36e59c700 1 --2- 172.21.15.156:0/3053899899 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3700d01b0 0x7fa3700b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:53.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.996+0000 7fa36e59c700 1 -- 172.21.15.156:0/3053899899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3700b29b0 msgr2=0x7fa3700cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:53.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.996+0000 7fa36e59c700 1 --2- 172.21.15.156:0/3053899899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3700b29b0 0x7fa3700cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:53.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.996+0000 7fa36e59c700 1 -- 172.21.15.156:0/3053899899 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa360009550 con 0x7fa3700b20b0 2023-11-28T13:12:53.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.996+0000 7fa36e59c700 1 --2- 172.21.15.156:0/3053899899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3700b20b0 0x7fa3700b7640 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa360009130 tx=0x7fa360004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:53.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.997+0000 7fa35effd700 1 -- 172.21.15.156:0/3053899899 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa360003e10 con 0x7fa3700b20b0 2023-11-28T13:12:53.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.997+0000 7fa35effd700 1 -- 172.21.15.156:0/3053899899 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36000b660 con 0x7fa3700b20b0 2023-11-28T13:12:53.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.997+0000 7fa374fff700 1 -- 172.21.15.156:0/3053899899 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3700afec0 con 0x7fa3700b20b0 2023-11-28T13:12:53.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.997+0000 7fa35effd700 1 -- 172.21.15.156:0/3053899899 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36000ba10 con 0x7fa3700b20b0 2023-11-28T13:12:53.005 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.997+0000 7fa374fff700 1 -- 172.21.15.156:0/3053899899 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3700afbe0 con 0x7fa3700b20b0 2023-11-28T13:12:53.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.998+0000 7fa374fff700 1 -- 172.21.15.156:0/3053899899 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3700b2600 con 0x7fa3700b20b0 2023-11-28T13:12:53.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:52.999+0000 7fa35effd700 1 -- 172.21.15.156:0/3053899899 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa360017020 con 0x7fa3700b20b0 2023-11-28T13:12:53.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.000+0000 7fa35effd700 1 --2- 172.21.15.156:0/3053899899 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa3500682f0 0x7fa35006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:53.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.000+0000 7fa35effd700 1 -- 172.21.15.156:0/3053899899 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fa360087620 con 0x7fa3700b20b0 2023-11-28T13:12:53.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.000+0000 7fa36dd9b700 1 --2- 172.21.15.156:0/3053899899 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa3500682f0 0x7fa35006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:53.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.001+0000 7fa36dd9b700 1 --2- 172.21.15.156:0/3053899899 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa3500682f0 0x7fa35006a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa3700b7400 tx=0x7fa364008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:53.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.004+0000 7fa35effd700 1 -- 172.21.15.156:0/3053899899 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa360053ca0 con 0x7fa3700b20b0 2023-11-28T13:12:53.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.238+0000 7fa374fff700 1 -- 172.21.15.156:0/3053899899 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa3700cbaa0 con 0x7fa3700b20b0 2023-11-28T13:12:53.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.238+0000 7fa35effd700 1 -- 172.21.15.156:0/3053899899 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fa3600572c0 con 0x7fa3700b20b0 2023-11-28T13:12:53.240 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:12:53.240 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:12:53.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.241+0000 7fa374fff700 1 -- 172.21.15.156:0/3053899899 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa3500682f0 msgr2=0x7fa35006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:53.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.241+0000 7fa374fff700 1 --2- 172.21.15.156:0/3053899899 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa3500682f0 0x7fa35006a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa3700b7400 tx=0x7fa364008040 comp rx=0 tx=0).stop 2023-11-28T13:12:53.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.241+0000 7fa374fff700 1 -- 172.21.15.156:0/3053899899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3700b20b0 msgr2=0x7fa3700b7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:53.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.241+0000 7fa374fff700 1 --2- 172.21.15.156:0/3053899899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3700b20b0 0x7fa3700b7640 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa360009130 tx=0x7fa360004750 comp rx=0 tx=0).stop 2023-11-28T13:12:53.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.241+0000 7fa374fff700 1 -- 172.21.15.156:0/3053899899 shutdown_connections 2023-11-28T13:12:53.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.241+0000 7fa374fff700 1 --2- 172.21.15.156:0/3053899899 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa3500682f0 0x7fa35006a7a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:53.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.241+0000 7fa374fff700 1 --2- 172.21.15.156:0/3053899899 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3700d01b0 0x7fa3700b8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:53.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.241+0000 7fa374fff700 1 --2- 172.21.15.156:0/3053899899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3700b20b0 0x7fa3700b7640 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:53.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.241+0000 7fa374fff700 1 --2- 172.21.15.156:0/3053899899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3700b29b0 0x7fa3700cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:53.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.241+0000 7fa374fff700 1 -- 172.21.15.156:0/3053899899 >> 172.21.15.156:0/3053899899 conn(0x7fa37009c3f0 msgr2=0x7fa3700d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:53.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.241+0000 7fa374fff700 1 -- 172.21.15.156:0/3053899899 shutdown_connections 2023-11-28T13:12:53.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:53.241+0000 7fa374fff700 1 -- 172.21.15.156:0/3053899899 wait complete. 2023-11-28T13:12:53.260 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:12:58.262 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:12:58.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.466+0000 7f6368bff700 1 Processor -- start 2023-11-28T13:12:58.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.466+0000 7f6368bff700 1 -- start start 2023-11-28T13:12:58.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.468+0000 7f6368bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6364070c50 0x7f63640b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:58.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.468+0000 7f6368bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63640b0bb0 0x7f636415add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:58.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.468+0000 7f6368bff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636415e1a0 con 0x7f6364158200 2023-11-28T13:12:58.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.468+0000 7f6368bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f636415f070 con 0x7f6364070c50 2023-11-28T13:12:58.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.468+0000 7f6368bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63640b0fd0 con 0x7f63640b0bb0 2023-11-28T13:12:58.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.468+0000 7f635bfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6364070c50 0x7f63640b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:58.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.468+0000 7f635bfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6364070c50 0x7f63640b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39900/0 (socket says 172.21.15.156:39900) 2023-11-28T13:12:58.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.468+0000 7f635bfff700 1 -- 172.21.15.156:0/329354915 learned_addr learned my addr 172.21.15.156:0/329354915 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:12:58.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.468+0000 7f6362d9d700 1 --2- 172.21.15.156:0/329354915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63640b0bb0 0x7f636415add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:58.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.468+0000 7f635bfff700 1 -- 172.21.15.156:0/329354915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63640b0bb0 msgr2=0x7f636415add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:12:58.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.468+0000 7f635bfff700 1 --2- 172.21.15.156:0/329354915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63640b0bb0 0x7f636415add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:58.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.468+0000 7f635bfff700 1 -- 172.21.15.156:0/329354915 >> v1:172.21.15.186:6789/0 conn(0x7f6364158200 legacy=0x7f63640caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:58.471 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.468+0000 7f635bfff700 1 -- 172.21.15.156:0/329354915 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63640cb180 con 0x7f6364070c50 2023-11-28T13:12:58.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.468+0000 7f6362d9d700 1 --2- 172.21.15.156:0/329354915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63640b0bb0 0x7f636415add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:12:58.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.469+0000 7f635bfff700 1 --2- 172.21.15.156:0/329354915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6364070c50 0x7f63640b0670 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f634c0098f0 tx=0x7f634c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0379d7145cfcd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:58.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.469+0000 7f6361d9b700 1 -- 172.21.15.156:0/329354915 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c00b8a0 con 0x7f6364070c50 2023-11-28T13:12:58.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.469+0000 7f6361d9b700 1 -- 172.21.15.156:0/329354915 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634c00ba00 con 0x7f6364070c50 2023-11-28T13:12:58.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.469+0000 7f6361d9b700 1 -- 172.21.15.156:0/329354915 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c00bd90 con 0x7f6364070c50 2023-11-28T13:12:58.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.469+0000 7f6368bff700 1 -- 172.21.15.156:0/329354915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6364070c50 msgr2=0x7f63640b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:58.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.469+0000 7f6368bff700 1 --2- 172.21.15.156:0/329354915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6364070c50 0x7f63640b0670 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f634c0098f0 tx=0x7f634c009c00 comp rx=0 tx=0).stop 2023-11-28T13:12:58.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.469+0000 7f6368bff700 1 -- 172.21.15.156:0/329354915 shutdown_connections 2023-11-28T13:12:58.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.470+0000 7f6368bff700 1 --2- 172.21.15.156:0/329354915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63640b0bb0 0x7f636415add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:58.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.470+0000 7f6368bff700 1 --2- 172.21.15.156:0/329354915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6364070c50 0x7f63640b0670 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:58.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.470+0000 7f6368bff700 1 -- 172.21.15.156:0/329354915 >> 172.21.15.156:0/329354915 conn(0x7f63640cb670 msgr2=0x7f63640cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:58.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.470+0000 7f6368bff700 1 -- 172.21.15.156:0/329354915 shutdown_connections 2023-11-28T13:12:58.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.470+0000 7f6368bff700 1 -- 172.21.15.156:0/329354915 wait complete. 2023-11-28T13:12:58.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.472+0000 7f6368bff700 1 Processor -- start 2023-11-28T13:12:58.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.472+0000 7f6368bff700 1 -- start start 2023-11-28T13:12:58.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.473+0000 7f6368bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6364070c50 0x7f63641568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:58.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.473+0000 7f6368bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63640b0bb0 0x7f63641f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:58.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.473+0000 7f6368bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6364158200 0x7f63640cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:58.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.473+0000 7f6368bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6364156df0 con 0x7f6364158200 2023-11-28T13:12:58.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.473+0000 7f6368bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6364156420 con 0x7f63640b0bb0 2023-11-28T13:12:58.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.473+0000 7f6368bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6364155810 con 0x7f6364070c50 2023-11-28T13:12:58.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.474+0000 7f6362d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6364158200 0x7f63640cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:58.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.474+0000 7f6362d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6364158200 0x7f63640cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39912/0 (socket says 172.21.15.156:39912) 2023-11-28T13:12:58.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.474+0000 7f6362d9d700 1 -- 172.21.15.156:0/282310024 learned_addr learned my addr 172.21.15.156:0/282310024 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:12:58.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.474+0000 7f635bfff700 1 --2- 172.21.15.156:0/282310024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63640b0bb0 0x7f63641f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:58.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.474+0000 7f636259c700 1 --2- 172.21.15.156:0/282310024 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6364070c50 0x7f63641568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:58.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.474+0000 7f6362d9d700 1 -- 172.21.15.156:0/282310024 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6364070c50 msgr2=0x7f63641568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:58.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.474+0000 7f6362d9d700 1 --2- 172.21.15.156:0/282310024 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6364070c50 0x7f63641568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:58.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.474+0000 7f6362d9d700 1 -- 172.21.15.156:0/282310024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63640b0bb0 msgr2=0x7f63641f1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:58.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.474+0000 7f6362d9d700 1 --2- 172.21.15.156:0/282310024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63640b0bb0 0x7f63641f1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:58.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.474+0000 7f6362d9d700 1 -- 172.21.15.156:0/282310024 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634c009550 con 0x7f6364158200 2023-11-28T13:12:58.477 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.474+0000 7f6362d9d700 1 --2- 172.21.15.156:0/282310024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6364158200 0x7f63640cda50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f635c009fa0 tx=0x7f635c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:58.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.475+0000 7f635affd700 1 -- 172.21.15.156:0/282310024 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c00ddb0 con 0x7f6364158200 2023-11-28T13:12:58.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.475+0000 7f635affd700 1 -- 172.21.15.156:0/282310024 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635c00f040 con 0x7f6364158200 2023-11-28T13:12:58.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.475+0000 7f6368bff700 1 -- 172.21.15.156:0/282310024 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63641585d0 con 0x7f6364158200 2023-11-28T13:12:58.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.475+0000 7f635affd700 1 -- 172.21.15.156:0/282310024 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c003c60 con 0x7f6364158200 2023-11-28T13:12:58.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.475+0000 7f6368bff700 1 -- 172.21.15.156:0/282310024 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63640710f0 con 0x7f6364158200 2023-11-28T13:12:58.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.476+0000 7f6368bff700 1 -- 172.21.15.156:0/282310024 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636415e3f0 con 0x7f6364158200 2023-11-28T13:12:58.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.477+0000 7f635affd700 1 -- 172.21.15.156:0/282310024 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f635c01c030 con 0x7f6364158200 2023-11-28T13:12:58.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.477+0000 7f635affd700 1 --2- 172.21.15.156:0/282310024 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63440682f0 0x7f634406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:12:58.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.478+0000 7f635affd700 1 -- 172.21.15.156:0/282310024 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f635c085d40 con 0x7f6364158200 2023-11-28T13:12:58.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.478+0000 7f636259c700 1 --2- 172.21.15.156:0/282310024 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63440682f0 0x7f634406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:12:58.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.479+0000 7f636259c700 1 --2- 172.21.15.156:0/282310024 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63440682f0 0x7f634406a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6354007fd0 tx=0x7f6354009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:12:58.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.482+0000 7f635affd700 1 -- 172.21.15.156:0/282310024 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f635c052420 con 0x7f6364158200 2023-11-28T13:12:58.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.715+0000 7f6368bff700 1 -- 172.21.15.156:0/282310024 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f636406fd60 con 0x7f6364158200 2023-11-28T13:12:58.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.715+0000 7f635affd700 1 -- 172.21.15.156:0/282310024 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f635c01a0c0 con 0x7f6364158200 2023-11-28T13:12:58.717 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:12:58.717 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:12:58.719 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.718+0000 7f6368bff700 1 -- 172.21.15.156:0/282310024 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63440682f0 msgr2=0x7f634406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:58.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.718+0000 7f6368bff700 1 --2- 172.21.15.156:0/282310024 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63440682f0 0x7f634406a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6354007fd0 tx=0x7f6354009040 comp rx=0 tx=0).stop 2023-11-28T13:12:58.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.718+0000 7f6368bff700 1 -- 172.21.15.156:0/282310024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6364158200 msgr2=0x7f63640cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:12:58.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.718+0000 7f6368bff700 1 --2- 172.21.15.156:0/282310024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6364158200 0x7f63640cda50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f635c009fa0 tx=0x7f635c00d3b0 comp rx=0 tx=0).stop 2023-11-28T13:12:58.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.718+0000 7f6368bff700 1 -- 172.21.15.156:0/282310024 shutdown_connections 2023-11-28T13:12:58.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.718+0000 7f6368bff700 1 --2- 172.21.15.156:0/282310024 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f63440682f0 0x7f634406a7a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:58.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.718+0000 7f6368bff700 1 --2- 172.21.15.156:0/282310024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6364158200 0x7f63640cda50 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:58.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.718+0000 7f6368bff700 1 --2- 172.21.15.156:0/282310024 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6364070c50 0x7f63641568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:58.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.718+0000 7f6368bff700 1 --2- 172.21.15.156:0/282310024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63640b0bb0 0x7f63641f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:12:58.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.718+0000 7f6368bff700 1 -- 172.21.15.156:0/282310024 >> 172.21.15.156:0/282310024 conn(0x7f63640cb670 msgr2=0x7f63640712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:12:58.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.718+0000 7f6368bff700 1 -- 172.21.15.156:0/282310024 shutdown_connections 2023-11-28T13:12:58.720 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:12:58.718+0000 7f6368bff700 1 -- 172.21.15.156:0/282310024 wait complete. 2023-11-28T13:12:58.738 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:13:03.739 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:13:03.889 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:13:03.896 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:13:03.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.952+0000 7f2ddadff700 1 Processor -- start 2023-11-28T13:13:03.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.952+0000 7f2ddadff700 1 -- start start 2023-11-28T13:13:03.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.953+0000 7f2ddadff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2dd415c9b0 0x7f2dd40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:03.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.953+0000 7f2ddadff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd40cd7c0 0x7f2dd40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:03.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.953+0000 7f2ddadff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd40d1950 con 0x7f2dd4161340 2023-11-28T13:13:03.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.953+0000 7f2ddadff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd40b5800 con 0x7f2dd415c9b0 2023-11-28T13:13:03.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.953+0000 7f2ddadff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd40ce140 con 0x7f2dd40cd7c0 2023-11-28T13:13:03.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.954+0000 7f2dd8b9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2dd415c9b0 0x7f2dd40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:03.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.954+0000 7f2dd8b9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2dd415c9b0 0x7f2dd40b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39928/0 (socket says 172.21.15.156:39928) 2023-11-28T13:13:03.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.954+0000 7f2dd8b9b700 1 -- 172.21.15.156:0/109360167 learned_addr learned my addr 172.21.15.156:0/109360167 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:13:03.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.954+0000 7f2dd939c700 1 --2- 172.21.15.156:0/109360167 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd40cd7c0 0x7f2dd40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:03.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.954+0000 7f2dd8b9b700 1 -- 172.21.15.156:0/109360167 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd40cd7c0 msgr2=0x7f2dd40cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:13:03.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.954+0000 7f2dd8b9b700 1 --2- 172.21.15.156:0/109360167 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd40cd7c0 0x7f2dd40cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:03.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.954+0000 7f2dd8b9b700 1 -- 172.21.15.156:0/109360167 >> v1:172.21.15.186:6789/0 conn(0x7f2dd4161340 legacy=0x7f2dd40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:03.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.954+0000 7f2dd8b9b700 1 -- 172.21.15.156:0/109360167 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd40d24f0 con 0x7f2dd415c9b0 2023-11-28T13:13:03.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.955+0000 7f2dd8b9b700 1 --2- 172.21.15.156:0/109360167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd415c9b0 0x7f2dd40b4bf0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80098f0 tx=0x7f2dc8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3ed64205cf6e68 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:03.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.955+0000 7f2dd37fe700 1 -- 172.21.15.156:0/109360167 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8011830 con 0x7f2dd415c9b0 2023-11-28T13:13:03.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.955+0000 7f2dd37fe700 1 -- 172.21.15.156:0/109360167 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc8011990 con 0x7f2dd415c9b0 2023-11-28T13:13:03.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.955+0000 7f2dd37fe700 1 -- 172.21.15.156:0/109360167 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8011d20 con 0x7f2dd415c9b0 2023-11-28T13:13:03.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.955+0000 7f2ddadff700 1 -- 172.21.15.156:0/109360167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd415c9b0 msgr2=0x7f2dd40b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:03.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.956+0000 7f2ddadff700 1 --2- 172.21.15.156:0/109360167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd415c9b0 0x7f2dd40b4bf0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80098f0 tx=0x7f2dc8009c00 comp rx=0 tx=0).stop 2023-11-28T13:13:03.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.956+0000 7f2ddadff700 1 -- 172.21.15.156:0/109360167 shutdown_connections 2023-11-28T13:13:03.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.956+0000 7f2ddadff700 1 --2- 172.21.15.156:0/109360167 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd40cd7c0 0x7f2dd40cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:03.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.956+0000 7f2ddadff700 1 --2- 172.21.15.156:0/109360167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd415c9b0 0x7f2dd40b4bf0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:03.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.956+0000 7f2ddadff700 1 -- 172.21.15.156:0/109360167 >> 172.21.15.156:0/109360167 conn(0x7f2dd4072600 msgr2=0x7f2dd40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:03.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.956+0000 7f2ddadff700 1 -- 172.21.15.156:0/109360167 shutdown_connections 2023-11-28T13:13:03.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.956+0000 7f2ddadff700 1 -- 172.21.15.156:0/109360167 wait complete. 2023-11-28T13:13:03.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.958+0000 7f2ddadff700 1 Processor -- start 2023-11-28T13:13:03.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.958+0000 7f2ddadff700 1 -- start start 2023-11-28T13:13:03.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.959+0000 7f2ddadff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd40cd7c0 0x7f2dd4147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:03.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.959+0000 7f2ddadff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd415c9b0 0x7f2dd41e6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:03.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.959+0000 7f2ddadff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dd4161340 0x7f2dd4159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:03.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.959+0000 7f2ddadff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd40c6780 con 0x7f2dd40cd7c0 2023-11-28T13:13:03.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.959+0000 7f2ddadff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd4147970 con 0x7f2dd4161340 2023-11-28T13:13:03.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.959+0000 7f2ddadff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd41e6d60 con 0x7f2dd415c9b0 2023-11-28T13:13:03.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.959+0000 7f2dd8b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd40cd7c0 0x7f2dd4147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:03.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.959+0000 7f2dd8b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd40cd7c0 0x7f2dd4147430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39940/0 (socket says 172.21.15.156:39940) 2023-11-28T13:13:03.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.959+0000 7f2dd8b9b700 1 -- 172.21.15.156:0/855704485 learned_addr learned my addr 172.21.15.156:0/855704485 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:13:03.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.959+0000 7f2dd939c700 1 --2- 172.21.15.156:0/855704485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dd4161340 0x7f2dd4159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:03.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.959+0000 7f2dd3fff700 1 --2- 172.21.15.156:0/855704485 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd415c9b0 0x7f2dd41e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:03.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.959+0000 7f2dd8b9b700 1 -- 172.21.15.156:0/855704485 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd415c9b0 msgr2=0x7f2dd41e6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:03.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.959+0000 7f2dd8b9b700 1 --2- 172.21.15.156:0/855704485 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd415c9b0 0x7f2dd41e6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:03.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.959+0000 7f2dd8b9b700 1 -- 172.21.15.156:0/855704485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dd4161340 msgr2=0x7f2dd4159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:03.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.959+0000 7f2dd8b9b700 1 --2- 172.21.15.156:0/855704485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dd4161340 0x7f2dd4159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:03.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.959+0000 7f2dd8b9b700 1 -- 172.21.15.156:0/855704485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc8009550 con 0x7f2dd40cd7c0 2023-11-28T13:13:03.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.960+0000 7f2dd8b9b700 1 --2- 172.21.15.156:0/855704485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd40cd7c0 0x7f2dd4147430 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009c40 tx=0x7f2dc8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:03.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.960+0000 7f2dd17fa700 1 -- 172.21.15.156:0/855704485 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8004510 con 0x7f2dd40cd7c0 2023-11-28T13:13:03.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.960+0000 7f2dd17fa700 1 -- 172.21.15.156:0/855704485 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc801e070 con 0x7f2dd40cd7c0 2023-11-28T13:13:03.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.960+0000 7f2dd17fa700 1 -- 172.21.15.156:0/855704485 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc80038e0 con 0x7f2dd40cd7c0 2023-11-28T13:13:03.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.960+0000 7f2ddadff700 1 -- 172.21.15.156:0/855704485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd40d1300 con 0x7f2dd40cd7c0 2023-11-28T13:13:03.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.960+0000 7f2ddadff700 1 -- 172.21.15.156:0/855704485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd40b5520 con 0x7f2dd40cd7c0 2023-11-28T13:13:03.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.962+0000 7f2ddadff700 1 -- 172.21.15.156:0/855704485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd40d15e0 con 0x7f2dd40cd7c0 2023-11-28T13:13:03.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.962+0000 7f2dd17fa700 1 -- 172.21.15.156:0/855704485 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2dc800f050 con 0x7f2dd40cd7c0 2023-11-28T13:13:03.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.963+0000 7f2dd17fa700 1 --2- 172.21.15.156:0/855704485 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2db80682f0 0x7f2db806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:03.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.964+0000 7f2dd17fa700 1 -- 172.21.15.156:0/855704485 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f2dc8086c80 con 0x7f2dd40cd7c0 2023-11-28T13:13:03.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.964+0000 7f2dd3fff700 1 --2- 172.21.15.156:0/855704485 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2db80682f0 0x7f2db806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:03.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.965+0000 7f2dd3fff700 1 --2- 172.21.15.156:0/855704485 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2db80682f0 0x7f2db806a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2dd40c9a50 tx=0x7f2dc0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:03.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:03.967+0000 7f2dd17fa700 1 -- 172.21.15.156:0/855704485 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2dc80532e0 con 0x7f2dd40cd7c0 2023-11-28T13:13:04.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:04.200+0000 7f2ddadff700 1 -- 172.21.15.156:0/855704485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2dd40745c0 con 0x7f2dd40cd7c0 2023-11-28T13:13:04.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:04.201+0000 7f2dd17fa700 1 -- 172.21.15.156:0/855704485 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f2dc80200c0 con 0x7f2dd40cd7c0 2023-11-28T13:13:04.202 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:13:04.202 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:13:04.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:04.203+0000 7f2ddadff700 1 -- 172.21.15.156:0/855704485 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2db80682f0 msgr2=0x7f2db806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:04.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:04.203+0000 7f2ddadff700 1 --2- 172.21.15.156:0/855704485 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2db80682f0 0x7f2db806a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2dd40c9a50 tx=0x7f2dc0009040 comp rx=0 tx=0).stop 2023-11-28T13:13:04.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:04.203+0000 7f2ddadff700 1 -- 172.21.15.156:0/855704485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd40cd7c0 msgr2=0x7f2dd4147430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:04.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:04.203+0000 7f2ddadff700 1 --2- 172.21.15.156:0/855704485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd40cd7c0 0x7f2dd4147430 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009c40 tx=0x7f2dc8004750 comp rx=0 tx=0).stop 2023-11-28T13:13:04.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:04.203+0000 7f2ddadff700 1 -- 172.21.15.156:0/855704485 shutdown_connections 2023-11-28T13:13:04.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:04.203+0000 7f2ddadff700 1 --2- 172.21.15.156:0/855704485 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f2db80682f0 0x7f2db806a7a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:04.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:04.203+0000 7f2ddadff700 1 --2- 172.21.15.156:0/855704485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dd4161340 0x7f2dd4159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:04.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:04.203+0000 7f2ddadff700 1 --2- 172.21.15.156:0/855704485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd40cd7c0 0x7f2dd4147430 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:04.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:04.203+0000 7f2ddadff700 1 --2- 172.21.15.156:0/855704485 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2dd415c9b0 0x7f2dd41e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:04.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:04.203+0000 7f2ddadff700 1 -- 172.21.15.156:0/855704485 >> 172.21.15.156:0/855704485 conn(0x7f2dd4072600 msgr2=0x7f2dd40b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:04.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:04.203+0000 7f2ddadff700 1 -- 172.21.15.156:0/855704485 shutdown_connections 2023-11-28T13:13:04.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:04.203+0000 7f2ddadff700 1 -- 172.21.15.156:0/855704485 wait complete. 2023-11-28T13:13:04.223 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:13:09.224 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:13:09.429 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.428+0000 7f32467ff700 1 Processor -- start 2023-11-28T13:13:09.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.428+0000 7f32467ff700 1 -- start start 2023-11-28T13:13:09.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.429+0000 7f32467ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f324015c9b0 0x7f32400b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:09.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.429+0000 7f32467ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3240161340 0x7f32400d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:09.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.429+0000 7f32467ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32400d1950 con 0x7f32400d2320 2023-11-28T13:13:09.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.429+0000 7f32467ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32400b5800 con 0x7f324015c9b0 2023-11-28T13:13:09.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.429+0000 7f32467ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32400b5520 con 0x7f3240161340 2023-11-28T13:13:09.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.429+0000 7f323ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f324015c9b0 0x7f32400b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:09.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.429+0000 7f323ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f324015c9b0 0x7f32400b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34028/0 (socket says 172.21.15.156:34028) 2023-11-28T13:13:09.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.429+0000 7f323ffff700 1 -- 172.21.15.156:0/2289432880 learned_addr learned my addr 172.21.15.156:0/2289432880 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:13:09.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.430+0000 7f323f7fe700 1 --2- 172.21.15.156:0/2289432880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3240161340 0x7f32400d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:09.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.430+0000 7f323ffff700 1 -- 172.21.15.156:0/2289432880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3240161340 msgr2=0x7f32400d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:13:09.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.430+0000 7f323ffff700 1 --2- 172.21.15.156:0/2289432880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3240161340 0x7f32400d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:09.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.430+0000 7f323ffff700 1 -- 172.21.15.156:0/2289432880 >> v1:172.21.15.186:6789/0 conn(0x7f32400d2320 legacy=0x7f32400cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:09.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.430+0000 7f323ffff700 1 -- 172.21.15.156:0/2289432880 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32400d1300 con 0x7f324015c9b0 2023-11-28T13:13:09.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.430+0000 7f323ffff700 1 --2- 172.21.15.156:0/2289432880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f324015c9b0 0x7f32400b4bf0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f32300098f0 tx=0x7f3230009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d3a6a60775c91c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:09.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.430+0000 7f323effd700 1 -- 172.21.15.156:0/2289432880 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323000b8a0 con 0x7f324015c9b0 2023-11-28T13:13:09.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.430+0000 7f323effd700 1 -- 172.21.15.156:0/2289432880 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323000ba00 con 0x7f324015c9b0 2023-11-28T13:13:09.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.430+0000 7f323effd700 1 -- 172.21.15.156:0/2289432880 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323000bd90 con 0x7f324015c9b0 2023-11-28T13:13:09.433 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.431+0000 7f32467ff700 1 -- 172.21.15.156:0/2289432880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f324015c9b0 msgr2=0x7f32400b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:09.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.431+0000 7f32467ff700 1 --2- 172.21.15.156:0/2289432880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f324015c9b0 0x7f32400b4bf0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f32300098f0 tx=0x7f3230009c00 comp rx=0 tx=0).stop 2023-11-28T13:13:09.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.431+0000 7f32467ff700 1 -- 172.21.15.156:0/2289432880 shutdown_connections 2023-11-28T13:13:09.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.431+0000 7f32467ff700 1 --2- 172.21.15.156:0/2289432880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f324015c9b0 0x7f32400b4bf0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:09.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.431+0000 7f32467ff700 1 --2- 172.21.15.156:0/2289432880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3240161340 0x7f32400d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:09.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.431+0000 7f32467ff700 1 -- 172.21.15.156:0/2289432880 >> 172.21.15.156:0/2289432880 conn(0x7f3240072600 msgr2=0x7f32400cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:09.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.431+0000 7f32467ff700 1 -- 172.21.15.156:0/2289432880 shutdown_connections 2023-11-28T13:13:09.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.431+0000 7f32467ff700 1 -- 172.21.15.156:0/2289432880 wait complete. 2023-11-28T13:13:09.434 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.433+0000 7f32467ff700 1 Processor -- start 2023-11-28T13:13:09.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.433+0000 7f32467ff700 1 -- start start 2023-11-28T13:13:09.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.434+0000 7f32467ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32400d2320 0x7f3240153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:09.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.434+0000 7f32467ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f324015c9b0 0x7f324014e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:09.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.434+0000 7f32467ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3240161340 0x7f32401516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:09.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.434+0000 7f32467ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3240151be0 con 0x7f324015c9b0 2023-11-28T13:13:09.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.434+0000 7f32467ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32400c6dd0 con 0x7f32400d2320 2023-11-28T13:13:09.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.434+0000 7f32467ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3240151e10 con 0x7f3240161340 2023-11-28T13:13:09.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.434+0000 7f323f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f324015c9b0 0x7f324014e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:09.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.434+0000 7f323f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f324015c9b0 0x7f324014e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34038/0 (socket says 172.21.15.156:34038) 2023-11-28T13:13:09.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.434+0000 7f323f7fe700 1 -- 172.21.15.156:0/2404933904 learned_addr learned my addr 172.21.15.156:0/2404933904 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:13:09.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.434+0000 7f323ffff700 1 --2- 172.21.15.156:0/2404933904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32400d2320 0x7f3240153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:09.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.434+0000 7f3244d9c700 1 --2- 172.21.15.156:0/2404933904 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3240161340 0x7f32401516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:09.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.434+0000 7f323f7fe700 1 -- 172.21.15.156:0/2404933904 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3240161340 msgr2=0x7f32401516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:09.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.434+0000 7f323f7fe700 1 --2- 172.21.15.156:0/2404933904 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3240161340 0x7f32401516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:09.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.434+0000 7f323f7fe700 1 -- 172.21.15.156:0/2404933904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32400d2320 msgr2=0x7f3240153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:09.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.434+0000 7f323f7fe700 1 --2- 172.21.15.156:0/2404933904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32400d2320 0x7f3240153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:09.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.435+0000 7f323f7fe700 1 -- 172.21.15.156:0/2404933904 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3230009550 con 0x7f324015c9b0 2023-11-28T13:13:09.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.435+0000 7f3244d9c700 1 --2- 172.21.15.156:0/2404933904 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3240161340 0x7f32401516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:13:09.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.435+0000 7f323f7fe700 1 --2- 172.21.15.156:0/2404933904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f324015c9b0 0x7f324014e8e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f323400e810 tx=0x7f323400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:09.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.435+0000 7f323cff9700 1 -- 172.21.15.156:0/2404933904 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234013610 con 0x7f324015c9b0 2023-11-28T13:13:09.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.435+0000 7f323cff9700 1 -- 172.21.15.156:0/2404933904 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3234013770 con 0x7f324015c9b0 2023-11-28T13:13:09.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.435+0000 7f32467ff700 1 -- 172.21.15.156:0/2404933904 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32400b5130 con 0x7f324015c9b0 2023-11-28T13:13:09.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.435+0000 7f323cff9700 1 -- 172.21.15.156:0/2404933904 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234013b20 con 0x7f324015c9b0 2023-11-28T13:13:09.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.435+0000 7f32467ff700 1 -- 172.21.15.156:0/2404933904 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32400d15e0 con 0x7f324015c9b0 2023-11-28T13:13:09.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.436+0000 7f32467ff700 1 -- 172.21.15.156:0/2404933904 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32400b52c0 con 0x7f324015c9b0 2023-11-28T13:13:09.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.437+0000 7f323cff9700 1 -- 172.21.15.156:0/2404933904 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3234023070 con 0x7f324015c9b0 2023-11-28T13:13:09.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.438+0000 7f323cff9700 1 --2- 172.21.15.156:0/2404933904 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f322c0682f0 0x7f322c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:09.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.438+0000 7f323cff9700 1 -- 172.21.15.156:0/2404933904 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f3234085a50 con 0x7f324015c9b0 2023-11-28T13:13:09.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.438+0000 7f323ffff700 1 --2- 172.21.15.156:0/2404933904 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f322c0682f0 0x7f322c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:09.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.439+0000 7f323ffff700 1 --2- 172.21.15.156:0/2404933904 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f322c0682f0 0x7f322c06a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3230014040 tx=0x7f3230008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:09.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.442+0000 7f323cff9700 1 -- 172.21.15.156:0/2404933904 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f32340520b0 con 0x7f324015c9b0 2023-11-28T13:13:09.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.678+0000 7f32467ff700 1 -- 172.21.15.156:0/2404933904 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3240067b70 con 0x7f324015c9b0 2023-11-28T13:13:09.680 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.678+0000 7f323cff9700 1 -- 172.21.15.156:0/2404933904 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f32340556d0 con 0x7f324015c9b0 2023-11-28T13:13:09.680 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:13:09.680 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:13:09.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.680+0000 7f32467ff700 1 -- 172.21.15.156:0/2404933904 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f322c0682f0 msgr2=0x7f322c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:09.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.680+0000 7f32467ff700 1 --2- 172.21.15.156:0/2404933904 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f322c0682f0 0x7f322c06a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3230014040 tx=0x7f3230008030 comp rx=0 tx=0).stop 2023-11-28T13:13:09.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.680+0000 7f32467ff700 1 -- 172.21.15.156:0/2404933904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f324015c9b0 msgr2=0x7f324014e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.680+0000 7f32467ff700 1 --2- 172.21.15.156:0/2404933904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f324015c9b0 0x7f324014e8e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f323400e810 tx=0x7f323400eb20 comp rx=0 tx=0).stop 2023-11-28T13:13:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.680+0000 7f32467ff700 1 -- 172.21.15.156:0/2404933904 shutdown_connections 2023-11-28T13:13:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.680+0000 7f32467ff700 1 --2- 172.21.15.156:0/2404933904 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f322c0682f0 0x7f322c06a7a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.680+0000 7f32467ff700 1 --2- 172.21.15.156:0/2404933904 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3240161340 0x7f32401516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.680+0000 7f32467ff700 1 --2- 172.21.15.156:0/2404933904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32400d2320 0x7f3240153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.680+0000 7f32467ff700 1 --2- 172.21.15.156:0/2404933904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f324015c9b0 0x7f324014e8e0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.680+0000 7f32467ff700 1 -- 172.21.15.156:0/2404933904 >> 172.21.15.156:0/2404933904 conn(0x7f3240072600 msgr2=0x7f32400b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.680+0000 7f32467ff700 1 -- 172.21.15.156:0/2404933904 shutdown_connections 2023-11-28T13:13:09.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:09.680+0000 7f32467ff700 1 -- 172.21.15.156:0/2404933904 wait complete. 2023-11-28T13:13:09.700 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:13:14.701 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:13:16.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.289+0000 7f42509ff700 1 Processor -- start 2023-11-28T13:13:16.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.290+0000 7f42509ff700 1 -- start start 2023-11-28T13:13:16.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.291+0000 7f42509ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f424c15c9b0 0x7f424c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:16.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.291+0000 7f42509ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f424c0cd7c0 0x7f424c0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:16.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.291+0000 7f42509ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f424c0ce160 con 0x7f424c161340 2023-11-28T13:13:16.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.291+0000 7f42509ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f424c0b5800 con 0x7f424c0cd7c0 2023-11-28T13:13:16.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.291+0000 7f42509ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424c0d1950 con 0x7f424c15c9b0 2023-11-28T13:13:16.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.291+0000 7f424ad9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f424c0cd7c0 0x7f424c0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:16.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.291+0000 7f424ad9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f424c0cd7c0 0x7f424c0cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:55346/0 (socket says 172.21.15.156:55346) 2023-11-28T13:13:16.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.291+0000 7f424ad9d700 1 -- 172.21.15.156:0/2475392077 learned_addr learned my addr 172.21.15.156:0/2475392077 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:13:16.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.291+0000 7f424a59c700 1 --2- 172.21.15.156:0/2475392077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f424c15c9b0 0x7f424c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:16.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.292+0000 7f424ad9d700 1 -- 172.21.15.156:0/2475392077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f424c15c9b0 msgr2=0x7f424c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:13:16.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.292+0000 7f424ad9d700 1 --2- 172.21.15.156:0/2475392077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f424c15c9b0 0x7f424c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:16.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.292+0000 7f424ad9d700 1 -- 172.21.15.156:0/2475392077 >> v1:172.21.15.186:6789/0 conn(0x7f424c161340 legacy=0x7f424c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:16.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.292+0000 7f424ad9d700 1 -- 172.21.15.156:0/2475392077 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424c0d24f0 con 0x7f424c0cd7c0 2023-11-28T13:13:16.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.292+0000 7f424ad9d700 1 --2- 172.21.15.156:0/2475392077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c0cd7c0 0x7f424c0cdc20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4244009e50 tx=0x7f424400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828d295df8fb60a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:16.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.292+0000 7f4249d9b700 1 -- 172.21.15.156:0/2475392077 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424400e070 con 0x7f424c0cd7c0 2023-11-28T13:13:16.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.292+0000 7f4249d9b700 1 -- 172.21.15.156:0/2475392077 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424400b910 con 0x7f424c0cd7c0 2023-11-28T13:13:16.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.292+0000 7f4249d9b700 1 -- 172.21.15.156:0/2475392077 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424400bcc0 con 0x7f424c0cd7c0 2023-11-28T13:13:16.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.293+0000 7f42509ff700 1 -- 172.21.15.156:0/2475392077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c0cd7c0 msgr2=0x7f424c0cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:16.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.293+0000 7f42509ff700 1 --2- 172.21.15.156:0/2475392077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c0cd7c0 0x7f424c0cdc20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4244009e50 tx=0x7f424400b330 comp rx=0 tx=0).stop 2023-11-28T13:13:16.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.293+0000 7f42509ff700 1 -- 172.21.15.156:0/2475392077 shutdown_connections 2023-11-28T13:13:16.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.293+0000 7f42509ff700 1 --2- 172.21.15.156:0/2475392077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c0cd7c0 0x7f424c0cdc20 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:16.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.293+0000 7f42509ff700 1 --2- 172.21.15.156:0/2475392077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f424c15c9b0 0x7f424c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:16.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.293+0000 7f42509ff700 1 -- 172.21.15.156:0/2475392077 >> 172.21.15.156:0/2475392077 conn(0x7f424c072600 msgr2=0x7f424c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:16.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.293+0000 7f42509ff700 1 -- 172.21.15.156:0/2475392077 shutdown_connections 2023-11-28T13:13:16.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.293+0000 7f42509ff700 1 -- 172.21.15.156:0/2475392077 wait complete. 2023-11-28T13:13:16.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.295+0000 7f42509ff700 1 Processor -- start 2023-11-28T13:13:16.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.295+0000 7f42509ff700 1 -- start start 2023-11-28T13:13:16.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.296+0000 7f42509ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f424c0cd7c0 0x7f424c148d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:16.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.296+0000 7f42509ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c15c9b0 0x7f424c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:16.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.296+0000 7f42509ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f424c161340 0x7f424c0b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:16.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.296+0000 7f42509ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424c1f6300 con 0x7f424c15c9b0 2023-11-28T13:13:16.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.296+0000 7f42509ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424c14c660 con 0x7f424c161340 2023-11-28T13:13:16.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.296+0000 7f42509ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424c149250 con 0x7f424c0cd7c0 2023-11-28T13:13:16.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.296+0000 7f4243fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c15c9b0 0x7f424c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:16.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.296+0000 7f4243fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c15c9b0 0x7f424c159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:55360/0 (socket says 172.21.15.156:55360) 2023-11-28T13:13:16.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.296+0000 7f4243fff700 1 -- 172.21.15.156:0/691796813 learned_addr learned my addr 172.21.15.156:0/691796813 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:13:16.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.296+0000 7f424a59c700 1 --2- 172.21.15.156:0/691796813 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f424c0cd7c0 0x7f424c148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:16.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.296+0000 7f424ad9d700 1 --2- 172.21.15.156:0/691796813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f424c161340 0x7f424c0b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:16.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.296+0000 7f4243fff700 1 -- 172.21.15.156:0/691796813 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f424c0cd7c0 msgr2=0x7f424c148d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:16.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.296+0000 7f4243fff700 1 --2- 172.21.15.156:0/691796813 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f424c0cd7c0 0x7f424c148d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:16.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.296+0000 7f4243fff700 1 -- 172.21.15.156:0/691796813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f424c161340 msgr2=0x7f424c0b7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:16.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.296+0000 7f4243fff700 1 --2- 172.21.15.156:0/691796813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f424c161340 0x7f424c0b7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:16.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.296+0000 7f4243fff700 1 -- 172.21.15.156:0/691796813 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4244009b30 con 0x7f424c15c9b0 2023-11-28T13:13:16.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.297+0000 7f4243fff700 1 --2- 172.21.15.156:0/691796813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c15c9b0 0x7f424c159ce0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f423400c450 tx=0x7f423400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:16.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.297+0000 7f4242ffd700 1 -- 172.21.15.156:0/691796813 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42340052d0 con 0x7f424c15c9b0 2023-11-28T13:13:16.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.297+0000 7f4242ffd700 1 -- 172.21.15.156:0/691796813 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4234005430 con 0x7f424c15c9b0 2023-11-28T13:13:16.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.297+0000 7f42509ff700 1 -- 172.21.15.156:0/691796813 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f424c0d1300 con 0x7f424c15c9b0 2023-11-28T13:13:16.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.297+0000 7f4242ffd700 1 -- 172.21.15.156:0/691796813 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42340038c0 con 0x7f424c15c9b0 2023-11-28T13:13:16.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.297+0000 7f42509ff700 1 -- 172.21.15.156:0/691796813 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f424c0b5520 con 0x7f424c15c9b0 2023-11-28T13:13:16.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.298+0000 7f42509ff700 1 -- 172.21.15.156:0/691796813 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f424c0d15e0 con 0x7f424c15c9b0 2023-11-28T13:13:16.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.299+0000 7f4242ffd700 1 -- 172.21.15.156:0/691796813 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4234019030 con 0x7f424c15c9b0 2023-11-28T13:13:16.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.300+0000 7f4242ffd700 1 --2- 172.21.15.156:0/691796813 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f422c0682f0 0x7f422c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:16.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.300+0000 7f4242ffd700 1 -- 172.21.15.156:0/691796813 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f4234084fb0 con 0x7f424c15c9b0 2023-11-28T13:13:16.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.300+0000 7f424a59c700 1 --2- 172.21.15.156:0/691796813 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f422c0682f0 0x7f422c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:16.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.301+0000 7f424a59c700 1 --2- 172.21.15.156:0/691796813 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f422c0682f0 0x7f422c06a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f423c009b40 tx=0x7f423c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:16.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.304+0000 7f4242ffd700 1 -- 172.21.15.156:0/691796813 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4234051690 con 0x7f424c15c9b0 2023-11-28T13:13:16.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.537+0000 7f42509ff700 1 -- 172.21.15.156:0/691796813 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f424c0745c0 con 0x7f424c15c9b0 2023-11-28T13:13:16.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.538+0000 7f4242ffd700 1 -- 172.21.15.156:0/691796813 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f42340170c0 con 0x7f424c15c9b0 2023-11-28T13:13:16.540 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:13:16.541 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:13:16.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.540+0000 7f42509ff700 1 -- 172.21.15.156:0/691796813 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f422c0682f0 msgr2=0x7f422c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:16.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.540+0000 7f42509ff700 1 --2- 172.21.15.156:0/691796813 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f422c0682f0 0x7f422c06a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f423c009b40 tx=0x7f423c009210 comp rx=0 tx=0).stop 2023-11-28T13:13:16.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.540+0000 7f42509ff700 1 -- 172.21.15.156:0/691796813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c15c9b0 msgr2=0x7f424c159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:16.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.540+0000 7f42509ff700 1 --2- 172.21.15.156:0/691796813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c15c9b0 0x7f424c159ce0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f423400c450 tx=0x7f423400c760 comp rx=0 tx=0).stop 2023-11-28T13:13:16.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.540+0000 7f42509ff700 1 -- 172.21.15.156:0/691796813 shutdown_connections 2023-11-28T13:13:16.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.540+0000 7f42509ff700 1 --2- 172.21.15.156:0/691796813 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f422c0682f0 0x7f422c06a7a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:16.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.540+0000 7f42509ff700 1 --2- 172.21.15.156:0/691796813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f424c161340 0x7f424c0b7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:16.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.540+0000 7f42509ff700 1 --2- 172.21.15.156:0/691796813 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f424c0cd7c0 0x7f424c148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:16.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.540+0000 7f42509ff700 1 --2- 172.21.15.156:0/691796813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f424c15c9b0 0x7f424c159ce0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:16.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.540+0000 7f42509ff700 1 -- 172.21.15.156:0/691796813 >> 172.21.15.156:0/691796813 conn(0x7f424c072600 msgr2=0x7f424c0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:16.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.540+0000 7f42509ff700 1 -- 172.21.15.156:0/691796813 shutdown_connections 2023-11-28T13:13:16.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:16.540+0000 7f42509ff700 1 -- 172.21.15.156:0/691796813 wait complete. 2023-11-28T13:13:16.561 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:13:21.565 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:13:21.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.769+0000 7f82a73ff700 1 Processor -- start 2023-11-28T13:13:21.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.769+0000 7f82a73ff700 1 -- start start 2023-11-28T13:13:21.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.770+0000 7f82a73ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f82a015c9b0 0x7f82a00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:21.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.770+0000 7f82a73ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f82a0161340 0x7f82a00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:21.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.770+0000 7f82a73ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a00d1950 con 0x7f82a00d2320 2023-11-28T13:13:21.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.770+0000 7f82a73ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a00b5800 con 0x7f82a0161340 2023-11-28T13:13:21.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.770+0000 7f82a73ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a00b5520 con 0x7f82a015c9b0 2023-11-28T13:13:21.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.770+0000 7f82a499a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f82a0161340 0x7f82a00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:21.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.770+0000 7f82a499a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f82a0161340 0x7f82a00d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:55372/0 (socket says 172.21.15.156:55372) 2023-11-28T13:13:21.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.770+0000 7f82a499a700 1 -- 172.21.15.156:0/2036341480 learned_addr learned my addr 172.21.15.156:0/2036341480 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:13:21.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.770+0000 7f82a519b700 1 --2- 172.21.15.156:0/2036341480 >> v2:172.21.15.186:3300/0 conn(0x7f82a015c9b0 0x7f82a00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:21.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.770+0000 7f82a499a700 1 -- 172.21.15.156:0/2036341480 >> v2:172.21.15.186:3300/0 conn(0x7f82a015c9b0 msgr2=0x7f82a00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:13:21.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.770+0000 7f82a499a700 1 --2- 172.21.15.156:0/2036341480 >> v2:172.21.15.186:3300/0 conn(0x7f82a015c9b0 0x7f82a00b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:21.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.771+0000 7f82a499a700 1 -- 172.21.15.156:0/2036341480 >> v1:172.21.15.186:6789/0 conn(0x7f82a00d2320 legacy=0x7f82a00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:21.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.771+0000 7f82a499a700 1 -- 172.21.15.156:0/2036341480 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a00d1300 con 0x7f82a0161340 2023-11-28T13:13:21.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.771+0000 7f82a499a700 1 --2- 172.21.15.156:0/2036341480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a0161340 0x7f82a00d1de0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f82a0067460 tx=0x7f8290009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce719677cd0b116 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:21.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.771+0000 7f8297fff700 1 -- 172.21.15.156:0/2036341480 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829000b870 con 0x7f82a0161340 2023-11-28T13:13:21.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.771+0000 7f8297fff700 1 -- 172.21.15.156:0/2036341480 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829000b9d0 con 0x7f82a0161340 2023-11-28T13:13:21.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.771+0000 7f8297fff700 1 -- 172.21.15.156:0/2036341480 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829000bd60 con 0x7f82a0161340 2023-11-28T13:13:21.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.772+0000 7f82a73ff700 1 -- 172.21.15.156:0/2036341480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a0161340 msgr2=0x7f82a00d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:21.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.772+0000 7f82a73ff700 1 --2- 172.21.15.156:0/2036341480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a0161340 0x7f82a00d1de0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f82a0067460 tx=0x7f8290009bd0 comp rx=0 tx=0).stop 2023-11-28T13:13:21.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.772+0000 7f82a73ff700 1 -- 172.21.15.156:0/2036341480 shutdown_connections 2023-11-28T13:13:21.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.772+0000 7f82a73ff700 1 --2- 172.21.15.156:0/2036341480 >> v2:172.21.15.186:3300/0 conn(0x7f82a015c9b0 0x7f82a00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:21.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.772+0000 7f82a73ff700 1 --2- 172.21.15.156:0/2036341480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a0161340 0x7f82a00d1de0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:21.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.772+0000 7f82a73ff700 1 -- 172.21.15.156:0/2036341480 >> 172.21.15.156:0/2036341480 conn(0x7f82a0072600 msgr2=0x7f82a00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:21.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.772+0000 7f82a73ff700 1 -- 172.21.15.156:0/2036341480 shutdown_connections 2023-11-28T13:13:21.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.772+0000 7f82a73ff700 1 -- 172.21.15.156:0/2036341480 wait complete. 2023-11-28T13:13:21.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.773+0000 7f82a73ff700 1 Processor -- start 2023-11-28T13:13:21.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.774+0000 7f82a73ff700 1 -- start start 2023-11-28T13:13:21.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.774+0000 7f82a73ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82a00d2320 0x7f82a0143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:21.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.775+0000 7f82a73ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82a015c9b0 0x7f82a00c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:21.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.775+0000 7f82a73ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a0161340 0x7f82a00c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:21.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.775+0000 7f82a73ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a0153580 con 0x7f82a0161340 2023-11-28T13:13:21.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.775+0000 7f82a73ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00c6170 con 0x7f82a015c9b0 2023-11-28T13:13:21.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.775+0000 7f82a73ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00c7680 con 0x7f82a00d2320 2023-11-28T13:13:21.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.775+0000 7f82a599c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a0161340 0x7f82a00c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:21.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.775+0000 7f82a599c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a0161340 0x7f82a00c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:55384/0 (socket says 172.21.15.156:55384) 2023-11-28T13:13:21.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.775+0000 7f82a599c700 1 -- 172.21.15.156:0/3876795453 learned_addr learned my addr 172.21.15.156:0/3876795453 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:13:21.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.775+0000 7f82a519b700 1 --2- 172.21.15.156:0/3876795453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82a00d2320 0x7f82a0143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:21.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.775+0000 7f82a499a700 1 --2- 172.21.15.156:0/3876795453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82a015c9b0 0x7f82a00c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:21.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.775+0000 7f82a599c700 1 -- 172.21.15.156:0/3876795453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82a00d2320 msgr2=0x7f82a0143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:21.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.775+0000 7f82a599c700 1 --2- 172.21.15.156:0/3876795453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82a00d2320 0x7f82a0143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:21.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.775+0000 7f82a599c700 1 -- 172.21.15.156:0/3876795453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82a015c9b0 msgr2=0x7f82a00c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:21.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.775+0000 7f82a599c700 1 --2- 172.21.15.156:0/3876795453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82a015c9b0 0x7f82a00c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:21.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.775+0000 7f82a599c700 1 -- 172.21.15.156:0/3876795453 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8290009550 con 0x7f82a0161340 2023-11-28T13:13:21.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.776+0000 7f82a599c700 1 --2- 172.21.15.156:0/3876795453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a0161340 0x7f82a00c6ca0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f829800cbc0 tx=0x7f829800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:21.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.776+0000 7f8295ffb700 1 -- 172.21.15.156:0/3876795453 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8298003c00 con 0x7f82a0161340 2023-11-28T13:13:21.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.776+0000 7f8295ffb700 1 -- 172.21.15.156:0/3876795453 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8298003d60 con 0x7f82a0161340 2023-11-28T13:13:21.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.776+0000 7f82a73ff700 1 -- 172.21.15.156:0/3876795453 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a00b5130 con 0x7f82a0161340 2023-11-28T13:13:21.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.776+0000 7f8295ffb700 1 -- 172.21.15.156:0/3876795453 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829800aa20 con 0x7f82a0161340 2023-11-28T13:13:21.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.776+0000 7f82a73ff700 1 -- 172.21.15.156:0/3876795453 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a00d15e0 con 0x7f82a0161340 2023-11-28T13:13:21.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.777+0000 7f82a73ff700 1 -- 172.21.15.156:0/3876795453 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a00b52c0 con 0x7f82a0161340 2023-11-28T13:13:21.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.778+0000 7f8295ffb700 1 -- 172.21.15.156:0/3876795453 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8298010050 con 0x7f82a0161340 2023-11-28T13:13:21.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.779+0000 7f8295ffb700 1 --2- 172.21.15.156:0/3876795453 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f82840682f0 0x7f828406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:21.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.779+0000 7f8295ffb700 1 -- 172.21.15.156:0/3876795453 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f8298085ba0 con 0x7f82a0161340 2023-11-28T13:13:21.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.779+0000 7f82a519b700 1 --2- 172.21.15.156:0/3876795453 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f82840682f0 0x7f828406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:21.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.780+0000 7f82a519b700 1 --2- 172.21.15.156:0/3876795453 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f82840682f0 0x7f828406a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f829c006b20 tx=0x7f829c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:21.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:21.783+0000 7f8295ffb700 1 -- 172.21.15.156:0/3876795453 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8298052280 con 0x7f82a0161340 2023-11-28T13:13:22.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:22.016+0000 7f82a73ff700 1 -- 172.21.15.156:0/3876795453 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f82a00745c0 con 0x7f82a0161340 2023-11-28T13:13:22.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:22.017+0000 7f8295ffb700 1 -- 172.21.15.156:0/3876795453 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f82980558a0 con 0x7f82a0161340 2023-11-28T13:13:22.018 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:13:22.018 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:13:22.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:22.019+0000 7f82a73ff700 1 -- 172.21.15.156:0/3876795453 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f82840682f0 msgr2=0x7f828406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:22.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:22.019+0000 7f82a73ff700 1 --2- 172.21.15.156:0/3876795453 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f82840682f0 0x7f828406a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f829c006b20 tx=0x7f829c008040 comp rx=0 tx=0).stop 2023-11-28T13:13:22.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:22.019+0000 7f82a73ff700 1 -- 172.21.15.156:0/3876795453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a0161340 msgr2=0x7f82a00c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:22.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:22.019+0000 7f82a73ff700 1 --2- 172.21.15.156:0/3876795453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a0161340 0x7f82a00c6ca0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f829800cbc0 tx=0x7f829800ced0 comp rx=0 tx=0).stop 2023-11-28T13:13:22.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:22.019+0000 7f82a73ff700 1 -- 172.21.15.156:0/3876795453 shutdown_connections 2023-11-28T13:13:22.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:22.019+0000 7f82a73ff700 1 --2- 172.21.15.156:0/3876795453 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f82840682f0 0x7f828406a7a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:22.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:22.019+0000 7f82a73ff700 1 --2- 172.21.15.156:0/3876795453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a0161340 0x7f82a00c6ca0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:22.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:22.019+0000 7f82a73ff700 1 --2- 172.21.15.156:0/3876795453 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82a00d2320 0x7f82a0143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:22.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:22.019+0000 7f82a73ff700 1 --2- 172.21.15.156:0/3876795453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82a015c9b0 0x7f82a00c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:22.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:22.019+0000 7f82a73ff700 1 -- 172.21.15.156:0/3876795453 >> 172.21.15.156:0/3876795453 conn(0x7f82a0072600 msgr2=0x7f82a015a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:22.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:22.019+0000 7f82a73ff700 1 -- 172.21.15.156:0/3876795453 shutdown_connections 2023-11-28T13:13:22.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:22.019+0000 7f82a73ff700 1 -- 172.21.15.156:0/3876795453 wait complete. 2023-11-28T13:13:22.039 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:13:27.041 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:13:27.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.245+0000 7f28fdfff700 1 Processor -- start 2023-11-28T13:13:27.247 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.245+0000 7f28fdfff700 1 -- start start 2023-11-28T13:13:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.246+0000 7f28fdfff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f28f815c9b0 0x7f28f80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.246+0000 7f28fdfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f28f80cd7c0 0x7f28f80cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.247+0000 7f28fdfff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f80d1950 con 0x7f28f8161340 2023-11-28T13:13:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.247+0000 7f28fdfff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f80b5800 con 0x7f28f80cd7c0 2023-11-28T13:13:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.247+0000 7f28fdfff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f80ce140 con 0x7f28f815c9b0 2023-11-28T13:13:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.247+0000 7f28f7fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f28f80cd7c0 0x7f28f80cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.247+0000 7f28f7fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f28f80cd7c0 0x7f28f80cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35984/0 (socket says 172.21.15.156:35984) 2023-11-28T13:13:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.247+0000 7f28f7fff700 1 -- 172.21.15.156:0/1579112716 learned_addr learned my addr 172.21.15.156:0/1579112716 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:13:27.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.247+0000 7f28f77fe700 1 --2- 172.21.15.156:0/1579112716 >> v2:172.21.15.186:3300/0 conn(0x7f28f815c9b0 0x7f28f80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:27.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.247+0000 7f28f7fff700 1 -- 172.21.15.156:0/1579112716 >> v2:172.21.15.186:3300/0 conn(0x7f28f815c9b0 msgr2=0x7f28f80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:13:27.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.247+0000 7f28f7fff700 1 --2- 172.21.15.156:0/1579112716 >> v2:172.21.15.186:3300/0 conn(0x7f28f815c9b0 0x7f28f80b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:27.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.247+0000 7f28f7fff700 1 -- 172.21.15.156:0/1579112716 >> v1:172.21.15.186:6789/0 conn(0x7f28f8161340 legacy=0x7f28f80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:27.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.247+0000 7f28f7fff700 1 -- 172.21.15.156:0/1579112716 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f80d24f0 con 0x7f28f80cd7c0 2023-11-28T13:13:27.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.248+0000 7f28f7fff700 1 --2- 172.21.15.156:0/1579112716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f80cd7c0 0x7f28f80cdc00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f28f8067460 tx=0x7f28f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe001c0ffa68234d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:27.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.248+0000 7f28f6ffd700 1 -- 172.21.15.156:0/1579112716 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f000e070 con 0x7f28f80cd7c0 2023-11-28T13:13:27.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.248+0000 7f28f6ffd700 1 -- 172.21.15.156:0/1579112716 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f000b9c0 con 0x7f28f80cd7c0 2023-11-28T13:13:27.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.248+0000 7f28f6ffd700 1 -- 172.21.15.156:0/1579112716 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f000bd70 con 0x7f28f80cd7c0 2023-11-28T13:13:27.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.248+0000 7f28fdfff700 1 -- 172.21.15.156:0/1579112716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f80cd7c0 msgr2=0x7f28f80cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:27.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.248+0000 7f28fdfff700 1 --2- 172.21.15.156:0/1579112716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f80cd7c0 0x7f28f80cdc00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f28f8067460 tx=0x7f28f000b330 comp rx=0 tx=0).stop 2023-11-28T13:13:27.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.248+0000 7f28fdfff700 1 -- 172.21.15.156:0/1579112716 shutdown_connections 2023-11-28T13:13:27.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.248+0000 7f28fdfff700 1 --2- 172.21.15.156:0/1579112716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f80cd7c0 0x7f28f80cdc00 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:27.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.248+0000 7f28fdfff700 1 --2- 172.21.15.156:0/1579112716 >> v2:172.21.15.186:3300/0 conn(0x7f28f815c9b0 0x7f28f80b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:27.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.248+0000 7f28fdfff700 1 -- 172.21.15.156:0/1579112716 >> 172.21.15.156:0/1579112716 conn(0x7f28f8072600 msgr2=0x7f28f80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:27.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.249+0000 7f28fdfff700 1 -- 172.21.15.156:0/1579112716 shutdown_connections 2023-11-28T13:13:27.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.249+0000 7f28fdfff700 1 -- 172.21.15.156:0/1579112716 wait complete. 2023-11-28T13:13:27.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.251+0000 7f28fdfff700 1 Processor -- start 2023-11-28T13:13:27.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.251+0000 7f28fdfff700 1 -- start start 2023-11-28T13:13:27.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.252+0000 7f28fdfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f80cd7c0 0x7f28f8156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:27.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.252+0000 7f28fdfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28f815c9b0 0x7f28f814a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:27.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.252+0000 7f28fdfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28f8161340 0x7f28f80c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:27.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.252+0000 7f28fdfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f814a990 con 0x7f28f80cd7c0 2023-11-28T13:13:27.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.252+0000 7f28fdfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80c7080 con 0x7f28f815c9b0 2023-11-28T13:13:27.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.252+0000 7f28fdfff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80c7360 con 0x7f28f8161340 2023-11-28T13:13:27.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.253+0000 7f28f77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f80cd7c0 0x7f28f8156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:27.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.253+0000 7f28f77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f80cd7c0 0x7f28f8156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35990/0 (socket says 172.21.15.156:35990) 2023-11-28T13:13:27.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.253+0000 7f28f77fe700 1 -- 172.21.15.156:0/2194346195 learned_addr learned my addr 172.21.15.156:0/2194346195 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:13:27.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.253+0000 7f28effff700 1 --2- 172.21.15.156:0/2194346195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28f815c9b0 0x7f28f814a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:27.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.253+0000 7f28f7fff700 1 --2- 172.21.15.156:0/2194346195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28f8161340 0x7f28f80c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:27.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.253+0000 7f28f77fe700 1 -- 172.21.15.156:0/2194346195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28f8161340 msgr2=0x7f28f80c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:27.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.253+0000 7f28f77fe700 1 --2- 172.21.15.156:0/2194346195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28f8161340 0x7f28f80c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:27.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.253+0000 7f28f77fe700 1 -- 172.21.15.156:0/2194346195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28f815c9b0 msgr2=0x7f28f814a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:27.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.253+0000 7f28f77fe700 1 --2- 172.21.15.156:0/2194346195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28f815c9b0 0x7f28f814a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:27.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.253+0000 7f28f77fe700 1 -- 172.21.15.156:0/2194346195 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f0009b30 con 0x7f28f80cd7c0 2023-11-28T13:13:27.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.253+0000 7f28effff700 1 --2- 172.21.15.156:0/2194346195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28f815c9b0 0x7f28f814a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:13:27.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.253+0000 7f28f77fe700 1 --2- 172.21.15.156:0/2194346195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f80cd7c0 0x7f28f8156f90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f28e800d800 tx=0x7f28e800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:27.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.253+0000 7f28f4ff9700 1 -- 172.21.15.156:0/2194346195 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e80125f0 con 0x7f28f80cd7c0 2023-11-28T13:13:27.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.253+0000 7f28f4ff9700 1 -- 172.21.15.156:0/2194346195 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e8012750 con 0x7f28f80cd7c0 2023-11-28T13:13:27.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.253+0000 7f28f4ff9700 1 -- 172.21.15.156:0/2194346195 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8012b00 con 0x7f28f80cd7c0 2023-11-28T13:13:27.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.253+0000 7f28fdfff700 1 -- 172.21.15.156:0/2194346195 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f80d1300 con 0x7f28f80cd7c0 2023-11-28T13:13:27.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.254+0000 7f28fdfff700 1 -- 172.21.15.156:0/2194346195 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f80b5520 con 0x7f28f80cd7c0 2023-11-28T13:13:27.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.255+0000 7f28fdfff700 1 -- 172.21.15.156:0/2194346195 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f80d15e0 con 0x7f28f80cd7c0 2023-11-28T13:13:27.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.255+0000 7f28f4ff9700 1 -- 172.21.15.156:0/2194346195 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f28e8012c60 con 0x7f28f80cd7c0 2023-11-28T13:13:27.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.256+0000 7f28f4ff9700 1 --2- 172.21.15.156:0/2194346195 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f28d80682f0 0x7f28d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:27.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.256+0000 7f28f4ff9700 1 -- 172.21.15.156:0/2194346195 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f28e8086310 con 0x7f28f80cd7c0 2023-11-28T13:13:27.258 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.257+0000 7f28effff700 1 --2- 172.21.15.156:0/2194346195 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f28d80682f0 0x7f28d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:27.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.257+0000 7f28effff700 1 --2- 172.21.15.156:0/2194346195 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f28d80682f0 0x7f28d806a7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f28f80cd790 tx=0x7f28e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:27.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.261+0000 7f28f4ff9700 1 -- 172.21.15.156:0/2194346195 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f28e80529f0 con 0x7f28f80cd7c0 2023-11-28T13:13:27.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.494+0000 7f28fdfff700 1 -- 172.21.15.156:0/2194346195 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f28f80745c0 con 0x7f28f80cd7c0 2023-11-28T13:13:27.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.494+0000 7f28f4ff9700 1 -- 172.21.15.156:0/2194346195 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f28e8056010 con 0x7f28f80cd7c0 2023-11-28T13:13:27.496 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:13:27.496 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:13:27.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.497+0000 7f28fdfff700 1 -- 172.21.15.156:0/2194346195 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f28d80682f0 msgr2=0x7f28d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:27.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.497+0000 7f28fdfff700 1 --2- 172.21.15.156:0/2194346195 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f28d80682f0 0x7f28d806a7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f28f80cd790 tx=0x7f28e0009040 comp rx=0 tx=0).stop 2023-11-28T13:13:27.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.497+0000 7f28fdfff700 1 -- 172.21.15.156:0/2194346195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f80cd7c0 msgr2=0x7f28f8156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.497+0000 7f28fdfff700 1 --2- 172.21.15.156:0/2194346195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f80cd7c0 0x7f28f8156f90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f28e800d800 tx=0x7f28e800dbc0 comp rx=0 tx=0).stop 2023-11-28T13:13:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.497+0000 7f28fdfff700 1 -- 172.21.15.156:0/2194346195 shutdown_connections 2023-11-28T13:13:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.497+0000 7f28fdfff700 1 --2- 172.21.15.156:0/2194346195 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f28d80682f0 0x7f28d806a7a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.497+0000 7f28fdfff700 1 --2- 172.21.15.156:0/2194346195 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28f8161340 0x7f28f80c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.497+0000 7f28fdfff700 1 --2- 172.21.15.156:0/2194346195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f80cd7c0 0x7f28f8156f90 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.497+0000 7f28fdfff700 1 --2- 172.21.15.156:0/2194346195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28f815c9b0 0x7f28f814a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.497+0000 7f28fdfff700 1 -- 172.21.15.156:0/2194346195 >> 172.21.15.156:0/2194346195 conn(0x7f28f8072600 msgr2=0x7f28f8159a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.497+0000 7f28fdfff700 1 -- 172.21.15.156:0/2194346195 shutdown_connections 2023-11-28T13:13:27.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:27.497+0000 7f28fdfff700 1 -- 172.21.15.156:0/2194346195 wait complete. 2023-11-28T13:13:27.517 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:13:32.518 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:13:32.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.723+0000 7f34755ff700 1 Processor -- start 2023-11-28T13:13:32.725 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.723+0000 7f34755ff700 1 -- start start 2023-11-28T13:13:32.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.724+0000 7f34755ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f347006fb30 0x7f3470158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:32.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.725+0000 7f34755ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f347006bf10 con 0x7f347015d1f0 2023-11-28T13:13:32.729 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.725+0000 7f34755ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34700cb0d0 con 0x7f34700cdda0 2023-11-28T13:13:32.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.725+0000 7f34755ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f347006ff00 con 0x7f347006fb30 2023-11-28T13:13:32.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.725+0000 7f3467fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f347015d1f0 0x7f34700b06e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:45536/0 (socket says 172.21.15.156:45536) 2023-11-28T13:13:32.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.725+0000 7f3467fff700 1 -- 172.21.15.156:0/841258923 learned_addr learned my addr 172.21.15.156:0/841258923 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:13:32.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.725+0000 7f346e7fc700 1 -- 172.21.15.156:0/841258923 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189872962 0 0) 0x7f347006bf10 con 0x7f347015d1f0 2023-11-28T13:13:32.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.725+0000 7f346e7fc700 1 -- 172.21.15.156:0/841258923 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f345c0035b0 con 0x7f347015d1f0 2023-11-28T13:13:32.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.725+0000 7f346effd700 1 --2- 172.21.15.156:0/841258923 >> v2:172.21.15.186:3300/0 conn(0x7f347006fb30 0x7f3470158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:32.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.725+0000 7f346e7fc700 1 -- 172.21.15.156:0/841258923 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4151000938 0 0) 0x7f345c0035b0 con 0x7f347015d1f0 2023-11-28T13:13:32.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f346e7fc700 1 -- 172.21.15.156:0/841258923 >> v2:172.21.15.186:3300/0 conn(0x7f347006fb30 msgr2=0x7f3470158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:13:32.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f346e7fc700 1 --2- 172.21.15.156:0/841258923 >> v2:172.21.15.186:3300/0 conn(0x7f347006fb30 0x7f3470158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:32.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f346e7fc700 1 -- 172.21.15.156:0/841258923 >> v1:172.21.15.186:6789/0 conn(0x7f34700cdda0 legacy=0x7f34700ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:32.730 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f346e7fc700 1 -- 172.21.15.156:0/841258923 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34700cb3b0 con 0x7f347015d1f0 2023-11-28T13:13:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f346e7fc700 1 -- 172.21.15.156:0/841258923 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f3458001670 con 0x7f347015d1f0 2023-11-28T13:13:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f346e7fc700 1 -- 172.21.15.156:0/841258923 >> v1:172.21.15.156:6789/0 conn(0x7f347015d1f0 legacy=0x7f34700b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f346e7fc700 1 --2- 172.21.15.156:0/841258923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f345c003a00 0x7f345c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f346e7fc700 1 --2- 172.21.15.156:0/841258923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f345c004850 0x7f345c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f346e7fc700 1 --2- 172.21.15.156:0/841258923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f345c005200 0x7f345c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f346e7fc700 1 -- 172.21.15.156:0/841258923 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347006ff00 con 0x7f345c004850 2023-11-28T13:13:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f346e7fc700 1 -- 172.21.15.156:0/841258923 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c00c080 con 0x7f345c003a00 2023-11-28T13:13:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f346e7fc700 1 -- 172.21.15.156:0/841258923 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345c00c240 con 0x7f345c005200 2023-11-28T13:13:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f3467fff700 1 --2- 172.21.15.156:0/841258923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f345c004850 0x7f345c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f346effd700 1 --2- 172.21.15.156:0/841258923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f345c003a00 0x7f345c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f3467fff700 1 -- 172.21.15.156:0/841258923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f345c005200 msgr2=0x7f345c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:13:32.731 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f346f7fe700 1 --2- 172.21.15.156:0/841258923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f345c005200 0x7f345c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:32.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f3467fff700 1 --2- 172.21.15.156:0/841258923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f345c005200 0x7f345c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:32.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f3467fff700 1 -- 172.21.15.156:0/841258923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f345c003a00 msgr2=0x7f345c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:13:32.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f3467fff700 1 --2- 172.21.15.156:0/841258923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f345c003a00 0x7f345c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:32.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f3467fff700 1 -- 172.21.15.156:0/841258923 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c0035b0 con 0x7f345c004850 2023-11-28T13:13:32.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f346effd700 1 --2- 172.21.15.156:0/841258923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f345c003a00 0x7f345c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:13:32.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f346f7fe700 1 --2- 172.21.15.156:0/841258923 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f345c005200 0x7f345c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:13:32.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.726+0000 7f346f7fe700 1 -- 172.21.15.156:0/841258923 reap_dead start 2023-11-28T13:13:32.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.727+0000 7f3467fff700 1 --2- 172.21.15.156:0/841258923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f345c004850 0x7f345c004cc0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3458007d30 tx=0x7f345c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9a3e6eb08a5229 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:32.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.727+0000 7f346e7fc700 1 -- 172.21.15.156:0/841258923 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345800cdb0 con 0x7f345c004850 2023-11-28T13:13:32.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.727+0000 7f346e7fc700 1 -- 172.21.15.156:0/841258923 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3458005960 con 0x7f345c004850 2023-11-28T13:13:32.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.727+0000 7f346e7fc700 1 -- 172.21.15.156:0/841258923 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458005d10 con 0x7f345c004850 2023-11-28T13:13:32.732 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.727+0000 7f34755ff700 1 -- 172.21.15.156:0/841258923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f345c004850 msgr2=0x7f345c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:32.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.727+0000 7f34755ff700 1 --2- 172.21.15.156:0/841258923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f345c004850 0x7f345c004cc0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3458007d30 tx=0x7f345c005880 comp rx=0 tx=0).stop 2023-11-28T13:13:32.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.727+0000 7f34755ff700 1 -- 172.21.15.156:0/841258923 shutdown_connections 2023-11-28T13:13:32.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.727+0000 7f34755ff700 1 --2- 172.21.15.156:0/841258923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f345c004850 0x7f345c004cc0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:32.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.727+0000 7f34755ff700 1 -- 172.21.15.156:0/841258923 >> 172.21.15.156:0/841258923 conn(0x7f34700cb670 msgr2=0x7f34700cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:32.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.727+0000 7f34755ff700 1 -- 172.21.15.156:0/841258923 shutdown_connections 2023-11-28T13:13:32.733 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.727+0000 7f34755ff700 1 -- 172.21.15.156:0/841258923 wait complete. 2023-11-28T13:13:32.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.729+0000 7f34755ff700 1 Processor -- start 2023-11-28T13:13:32.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.729+0000 7f34755ff700 1 -- start start 2023-11-28T13:13:32.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.730+0000 7f34755ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f345c004850 0x7f34701490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:32.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.730+0000 7f34755ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34700b3990 0x7f34700b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:32.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.730+0000 7f34755ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34700c5ca0 0x7f34700c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:32.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.730+0000 7f34755ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34701495e0 con 0x7f34700c5ca0 2023-11-28T13:13:32.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.730+0000 7f34755ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34700b4340 con 0x7f34700b3990 2023-11-28T13:13:32.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.730+0000 7f34755ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34700c69f0 con 0x7f345c004850 2023-11-28T13:13:32.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.731+0000 7f346f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34700c5ca0 0x7f34700c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:32.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.731+0000 7f346effd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f345c004850 0x7f34701490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:32.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.731+0000 7f346f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34700c5ca0 0x7f34700c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36008/0 (socket says 172.21.15.156:36008) 2023-11-28T13:13:32.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.731+0000 7f346f7fe700 1 -- 172.21.15.156:0/110919692 learned_addr learned my addr 172.21.15.156:0/110919692 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:13:32.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.731+0000 7f3467fff700 1 --2- 172.21.15.156:0/110919692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34700b3990 0x7f34700b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:32.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.731+0000 7f346f7fe700 1 -- 172.21.15.156:0/110919692 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f345c004850 msgr2=0x7f34701490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:32.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.731+0000 7f346f7fe700 1 --2- 172.21.15.156:0/110919692 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f345c004850 0x7f34701490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:32.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.731+0000 7f346f7fe700 1 -- 172.21.15.156:0/110919692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34700b3990 msgr2=0x7f34700b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:32.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.731+0000 7f346f7fe700 1 --2- 172.21.15.156:0/110919692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34700b3990 0x7f34700b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:32.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.731+0000 7f346f7fe700 1 -- 172.21.15.156:0/110919692 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347006bba0 con 0x7f34700c5ca0 2023-11-28T13:13:32.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.731+0000 7f346f7fe700 1 --2- 172.21.15.156:0/110919692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34700c5ca0 0x7f34700c6110 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f347006e100 tx=0x7f346800dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:32.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.731+0000 7f34677fe700 1 -- 172.21.15.156:0/110919692 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468003e10 con 0x7f34700c5ca0 2023-11-28T13:13:32.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.731+0000 7f34677fe700 1 -- 172.21.15.156:0/110919692 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3468005070 con 0x7f34700c5ca0 2023-11-28T13:13:32.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.731+0000 7f34755ff700 1 -- 172.21.15.156:0/110919692 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347006b8c0 con 0x7f34700c5ca0 2023-11-28T13:13:32.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.731+0000 7f34677fe700 1 -- 172.21.15.156:0/110919692 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468005420 con 0x7f34700c5ca0 2023-11-28T13:13:32.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.732+0000 7f34755ff700 1 -- 172.21.15.156:0/110919692 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347015d690 con 0x7f34700c5ca0 2023-11-28T13:13:32.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.732+0000 7f34755ff700 1 -- 172.21.15.156:0/110919692 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34700745c0 con 0x7f34700c5ca0 2023-11-28T13:13:32.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.733+0000 7f34677fe700 1 -- 172.21.15.156:0/110919692 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f346801a030 con 0x7f34700c5ca0 2023-11-28T13:13:32.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.735+0000 7f34677fe700 1 --2- 172.21.15.156:0/110919692 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f34500682f0 0x7f345006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:32.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.735+0000 7f34677fe700 1 -- 172.21.15.156:0/110919692 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f3468085740 con 0x7f34700c5ca0 2023-11-28T13:13:32.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.735+0000 7f346effd700 1 --2- 172.21.15.156:0/110919692 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f34500682f0 0x7f345006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:32.746 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.736+0000 7f346effd700 1 --2- 172.21.15.156:0/110919692 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f34500682f0 0x7f345006a7a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3460009c60 tx=0x7f3460008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:32.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.740+0000 7f34677fe700 1 -- 172.21.15.156:0/110919692 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3468051e20 con 0x7f34700c5ca0 2023-11-28T13:13:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.973+0000 7f34755ff700 1 -- 172.21.15.156:0/110919692 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f34700b4b70 con 0x7f34700c5ca0 2023-11-28T13:13:32.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.974+0000 7f34677fe700 1 -- 172.21.15.156:0/110919692 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f34680180c0 con 0x7f34700c5ca0 2023-11-28T13:13:32.975 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:13:32.976 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:13:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.976+0000 7f34755ff700 1 -- 172.21.15.156:0/110919692 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f34500682f0 msgr2=0x7f345006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.976+0000 7f34755ff700 1 --2- 172.21.15.156:0/110919692 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f34500682f0 0x7f345006a7a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3460009c60 tx=0x7f3460008040 comp rx=0 tx=0).stop 2023-11-28T13:13:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.976+0000 7f34755ff700 1 -- 172.21.15.156:0/110919692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34700c5ca0 msgr2=0x7f34700c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.976+0000 7f34755ff700 1 --2- 172.21.15.156:0/110919692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34700c5ca0 0x7f34700c6110 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f347006e100 tx=0x7f346800dcd0 comp rx=0 tx=0).stop 2023-11-28T13:13:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.976+0000 7f34755ff700 1 -- 172.21.15.156:0/110919692 shutdown_connections 2023-11-28T13:13:32.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.976+0000 7f34755ff700 1 --2- 172.21.15.156:0/110919692 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f34500682f0 0x7f345006a7a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:32.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.976+0000 7f34755ff700 1 --2- 172.21.15.156:0/110919692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34700c5ca0 0x7f34700c6110 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:32.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.976+0000 7f34755ff700 1 --2- 172.21.15.156:0/110919692 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f345c004850 0x7f34701490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:32.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.976+0000 7f34755ff700 1 --2- 172.21.15.156:0/110919692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34700b3990 0x7f34700b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:32.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.976+0000 7f34755ff700 1 -- 172.21.15.156:0/110919692 >> 172.21.15.156:0/110919692 conn(0x7f34700cb670 msgr2=0x7f34700aed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:32.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.976+0000 7f34755ff700 1 -- 172.21.15.156:0/110919692 shutdown_connections 2023-11-28T13:13:32.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:32.976+0000 7f34755ff700 1 -- 172.21.15.156:0/110919692 wait complete. 2023-11-28T13:13:32.996 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:13:34.011 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:13:34.017 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:13:37.998 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:13:38.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.204+0000 7f2a1c5ff700 1 Processor -- start 2023-11-28T13:13:38.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.204+0000 7f2a1c5ff700 1 -- start start 2023-11-28T13:13:38.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.205+0000 7f2a1c5ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2a1415c9b0 0x7f2a140b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:38.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.205+0000 7f2a1c5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a140cd7c0 0x7f2a140cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:38.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.205+0000 7f2a1c5ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a140d1950 con 0x7f2a14161340 2023-11-28T13:13:38.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.205+0000 7f2a1c5ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a140b5800 con 0x7f2a1415c9b0 2023-11-28T13:13:38.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.205+0000 7f2a1c5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a140ce140 con 0x7f2a140cd7c0 2023-11-28T13:13:38.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.206+0000 7f2a19b9a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f2a14161340 0x7f2a140d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:53274/0 (socket says 172.21.15.156:53274) 2023-11-28T13:13:38.207 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.206+0000 7f2a19b9a700 1 -- 172.21.15.156:0/3203049424 learned_addr learned my addr 172.21.15.156:0/3203049424 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:13:38.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.206+0000 7f2a1ab9c700 1 --2- 172.21.15.156:0/3203049424 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a140cd7c0 0x7f2a140cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:38.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.206+0000 7f2a1a39b700 1 --2- 172.21.15.156:0/3203049424 >> v2:172.21.15.186:3300/0 conn(0x7f2a1415c9b0 0x7f2a140b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:38.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.206+0000 7f2a19399700 1 -- 172.21.15.156:0/3203049424 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1047629111 0 0) 0x7f2a140d1950 con 0x7f2a14161340 2023-11-28T13:13:38.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.206+0000 7f2a19399700 1 -- 172.21.15.156:0/3203049424 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a000035b0 con 0x7f2a14161340 2023-11-28T13:13:38.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.206+0000 7f2a1a39b700 1 -- 172.21.15.156:0/3203049424 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a140cd7c0 msgr2=0x7f2a140cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:13:38.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.206+0000 7f2a1a39b700 1 --2- 172.21.15.156:0/3203049424 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a140cd7c0 0x7f2a140cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:38.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.206+0000 7f2a1a39b700 1 -- 172.21.15.156:0/3203049424 >> v1:172.21.15.156:6789/0 conn(0x7f2a14161340 legacy=0x7f2a140d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:38.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.206+0000 7f2a1a39b700 1 -- 172.21.15.156:0/3203049424 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a140d24f0 con 0x7f2a1415c9b0 2023-11-28T13:13:38.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.206+0000 7f2a1ab9c700 1 --2- 172.21.15.156:0/3203049424 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a140cd7c0 0x7f2a140cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:13:38.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.207+0000 7f2a1a39b700 1 --2- 172.21.15.156:0/3203049424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1415c9b0 0x7f2a140b4bf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009840 tx=0x7f2a10009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2213f08a0e0e993 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:38.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.207+0000 7f2a19399700 1 -- 172.21.15.156:0/3203049424 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a10011830 con 0x7f2a1415c9b0 2023-11-28T13:13:38.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.207+0000 7f2a19399700 1 -- 172.21.15.156:0/3203049424 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a10011990 con 0x7f2a1415c9b0 2023-11-28T13:13:38.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.207+0000 7f2a19399700 1 -- 172.21.15.156:0/3203049424 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a10011d00 con 0x7f2a1415c9b0 2023-11-28T13:13:38.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.208+0000 7f2a1c5ff700 1 -- 172.21.15.156:0/3203049424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1415c9b0 msgr2=0x7f2a140b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:38.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.208+0000 7f2a1c5ff700 1 --2- 172.21.15.156:0/3203049424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1415c9b0 0x7f2a140b4bf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009840 tx=0x7f2a10009b50 comp rx=0 tx=0).stop 2023-11-28T13:13:38.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.208+0000 7f2a1c5ff700 1 -- 172.21.15.156:0/3203049424 shutdown_connections 2023-11-28T13:13:38.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.208+0000 7f2a1c5ff700 1 --2- 172.21.15.156:0/3203049424 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a140cd7c0 0x7f2a140cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:38.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.208+0000 7f2a1c5ff700 1 --2- 172.21.15.156:0/3203049424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1415c9b0 0x7f2a140b4bf0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:38.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.208+0000 7f2a1c5ff700 1 -- 172.21.15.156:0/3203049424 >> 172.21.15.156:0/3203049424 conn(0x7f2a14072600 msgr2=0x7f2a140cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:38.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.208+0000 7f2a1c5ff700 1 -- 172.21.15.156:0/3203049424 shutdown_connections 2023-11-28T13:13:38.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.208+0000 7f2a1c5ff700 1 -- 172.21.15.156:0/3203049424 wait complete. 2023-11-28T13:13:38.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.210+0000 7f2a1c5ff700 1 Processor -- start 2023-11-28T13:13:38.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.210+0000 7f2a1c5ff700 1 -- start start 2023-11-28T13:13:38.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.211+0000 7f2a1c5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a140cd7c0 0x7f2a14157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:38.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.211+0000 7f2a1c5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1415c9b0 0x7f2a140c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:38.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.211+0000 7f2a1c5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a14161340 0x7f2a1415e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:38.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.211+0000 7f2a1c5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a140cd630 con 0x7f2a14161340 2023-11-28T13:13:38.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.211+0000 7f2a1c5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a141e73c0 con 0x7f2a1415c9b0 2023-11-28T13:13:38.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.211+0000 7f2a1c5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a141f6470 con 0x7f2a140cd7c0 2023-11-28T13:13:38.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.211+0000 7f2a1ab9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a14161340 0x7f2a1415e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:38.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.211+0000 7f2a1ab9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a14161340 0x7f2a1415e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50214/0 (socket says 172.21.15.156:50214) 2023-11-28T13:13:38.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.211+0000 7f2a1ab9c700 1 -- 172.21.15.156:0/3980544342 learned_addr learned my addr 172.21.15.156:0/3980544342 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:13:38.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.211+0000 7f2a1a39b700 1 --2- 172.21.15.156:0/3980544342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a140cd7c0 0x7f2a14157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:38.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.211+0000 7f2a19b9a700 1 --2- 172.21.15.156:0/3980544342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1415c9b0 0x7f2a140c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:38.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.211+0000 7f2a1ab9c700 1 -- 172.21.15.156:0/3980544342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a140cd7c0 msgr2=0x7f2a14157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:38.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.211+0000 7f2a1ab9c700 1 --2- 172.21.15.156:0/3980544342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a140cd7c0 0x7f2a14157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:38.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.211+0000 7f2a1ab9c700 1 -- 172.21.15.156:0/3980544342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1415c9b0 msgr2=0x7f2a140c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:38.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.211+0000 7f2a1ab9c700 1 --2- 172.21.15.156:0/3980544342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1415c9b0 0x7f2a140c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:38.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.211+0000 7f2a1ab9c700 1 -- 172.21.15.156:0/3980544342 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c009a70 con 0x7f2a14161340 2023-11-28T13:13:38.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.212+0000 7f2a19b9a700 1 --2- 172.21.15.156:0/3980544342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1415c9b0 0x7f2a140c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:13:38.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.212+0000 7f2a1ab9c700 1 --2- 172.21.15.156:0/3980544342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a14161340 0x7f2a1415e960 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00efc0 tx=0x7f2a0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:38.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.212+0000 7f2a0affd700 1 -- 172.21.15.156:0/3980544342 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c00cb40 con 0x7f2a14161340 2023-11-28T13:13:38.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.212+0000 7f2a0affd700 1 -- 172.21.15.156:0/3980544342 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0c00f040 con 0x7f2a14161340 2023-11-28T13:13:38.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.212+0000 7f2a1c5ff700 1 -- 172.21.15.156:0/3980544342 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a100094a0 con 0x7f2a14161340 2023-11-28T13:13:38.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.212+0000 7f2a0affd700 1 -- 172.21.15.156:0/3980544342 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c003a20 con 0x7f2a14161340 2023-11-28T13:13:38.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.212+0000 7f2a1c5ff700 1 -- 172.21.15.156:0/3980544342 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a140d1300 con 0x7f2a14161340 2023-11-28T13:13:38.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.213+0000 7f2a1c5ff700 1 -- 172.21.15.156:0/3980544342 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a140b5520 con 0x7f2a14161340 2023-11-28T13:13:38.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.214+0000 7f2a0affd700 1 -- 172.21.15.156:0/3980544342 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2a0c014480 con 0x7f2a14161340 2023-11-28T13:13:38.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.215+0000 7f2a0affd700 1 --2- 172.21.15.156:0/3980544342 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f29f80682f0 0x7f29f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:38.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.215+0000 7f2a0affd700 1 -- 172.21.15.156:0/3980544342 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f2a0c0868d0 con 0x7f2a14161340 2023-11-28T13:13:38.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.216+0000 7f2a1a39b700 1 --2- 172.21.15.156:0/3980544342 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f29f80682f0 0x7f29f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:38.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.217+0000 7f2a1a39b700 1 --2- 172.21.15.156:0/3980544342 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f29f80682f0 0x7f29f806a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009fd0 tx=0x7f2a1001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:38.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.220+0000 7f2a0affd700 1 -- 172.21.15.156:0/3980544342 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2a0c052f50 con 0x7f2a14161340 2023-11-28T13:13:38.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.455+0000 7f2a1c5ff700 1 -- 172.21.15.156:0/3980544342 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2a140d15e0 con 0x7f2a14161340 2023-11-28T13:13:38.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.455+0000 7f2a0affd700 1 -- 172.21.15.156:0/3980544342 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f2a0c009cd0 con 0x7f2a14161340 2023-11-28T13:13:38.456 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:13:38.456 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:13:38.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.457+0000 7f2a1c5ff700 1 -- 172.21.15.156:0/3980544342 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f29f80682f0 msgr2=0x7f29f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:38.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.457+0000 7f2a1c5ff700 1 --2- 172.21.15.156:0/3980544342 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f29f80682f0 0x7f29f806a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009fd0 tx=0x7f2a1001c040 comp rx=0 tx=0).stop 2023-11-28T13:13:38.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.457+0000 7f2a1c5ff700 1 -- 172.21.15.156:0/3980544342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a14161340 msgr2=0x7f2a1415e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:38.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.457+0000 7f2a1c5ff700 1 --2- 172.21.15.156:0/3980544342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a14161340 0x7f2a1415e960 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00efc0 tx=0x7f2a0c00c3a0 comp rx=0 tx=0).stop 2023-11-28T13:13:38.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.458+0000 7f2a1c5ff700 1 -- 172.21.15.156:0/3980544342 shutdown_connections 2023-11-28T13:13:38.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.458+0000 7f2a1c5ff700 1 --2- 172.21.15.156:0/3980544342 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f29f80682f0 0x7f29f806a7a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:38.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.458+0000 7f2a1c5ff700 1 --2- 172.21.15.156:0/3980544342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a14161340 0x7f2a1415e960 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:38.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.458+0000 7f2a1c5ff700 1 --2- 172.21.15.156:0/3980544342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a140cd7c0 0x7f2a14157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:38.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.458+0000 7f2a1c5ff700 1 --2- 172.21.15.156:0/3980544342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1415c9b0 0x7f2a140c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:38.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.458+0000 7f2a1c5ff700 1 -- 172.21.15.156:0/3980544342 >> 172.21.15.156:0/3980544342 conn(0x7f2a14072600 msgr2=0x7f2a140b7450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:38.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.458+0000 7f2a1c5ff700 1 -- 172.21.15.156:0/3980544342 shutdown_connections 2023-11-28T13:13:38.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:38.458+0000 7f2a1c5ff700 1 -- 172.21.15.156:0/3980544342 wait complete. 2023-11-28T13:13:38.477 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:13:43.478 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:13:43.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.682+0000 7fbee27ff700 1 Processor -- start 2023-11-28T13:13:43.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.683+0000 7fbee27ff700 1 -- start start 2023-11-28T13:13:43.684 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.684+0000 7fbee27ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbedc159860 0x7fbedc0c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:43.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.684+0000 7fbee27ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbedc0b0d80 0x7fbedc0c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:43.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.684+0000 7fbee27ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbedc0ccc00 con 0x7fbedc159490 2023-11-28T13:13:43.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.684+0000 7fbee27ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbedc0ccec0 con 0x7fbedc159860 2023-11-28T13:13:43.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.684+0000 7fbee27ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbedc0cd1a0 con 0x7fbedc0b0d80 2023-11-28T13:13:43.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.684+0000 7fbedb7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbedc159860 0x7fbedc0c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:43.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.684+0000 7fbedb7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbedc159860 0x7fbedc0c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50218/0 (socket says 172.21.15.156:50218) 2023-11-28T13:13:43.685 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.684+0000 7fbedb7fe700 1 -- 172.21.15.156:0/1569800204 learned_addr learned my addr 172.21.15.156:0/1569800204 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:13:43.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.685+0000 7fbee0d9c700 1 --2- 172.21.15.156:0/1569800204 >> v2:172.21.15.186:3300/0 conn(0x7fbedc0b0d80 0x7fbedc0c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:43.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.685+0000 7fbedb7fe700 1 -- 172.21.15.156:0/1569800204 >> v2:172.21.15.186:3300/0 conn(0x7fbedc0b0d80 msgr2=0x7fbedc0c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:13:43.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.685+0000 7fbedb7fe700 1 --2- 172.21.15.156:0/1569800204 >> v2:172.21.15.186:3300/0 conn(0x7fbedc0b0d80 0x7fbedc0c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:43.687 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.685+0000 7fbedb7fe700 1 -- 172.21.15.156:0/1569800204 >> v1:172.21.15.186:6789/0 conn(0x7fbedc159490 legacy=0x7fbedc0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:13:43.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.685+0000 7fbedb7fe700 1 -- 172.21.15.156:0/1569800204 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc0cd480 con 0x7fbedc159860 2023-11-28T13:13:43.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.685+0000 7fbee0d9c700 1 --2- 172.21.15.156:0/1569800204 >> v2:172.21.15.186:3300/0 conn(0x7fbedc0b0d80 0x7fbedc0c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:13:43.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.685+0000 7fbedb7fe700 1 --2- 172.21.15.156:0/1569800204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbedc159860 0x7fbedc0c94c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0c5be0 tx=0x7fbed0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43101f74a2c3c59 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:43.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.685+0000 7fbedaffd700 1 -- 172.21.15.156:0/1569800204 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed000b870 con 0x7fbedc159860 2023-11-28T13:13:43.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.685+0000 7fbedaffd700 1 -- 172.21.15.156:0/1569800204 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed000b9d0 con 0x7fbedc159860 2023-11-28T13:13:43.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.685+0000 7fbedaffd700 1 -- 172.21.15.156:0/1569800204 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed000bd60 con 0x7fbedc159860 2023-11-28T13:13:43.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.686+0000 7fbee27ff700 1 -- 172.21.15.156:0/1569800204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbedc159860 msgr2=0x7fbedc0c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:43.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.686+0000 7fbee27ff700 1 --2- 172.21.15.156:0/1569800204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbedc159860 0x7fbedc0c94c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0c5be0 tx=0x7fbed0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:13:43.688 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.686+0000 7fbee27ff700 1 -- 172.21.15.156:0/1569800204 shutdown_connections 2023-11-28T13:13:43.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.686+0000 7fbee27ff700 1 --2- 172.21.15.156:0/1569800204 >> v2:172.21.15.186:3300/0 conn(0x7fbedc0b0d80 0x7fbedc0c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:43.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.686+0000 7fbee27ff700 1 --2- 172.21.15.156:0/1569800204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbedc159860 0x7fbedc0c94c0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:43.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.686+0000 7fbee27ff700 1 -- 172.21.15.156:0/1569800204 >> 172.21.15.156:0/1569800204 conn(0x7fbedc0cb670 msgr2=0x7fbedc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:43.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.686+0000 7fbee27ff700 1 -- 172.21.15.156:0/1569800204 shutdown_connections 2023-11-28T13:13:43.689 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.686+0000 7fbee27ff700 1 -- 172.21.15.156:0/1569800204 wait complete. 2023-11-28T13:13:43.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.688+0000 7fbee27ff700 1 Processor -- start 2023-11-28T13:13:43.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.688+0000 7fbee27ff700 1 -- start start 2023-11-28T13:13:43.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.689+0000 7fbee27ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbedc0b0d80 0x7fbedc153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:43.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.689+0000 7fbee27ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbedc159490 0x7fbedc15d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:43.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.689+0000 7fbee27ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbedc15d6a0 0x7fbedc0c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:43.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.689+0000 7fbee27ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0c9980 con 0x7fbedc0b0d80 2023-11-28T13:13:43.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.689+0000 7fbee27ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc15db20 con 0x7fbedc159490 2023-11-28T13:13:43.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.689+0000 7fbee27ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0c88e0 con 0x7fbedc15d6a0 2023-11-28T13:13:43.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.689+0000 7fbedbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbedc0b0d80 0x7fbedc153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:43.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.689+0000 7fbedbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbedc0b0d80 0x7fbedc153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50226/0 (socket says 172.21.15.156:50226) 2023-11-28T13:13:43.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.689+0000 7fbedbfff700 1 -- 172.21.15.156:0/3817230044 learned_addr learned my addr 172.21.15.156:0/3817230044 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:13:43.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.689+0000 7fbedb7fe700 1 --2- 172.21.15.156:0/3817230044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbedc159490 0x7fbedc15d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:43.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.689+0000 7fbee0d9c700 1 --2- 172.21.15.156:0/3817230044 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbedc15d6a0 0x7fbedc0c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:43.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.689+0000 7fbedbfff700 1 -- 172.21.15.156:0/3817230044 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbedc15d6a0 msgr2=0x7fbedc0c7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:43.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.689+0000 7fbedbfff700 1 --2- 172.21.15.156:0/3817230044 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbedc15d6a0 0x7fbedc0c7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:43.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.689+0000 7fbedbfff700 1 -- 172.21.15.156:0/3817230044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbedc159490 msgr2=0x7fbedc15d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:43.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.689+0000 7fbedbfff700 1 --2- 172.21.15.156:0/3817230044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbedc159490 0x7fbedc15d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:43.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.689+0000 7fbedbfff700 1 -- 172.21.15.156:0/3817230044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed0009550 con 0x7fbedc0b0d80 2023-11-28T13:13:43.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.689+0000 7fbee0d9c700 1 --2- 172.21.15.156:0/3817230044 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbedc15d6a0 0x7fbedc0c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:13:43.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.690+0000 7fbedbfff700 1 --2- 172.21.15.156:0/3817230044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbedc0b0d80 0x7fbedc153b00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00c4e0 tx=0x7fbecc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:43.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.690+0000 7fbed8ff9700 1 -- 172.21.15.156:0/3817230044 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc0052d0 con 0x7fbedc0b0d80 2023-11-28T13:13:43.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.690+0000 7fbed8ff9700 1 -- 172.21.15.156:0/3817230044 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbecc005430 con 0x7fbedc0b0d80 2023-11-28T13:13:43.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.690+0000 7fbee27ff700 1 -- 172.21.15.156:0/3817230044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbedc0c9f40 con 0x7fbedc0b0d80 2023-11-28T13:13:43.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.690+0000 7fbed8ff9700 1 -- 172.21.15.156:0/3817230044 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc0038c0 con 0x7fbedc0b0d80 2023-11-28T13:13:43.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.690+0000 7fbee27ff700 1 -- 172.21.15.156:0/3817230044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbedc0cd760 con 0x7fbedc0b0d80 2023-11-28T13:13:43.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.691+0000 7fbee27ff700 1 -- 172.21.15.156:0/3817230044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbedc0cdc80 con 0x7fbedc0b0d80 2023-11-28T13:13:43.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.692+0000 7fbed8ff9700 1 -- 172.21.15.156:0/3817230044 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbecc019030 con 0x7fbedc0b0d80 2023-11-28T13:13:43.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.693+0000 7fbed8ff9700 1 --2- 172.21.15.156:0/3817230044 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbec80682f0 0x7fbec806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:43.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.693+0000 7fbed8ff9700 1 -- 172.21.15.156:0/3817230044 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fbecc085130 con 0x7fbedc0b0d80 2023-11-28T13:13:43.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.694+0000 7fbedb7fe700 1 --2- 172.21.15.156:0/3817230044 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbec80682f0 0x7fbec806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:43.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.694+0000 7fbedb7fe700 1 --2- 172.21.15.156:0/3817230044 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbec80682f0 0x7fbec806a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbed0014040 tx=0x7fbed0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:43.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.697+0000 7fbed8ff9700 1 -- 172.21.15.156:0/3817230044 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbecc051810 con 0x7fbedc0b0d80 2023-11-28T13:13:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.930+0000 7fbee27ff700 1 -- 172.21.15.156:0/3817230044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbedc0745c0 con 0x7fbedc0b0d80 2023-11-28T13:13:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.931+0000 7fbed8ff9700 1 -- 172.21.15.156:0/3817230044 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fbecc054e30 con 0x7fbedc0b0d80 2023-11-28T13:13:43.931 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:13:43.931 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:13:43.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.933+0000 7fbee27ff700 1 -- 172.21.15.156:0/3817230044 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbec80682f0 msgr2=0x7fbec806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:43.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.933+0000 7fbee27ff700 1 --2- 172.21.15.156:0/3817230044 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbec80682f0 0x7fbec806a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbed0014040 tx=0x7fbed0008030 comp rx=0 tx=0).stop 2023-11-28T13:13:43.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.933+0000 7fbee27ff700 1 -- 172.21.15.156:0/3817230044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbedc0b0d80 msgr2=0x7fbedc153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:43.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.933+0000 7fbee27ff700 1 --2- 172.21.15.156:0/3817230044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbedc0b0d80 0x7fbedc153b00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00c4e0 tx=0x7fbecc00c7f0 comp rx=0 tx=0).stop 2023-11-28T13:13:43.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.933+0000 7fbee27ff700 1 -- 172.21.15.156:0/3817230044 shutdown_connections 2023-11-28T13:13:43.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.933+0000 7fbee27ff700 1 --2- 172.21.15.156:0/3817230044 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbec80682f0 0x7fbec806a7a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:43.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.933+0000 7fbee27ff700 1 --2- 172.21.15.156:0/3817230044 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbedc15d6a0 0x7fbedc0c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:43.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.933+0000 7fbee27ff700 1 --2- 172.21.15.156:0/3817230044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbedc0b0d80 0x7fbedc153b00 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:43.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.933+0000 7fbee27ff700 1 --2- 172.21.15.156:0/3817230044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbedc159490 0x7fbedc15d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:43.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.933+0000 7fbee27ff700 1 -- 172.21.15.156:0/3817230044 >> 172.21.15.156:0/3817230044 conn(0x7fbedc0cb670 msgr2=0x7fbedc15a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:43.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.933+0000 7fbee27ff700 1 -- 172.21.15.156:0/3817230044 shutdown_connections 2023-11-28T13:13:43.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:43.933+0000 7fbee27ff700 1 -- 172.21.15.156:0/3817230044 wait complete. 2023-11-28T13:13:43.952 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:13:48.953 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:13:49.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.159+0000 7f475999d700 1 Processor -- start 2023-11-28T13:13:49.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.159+0000 7f475999d700 1 -- start start 2023-11-28T13:13:49.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.160+0000 7f475999d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47540b2560 0x7f47540b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:49.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.160+0000 7f475999d700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f47540b15e0 0x7f47540b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:49.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.160+0000 7f475999d700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47540b0500 con 0x7f47540b2010 2023-11-28T13:13:49.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.160+0000 7f475999d700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47540b4810 con 0x7f47540b15e0 2023-11-28T13:13:49.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.160+0000 7f475999d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47540afbd0 con 0x7f47540b2560 2023-11-28T13:13:49.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.161+0000 7f475899b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47540b2560 0x7f47540b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:49.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.161+0000 7f475899b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47540b2560 0x7f47540b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:41904/0 (socket says 172.21.15.156:41904) 2023-11-28T13:13:49.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.161+0000 7f475899b700 1 -- 172.21.15.156:0/1585547422 learned_addr learned my addr 172.21.15.156:0/1585547422 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:13:49.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.161+0000 7f475919c700 1 --2- 172.21.15.156:0/1585547422 >> v2:172.21.15.186:3300/0 conn(0x7f47540b15e0 0x7f47540b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:49.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.161+0000 7f47537fe700 1 -- 172.21.15.156:0/1585547422 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423562246 0 0) 0x7f47540b0500 con 0x7f47540b2010 2023-11-28T13:13:49.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.161+0000 7f47537fe700 1 -- 172.21.15.156:0/1585547422 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47440035b0 con 0x7f47540b2010 2023-11-28T13:13:49.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.161+0000 7f475899b700 1 -- 172.21.15.156:0/1585547422 >> v2:172.21.15.186:3300/0 conn(0x7f47540b15e0 msgr2=0x7f47540b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:13:49.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.161+0000 7f475899b700 1 --2- 172.21.15.156:0/1585547422 >> v2:172.21.15.186:3300/0 conn(0x7f47540b15e0 0x7f47540b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:49.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.161+0000 7f475899b700 1 -- 172.21.15.156:0/1585547422 >> v1:172.21.15.186:6789/0 conn(0x7f47540b2010 legacy=0x7f47540b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:49.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.161+0000 7f475899b700 1 -- 172.21.15.156:0/1585547422 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47540afeb0 con 0x7f47540b2560 2023-11-28T13:13:49.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.161+0000 7f475919c700 1 --2- 172.21.15.156:0/1585547422 >> v2:172.21.15.186:3300/0 conn(0x7f47540b15e0 0x7f47540b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:13:49.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.161+0000 7f475899b700 1 --2- 172.21.15.156:0/1585547422 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47540b2560 0x7f47540b0990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f47480098f0 tx=0x7f4748009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d66954fde1175b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:49.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.162+0000 7f47537fe700 1 -- 172.21.15.156:0/1585547422 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748011830 con 0x7f47540b2560 2023-11-28T13:13:49.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.162+0000 7f47537fe700 1 -- 172.21.15.156:0/1585547422 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4748011990 con 0x7f47540b2560 2023-11-28T13:13:49.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.162+0000 7f47537fe700 1 -- 172.21.15.156:0/1585547422 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748011d20 con 0x7f47540b2560 2023-11-28T13:13:49.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.162+0000 7f475999d700 1 -- 172.21.15.156:0/1585547422 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47540b2560 msgr2=0x7f47540b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:49.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.162+0000 7f475999d700 1 --2- 172.21.15.156:0/1585547422 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47540b2560 0x7f47540b0990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f47480098f0 tx=0x7f4748009c00 comp rx=0 tx=0).stop 2023-11-28T13:13:49.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.162+0000 7f475999d700 1 -- 172.21.15.156:0/1585547422 shutdown_connections 2023-11-28T13:13:49.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.162+0000 7f475999d700 1 --2- 172.21.15.156:0/1585547422 >> v2:172.21.15.186:3300/0 conn(0x7f47540b15e0 0x7f47540b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:49.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.162+0000 7f475999d700 1 --2- 172.21.15.156:0/1585547422 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47540b2560 0x7f47540b0990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:49.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.162+0000 7f475999d700 1 -- 172.21.15.156:0/1585547422 >> 172.21.15.156:0/1585547422 conn(0x7f475409c1d0 msgr2=0x7f47540d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:49.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.162+0000 7f475999d700 1 -- 172.21.15.156:0/1585547422 shutdown_connections 2023-11-28T13:13:49.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.162+0000 7f475999d700 1 -- 172.21.15.156:0/1585547422 wait complete. 2023-11-28T13:13:49.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.164+0000 7f475999d700 1 Processor -- start 2023-11-28T13:13:49.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.164+0000 7f475999d700 1 -- start start 2023-11-28T13:13:49.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.165+0000 7f475999d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47540b15e0 0x7f47540c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:49.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.165+0000 7f475999d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47540b2010 0x7f47540b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:49.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.165+0000 7f475999d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47540b2560 0x7f47540c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:49.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.165+0000 7f475999d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47540b8a50 con 0x7f47540b2560 2023-11-28T13:13:49.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.165+0000 7f475999d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47540b7e40 con 0x7f47540b15e0 2023-11-28T13:13:49.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.165+0000 7f475999d700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47540b8120 con 0x7f47540b2010 2023-11-28T13:13:49.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.165+0000 7f475919c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47540b2560 0x7f47540c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:49.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.165+0000 7f475899b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47540b15e0 0x7f47540c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:49.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.165+0000 7f475919c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47540b2560 0x7f47540c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:54972/0 (socket says 172.21.15.156:54972) 2023-11-28T13:13:49.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.165+0000 7f475919c700 1 -- 172.21.15.156:0/2886552547 learned_addr learned my addr 172.21.15.156:0/2886552547 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:13:49.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.165+0000 7f4753fff700 1 --2- 172.21.15.156:0/2886552547 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47540b2010 0x7f47540b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:49.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.166+0000 7f475919c700 1 -- 172.21.15.156:0/2886552547 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47540b2010 msgr2=0x7f47540b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:49.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.166+0000 7f475919c700 1 --2- 172.21.15.156:0/2886552547 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47540b2010 0x7f47540b7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:49.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.166+0000 7f475919c700 1 -- 172.21.15.156:0/2886552547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47540b15e0 msgr2=0x7f47540c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:49.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.166+0000 7f475919c700 1 --2- 172.21.15.156:0/2886552547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47540b15e0 0x7f47540c8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:49.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.166+0000 7f475919c700 1 -- 172.21.15.156:0/2886552547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474c009a70 con 0x7f47540b2560 2023-11-28T13:13:49.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.166+0000 7f475899b700 1 --2- 172.21.15.156:0/2886552547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47540b15e0 0x7f47540c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:13:49.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.166+0000 7f475919c700 1 --2- 172.21.15.156:0/2886552547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47540b2560 0x7f47540c9340 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f474c00efc0 tx=0x7f474c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:49.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.166+0000 7f47517fa700 1 -- 172.21.15.156:0/2886552547 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c00ca90 con 0x7f47540b2560 2023-11-28T13:13:49.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.166+0000 7f47517fa700 1 -- 172.21.15.156:0/2886552547 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474c00f040 con 0x7f47540b2560 2023-11-28T13:13:49.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.166+0000 7f475999d700 1 -- 172.21.15.156:0/2886552547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4748009550 con 0x7f47540b2560 2023-11-28T13:13:49.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.167+0000 7f47517fa700 1 -- 172.21.15.156:0/2886552547 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c00ce10 con 0x7f47540b2560 2023-11-28T13:13:49.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.167+0000 7f475999d700 1 -- 172.21.15.156:0/2886552547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47540d0070 con 0x7f47540b2560 2023-11-28T13:13:49.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.168+0000 7f475999d700 1 -- 172.21.15.156:0/2886552547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47540b0190 con 0x7f47540b2560 2023-11-28T13:13:49.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.168+0000 7f47517fa700 1 -- 172.21.15.156:0/2886552547 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f474c014480 con 0x7f47540b2560 2023-11-28T13:13:49.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.170+0000 7f47517fa700 1 --2- 172.21.15.156:0/2886552547 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f47380682f0 0x7f473806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:49.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.170+0000 7f47517fa700 1 -- 172.21.15.156:0/2886552547 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f474c0865e0 con 0x7f47540b2560 2023-11-28T13:13:49.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.170+0000 7f475899b700 1 --2- 172.21.15.156:0/2886552547 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f47380682f0 0x7f473806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:49.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.171+0000 7f475899b700 1 --2- 172.21.15.156:0/2886552547 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f47380682f0 0x7f473806a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4748009c40 tx=0x7f4748008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:49.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.174+0000 7f47517fa700 1 -- 172.21.15.156:0/2886552547 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f474c052c60 con 0x7f47540b2560 2023-11-28T13:13:49.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.404+0000 7f475999d700 1 -- 172.21.15.156:0/2886552547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f47540d0200 con 0x7f47540b2560 2023-11-28T13:13:49.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.404+0000 7f47517fa700 1 -- 172.21.15.156:0/2886552547 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f474c009d70 con 0x7f47540b2560 2023-11-28T13:13:49.405 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:13:49.405 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:13:49.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.406+0000 7f475999d700 1 -- 172.21.15.156:0/2886552547 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f47380682f0 msgr2=0x7f473806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:49.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.406+0000 7f475999d700 1 --2- 172.21.15.156:0/2886552547 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f47380682f0 0x7f473806a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4748009c40 tx=0x7f4748008030 comp rx=0 tx=0).stop 2023-11-28T13:13:49.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.406+0000 7f475999d700 1 -- 172.21.15.156:0/2886552547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47540b2560 msgr2=0x7f47540c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:49.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.406+0000 7f475999d700 1 --2- 172.21.15.156:0/2886552547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47540b2560 0x7f47540c9340 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f474c00efc0 tx=0x7f474c00c3a0 comp rx=0 tx=0).stop 2023-11-28T13:13:49.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.406+0000 7f475999d700 1 -- 172.21.15.156:0/2886552547 shutdown_connections 2023-11-28T13:13:49.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.406+0000 7f475999d700 1 --2- 172.21.15.156:0/2886552547 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f47380682f0 0x7f473806a7a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:49.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.406+0000 7f475999d700 1 --2- 172.21.15.156:0/2886552547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47540b2560 0x7f47540c9340 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:49.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.406+0000 7f475999d700 1 --2- 172.21.15.156:0/2886552547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47540b15e0 0x7f47540c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:49.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.406+0000 7f475999d700 1 --2- 172.21.15.156:0/2886552547 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f47540b2010 0x7f47540b7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:49.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.406+0000 7f475999d700 1 -- 172.21.15.156:0/2886552547 >> 172.21.15.156:0/2886552547 conn(0x7f475409c1d0 msgr2=0x7f47540d0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:49.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.406+0000 7f475999d700 1 -- 172.21.15.156:0/2886552547 shutdown_connections 2023-11-28T13:13:49.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:49.407+0000 7f475999d700 1 -- 172.21.15.156:0/2886552547 wait complete. 2023-11-28T13:13:49.425 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:13:54.427 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:13:54.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.633+0000 7fa3dc3ff700 1 Processor -- start 2023-11-28T13:13:54.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.633+0000 7fa3dc3ff700 1 -- start start 2023-11-28T13:13:54.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.634+0000 7fa3dc3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3d415c9b0 0x7fa3d40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:54.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.634+0000 7fa3dc3ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa3d40cd7c0 0x7fa3d40cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:54.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.635+0000 7fa3dc3ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d40ce160 con 0x7fa3d4161340 2023-11-28T13:13:54.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.635+0000 7fa3dc3ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d40b5800 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.635+0000 7fa3dc3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40d1950 con 0x7fa3d415c9b0 2023-11-28T13:13:54.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.635+0000 7fa3d999a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa3d4161340 0x7fa3d40d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:54724/0 (socket says 172.21.15.156:54724) 2023-11-28T13:13:54.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.635+0000 7fa3d999a700 1 -- 172.21.15.156:0/3519887212 learned_addr learned my addr 172.21.15.156:0/3519887212 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:13:54.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.635+0000 7fa3da99c700 1 --2- 172.21.15.156:0/3519887212 >> v2:172.21.15.156:3300/0 conn(0x7fa3d40cd7c0 0x7fa3d40cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:54.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.635+0000 7fa3da19b700 1 --2- 172.21.15.156:0/3519887212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3d415c9b0 0x7fa3d40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:54.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.635+0000 7fa3da99c700 1 -- 172.21.15.156:0/3519887212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3d415c9b0 msgr2=0x7fa3d40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:13:54.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.635+0000 7fa3da99c700 1 --2- 172.21.15.156:0/3519887212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3d415c9b0 0x7fa3d40b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:54.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.635+0000 7fa3da99c700 1 -- 172.21.15.156:0/3519887212 >> v1:172.21.15.156:6789/0 conn(0x7fa3d4161340 legacy=0x7fa3d40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:54.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.635+0000 7fa3da99c700 1 -- 172.21.15.156:0/3519887212 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d40d24f0 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.635+0000 7fa3da19b700 1 --2- 172.21.15.156:0/3519887212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3d415c9b0 0x7fa3d40b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:13:54.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.636+0000 7fa3da99c700 1 --2- 172.21.15.156:0/3519887212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d40cd7c0 0x7fa3d40cdc20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009d90 tx=0x7fa3cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56c45e0b912d88 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:54.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.636+0000 7fa3d9199700 1 -- 172.21.15.156:0/3519887212 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00e070 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.636+0000 7fa3d9199700 1 -- 172.21.15.156:0/3519887212 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc00bbd0 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.636+0000 7fa3d9199700 1 -- 172.21.15.156:0/3519887212 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc012430 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.636+0000 7fa3dc3ff700 1 -- 172.21.15.156:0/3519887212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d40cd7c0 msgr2=0x7fa3d40cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:54.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.636+0000 7fa3dc3ff700 1 --2- 172.21.15.156:0/3519887212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d40cd7c0 0x7fa3d40cdc20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009d90 tx=0x7fa3cc00b330 comp rx=0 tx=0).stop 2023-11-28T13:13:54.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.636+0000 7fa3dc3ff700 1 -- 172.21.15.156:0/3519887212 shutdown_connections 2023-11-28T13:13:54.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.636+0000 7fa3dc3ff700 1 --2- 172.21.15.156:0/3519887212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d40cd7c0 0x7fa3d40cdc20 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:54.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.636+0000 7fa3dc3ff700 1 --2- 172.21.15.156:0/3519887212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3d415c9b0 0x7fa3d40b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:54.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.637+0000 7fa3dc3ff700 1 -- 172.21.15.156:0/3519887212 >> 172.21.15.156:0/3519887212 conn(0x7fa3d4072600 msgr2=0x7fa3d40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:54.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.637+0000 7fa3dc3ff700 1 -- 172.21.15.156:0/3519887212 shutdown_connections 2023-11-28T13:13:54.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.637+0000 7fa3dc3ff700 1 -- 172.21.15.156:0/3519887212 wait complete. 2023-11-28T13:13:54.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.638+0000 7fa3dc3ff700 1 Processor -- start 2023-11-28T13:13:54.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.638+0000 7fa3dc3ff700 1 -- start start 2023-11-28T13:13:54.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.640+0000 7fa3dc3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d40cd7c0 0x7fa3d41577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:54.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.640+0000 7fa3dc3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3d415c9b0 0x7fa3d415a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:54.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.640+0000 7fa3dc3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3d4161340 0x7fa3d415ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:54.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.640+0000 7fa3dc3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d4156e20 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.640+0000 7fa3dc3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40c7880 con 0x7fa3d415c9b0 2023-11-28T13:13:54.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.640+0000 7fa3dc3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40c7b60 con 0x7fa3d4161340 2023-11-28T13:13:54.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.640+0000 7fa3da19b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d40cd7c0 0x7fa3d41577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:54.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.640+0000 7fa3da19b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d40cd7c0 0x7fa3d41577e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:55000/0 (socket says 172.21.15.156:55000) 2023-11-28T13:13:54.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.640+0000 7fa3da19b700 1 -- 172.21.15.156:0/3916477760 learned_addr learned my addr 172.21.15.156:0/3916477760 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:13:54.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.640+0000 7fa3da19b700 1 -- 172.21.15.156:0/3916477760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3d4161340 msgr2=0x7fa3d415ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:54.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.640+0000 7fa3da19b700 1 --2- 172.21.15.156:0/3916477760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3d4161340 0x7fa3d415ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:54.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.640+0000 7fa3d999a700 1 --2- 172.21.15.156:0/3916477760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3d415c9b0 0x7fa3d415a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:54.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.640+0000 7fa3da19b700 1 -- 172.21.15.156:0/3916477760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3d415c9b0 msgr2=0x7fa3d415a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:54.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.640+0000 7fa3da19b700 1 --2- 172.21.15.156:0/3916477760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3d415c9b0 0x7fa3d415a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:54.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.640+0000 7fa3da19b700 1 -- 172.21.15.156:0/3916477760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc009a70 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.640+0000 7fa3d999a700 1 --2- 172.21.15.156:0/3916477760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3d415c9b0 0x7fa3d415a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:13:54.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.641+0000 7fa3da19b700 1 --2- 172.21.15.156:0/3916477760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d40cd7c0 0x7fa3d41577e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000e760 tx=0x7fa3d000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:54.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.641+0000 7fa3caffd700 1 -- 172.21.15.156:0/3916477760 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0013610 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.641+0000 7fa3caffd700 1 -- 172.21.15.156:0/3916477760 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d0013770 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.641+0000 7fa3caffd700 1 -- 172.21.15.156:0/3916477760 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0013b20 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.641+0000 7fa3dc3ff700 1 -- 172.21.15.156:0/3916477760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d40d1300 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.643 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.641+0000 7fa3dc3ff700 1 -- 172.21.15.156:0/3916477760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d40b5520 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.642+0000 7fa3dc3ff700 1 -- 172.21.15.156:0/3916477760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d40d15e0 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.643+0000 7fa3caffd700 1 -- 172.21.15.156:0/3916477760 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa3d0023070 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.644+0000 7fa3caffd700 1 --2- 172.21.15.156:0/3916477760 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa3b80682f0 0x7fa3b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:13:54.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.644+0000 7fa3caffd700 1 -- 172.21.15.156:0/3916477760 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fa3d0085a60 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.644+0000 7fa3d999a700 1 --2- 172.21.15.156:0/3916477760 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa3b80682f0 0x7fa3b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:13:54.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.645+0000 7fa3d999a700 1 --2- 172.21.15.156:0/3916477760 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa3b80682f0 0x7fa3b806a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa3d41f6310 tx=0x7fa3c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:13:54.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.648+0000 7fa3caffd700 1 -- 172.21.15.156:0/3916477760 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa3d00520b0 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.881+0000 7fa3dc3ff700 1 -- 172.21.15.156:0/3916477760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa3d40745c0 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.881+0000 7fa3caffd700 1 -- 172.21.15.156:0/3916477760 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fa3d00210c0 con 0x7fa3d40cd7c0 2023-11-28T13:13:54.882 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:13:54.882 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:13:54.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.883+0000 7fa3dc3ff700 1 -- 172.21.15.156:0/3916477760 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa3b80682f0 msgr2=0x7fa3b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:54.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.883+0000 7fa3dc3ff700 1 --2- 172.21.15.156:0/3916477760 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa3b80682f0 0x7fa3b806a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa3d41f6310 tx=0x7fa3c4009040 comp rx=0 tx=0).stop 2023-11-28T13:13:54.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.883+0000 7fa3dc3ff700 1 -- 172.21.15.156:0/3916477760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d40cd7c0 msgr2=0x7fa3d41577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:13:54.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.883+0000 7fa3dc3ff700 1 --2- 172.21.15.156:0/3916477760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d40cd7c0 0x7fa3d41577e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000e760 tx=0x7fa3d000eb20 comp rx=0 tx=0).stop 2023-11-28T13:13:54.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.883+0000 7fa3dc3ff700 1 -- 172.21.15.156:0/3916477760 shutdown_connections 2023-11-28T13:13:54.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.883+0000 7fa3dc3ff700 1 --2- 172.21.15.156:0/3916477760 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fa3b80682f0 0x7fa3b806a7a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:54.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.883+0000 7fa3dc3ff700 1 --2- 172.21.15.156:0/3916477760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa3d4161340 0x7fa3d415ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:54.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.883+0000 7fa3dc3ff700 1 --2- 172.21.15.156:0/3916477760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d40cd7c0 0x7fa3d41577e0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:54.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.883+0000 7fa3dc3ff700 1 --2- 172.21.15.156:0/3916477760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3d415c9b0 0x7fa3d415a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:13:54.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.883+0000 7fa3dc3ff700 1 -- 172.21.15.156:0/3916477760 >> 172.21.15.156:0/3916477760 conn(0x7fa3d4072600 msgr2=0x7fa3d40b82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:13:54.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.884+0000 7fa3dc3ff700 1 -- 172.21.15.156:0/3916477760 shutdown_connections 2023-11-28T13:13:54.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:13:54.884+0000 7fa3dc3ff700 1 -- 172.21.15.156:0/3916477760 wait complete. 2023-11-28T13:13:54.902 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:13:59.903 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:14:00.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.108+0000 7fe6227ff700 1 Processor -- start 2023-11-28T13:14:00.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.109+0000 7fe6227ff700 1 -- start start 2023-11-28T13:14:00.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.110+0000 7fe6227ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe61c161340 0x7fe61c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:00.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.110+0000 7fe6227ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61c0d1950 con 0x7fe61c15c9b0 2023-11-28T13:14:00.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.110+0000 7fe6227ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61c0b5520 con 0x7fe61c0d2320 2023-11-28T13:14:00.111 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.110+0000 7fe6227ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe61c0b5800 con 0x7fe61c161340 2023-11-28T13:14:00.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.110+0000 7fe61bfff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fe61c15c9b0 0x7fe61c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:56578/0 (socket says 172.21.15.156:56578) 2023-11-28T13:14:00.112 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.110+0000 7fe61bfff700 1 -- 172.21.15.156:0/2760056212 learned_addr learned my addr 172.21.15.156:0/2760056212 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:14:00.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.110+0000 7fe61b7fe700 1 --2- 172.21.15.156:0/2760056212 >> v2:172.21.15.186:3300/0 conn(0x7fe61c161340 0x7fe61c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:00.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.110+0000 7fe61affd700 1 -- 172.21.15.156:0/2760056212 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3653065654 0 0) 0x7fe61c0d1950 con 0x7fe61c15c9b0 2023-11-28T13:14:00.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.110+0000 7fe61affd700 1 -- 172.21.15.156:0/2760056212 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6040035b0 con 0x7fe61c15c9b0 2023-11-28T13:14:00.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61affd700 1 -- 172.21.15.156:0/2760056212 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562550683 0 0) 0x7fe61c0b5520 con 0x7fe61c0d2320 2023-11-28T13:14:00.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61affd700 1 -- 172.21.15.156:0/2760056212 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe61c0d1950 con 0x7fe61c0d2320 2023-11-28T13:14:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61affd700 1 -- 172.21.15.156:0/2760056212 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 300331542 0 0) 0x7fe6040035b0 con 0x7fe61c15c9b0 2023-11-28T13:14:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61affd700 1 -- 172.21.15.156:0/2760056212 >> v2:172.21.15.186:3300/0 conn(0x7fe61c161340 msgr2=0x7fe61c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:14:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61affd700 1 --2- 172.21.15.156:0/2760056212 >> v2:172.21.15.186:3300/0 conn(0x7fe61c161340 0x7fe61c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61affd700 1 -- 172.21.15.156:0/2760056212 >> v1:172.21.15.186:6789/0 conn(0x7fe61c0d2320 legacy=0x7fe61c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61affd700 1 -- 172.21.15.156:0/2760056212 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c0d1300 con 0x7fe61c15c9b0 2023-11-28T13:14:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61affd700 1 -- 172.21.15.156:0/2760056212 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7fe60c001670 con 0x7fe61c15c9b0 2023-11-28T13:14:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61affd700 1 -- 172.21.15.156:0/2760056212 >> v1:172.21.15.156:6789/0 conn(0x7fe61c15c9b0 legacy=0x7fe61c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61affd700 1 --2- 172.21.15.156:0/2760056212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe604003be0 0x7fe604003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:00.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61affd700 1 --2- 172.21.15.156:0/2760056212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe604004850 0x7fe604004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61affd700 1 --2- 172.21.15.156:0/2760056212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe604005250 0x7fe60400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61affd700 1 -- 172.21.15.156:0/2760056212 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c0b5800 con 0x7fe604003be0 2023-11-28T13:14:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61affd700 1 -- 172.21.15.156:0/2760056212 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60400c160 con 0x7fe604005250 2023-11-28T13:14:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61affd700 1 -- 172.21.15.156:0/2760056212 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60400c320 con 0x7fe604004850 2023-11-28T13:14:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61bfff700 1 --2- 172.21.15.156:0/2760056212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe604003be0 0x7fe604003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe620d9c700 1 --2- 172.21.15.156:0/2760056212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe604005250 0x7fe60400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61b7fe700 1 --2- 172.21.15.156:0/2760056212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe604004850 0x7fe604004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61bfff700 1 -- 172.21.15.156:0/2760056212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe604004850 msgr2=0x7fe604004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:14:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.111+0000 7fe61bfff700 1 --2- 172.21.15.156:0/2760056212 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe604004850 0x7fe604004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:00.117 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.112+0000 7fe61bfff700 1 -- 172.21.15.156:0/2760056212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe604005250 msgr2=0x7fe60400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:14:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.112+0000 7fe61bfff700 1 --2- 172.21.15.156:0/2760056212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe604005250 0x7fe60400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.112+0000 7fe61bfff700 1 -- 172.21.15.156:0/2760056212 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6040035b0 con 0x7fe604003be0 2023-11-28T13:14:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.112+0000 7fe620d9c700 1 --2- 172.21.15.156:0/2760056212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe604005250 0x7fe60400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:14:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.112+0000 7fe620d9c700 1 -- 172.21.15.156:0/2760056212 reap_dead start 2023-11-28T13:14:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.112+0000 7fe61bfff700 1 --2- 172.21.15.156:0/2760056212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe604003be0 0x7fe604003fd0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe60c001eb0 tx=0x7fe604005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f642b67f550e8b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.112+0000 7fe61affd700 1 -- 172.21.15.156:0/2760056212 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00cc70 con 0x7fe604003be0 2023-11-28T13:14:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.112+0000 7fe61affd700 1 -- 172.21.15.156:0/2760056212 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c00cdf0 con 0x7fe604003be0 2023-11-28T13:14:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.112+0000 7fe61affd700 1 -- 172.21.15.156:0/2760056212 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c005ba0 con 0x7fe604003be0 2023-11-28T13:14:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.112+0000 7fe6227ff700 1 -- 172.21.15.156:0/2760056212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe604003be0 msgr2=0x7fe604003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.112+0000 7fe6227ff700 1 --2- 172.21.15.156:0/2760056212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe604003be0 0x7fe604003fd0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe60c001eb0 tx=0x7fe604005880 comp rx=0 tx=0).stop 2023-11-28T13:14:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.113+0000 7fe6227ff700 1 -- 172.21.15.156:0/2760056212 shutdown_connections 2023-11-28T13:14:00.118 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.113+0000 7fe6227ff700 1 --2- 172.21.15.156:0/2760056212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe604003be0 0x7fe604003fd0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:00.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.113+0000 7fe6227ff700 1 -- 172.21.15.156:0/2760056212 >> 172.21.15.156:0/2760056212 conn(0x7fe61c072600 msgr2=0x7fe61c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:00.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.113+0000 7fe6227ff700 1 -- 172.21.15.156:0/2760056212 shutdown_connections 2023-11-28T13:14:00.119 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.113+0000 7fe6227ff700 1 -- 172.21.15.156:0/2760056212 wait complete. 2023-11-28T13:14:00.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.114+0000 7fe6227ff700 1 Processor -- start 2023-11-28T13:14:00.121 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.115+0000 7fe6227ff700 1 -- start start 2023-11-28T13:14:00.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.115+0000 7fe6227ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe604003be0 0x7fe61c1f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:00.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.115+0000 7fe6227ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe61c1f6df0 0x7fe61c151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:00.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.115+0000 7fe6227ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe61c1596d0 0x7fe61c159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:00.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.115+0000 7fe6227ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c15a080 con 0x7fe61c1596d0 2023-11-28T13:14:00.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.115+0000 7fe6227ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c1f3c10 con 0x7fe604003be0 2023-11-28T13:14:00.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.115+0000 7fe6227ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c1f6960 con 0x7fe61c1f6df0 2023-11-28T13:14:00.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.116+0000 7fe620d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe61c1596d0 0x7fe61c159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:00.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.116+0000 7fe620d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe61c1596d0 0x7fe61c159b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:52042/0 (socket says 172.21.15.156:52042) 2023-11-28T13:14:00.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.116+0000 7fe620d9c700 1 -- 172.21.15.156:0/3955373011 learned_addr learned my addr 172.21.15.156:0/3955373011 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:00.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.116+0000 7fe61bfff700 1 --2- 172.21.15.156:0/3955373011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe604003be0 0x7fe61c1f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:00.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.116+0000 7fe61b7fe700 1 --2- 172.21.15.156:0/3955373011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe61c1f6df0 0x7fe61c151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:00.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.116+0000 7fe620d9c700 1 -- 172.21.15.156:0/3955373011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe61c1f6df0 msgr2=0x7fe61c151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:00.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.116+0000 7fe620d9c700 1 --2- 172.21.15.156:0/3955373011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe61c1f6df0 0x7fe61c151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:00.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.116+0000 7fe620d9c700 1 -- 172.21.15.156:0/3955373011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe604003be0 msgr2=0x7fe61c1f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:00.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.116+0000 7fe620d9c700 1 --2- 172.21.15.156:0/3955373011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe604003be0 0x7fe61c1f36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:00.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.116+0000 7fe620d9c700 1 -- 172.21.15.156:0/3955373011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c15cd80 con 0x7fe61c1596d0 2023-11-28T13:14:00.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.116+0000 7fe620d9c700 1 --2- 172.21.15.156:0/3955373011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe61c1596d0 0x7fe61c159b40 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe61c08bfd0 tx=0x7fe61400c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:00.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.116+0000 7fe618ff9700 1 -- 172.21.15.156:0/3955373011 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe614003c10 con 0x7fe61c1596d0 2023-11-28T13:14:00.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.116+0000 7fe618ff9700 1 -- 172.21.15.156:0/3955373011 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe614003d70 con 0x7fe61c1596d0 2023-11-28T13:14:00.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.116+0000 7fe6227ff700 1 -- 172.21.15.156:0/3955373011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61c0d15e0 con 0x7fe61c1596d0 2023-11-28T13:14:00.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.117+0000 7fe618ff9700 1 -- 172.21.15.156:0/3955373011 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61400c920 con 0x7fe61c1596d0 2023-11-28T13:14:00.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.117+0000 7fe6227ff700 1 -- 172.21.15.156:0/3955373011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61c15cf10 con 0x7fe61c1596d0 2023-11-28T13:14:00.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.118+0000 7fe6227ff700 1 -- 172.21.15.156:0/3955373011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61c0745c0 con 0x7fe61c1596d0 2023-11-28T13:14:00.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.118+0000 7fe618ff9700 1 -- 172.21.15.156:0/3955373011 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe61400ca80 con 0x7fe61c1596d0 2023-11-28T13:14:00.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.119+0000 7fe618ff9700 1 --2- 172.21.15.156:0/3955373011 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6080682f0 0x7fe60806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:00.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.119+0000 7fe618ff9700 1 -- 172.21.15.156:0/3955373011 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fe6140860b0 con 0x7fe61c1596d0 2023-11-28T13:14:00.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.120+0000 7fe61bfff700 1 --2- 172.21.15.156:0/3955373011 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6080682f0 0x7fe60806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:00.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.120+0000 7fe61bfff700 1 --2- 172.21.15.156:0/3955373011 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6080682f0 0x7fe60806a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe60c000c00 tx=0x7fe60c007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:00.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.124+0000 7fe618ff9700 1 -- 172.21.15.156:0/3955373011 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe614052790 con 0x7fe61c1596d0 2023-11-28T13:14:00.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.357+0000 7fe6227ff700 1 -- 172.21.15.156:0/3955373011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe61c1f6aa0 con 0x7fe61c1596d0 2023-11-28T13:14:00.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.358+0000 7fe618ff9700 1 -- 172.21.15.156:0/3955373011 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fe614055db0 con 0x7fe61c1596d0 2023-11-28T13:14:00.360 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:14:00.360 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:14:00.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.360+0000 7fe6227ff700 1 -- 172.21.15.156:0/3955373011 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6080682f0 msgr2=0x7fe60806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:00.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.360+0000 7fe6227ff700 1 --2- 172.21.15.156:0/3955373011 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6080682f0 0x7fe60806a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe60c000c00 tx=0x7fe60c007000 comp rx=0 tx=0).stop 2023-11-28T13:14:00.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.360+0000 7fe6227ff700 1 -- 172.21.15.156:0/3955373011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe61c1596d0 msgr2=0x7fe61c159b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:00.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.360+0000 7fe6227ff700 1 --2- 172.21.15.156:0/3955373011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe61c1596d0 0x7fe61c159b40 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe61c08bfd0 tx=0x7fe61400c0e0 comp rx=0 tx=0).stop 2023-11-28T13:14:00.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.360+0000 7fe6227ff700 1 -- 172.21.15.156:0/3955373011 shutdown_connections 2023-11-28T13:14:00.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.360+0000 7fe6227ff700 1 --2- 172.21.15.156:0/3955373011 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe6080682f0 0x7fe60806a7a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:00.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.360+0000 7fe6227ff700 1 --2- 172.21.15.156:0/3955373011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe61c1596d0 0x7fe61c159b40 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:00.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.360+0000 7fe6227ff700 1 --2- 172.21.15.156:0/3955373011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe604003be0 0x7fe61c1f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:00.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.360+0000 7fe6227ff700 1 --2- 172.21.15.156:0/3955373011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe61c1f6df0 0x7fe61c151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:00.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.360+0000 7fe6227ff700 1 -- 172.21.15.156:0/3955373011 >> 172.21.15.156:0/3955373011 conn(0x7fe61c072600 msgr2=0x7fe61c160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:00.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.360+0000 7fe6227ff700 1 -- 172.21.15.156:0/3955373011 shutdown_connections 2023-11-28T13:14:00.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:00.360+0000 7fe6227ff700 1 -- 172.21.15.156:0/3955373011 wait complete. 2023-11-28T13:14:00.380 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:14:04.134 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:14:04.139 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:14:05.381 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:14:05.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.584+0000 7f0796bff700 1 Processor -- start 2023-11-28T13:14:05.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.584+0000 7f0796bff700 1 -- start start 2023-11-28T13:14:05.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.585+0000 7f0796bff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f079015c9b0 0x7f07900b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:05.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.586+0000 7f0796bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0790161340 0x7f07900d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:05.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.586+0000 7f0796bff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07900d1950 con 0x7f07900d2320 2023-11-28T13:14:05.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.586+0000 7f0796bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07900b5800 con 0x7f0790161340 2023-11-28T13:14:05.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.586+0000 7f0796bff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07900b5520 con 0x7f079015c9b0 2023-11-28T13:14:05.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.586+0000 7f078ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0790161340 0x7f07900d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:05.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.586+0000 7f078ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0790161340 0x7f07900d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:52048/0 (socket says 172.21.15.156:52048) 2023-11-28T13:14:05.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.586+0000 7f078ffff700 1 -- 172.21.15.156:0/2291558468 learned_addr learned my addr 172.21.15.156:0/2291558468 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:05.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.586+0000 7f079499b700 1 --2- 172.21.15.156:0/2291558468 >> v2:172.21.15.186:3300/0 conn(0x7f079015c9b0 0x7f07900b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:05.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.586+0000 7f078ffff700 1 -- 172.21.15.156:0/2291558468 >> v2:172.21.15.186:3300/0 conn(0x7f079015c9b0 msgr2=0x7f07900b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:14:05.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.586+0000 7f078ffff700 1 --2- 172.21.15.156:0/2291558468 >> v2:172.21.15.186:3300/0 conn(0x7f079015c9b0 0x7f07900b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:05.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.586+0000 7f078ffff700 1 -- 172.21.15.156:0/2291558468 >> v1:172.21.15.186:6789/0 conn(0x7f07900d2320 legacy=0x7f07900cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:05.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.586+0000 7f078ffff700 1 -- 172.21.15.156:0/2291558468 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07900d1300 con 0x7f0790161340 2023-11-28T13:14:05.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.587+0000 7f078ffff700 1 --2- 172.21.15.156:0/2291558468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0790161340 0x7f07900d1de0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0790067460 tx=0x7f077c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9752e95d2cca2f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:05.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.587+0000 7f078f7fe700 1 -- 172.21.15.156:0/2291558468 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c00b870 con 0x7f0790161340 2023-11-28T13:14:05.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.587+0000 7f078f7fe700 1 -- 172.21.15.156:0/2291558468 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077c00b9d0 con 0x7f0790161340 2023-11-28T13:14:05.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.587+0000 7f078f7fe700 1 -- 172.21.15.156:0/2291558468 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c00bd60 con 0x7f0790161340 2023-11-28T13:14:05.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.587+0000 7f0796bff700 1 -- 172.21.15.156:0/2291558468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0790161340 msgr2=0x7f07900d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:05.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.587+0000 7f0796bff700 1 --2- 172.21.15.156:0/2291558468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0790161340 0x7f07900d1de0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0790067460 tx=0x7f077c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:14:05.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.587+0000 7f0796bff700 1 -- 172.21.15.156:0/2291558468 shutdown_connections 2023-11-28T13:14:05.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.587+0000 7f0796bff700 1 --2- 172.21.15.156:0/2291558468 >> v2:172.21.15.186:3300/0 conn(0x7f079015c9b0 0x7f07900b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:05.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.587+0000 7f0796bff700 1 --2- 172.21.15.156:0/2291558468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0790161340 0x7f07900d1de0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:05.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.587+0000 7f0796bff700 1 -- 172.21.15.156:0/2291558468 >> 172.21.15.156:0/2291558468 conn(0x7f0790072600 msgr2=0x7f07900cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:05.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.588+0000 7f0796bff700 1 -- 172.21.15.156:0/2291558468 shutdown_connections 2023-11-28T13:14:05.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.588+0000 7f0796bff700 1 -- 172.21.15.156:0/2291558468 wait complete. 2023-11-28T13:14:05.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.589+0000 7f0796bff700 1 Processor -- start 2023-11-28T13:14:05.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.589+0000 7f0796bff700 1 -- start start 2023-11-28T13:14:05.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.590+0000 7f0796bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07900d2320 0x7f0790143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:05.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.590+0000 7f0796bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f079015c9b0 0x7f07900c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:05.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.590+0000 7f0796bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0790161340 0x7f07900c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:05.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.590+0000 7f0796bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0790153580 con 0x7f0790161340 2023-11-28T13:14:05.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.590+0000 7f0796bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07900c6170 con 0x7f07900d2320 2023-11-28T13:14:05.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.590+0000 7f0796bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07900c7680 con 0x7f079015c9b0 2023-11-28T13:14:05.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.591+0000 7f079499b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07900d2320 0x7f0790143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:05.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.591+0000 7f079519c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0790161340 0x7f07900c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:05.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.591+0000 7f079499b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07900d2320 0x7f0790143e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:35142/0 (socket says 172.21.15.156:35142) 2023-11-28T13:14:05.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.591+0000 7f079499b700 1 -- 172.21.15.156:0/4264566482 learned_addr learned my addr 172.21.15.156:0/4264566482 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:05.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.591+0000 7f078ffff700 1 --2- 172.21.15.156:0/4264566482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f079015c9b0 0x7f07900c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:05.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.591+0000 7f079519c700 1 -- 172.21.15.156:0/4264566482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f079015c9b0 msgr2=0x7f07900c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:05.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.591+0000 7f079519c700 1 --2- 172.21.15.156:0/4264566482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f079015c9b0 0x7f07900c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:05.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.591+0000 7f079519c700 1 -- 172.21.15.156:0/4264566482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07900d2320 msgr2=0x7f0790143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:05.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.591+0000 7f079519c700 1 --2- 172.21.15.156:0/4264566482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07900d2320 0x7f0790143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:05.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.591+0000 7f079519c700 1 -- 172.21.15.156:0/4264566482 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c009550 con 0x7f0790161340 2023-11-28T13:14:05.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.591+0000 7f079499b700 1 --2- 172.21.15.156:0/4264566482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07900d2320 0x7f0790143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:14:05.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.591+0000 7f079519c700 1 --2- 172.21.15.156:0/4264566482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0790161340 0x7f07900c6ca0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0790067550 tx=0x7f078800cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:05.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.592+0000 7f078d7fa700 1 -- 172.21.15.156:0/4264566482 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788003c00 con 0x7f0790161340 2023-11-28T13:14:05.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.592+0000 7f078d7fa700 1 -- 172.21.15.156:0/4264566482 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0788003d60 con 0x7f0790161340 2023-11-28T13:14:05.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.592+0000 7f0796bff700 1 -- 172.21.15.156:0/4264566482 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07900b5130 con 0x7f0790161340 2023-11-28T13:14:05.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.592+0000 7f078d7fa700 1 -- 172.21.15.156:0/4264566482 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078800aa20 con 0x7f0790161340 2023-11-28T13:14:05.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.592+0000 7f0796bff700 1 -- 172.21.15.156:0/4264566482 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07900d15e0 con 0x7f0790161340 2023-11-28T13:14:05.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.593+0000 7f0796bff700 1 -- 172.21.15.156:0/4264566482 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07900b52c0 con 0x7f0790161340 2023-11-28T13:14:05.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.594+0000 7f078d7fa700 1 -- 172.21.15.156:0/4264566482 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0788010050 con 0x7f0790161340 2023-11-28T13:14:05.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.595+0000 7f078d7fa700 1 --2- 172.21.15.156:0/4264566482 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f07740682f0 0x7f077406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:05.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.595+0000 7f078d7fa700 1 -- 172.21.15.156:0/4264566482 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f0788085b00 con 0x7f0790161340 2023-11-28T13:14:05.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.595+0000 7f079499b700 1 --2- 172.21.15.156:0/4264566482 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f07740682f0 0x7f077406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:05.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.596+0000 7f079499b700 1 --2- 172.21.15.156:0/4264566482 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f07740682f0 0x7f077406a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0784009e80 tx=0x7f0784008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:05.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.599+0000 7f078d7fa700 1 -- 172.21.15.156:0/4264566482 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0788087e60 con 0x7f0790161340 2023-11-28T13:14:05.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.830+0000 7f0796bff700 1 -- 172.21.15.156:0/4264566482 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f07900745c0 con 0x7f0790161340 2023-11-28T13:14:05.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.831+0000 7f078d7fa700 1 -- 172.21.15.156:0/4264566482 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f07880521e0 con 0x7f0790161340 2023-11-28T13:14:05.833 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:14:05.833 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:14:05.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.833+0000 7f0796bff700 1 -- 172.21.15.156:0/4264566482 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f07740682f0 msgr2=0x7f077406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:05.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.833+0000 7f0796bff700 1 --2- 172.21.15.156:0/4264566482 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f07740682f0 0x7f077406a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0784009e80 tx=0x7f0784008040 comp rx=0 tx=0).stop 2023-11-28T13:14:05.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.833+0000 7f0796bff700 1 -- 172.21.15.156:0/4264566482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0790161340 msgr2=0x7f07900c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:05.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.833+0000 7f0796bff700 1 --2- 172.21.15.156:0/4264566482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0790161340 0x7f07900c6ca0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0790067550 tx=0x7f078800cdc0 comp rx=0 tx=0).stop 2023-11-28T13:14:05.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.833+0000 7f0796bff700 1 -- 172.21.15.156:0/4264566482 shutdown_connections 2023-11-28T13:14:05.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.834+0000 7f0796bff700 1 --2- 172.21.15.156:0/4264566482 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f07740682f0 0x7f077406a7a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:05.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.834+0000 7f0796bff700 1 --2- 172.21.15.156:0/4264566482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0790161340 0x7f07900c6ca0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:05.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.834+0000 7f0796bff700 1 --2- 172.21.15.156:0/4264566482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07900d2320 0x7f0790143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:05.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.834+0000 7f0796bff700 1 --2- 172.21.15.156:0/4264566482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f079015c9b0 0x7f07900c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:05.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.834+0000 7f0796bff700 1 -- 172.21.15.156:0/4264566482 >> 172.21.15.156:0/4264566482 conn(0x7f0790072600 msgr2=0x7f079015a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:05.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.834+0000 7f0796bff700 1 -- 172.21.15.156:0/4264566482 shutdown_connections 2023-11-28T13:14:05.838 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:05.834+0000 7f0796bff700 1 -- 172.21.15.156:0/4264566482 wait complete. 2023-11-28T13:14:05.854 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:14:10.855 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:14:11.060 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.059+0000 7f4d059ff700 1 Processor -- start 2023-11-28T13:14:11.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.059+0000 7f4d059ff700 1 -- start start 2023-11-28T13:14:11.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.060+0000 7f4d059ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4d0015c9b0 0x7f4d000b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:11.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.060+0000 7f4d059ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4d00161340 0x7f4d000d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:11.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.060+0000 7f4d059ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d000d1950 con 0x7f4d000d2320 2023-11-28T13:14:11.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.060+0000 7f4d059ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d000b5800 con 0x7f4d00161340 2023-11-28T13:14:11.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.060+0000 7f4d059ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d000b5520 con 0x7f4d0015c9b0 2023-11-28T13:14:11.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.060+0000 7f4cf7fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4d00161340 0x7f4d000d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:11.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.060+0000 7f4cf7fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4d00161340 0x7f4d000d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59524/0 (socket says 172.21.15.156:59524) 2023-11-28T13:14:11.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.060+0000 7f4cf7fff700 1 -- 172.21.15.156:0/3056994697 learned_addr learned my addr 172.21.15.156:0/3056994697 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:11.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.060+0000 7f4cfeffd700 1 --2- 172.21.15.156:0/3056994697 >> v2:172.21.15.186:3300/0 conn(0x7f4d0015c9b0 0x7f4d000b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:11.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.061+0000 7f4cf7fff700 1 -- 172.21.15.156:0/3056994697 >> v2:172.21.15.186:3300/0 conn(0x7f4d0015c9b0 msgr2=0x7f4d000b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:14:11.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.061+0000 7f4cf7fff700 1 --2- 172.21.15.156:0/3056994697 >> v2:172.21.15.186:3300/0 conn(0x7f4d0015c9b0 0x7f4d000b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:11.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.061+0000 7f4cf7fff700 1 -- 172.21.15.156:0/3056994697 >> v1:172.21.15.186:6789/0 conn(0x7f4d000d2320 legacy=0x7f4d000cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:11.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.061+0000 7f4cf7fff700 1 -- 172.21.15.156:0/3056994697 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d000d1300 con 0x7f4d00161340 2023-11-28T13:14:11.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.061+0000 7f4cf7fff700 1 --2- 172.21.15.156:0/3056994697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d00161340 0x7f4d000d1de0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4d00067460 tx=0x7f4ce8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cce625e10004106 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:11.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.061+0000 7f4cfe7fc700 1 -- 172.21.15.156:0/3056994697 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce800b870 con 0x7f4d00161340 2023-11-28T13:14:11.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.061+0000 7f4cfe7fc700 1 -- 172.21.15.156:0/3056994697 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce800b9d0 con 0x7f4d00161340 2023-11-28T13:14:11.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.061+0000 7f4cfe7fc700 1 -- 172.21.15.156:0/3056994697 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce800bd60 con 0x7f4d00161340 2023-11-28T13:14:11.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.062+0000 7f4d059ff700 1 -- 172.21.15.156:0/3056994697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d00161340 msgr2=0x7f4d000d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:11.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.062+0000 7f4d059ff700 1 --2- 172.21.15.156:0/3056994697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d00161340 0x7f4d000d1de0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4d00067460 tx=0x7f4ce8009bd0 comp rx=0 tx=0).stop 2023-11-28T13:14:11.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.062+0000 7f4d059ff700 1 -- 172.21.15.156:0/3056994697 shutdown_connections 2023-11-28T13:14:11.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.062+0000 7f4d059ff700 1 --2- 172.21.15.156:0/3056994697 >> v2:172.21.15.186:3300/0 conn(0x7f4d0015c9b0 0x7f4d000b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:11.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.062+0000 7f4d059ff700 1 --2- 172.21.15.156:0/3056994697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d00161340 0x7f4d000d1de0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:11.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.062+0000 7f4d059ff700 1 -- 172.21.15.156:0/3056994697 >> 172.21.15.156:0/3056994697 conn(0x7f4d00072600 msgr2=0x7f4d000cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:11.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.062+0000 7f4d059ff700 1 -- 172.21.15.156:0/3056994697 shutdown_connections 2023-11-28T13:14:11.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.062+0000 7f4d059ff700 1 -- 172.21.15.156:0/3056994697 wait complete. 2023-11-28T13:14:11.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.064+0000 7f4d059ff700 1 Processor -- start 2023-11-28T13:14:11.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.064+0000 7f4d059ff700 1 -- start start 2023-11-28T13:14:11.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.065+0000 7f4d059ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000d2320 0x7f4d0014ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:11.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.065+0000 7f4d059ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d0015c9b0 0x7f4d0015e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:11.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.065+0000 7f4d059ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d00161340 0x7f4d0015e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:11.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.065+0000 7f4d059ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d001f5e30 con 0x7f4d0015c9b0 2023-11-28T13:14:11.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.065+0000 7f4d059ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d001f60d0 con 0x7f4d000d2320 2023-11-28T13:14:11.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.065+0000 7f4d059ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d001f63b0 con 0x7f4d00161340 2023-11-28T13:14:11.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.065+0000 7f4cf7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d0015c9b0 0x7f4d0015e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:11.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.065+0000 7f4cf7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d0015c9b0 0x7f4d0015e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59530/0 (socket says 172.21.15.156:59530) 2023-11-28T13:14:11.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.065+0000 7f4cf7fff700 1 -- 172.21.15.156:0/4097246569 learned_addr learned my addr 172.21.15.156:0/4097246569 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:11.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.065+0000 7f4cfeffd700 1 --2- 172.21.15.156:0/4097246569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000d2320 0x7f4d0014ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:11.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.065+0000 7f4cff7fe700 1 --2- 172.21.15.156:0/4097246569 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d00161340 0x7f4d0015e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:11.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.065+0000 7f4cf7fff700 1 -- 172.21.15.156:0/4097246569 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d00161340 msgr2=0x7f4d0015e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:11.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.065+0000 7f4cf7fff700 1 --2- 172.21.15.156:0/4097246569 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d00161340 0x7f4d0015e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:11.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.065+0000 7f4cf7fff700 1 -- 172.21.15.156:0/4097246569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000d2320 msgr2=0x7f4d0014ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:11.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.065+0000 7f4cf7fff700 1 --2- 172.21.15.156:0/4097246569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000d2320 0x7f4d0014ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:11.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.065+0000 7f4cf7fff700 1 -- 172.21.15.156:0/4097246569 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce8009550 con 0x7f4d0015c9b0 2023-11-28T13:14:11.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.065+0000 7f4cfeffd700 1 --2- 172.21.15.156:0/4097246569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000d2320 0x7f4d0014ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:14:11.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.066+0000 7f4cf7fff700 1 --2- 172.21.15.156:0/4097246569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d0015c9b0 0x7f4d0015e260 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8000c00 tx=0x7f4ce8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:11.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.066+0000 7f4cfcff9700 1 -- 172.21.15.156:0/4097246569 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8004540 con 0x7f4d0015c9b0 2023-11-28T13:14:11.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.066+0000 7f4cfcff9700 1 -- 172.21.15.156:0/4097246569 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce80036a0 con 0x7f4d0015c9b0 2023-11-28T13:14:11.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.066+0000 7f4d059ff700 1 -- 172.21.15.156:0/4097246569 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d000b5130 con 0x7f4d0015c9b0 2023-11-28T13:14:11.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.066+0000 7f4cfcff9700 1 -- 172.21.15.156:0/4097246569 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8003a30 con 0x7f4d0015c9b0 2023-11-28T13:14:11.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.066+0000 7f4d059ff700 1 -- 172.21.15.156:0/4097246569 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d000d15e0 con 0x7f4d0015c9b0 2023-11-28T13:14:11.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.067+0000 7f4d059ff700 1 -- 172.21.15.156:0/4097246569 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d000b52c0 con 0x7f4d0015c9b0 2023-11-28T13:14:11.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.068+0000 7f4cfcff9700 1 -- 172.21.15.156:0/4097246569 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4ce800f050 con 0x7f4d0015c9b0 2023-11-28T13:14:11.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.069+0000 7f4cfcff9700 1 --2- 172.21.15.156:0/4097246569 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4ce0068320 0x7f4ce006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:11.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.069+0000 7f4cfcff9700 1 -- 172.21.15.156:0/4097246569 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f4ce80877a0 con 0x7f4d0015c9b0 2023-11-28T13:14:11.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.069+0000 7f4cfeffd700 1 --2- 172.21.15.156:0/4097246569 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4ce0068320 0x7f4ce006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:11.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.070+0000 7f4cfeffd700 1 --2- 172.21.15.156:0/4097246569 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4ce0068320 0x7f4ce006a7d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00099f0 tx=0x7f4cf0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:11.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.074+0000 7f4cfcff9700 1 -- 172.21.15.156:0/4097246569 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4ce8053e80 con 0x7f4d0015c9b0 2023-11-28T13:14:11.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.304+0000 7f4d059ff700 1 -- 172.21.15.156:0/4097246569 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4d000745c0 con 0x7f4d0015c9b0 2023-11-28T13:14:11.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.305+0000 7f4cfcff9700 1 -- 172.21.15.156:0/4097246569 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f4ce8085e60 con 0x7f4d0015c9b0 2023-11-28T13:14:11.307 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:14:11.307 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:14:11.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.307+0000 7f4d059ff700 1 -- 172.21.15.156:0/4097246569 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4ce0068320 msgr2=0x7f4ce006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:11.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.307+0000 7f4d059ff700 1 --2- 172.21.15.156:0/4097246569 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4ce0068320 0x7f4ce006a7d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00099f0 tx=0x7f4cf0008040 comp rx=0 tx=0).stop 2023-11-28T13:14:11.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.307+0000 7f4d059ff700 1 -- 172.21.15.156:0/4097246569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d0015c9b0 msgr2=0x7f4d0015e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:11.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.307+0000 7f4d059ff700 1 --2- 172.21.15.156:0/4097246569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d0015c9b0 0x7f4d0015e260 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8000c00 tx=0x7f4ce8004750 comp rx=0 tx=0).stop 2023-11-28T13:14:11.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.307+0000 7f4d059ff700 1 -- 172.21.15.156:0/4097246569 shutdown_connections 2023-11-28T13:14:11.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.307+0000 7f4d059ff700 1 --2- 172.21.15.156:0/4097246569 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f4ce0068320 0x7f4ce006a7d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:11.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.307+0000 7f4d059ff700 1 --2- 172.21.15.156:0/4097246569 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4d00161340 0x7f4d0015e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:11.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.307+0000 7f4d059ff700 1 --2- 172.21.15.156:0/4097246569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d000d2320 0x7f4d0014ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:11.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.307+0000 7f4d059ff700 1 --2- 172.21.15.156:0/4097246569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d0015c9b0 0x7f4d0015e260 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:11.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.307+0000 7f4d059ff700 1 -- 172.21.15.156:0/4097246569 >> 172.21.15.156:0/4097246569 conn(0x7f4d00072600 msgr2=0x7f4d000b8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:11.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.307+0000 7f4d059ff700 1 -- 172.21.15.156:0/4097246569 shutdown_connections 2023-11-28T13:14:11.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:11.307+0000 7f4d059ff700 1 -- 172.21.15.156:0/4097246569 wait complete. 2023-11-28T13:14:11.328 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:14:16.329 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:14:16.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.532+0000 7f12d95ff700 1 Processor -- start 2023-11-28T13:14:16.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.533+0000 7f12d95ff700 1 -- start start 2023-11-28T13:14:16.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.534+0000 7f12d95ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f12d415c9b0 0x7f12d40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:16.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.534+0000 7f12d95ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12d40cd7c0 0x7f12d40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:16.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.534+0000 7f12d95ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d40d1950 con 0x7f12d4161340 2023-11-28T13:14:16.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.534+0000 7f12d95ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d40b5800 con 0x7f12d415c9b0 2023-11-28T13:14:16.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.534+0000 7f12d95ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40ce140 con 0x7f12d40cd7c0 2023-11-28T13:14:16.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.534+0000 7f12d2ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f12d415c9b0 0x7f12d40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:16.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.534+0000 7f12d2ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f12d415c9b0 0x7f12d40b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36000/0 (socket says 172.21.15.156:36000) 2023-11-28T13:14:16.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.534+0000 7f12d2ffd700 1 -- 172.21.15.156:0/163981165 learned_addr learned my addr 172.21.15.156:0/163981165 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:16.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.534+0000 7f12d37fe700 1 --2- 172.21.15.156:0/163981165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12d40cd7c0 0x7f12d40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:16.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.535+0000 7f12d2ffd700 1 -- 172.21.15.156:0/163981165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12d40cd7c0 msgr2=0x7f12d40cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:14:16.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.535+0000 7f12d2ffd700 1 --2- 172.21.15.156:0/163981165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12d40cd7c0 0x7f12d40cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:16.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.535+0000 7f12d2ffd700 1 -- 172.21.15.156:0/163981165 >> v1:172.21.15.186:6789/0 conn(0x7f12d4161340 legacy=0x7f12d40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:16.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.535+0000 7f12d2ffd700 1 -- 172.21.15.156:0/163981165 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d40d24f0 con 0x7f12d415c9b0 2023-11-28T13:14:16.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.535+0000 7f12d2ffd700 1 --2- 172.21.15.156:0/163981165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12d415c9b0 0x7f12d40b4bf0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f12c40098f0 tx=0x7f12c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104f4825c8d69f5e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:16.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.535+0000 7f12d27fc700 1 -- 172.21.15.156:0/163981165 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c400b8a0 con 0x7f12d415c9b0 2023-11-28T13:14:16.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.535+0000 7f12d27fc700 1 -- 172.21.15.156:0/163981165 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c400ba00 con 0x7f12d415c9b0 2023-11-28T13:14:16.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.535+0000 7f12d27fc700 1 -- 172.21.15.156:0/163981165 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c400bd90 con 0x7f12d415c9b0 2023-11-28T13:14:16.538 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.536+0000 7f12d95ff700 1 -- 172.21.15.156:0/163981165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12d415c9b0 msgr2=0x7f12d40b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:16.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.536+0000 7f12d95ff700 1 --2- 172.21.15.156:0/163981165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12d415c9b0 0x7f12d40b4bf0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f12c40098f0 tx=0x7f12c4009c00 comp rx=0 tx=0).stop 2023-11-28T13:14:16.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.536+0000 7f12d95ff700 1 -- 172.21.15.156:0/163981165 shutdown_connections 2023-11-28T13:14:16.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.536+0000 7f12d95ff700 1 --2- 172.21.15.156:0/163981165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12d40cd7c0 0x7f12d40cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:16.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.536+0000 7f12d95ff700 1 --2- 172.21.15.156:0/163981165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12d415c9b0 0x7f12d40b4bf0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:16.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.536+0000 7f12d95ff700 1 -- 172.21.15.156:0/163981165 >> 172.21.15.156:0/163981165 conn(0x7f12d4072600 msgr2=0x7f12d40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:16.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.536+0000 7f12d95ff700 1 -- 172.21.15.156:0/163981165 shutdown_connections 2023-11-28T13:14:16.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.536+0000 7f12d95ff700 1 -- 172.21.15.156:0/163981165 wait complete. 2023-11-28T13:14:16.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.538+0000 7f12d95ff700 1 Processor -- start 2023-11-28T13:14:16.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.538+0000 7f12d95ff700 1 -- start start 2023-11-28T13:14:16.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.540+0000 7f12d95ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12d40cd7c0 0x7f12d40c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:16.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.540+0000 7f12d95ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12d415c9b0 0x7f12d4153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:16.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.540+0000 7f12d95ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12d4161340 0x7f12d406a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:16.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.540+0000 7f12d95ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d4154ef0 con 0x7f12d4161340 2023-11-28T13:14:16.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.540+0000 7f12d95ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d4155d50 con 0x7f12d415c9b0 2023-11-28T13:14:16.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.540+0000 7f12d95ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d4155140 con 0x7f12d40cd7c0 2023-11-28T13:14:16.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.540+0000 7f12d37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12d4161340 0x7f12d406a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:16.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.540+0000 7f12d37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12d4161340 0x7f12d406a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36006/0 (socket says 172.21.15.156:36006) 2023-11-28T13:14:16.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.540+0000 7f12d37fe700 1 -- 172.21.15.156:0/3226661321 learned_addr learned my addr 172.21.15.156:0/3226661321 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:16.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.540+0000 7f12d2ffd700 1 --2- 172.21.15.156:0/3226661321 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12d40cd7c0 0x7f12d40c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:16.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.540+0000 7f12cbfff700 1 --2- 172.21.15.156:0/3226661321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12d415c9b0 0x7f12d4153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:16.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.540+0000 7f12d37fe700 1 -- 172.21.15.156:0/3226661321 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12d40cd7c0 msgr2=0x7f12d40c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:16.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.540+0000 7f12d37fe700 1 --2- 172.21.15.156:0/3226661321 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12d40cd7c0 0x7f12d40c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:16.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.540+0000 7f12d37fe700 1 -- 172.21.15.156:0/3226661321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12d415c9b0 msgr2=0x7f12d4153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:16.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.540+0000 7f12d37fe700 1 --2- 172.21.15.156:0/3226661321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12d415c9b0 0x7f12d4153c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:16.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.540+0000 7f12d37fe700 1 -- 172.21.15.156:0/3226661321 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c4009550 con 0x7f12d4161340 2023-11-28T13:14:16.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.541+0000 7f12d37fe700 1 --2- 172.21.15.156:0/3226661321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12d4161340 0x7f12d406a900 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f12cc007e90 tx=0x7f12cc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:16.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.541+0000 7f12cb7fe700 1 -- 172.21.15.156:0/3226661321 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc00ede0 con 0x7f12d4161340 2023-11-28T13:14:16.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.541+0000 7f12cb7fe700 1 -- 172.21.15.156:0/3226661321 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc014420 con 0x7f12d4161340 2023-11-28T13:14:16.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.541+0000 7f12d95ff700 1 -- 172.21.15.156:0/3226661321 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d40d1300 con 0x7f12d4161340 2023-11-28T13:14:16.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.541+0000 7f12cb7fe700 1 -- 172.21.15.156:0/3226661321 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc0147d0 con 0x7f12d4161340 2023-11-28T13:14:16.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.541+0000 7f12d95ff700 1 -- 172.21.15.156:0/3226661321 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d40b5520 con 0x7f12d4161340 2023-11-28T13:14:16.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.542+0000 7f12d95ff700 1 -- 172.21.15.156:0/3226661321 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d40d15e0 con 0x7f12d4161340 2023-11-28T13:14:16.545 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.543+0000 7f12cb7fe700 1 -- 172.21.15.156:0/3226661321 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f12cc012050 con 0x7f12d4161340 2023-11-28T13:14:16.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.544+0000 7f12cb7fe700 1 --2- 172.21.15.156:0/3226661321 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f12b40682f0 0x7f12b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:16.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.544+0000 7f12cb7fe700 1 -- 172.21.15.156:0/3226661321 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f12cc0863b0 con 0x7f12d4161340 2023-11-28T13:14:16.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.544+0000 7f12d2ffd700 1 --2- 172.21.15.156:0/3226661321 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f12b40682f0 0x7f12b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:16.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.545+0000 7f12d2ffd700 1 --2- 172.21.15.156:0/3226661321 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f12b40682f0 0x7f12b406a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f12c4014040 tx=0x7f12c4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:16.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.548+0000 7f12cb7fe700 1 -- 172.21.15.156:0/3226661321 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f12cc052a90 con 0x7f12d4161340 2023-11-28T13:14:16.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.785+0000 7f12d95ff700 1 -- 172.21.15.156:0/3226661321 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f12d40745c0 con 0x7f12d4161340 2023-11-28T13:14:16.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.786+0000 7f12cb7fe700 1 -- 172.21.15.156:0/3226661321 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f12cc0560b0 con 0x7f12d4161340 2023-11-28T13:14:16.787 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:14:16.787 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:14:16.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.788+0000 7f12d95ff700 1 -- 172.21.15.156:0/3226661321 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f12b40682f0 msgr2=0x7f12b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:16.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.788+0000 7f12d95ff700 1 --2- 172.21.15.156:0/3226661321 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f12b40682f0 0x7f12b406a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f12c4014040 tx=0x7f12c4008030 comp rx=0 tx=0).stop 2023-11-28T13:14:16.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.788+0000 7f12d95ff700 1 -- 172.21.15.156:0/3226661321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12d4161340 msgr2=0x7f12d406a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:16.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.788+0000 7f12d95ff700 1 --2- 172.21.15.156:0/3226661321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12d4161340 0x7f12d406a900 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f12cc007e90 tx=0x7f12cc00e3c0 comp rx=0 tx=0).stop 2023-11-28T13:14:16.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.788+0000 7f12d95ff700 1 -- 172.21.15.156:0/3226661321 shutdown_connections 2023-11-28T13:14:16.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.788+0000 7f12d95ff700 1 --2- 172.21.15.156:0/3226661321 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f12b40682f0 0x7f12b406a7a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:16.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.788+0000 7f12d95ff700 1 --2- 172.21.15.156:0/3226661321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12d4161340 0x7f12d406a900 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:16.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.788+0000 7f12d95ff700 1 --2- 172.21.15.156:0/3226661321 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12d40cd7c0 0x7f12d40c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:16.790 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.788+0000 7f12d95ff700 1 --2- 172.21.15.156:0/3226661321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12d415c9b0 0x7f12d4153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:16.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.788+0000 7f12d95ff700 1 -- 172.21.15.156:0/3226661321 >> 172.21.15.156:0/3226661321 conn(0x7f12d4072600 msgr2=0x7f12d40b8800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:16.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.788+0000 7f12d95ff700 1 -- 172.21.15.156:0/3226661321 shutdown_connections 2023-11-28T13:14:16.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:16.788+0000 7f12d95ff700 1 -- 172.21.15.156:0/3226661321 wait complete. 2023-11-28T13:14:16.808 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:14:21.809 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:14:22.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.013+0000 7fab233ff700 1 Processor -- start 2023-11-28T13:14:22.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.013+0000 7fab233ff700 1 -- start start 2023-11-28T13:14:22.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.014+0000 7fab233ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fab1c15c9b0 0x7fab1c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:22.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.015+0000 7fab233ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab1c0cd7c0 0x7fab1c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:22.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.015+0000 7fab233ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab1c0d1950 con 0x7fab1c161340 2023-11-28T13:14:22.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.015+0000 7fab233ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab1c0b5800 con 0x7fab1c15c9b0 2023-11-28T13:14:22.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.015+0000 7fab233ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c0ce140 con 0x7fab1c0cd7c0 2023-11-28T13:14:22.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.015+0000 7fab2119b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fab1c15c9b0 0x7fab1c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:22.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.015+0000 7fab2119b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fab1c15c9b0 0x7fab1c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36014/0 (socket says 172.21.15.156:36014) 2023-11-28T13:14:22.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.015+0000 7fab2119b700 1 -- 172.21.15.156:0/2135943134 learned_addr learned my addr 172.21.15.156:0/2135943134 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:22.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.015+0000 7fab2199c700 1 --2- 172.21.15.156:0/2135943134 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab1c0cd7c0 0x7fab1c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:22.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.015+0000 7fab2119b700 1 -- 172.21.15.156:0/2135943134 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab1c0cd7c0 msgr2=0x7fab1c0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:14:22.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.015+0000 7fab2119b700 1 --2- 172.21.15.156:0/2135943134 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab1c0cd7c0 0x7fab1c0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:22.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.015+0000 7fab2119b700 1 -- 172.21.15.156:0/2135943134 >> v1:172.21.15.186:6789/0 conn(0x7fab1c161340 legacy=0x7fab1c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:14:22.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.015+0000 7fab2119b700 1 -- 172.21.15.156:0/2135943134 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c0d24f0 con 0x7fab1c15c9b0 2023-11-28T13:14:22.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.016+0000 7fab2119b700 1 --2- 172.21.15.156:0/2135943134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1c15c9b0 0x7fab1c0b4bf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fab180098f0 tx=0x7fab18009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cafe5b96d670542 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:22.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.016+0000 7fab13fff700 1 -- 172.21.15.156:0/2135943134 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1800b8a0 con 0x7fab1c15c9b0 2023-11-28T13:14:22.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.016+0000 7fab13fff700 1 -- 172.21.15.156:0/2135943134 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1800ba00 con 0x7fab1c15c9b0 2023-11-28T13:14:22.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.016+0000 7fab13fff700 1 -- 172.21.15.156:0/2135943134 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1800bd90 con 0x7fab1c15c9b0 2023-11-28T13:14:22.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.016+0000 7fab233ff700 1 -- 172.21.15.156:0/2135943134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1c15c9b0 msgr2=0x7fab1c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:22.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.016+0000 7fab233ff700 1 --2- 172.21.15.156:0/2135943134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1c15c9b0 0x7fab1c0b4bf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fab180098f0 tx=0x7fab18009c00 comp rx=0 tx=0).stop 2023-11-28T13:14:22.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.016+0000 7fab233ff700 1 -- 172.21.15.156:0/2135943134 shutdown_connections 2023-11-28T13:14:22.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.016+0000 7fab233ff700 1 --2- 172.21.15.156:0/2135943134 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab1c0cd7c0 0x7fab1c0cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:22.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.016+0000 7fab233ff700 1 --2- 172.21.15.156:0/2135943134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1c15c9b0 0x7fab1c0b4bf0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:22.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.016+0000 7fab233ff700 1 -- 172.21.15.156:0/2135943134 >> 172.21.15.156:0/2135943134 conn(0x7fab1c072600 msgr2=0x7fab1c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:22.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.016+0000 7fab233ff700 1 -- 172.21.15.156:0/2135943134 shutdown_connections 2023-11-28T13:14:22.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.017+0000 7fab233ff700 1 -- 172.21.15.156:0/2135943134 wait complete. 2023-11-28T13:14:22.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.018+0000 7fab233ff700 1 Processor -- start 2023-11-28T13:14:22.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.018+0000 7fab233ff700 1 -- start start 2023-11-28T13:14:22.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.019+0000 7fab233ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab1c0cd7c0 0x7fab1c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:22.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.019+0000 7fab233ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab1c15c9b0 0x7fab1c1596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:22.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.019+0000 7fab233ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1c161340 0x7fab1c153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:22.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.019+0000 7fab233ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c0c8670 con 0x7fab1c161340 2023-11-28T13:14:22.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.019+0000 7fab233ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c0c8950 con 0x7fab1c15c9b0 2023-11-28T13:14:22.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.019+0000 7fab233ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c0c8c30 con 0x7fab1c0cd7c0 2023-11-28T13:14:22.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.020+0000 7fab2199c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1c161340 0x7fab1c153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:22.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.020+0000 7fab2199c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1c161340 0x7fab1c153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36020/0 (socket says 172.21.15.156:36020) 2023-11-28T13:14:22.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.020+0000 7fab2199c700 1 -- 172.21.15.156:0/3628223166 learned_addr learned my addr 172.21.15.156:0/3628223166 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:22.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.020+0000 7fab2119b700 1 --2- 172.21.15.156:0/3628223166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab1c0cd7c0 0x7fab1c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:22.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.020+0000 7fab2099a700 1 --2- 172.21.15.156:0/3628223166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab1c15c9b0 0x7fab1c1596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:22.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.020+0000 7fab2199c700 1 -- 172.21.15.156:0/3628223166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab1c0cd7c0 msgr2=0x7fab1c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:22.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.020+0000 7fab2199c700 1 --2- 172.21.15.156:0/3628223166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab1c0cd7c0 0x7fab1c0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:22.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.020+0000 7fab2199c700 1 -- 172.21.15.156:0/3628223166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab1c15c9b0 msgr2=0x7fab1c1596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:22.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.020+0000 7fab2199c700 1 --2- 172.21.15.156:0/3628223166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab1c15c9b0 0x7fab1c1596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:22.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.020+0000 7fab2199c700 1 -- 172.21.15.156:0/3628223166 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab18009550 con 0x7fab1c161340 2023-11-28T13:14:22.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.020+0000 7fab2199c700 1 --2- 172.21.15.156:0/3628223166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1c161340 0x7fab1c153860 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fab14007e60 tx=0x7fab1400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:22.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.020+0000 7fab11ffb700 1 -- 172.21.15.156:0/3628223166 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1400ed30 con 0x7fab1c161340 2023-11-28T13:14:22.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.021+0000 7fab11ffb700 1 -- 172.21.15.156:0/3628223166 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab14014420 con 0x7fab1c161340 2023-11-28T13:14:22.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.021+0000 7fab233ff700 1 -- 172.21.15.156:0/3628223166 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c0d1300 con 0x7fab1c161340 2023-11-28T13:14:22.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.021+0000 7fab11ffb700 1 -- 172.21.15.156:0/3628223166 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab140147d0 con 0x7fab1c161340 2023-11-28T13:14:22.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.021+0000 7fab233ff700 1 -- 172.21.15.156:0/3628223166 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c0b5520 con 0x7fab1c161340 2023-11-28T13:14:22.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.022+0000 7fab233ff700 1 -- 172.21.15.156:0/3628223166 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1c0d15e0 con 0x7fab1c161340 2023-11-28T13:14:22.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.023+0000 7fab11ffb700 1 -- 172.21.15.156:0/3628223166 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fab14012050 con 0x7fab1c161340 2023-11-28T13:14:22.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.023+0000 7fab11ffb700 1 --2- 172.21.15.156:0/3628223166 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fab000682f0 0x7fab0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:22.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.023+0000 7fab11ffb700 1 -- 172.21.15.156:0/3628223166 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fab14086330 con 0x7fab1c161340 2023-11-28T13:14:22.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.024+0000 7fab2119b700 1 --2- 172.21.15.156:0/3628223166 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fab000682f0 0x7fab0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:22.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.025+0000 7fab2119b700 1 --2- 172.21.15.156:0/3628223166 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fab000682f0 0x7fab0006a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fab18014040 tx=0x7fab18008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:22.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.028+0000 7fab11ffb700 1 -- 172.21.15.156:0/3628223166 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fab14052a10 con 0x7fab1c161340 2023-11-28T13:14:22.263 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.262+0000 7fab233ff700 1 -- 172.21.15.156:0/3628223166 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fab1c0745c0 con 0x7fab1c161340 2023-11-28T13:14:22.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.262+0000 7fab11ffb700 1 -- 172.21.15.156:0/3628223166 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fab14056030 con 0x7fab1c161340 2023-11-28T13:14:22.264 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:14:22.264 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:14:22.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.265+0000 7fab233ff700 1 -- 172.21.15.156:0/3628223166 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fab000682f0 msgr2=0x7fab0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:22.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.265+0000 7fab233ff700 1 --2- 172.21.15.156:0/3628223166 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fab000682f0 0x7fab0006a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fab18014040 tx=0x7fab18008030 comp rx=0 tx=0).stop 2023-11-28T13:14:22.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.265+0000 7fab233ff700 1 -- 172.21.15.156:0/3628223166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1c161340 msgr2=0x7fab1c153860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:22.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.265+0000 7fab233ff700 1 --2- 172.21.15.156:0/3628223166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1c161340 0x7fab1c153860 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fab14007e60 tx=0x7fab1400e3c0 comp rx=0 tx=0).stop 2023-11-28T13:14:22.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.265+0000 7fab233ff700 1 -- 172.21.15.156:0/3628223166 shutdown_connections 2023-11-28T13:14:22.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.265+0000 7fab233ff700 1 --2- 172.21.15.156:0/3628223166 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fab000682f0 0x7fab0006a7a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:22.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.265+0000 7fab233ff700 1 --2- 172.21.15.156:0/3628223166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1c161340 0x7fab1c153860 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:22.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.265+0000 7fab233ff700 1 --2- 172.21.15.156:0/3628223166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fab1c0cd7c0 0x7fab1c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:22.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.265+0000 7fab233ff700 1 --2- 172.21.15.156:0/3628223166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab1c15c9b0 0x7fab1c1596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:22.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.265+0000 7fab233ff700 1 -- 172.21.15.156:0/3628223166 >> 172.21.15.156:0/3628223166 conn(0x7fab1c072600 msgr2=0x7fab1c06a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:22.268 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.265+0000 7fab233ff700 1 -- 172.21.15.156:0/3628223166 shutdown_connections 2023-11-28T13:14:22.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:22.265+0000 7fab233ff700 1 -- 172.21.15.156:0/3628223166 wait complete. 2023-11-28T13:14:22.285 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:14:27.287 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:14:27.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.490+0000 7f95147ff700 1 Processor -- start 2023-11-28T13:14:27.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.491+0000 7f95147ff700 1 -- start start 2023-11-28T13:14:27.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.492+0000 7f95147ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f950c15ca50 0x7f950c06b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:27.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.492+0000 7f95147ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f950c06b8c0 0x7f950c0cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:27.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.492+0000 7f95147ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f950c0ccc40 0x7f950c0cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:27.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.492+0000 7f95147ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f950c06bd60 con 0x7f950c06b8c0 2023-11-28T13:14:27.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.492+0000 7f95147ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f950c0cd470 con 0x7f950c15ca50 2023-11-28T13:14:27.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.492+0000 7f95147ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f950c0cf5e0 con 0x7f950c0ccc40 2023-11-28T13:14:27.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.492+0000 7f9511d9a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f950c06b8c0 0x7f950c0cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:27.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.492+0000 7f9511d9a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f950c06b8c0 0x7f950c0cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37944/0 (socket says 172.21.15.156:37944) 2023-11-28T13:14:27.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.492+0000 7f9511d9a700 1 -- 172.21.15.156:0/1257338837 learned_addr learned my addr 172.21.15.156:0/1257338837 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:27.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.492+0000 7f9511d9a700 1 -- 172.21.15.156:0/1257338837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f950c0ccc40 msgr2=0x7f950c0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:14:27.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.492+0000 7f9511d9a700 1 --2- 172.21.15.156:0/1257338837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f950c0ccc40 0x7f950c0cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:27.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.492+0000 7f9511d9a700 1 -- 172.21.15.156:0/1257338837 >> v2:172.21.15.186:3300/0 conn(0x7f950c15ca50 msgr2=0x7f950c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:14:27.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.492+0000 7f9511d9a700 1 --2- 172.21.15.156:0/1257338837 >> v2:172.21.15.186:3300/0 conn(0x7f950c15ca50 0x7f950c06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:27.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.492+0000 7f9511d9a700 1 -- 172.21.15.156:0/1257338837 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950c0cecb0 con 0x7f950c06b8c0 2023-11-28T13:14:27.496 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.493+0000 7f9511d9a700 1 --2- 172.21.15.156:0/1257338837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f950c06b8c0 0x7f950c0cc700 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f950c071240 tx=0x7f94fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ece11877aa4c67 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:27.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.493+0000 7f9511599700 1 -- 172.21.15.156:0/1257338837 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc00b870 con 0x7f950c06b8c0 2023-11-28T13:14:27.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.493+0000 7f9511599700 1 -- 172.21.15.156:0/1257338837 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94fc00b9d0 con 0x7f950c06b8c0 2023-11-28T13:14:27.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.493+0000 7f9511599700 1 -- 172.21.15.156:0/1257338837 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc00bd60 con 0x7f950c06b8c0 2023-11-28T13:14:27.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.493+0000 7f95147ff700 1 -- 172.21.15.156:0/1257338837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f950c06b8c0 msgr2=0x7f950c0cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:27.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.493+0000 7f95147ff700 1 --2- 172.21.15.156:0/1257338837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f950c06b8c0 0x7f950c0cc700 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f950c071240 tx=0x7f94fc009bd0 comp rx=0 tx=0).stop 2023-11-28T13:14:27.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.494+0000 7f95147ff700 1 -- 172.21.15.156:0/1257338837 shutdown_connections 2023-11-28T13:14:27.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.494+0000 7f95147ff700 1 --2- 172.21.15.156:0/1257338837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f950c0ccc40 0x7f950c0cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:27.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.494+0000 7f95147ff700 1 --2- 172.21.15.156:0/1257338837 >> v2:172.21.15.186:3300/0 conn(0x7f950c15ca50 0x7f950c06b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:27.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.494+0000 7f95147ff700 1 --2- 172.21.15.156:0/1257338837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f950c06b8c0 0x7f950c0cc700 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:27.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.494+0000 7f95147ff700 1 -- 172.21.15.156:0/1257338837 >> 172.21.15.156:0/1257338837 conn(0x7f950c09a230 msgr2=0x7f950c08afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:27.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.494+0000 7f95147ff700 1 -- 172.21.15.156:0/1257338837 shutdown_connections 2023-11-28T13:14:27.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.494+0000 7f95147ff700 1 -- 172.21.15.156:0/1257338837 wait complete. 2023-11-28T13:14:27.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.495+0000 7f95147ff700 1 Processor -- start 2023-11-28T13:14:27.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.496+0000 7f95147ff700 1 -- start start 2023-11-28T13:14:27.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.496+0000 7f95147ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f950c06b8c0 0x7f950c159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:27.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.496+0000 7f95147ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f950c0ccc40 0x7f950c15f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:27.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.496+0000 7f95147ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f950c15ca50 0x7f950c0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:27.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.496+0000 7f95147ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950c15d150 con 0x7f950c0ccc40 2023-11-28T13:14:27.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.496+0000 7f95147ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950c15f630 con 0x7f950c06b8c0 2023-11-28T13:14:27.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.496+0000 7f95147ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f950c160100 con 0x7f950c15ca50 2023-11-28T13:14:27.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.497+0000 7f9511d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f950c0ccc40 0x7f950c15f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:27.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.497+0000 7f9511d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f950c0ccc40 0x7f950c15f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37956/0 (socket says 172.21.15.156:37956) 2023-11-28T13:14:27.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.497+0000 7f9511d9a700 1 -- 172.21.15.156:0/1038652258 learned_addr learned my addr 172.21.15.156:0/1038652258 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:27.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.497+0000 7f9512d9c700 1 --2- 172.21.15.156:0/1038652258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f950c15ca50 0x7f950c0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:27.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.497+0000 7f951259b700 1 --2- 172.21.15.156:0/1038652258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f950c06b8c0 0x7f950c159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:27.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.497+0000 7f9511d9a700 1 -- 172.21.15.156:0/1038652258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f950c15ca50 msgr2=0x7f950c0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:27.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.497+0000 7f9511d9a700 1 --2- 172.21.15.156:0/1038652258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f950c15ca50 0x7f950c0b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:27.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.497+0000 7f9511d9a700 1 -- 172.21.15.156:0/1038652258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f950c06b8c0 msgr2=0x7f950c159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:27.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.497+0000 7f9511d9a700 1 --2- 172.21.15.156:0/1038652258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f950c06b8c0 0x7f950c159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:27.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.497+0000 7f9511d9a700 1 -- 172.21.15.156:0/1038652258 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc009550 con 0x7f950c0ccc40 2023-11-28T13:14:27.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.497+0000 7f9512d9c700 1 --2- 172.21.15.156:0/1038652258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f950c15ca50 0x7f950c0b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:14:27.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.497+0000 7f9511d9a700 1 --2- 172.21.15.156:0/1038652258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f950c0ccc40 0x7f950c15f0f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f950c06d870 tx=0x7f94fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:27.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.497+0000 7f9502ffd700 1 -- 172.21.15.156:0/1038652258 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc004310 con 0x7f950c0ccc40 2023-11-28T13:14:27.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.498+0000 7f9502ffd700 1 -- 172.21.15.156:0/1038652258 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94fc01f070 con 0x7f950c0ccc40 2023-11-28T13:14:27.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.498+0000 7f9502ffd700 1 -- 172.21.15.156:0/1038652258 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc0036a0 con 0x7f950c0ccc40 2023-11-28T13:14:27.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.498+0000 7f95147ff700 1 -- 172.21.15.156:0/1038652258 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f950c0cf270 con 0x7f950c0ccc40 2023-11-28T13:14:27.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.498+0000 7f95147ff700 1 -- 172.21.15.156:0/1038652258 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f950c0cef90 con 0x7f950c0ccc40 2023-11-28T13:14:27.502 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.499+0000 7f95147ff700 1 -- 172.21.15.156:0/1038652258 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f950c0cd0c0 con 0x7f950c0ccc40 2023-11-28T13:14:27.503 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.500+0000 7f9502ffd700 1 -- 172.21.15.156:0/1038652258 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f94fc02a020 con 0x7f950c0ccc40 2023-11-28T13:14:27.503 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.500+0000 7f9502ffd700 1 --2- 172.21.15.156:0/1038652258 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f94f00682f0 0x7f94f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:27.503 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.500+0000 7f9502ffd700 1 -- 172.21.15.156:0/1038652258 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f94fc087460 con 0x7f950c0ccc40 2023-11-28T13:14:27.503 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.501+0000 7f951259b700 1 --2- 172.21.15.156:0/1038652258 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f94f00682f0 0x7f94f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:27.504 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.502+0000 7f951259b700 1 --2- 172.21.15.156:0/1038652258 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f94f00682f0 0x7f94f006a7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9508006fd0 tx=0x7f9508008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:27.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.505+0000 7f9502ffd700 1 -- 172.21.15.156:0/1038652258 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f94fc053b40 con 0x7f950c0ccc40 2023-11-28T13:14:27.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.739+0000 7f95147ff700 1 -- 172.21.15.156:0/1038652258 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f950c0745c0 con 0x7f950c0ccc40 2023-11-28T13:14:27.741 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.740+0000 7f9502ffd700 1 -- 172.21.15.156:0/1038652258 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f94fc057160 con 0x7f950c0ccc40 2023-11-28T13:14:27.741 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:14:27.741 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:14:27.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.742+0000 7f95147ff700 1 -- 172.21.15.156:0/1038652258 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f94f00682f0 msgr2=0x7f94f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:27.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.742+0000 7f95147ff700 1 --2- 172.21.15.156:0/1038652258 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f94f00682f0 0x7f94f006a7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9508006fd0 tx=0x7f9508008040 comp rx=0 tx=0).stop 2023-11-28T13:14:27.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.742+0000 7f95147ff700 1 -- 172.21.15.156:0/1038652258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f950c0ccc40 msgr2=0x7f950c15f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:27.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.742+0000 7f95147ff700 1 --2- 172.21.15.156:0/1038652258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f950c0ccc40 0x7f950c15f0f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f950c06d870 tx=0x7f94fc004750 comp rx=0 tx=0).stop 2023-11-28T13:14:27.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.742+0000 7f95147ff700 1 -- 172.21.15.156:0/1038652258 shutdown_connections 2023-11-28T13:14:27.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.742+0000 7f95147ff700 1 --2- 172.21.15.156:0/1038652258 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f94f00682f0 0x7f94f006a7a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:27.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.742+0000 7f95147ff700 1 --2- 172.21.15.156:0/1038652258 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f950c15ca50 0x7f950c0b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:27.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.742+0000 7f95147ff700 1 --2- 172.21.15.156:0/1038652258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f950c06b8c0 0x7f950c159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:27.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.742+0000 7f95147ff700 1 --2- 172.21.15.156:0/1038652258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f950c0ccc40 0x7f950c15f0f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:27.744 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.742+0000 7f95147ff700 1 -- 172.21.15.156:0/1038652258 >> 172.21.15.156:0/1038652258 conn(0x7f950c09a230 msgr2=0x7f950c0adb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:27.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.742+0000 7f95147ff700 1 -- 172.21.15.156:0/1038652258 shutdown_connections 2023-11-28T13:14:27.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:27.742+0000 7f95147ff700 1 -- 172.21.15.156:0/1038652258 wait complete. 2023-11-28T13:14:27.763 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:14:32.764 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:14:32.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.968+0000 7f6f8b7ff700 1 Processor -- start 2023-11-28T13:14:32.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.968+0000 7f6f8b7ff700 1 -- start start 2023-11-28T13:14:32.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.969+0000 7f6f8b7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8415c9b0 0x7f6f840b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:32.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.969+0000 7f6f8b7ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6f84161340 0x7f6f840d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:32.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.969+0000 7f6f8b7ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f840d1950 con 0x7f6f840d2320 2023-11-28T13:14:32.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.969+0000 7f6f8b7ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f840b5800 con 0x7f6f84161340 2023-11-28T13:14:32.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.969+0000 7f6f8b7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f840b5520 con 0x7f6f8415c9b0 2023-11-28T13:14:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.970+0000 7f6f8959b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8415c9b0 0x7f6f840b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.970+0000 7f6f8959b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8415c9b0 0x7f6f840b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:43358/0 (socket says 172.21.15.156:43358) 2023-11-28T13:14:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.970+0000 7f6f8959b700 1 -- 172.21.15.156:0/1143218 learned_addr learned my addr 172.21.15.156:0/1143218 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.970+0000 7f6f88d9a700 1 --2- 172.21.15.156:0/1143218 >> v2:172.21.15.186:3300/0 conn(0x7f6f84161340 0x7f6f840d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.970+0000 7f6f7bfff700 1 -- 172.21.15.156:0/1143218 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1997437475 0 0) 0x7f6f840d1950 con 0x7f6f840d2320 2023-11-28T13:14:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.970+0000 7f6f8959b700 1 -- 172.21.15.156:0/1143218 >> v2:172.21.15.186:3300/0 conn(0x7f6f84161340 msgr2=0x7f6f840d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:14:32.974 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.970+0000 7f6f8959b700 1 --2- 172.21.15.156:0/1143218 >> v2:172.21.15.186:3300/0 conn(0x7f6f84161340 0x7f6f840d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:32.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.970+0000 7f6f8959b700 1 -- 172.21.15.156:0/1143218 >> v1:172.21.15.186:6789/0 conn(0x7f6f840d2320 legacy=0x7f6f840cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:32.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.971+0000 7f6f8959b700 1 -- 172.21.15.156:0/1143218 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f840d1300 con 0x7f6f8415c9b0 2023-11-28T13:14:32.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.971+0000 7f6f8959b700 1 --2- 172.21.15.156:0/1143218 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8415c9b0 0x7f6f840b4bf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6f800098f0 tx=0x7f6f80009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f1bf15e2a9836ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:32.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.971+0000 7f6f7bfff700 1 -- 172.21.15.156:0/1143218 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80011830 con 0x7f6f8415c9b0 2023-11-28T13:14:32.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.971+0000 7f6f7bfff700 1 -- 172.21.15.156:0/1143218 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f80011990 con 0x7f6f8415c9b0 2023-11-28T13:14:32.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.971+0000 7f6f7bfff700 1 -- 172.21.15.156:0/1143218 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80011d20 con 0x7f6f8415c9b0 2023-11-28T13:14:32.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.972+0000 7f6f8b7ff700 1 -- 172.21.15.156:0/1143218 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8415c9b0 msgr2=0x7f6f840b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:32.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.972+0000 7f6f8b7ff700 1 --2- 172.21.15.156:0/1143218 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8415c9b0 0x7f6f840b4bf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6f800098f0 tx=0x7f6f80009c00 comp rx=0 tx=0).stop 2023-11-28T13:14:32.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.972+0000 7f6f8b7ff700 1 -- 172.21.15.156:0/1143218 shutdown_connections 2023-11-28T13:14:32.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.972+0000 7f6f8b7ff700 1 --2- 172.21.15.156:0/1143218 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8415c9b0 0x7f6f840b4bf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:32.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.972+0000 7f6f8b7ff700 1 --2- 172.21.15.156:0/1143218 >> v2:172.21.15.186:3300/0 conn(0x7f6f84161340 0x7f6f840d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:32.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.972+0000 7f6f8b7ff700 1 -- 172.21.15.156:0/1143218 >> 172.21.15.156:0/1143218 conn(0x7f6f84072600 msgr2=0x7f6f840cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:32.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.972+0000 7f6f8b7ff700 1 -- 172.21.15.156:0/1143218 shutdown_connections 2023-11-28T13:14:32.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.972+0000 7f6f8b7ff700 1 -- 172.21.15.156:0/1143218 wait complete. 2023-11-28T13:14:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.973+0000 7f6f8b7ff700 1 Processor -- start 2023-11-28T13:14:32.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.974+0000 7f6f8b7ff700 1 -- start start 2023-11-28T13:14:32.979 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.974+0000 7f6f8b7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f840d2320 0x7f6f84146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:32.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.974+0000 7f6f8b7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8415c9b0 0x7f6f840b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:32.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.974+0000 7f6f8b7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f84161340 0x7f6f840b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:32.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.974+0000 7f6f8b7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f841559e0 con 0x7f6f84161340 2023-11-28T13:14:32.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.974+0000 7f6f8b7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f84155cc0 con 0x7f6f840d2320 2023-11-28T13:14:32.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.974+0000 7f6f8b7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f84155fa0 con 0x7f6f8415c9b0 2023-11-28T13:14:32.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.975+0000 7f6f89d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f84161340 0x7f6f840b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:32.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.975+0000 7f6f89d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f84161340 0x7f6f840b7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37966/0 (socket says 172.21.15.156:37966) 2023-11-28T13:14:32.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.975+0000 7f6f89d9c700 1 -- 172.21.15.156:0/2301237527 learned_addr learned my addr 172.21.15.156:0/2301237527 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:32.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.975+0000 7f6f88d9a700 1 --2- 172.21.15.156:0/2301237527 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8415c9b0 0x7f6f840b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:32.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.975+0000 7f6f8959b700 1 --2- 172.21.15.156:0/2301237527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f840d2320 0x7f6f84146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:32.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.975+0000 7f6f89d9c700 1 -- 172.21.15.156:0/2301237527 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8415c9b0 msgr2=0x7f6f840b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:32.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.975+0000 7f6f89d9c700 1 --2- 172.21.15.156:0/2301237527 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8415c9b0 0x7f6f840b75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:32.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.975+0000 7f6f89d9c700 1 -- 172.21.15.156:0/2301237527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f840d2320 msgr2=0x7f6f84146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:32.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.975+0000 7f6f89d9c700 1 --2- 172.21.15.156:0/2301237527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f840d2320 0x7f6f84146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:32.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.975+0000 7f6f89d9c700 1 -- 172.21.15.156:0/2301237527 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f80009550 con 0x7f6f84161340 2023-11-28T13:14:32.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.975+0000 7f6f89d9c700 1 --2- 172.21.15.156:0/2301237527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f84161340 0x7f6f840b7ae0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00c7a0 tx=0x7f6f7c00cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:32.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.976+0000 7f6f79ffb700 1 -- 172.21.15.156:0/2301237527 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c003e30 con 0x7f6f84161340 2023-11-28T13:14:32.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.976+0000 7f6f79ffb700 1 -- 172.21.15.156:0/2301237527 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7c005070 con 0x7f6f84161340 2023-11-28T13:14:32.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.976+0000 7f6f8b7ff700 1 -- 172.21.15.156:0/2301237527 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f840b5130 con 0x7f6f84161340 2023-11-28T13:14:32.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.976+0000 7f6f79ffb700 1 -- 172.21.15.156:0/2301237527 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c005420 con 0x7f6f84161340 2023-11-28T13:14:32.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.976+0000 7f6f8b7ff700 1 -- 172.21.15.156:0/2301237527 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f840d15e0 con 0x7f6f84161340 2023-11-28T13:14:32.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.977+0000 7f6f8b7ff700 1 -- 172.21.15.156:0/2301237527 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f840b52c0 con 0x7f6f84161340 2023-11-28T13:14:32.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.978+0000 7f6f79ffb700 1 -- 172.21.15.156:0/2301237527 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6f7c019030 con 0x7f6f84161340 2023-11-28T13:14:32.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.978+0000 7f6f79ffb700 1 --2- 172.21.15.156:0/2301237527 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6f68068340 0x7f6f6806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:32.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.979+0000 7f6f79ffb700 1 -- 172.21.15.156:0/2301237527 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f6f7c085230 con 0x7f6f84161340 2023-11-28T13:14:32.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.979+0000 7f6f8959b700 1 --2- 172.21.15.156:0/2301237527 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6f68068340 0x7f6f6806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:32.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.980+0000 7f6f8959b700 1 --2- 172.21.15.156:0/2301237527 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6f68068340 0x7f6f6806a7f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009c40 tx=0x7f6f80008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:32.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:32.983+0000 7f6f79ffb700 1 -- 172.21.15.156:0/2301237527 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6f7c051990 con 0x7f6f84161340 2023-11-28T13:14:33.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:33.214+0000 7f6f8b7ff700 1 -- 172.21.15.156:0/2301237527 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6f840745c0 con 0x7f6f84161340 2023-11-28T13:14:33.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:33.215+0000 7f6f79ffb700 1 -- 172.21.15.156:0/2301237527 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f6f7c0170c0 con 0x7f6f84161340 2023-11-28T13:14:33.216 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:14:33.216 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:14:33.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:33.217+0000 7f6f8b7ff700 1 -- 172.21.15.156:0/2301237527 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6f68068340 msgr2=0x7f6f6806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:33.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:33.217+0000 7f6f8b7ff700 1 --2- 172.21.15.156:0/2301237527 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6f68068340 0x7f6f6806a7f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009c40 tx=0x7f6f80008030 comp rx=0 tx=0).stop 2023-11-28T13:14:33.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:33.217+0000 7f6f8b7ff700 1 -- 172.21.15.156:0/2301237527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f84161340 msgr2=0x7f6f840b7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:33.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:33.217+0000 7f6f8b7ff700 1 --2- 172.21.15.156:0/2301237527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f84161340 0x7f6f840b7ae0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00c7a0 tx=0x7f6f7c00cab0 comp rx=0 tx=0).stop 2023-11-28T13:14:33.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:33.217+0000 7f6f8b7ff700 1 -- 172.21.15.156:0/2301237527 shutdown_connections 2023-11-28T13:14:33.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:33.217+0000 7f6f8b7ff700 1 --2- 172.21.15.156:0/2301237527 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6f68068340 0x7f6f6806a7f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:33.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:33.217+0000 7f6f8b7ff700 1 --2- 172.21.15.156:0/2301237527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f84161340 0x7f6f840b7ae0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:33.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:33.217+0000 7f6f8b7ff700 1 --2- 172.21.15.156:0/2301237527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f840d2320 0x7f6f84146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:33.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:33.217+0000 7f6f8b7ff700 1 --2- 172.21.15.156:0/2301237527 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8415c9b0 0x7f6f840b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:33.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:33.217+0000 7f6f8b7ff700 1 -- 172.21.15.156:0/2301237527 >> 172.21.15.156:0/2301237527 conn(0x7f6f84072600 msgr2=0x7f6f8415cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:33.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:33.217+0000 7f6f8b7ff700 1 -- 172.21.15.156:0/2301237527 shutdown_connections 2023-11-28T13:14:33.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:33.217+0000 7f6f8b7ff700 1 -- 172.21.15.156:0/2301237527 wait complete. 2023-11-28T13:14:33.237 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:14:34.248 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:14:34.309 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:14:38.238 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:14:38.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.442+0000 7f02ea1ff700 1 Processor -- start 2023-11-28T13:14:38.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.442+0000 7f02ea1ff700 1 -- start start 2023-11-28T13:14:38.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.443+0000 7f02ea1ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f02e415c9b0 0x7f02e40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:38.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.443+0000 7f02ea1ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f02e4161340 0x7f02e40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:38.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.443+0000 7f02ea1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02e40d2320 0x7f02e40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:38.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.443+0000 7f02ea1ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e40b5800 con 0x7f02e415c9b0 2023-11-28T13:14:38.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.443+0000 7f02ea1ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e40d1950 con 0x7f02e4161340 2023-11-28T13:14:38.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.443+0000 7f02ea1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e40b5520 con 0x7f02e40d2320 2023-11-28T13:14:38.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.443+0000 7f02e37fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f02e415c9b0 0x7f02e40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:38.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.444+0000 7f02e37fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f02e415c9b0 0x7f02e40b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37860/0 (socket says 172.21.15.156:37860) 2023-11-28T13:14:38.445 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.444+0000 7f02e37fe700 1 -- 172.21.15.156:0/261042617 learned_addr learned my addr 172.21.15.156:0/261042617 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:38.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.444+0000 7f02e2ffd700 1 --2- 172.21.15.156:0/261042617 >> v2:172.21.15.186:3300/0 conn(0x7f02e4161340 0x7f02e40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:38.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.444+0000 7f02e3fff700 1 --2- 172.21.15.156:0/261042617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02e40d2320 0x7f02e40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:38.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.444+0000 7f02e37fe700 1 -- 172.21.15.156:0/261042617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02e40d2320 msgr2=0x7f02e40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:14:38.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.444+0000 7f02e37fe700 1 --2- 172.21.15.156:0/261042617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02e40d2320 0x7f02e40cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:38.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.444+0000 7f02e37fe700 1 -- 172.21.15.156:0/261042617 >> v2:172.21.15.186:3300/0 conn(0x7f02e4161340 msgr2=0x7f02e40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:14:38.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.444+0000 7f02e37fe700 1 --2- 172.21.15.156:0/261042617 >> v2:172.21.15.186:3300/0 conn(0x7f02e4161340 0x7f02e40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:38.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.444+0000 7f02e37fe700 1 -- 172.21.15.156:0/261042617 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e40d1300 con 0x7f02e415c9b0 2023-11-28T13:14:38.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.444+0000 7f02e2ffd700 1 --2- 172.21.15.156:0/261042617 >> v2:172.21.15.186:3300/0 conn(0x7f02e4161340 0x7f02e40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:14:38.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.444+0000 7f02e3fff700 1 --2- 172.21.15.156:0/261042617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02e40d2320 0x7f02e40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:14:38.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.444+0000 7f02e37fe700 1 --2- 172.21.15.156:0/261042617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02e415c9b0 0x7f02e40b4bf0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f02e40685c0 tx=0x7f02d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9fc46d71b488d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:38.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.444+0000 7f02e27fc700 1 -- 172.21.15.156:0/261042617 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d400b870 con 0x7f02e415c9b0 2023-11-28T13:14:38.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.445+0000 7f02e27fc700 1 -- 172.21.15.156:0/261042617 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d400b9d0 con 0x7f02e415c9b0 2023-11-28T13:14:38.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.445+0000 7f02e27fc700 1 -- 172.21.15.156:0/261042617 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d400bd60 con 0x7f02e415c9b0 2023-11-28T13:14:38.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.445+0000 7f02ea1ff700 1 -- 172.21.15.156:0/261042617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02e415c9b0 msgr2=0x7f02e40b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:38.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.445+0000 7f02ea1ff700 1 --2- 172.21.15.156:0/261042617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02e415c9b0 0x7f02e40b4bf0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f02e40685c0 tx=0x7f02d4009bd0 comp rx=0 tx=0).stop 2023-11-28T13:14:38.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.445+0000 7f02ea1ff700 1 -- 172.21.15.156:0/261042617 shutdown_connections 2023-11-28T13:14:38.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.445+0000 7f02ea1ff700 1 --2- 172.21.15.156:0/261042617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02e40d2320 0x7f02e40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:38.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.445+0000 7f02ea1ff700 1 --2- 172.21.15.156:0/261042617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02e415c9b0 0x7f02e40b4bf0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:38.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.445+0000 7f02ea1ff700 1 --2- 172.21.15.156:0/261042617 >> v2:172.21.15.186:3300/0 conn(0x7f02e4161340 0x7f02e40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:38.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.445+0000 7f02ea1ff700 1 -- 172.21.15.156:0/261042617 >> 172.21.15.156:0/261042617 conn(0x7f02e4072600 msgr2=0x7f02e40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:38.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.445+0000 7f02ea1ff700 1 -- 172.21.15.156:0/261042617 shutdown_connections 2023-11-28T13:14:38.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.445+0000 7f02ea1ff700 1 -- 172.21.15.156:0/261042617 wait complete. 2023-11-28T13:14:38.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.447+0000 7f02ea1ff700 1 Processor -- start 2023-11-28T13:14:38.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.447+0000 7f02ea1ff700 1 -- start start 2023-11-28T13:14:38.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.448+0000 7f02ea1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02e40d2320 0x7f02e4150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:38.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.448+0000 7f02ea1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02e415c9b0 0x7f02e414d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:38.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.448+0000 7f02ea1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02e4161340 0x7f02e414dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:38.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.448+0000 7f02ea1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e41e8f60 con 0x7f02e4161340 2023-11-28T13:14:38.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.448+0000 7f02ea1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e41e65a0 con 0x7f02e40d2320 2023-11-28T13:14:38.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.448+0000 7f02ea1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e41e6880 con 0x7f02e415c9b0 2023-11-28T13:14:38.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.448+0000 7f02e3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02e4161340 0x7f02e414dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:38.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.448+0000 7f02e37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02e40d2320 0x7f02e4150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:38.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.448+0000 7f02e2ffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02e415c9b0 0x7f02e414d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:38.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.448+0000 7f02e37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02e40d2320 0x7f02e4150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:53014/0 (socket says 172.21.15.156:53014) 2023-11-28T13:14:38.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.448+0000 7f02e37fe700 1 -- 172.21.15.156:0/1985287990 learned_addr learned my addr 172.21.15.156:0/1985287990 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:38.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.449+0000 7f02e3fff700 1 -- 172.21.15.156:0/1985287990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02e415c9b0 msgr2=0x7f02e414d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:38.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.449+0000 7f02e3fff700 1 --2- 172.21.15.156:0/1985287990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02e415c9b0 0x7f02e414d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:38.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.449+0000 7f02e3fff700 1 -- 172.21.15.156:0/1985287990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02e40d2320 msgr2=0x7f02e4150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:38.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.449+0000 7f02e3fff700 1 --2- 172.21.15.156:0/1985287990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02e40d2320 0x7f02e4150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:38.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.449+0000 7f02e3fff700 1 -- 172.21.15.156:0/1985287990 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d4009550 con 0x7f02e4161340 2023-11-28T13:14:38.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.449+0000 7f02e3fff700 1 --2- 172.21.15.156:0/1985287990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02e4161340 0x7f02e414dc10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00cc60 tx=0x7f02dc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:38.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.449+0000 7f02cbfff700 1 -- 172.21.15.156:0/1985287990 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc00ace0 con 0x7f02e4161340 2023-11-28T13:14:38.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.449+0000 7f02cbfff700 1 -- 172.21.15.156:0/1985287990 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc00ae60 con 0x7f02e4161340 2023-11-28T13:14:38.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.449+0000 7f02cbfff700 1 -- 172.21.15.156:0/1985287990 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc00f6c0 con 0x7f02e4161340 2023-11-28T13:14:38.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.449+0000 7f02ea1ff700 1 -- 172.21.15.156:0/1985287990 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e40b5130 con 0x7f02e4161340 2023-11-28T13:14:38.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.449+0000 7f02ea1ff700 1 -- 172.21.15.156:0/1985287990 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e40d15e0 con 0x7f02e4161340 2023-11-28T13:14:38.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.450+0000 7f02ea1ff700 1 -- 172.21.15.156:0/1985287990 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e40b52c0 con 0x7f02e4161340 2023-11-28T13:14:38.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.451+0000 7f02cbfff700 1 -- 172.21.15.156:0/1985287990 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f02dc009030 con 0x7f02e4161340 2023-11-28T13:14:38.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.452+0000 7f02cbfff700 1 --2- 172.21.15.156:0/1985287990 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02d0068340 0x7f02d006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:38.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.452+0000 7f02cbfff700 1 -- 172.21.15.156:0/1985287990 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f02dc0863c0 con 0x7f02e4161340 2023-11-28T13:14:38.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.453+0000 7f02e37fe700 1 --2- 172.21.15.156:0/1985287990 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02d0068340 0x7f02d006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:38.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.454+0000 7f02e37fe700 1 --2- 172.21.15.156:0/1985287990 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02d0068340 0x7f02d006a7f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009130 tx=0x7f02d4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:38.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.457+0000 7f02cbfff700 1 -- 172.21.15.156:0/1985287990 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f02dc052bb0 con 0x7f02e4161340 2023-11-28T13:14:38.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.692+0000 7f02ea1ff700 1 -- 172.21.15.156:0/1985287990 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f02e40745c0 con 0x7f02e4161340 2023-11-28T13:14:38.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.693+0000 7f02cbfff700 1 -- 172.21.15.156:0/1985287990 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f02dc0561d0 con 0x7f02e4161340 2023-11-28T13:14:38.694 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:14:38.694 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:14:38.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.695+0000 7f02ea1ff700 1 -- 172.21.15.156:0/1985287990 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02d0068340 msgr2=0x7f02d006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:38.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.695+0000 7f02ea1ff700 1 --2- 172.21.15.156:0/1985287990 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02d0068340 0x7f02d006a7f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009130 tx=0x7f02d4008030 comp rx=0 tx=0).stop 2023-11-28T13:14:38.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.695+0000 7f02ea1ff700 1 -- 172.21.15.156:0/1985287990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02e4161340 msgr2=0x7f02e414dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:38.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.695+0000 7f02ea1ff700 1 --2- 172.21.15.156:0/1985287990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02e4161340 0x7f02e414dc10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00cc60 tx=0x7f02dc00cf70 comp rx=0 tx=0).stop 2023-11-28T13:14:38.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.695+0000 7f02ea1ff700 1 -- 172.21.15.156:0/1985287990 shutdown_connections 2023-11-28T13:14:38.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.695+0000 7f02ea1ff700 1 --2- 172.21.15.156:0/1985287990 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f02d0068340 0x7f02d006a7f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:38.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.695+0000 7f02ea1ff700 1 --2- 172.21.15.156:0/1985287990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02e4161340 0x7f02e414dc10 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:38.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.695+0000 7f02ea1ff700 1 --2- 172.21.15.156:0/1985287990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02e40d2320 0x7f02e4150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:38.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.695+0000 7f02ea1ff700 1 --2- 172.21.15.156:0/1985287990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f02e415c9b0 0x7f02e414d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:38.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.695+0000 7f02ea1ff700 1 -- 172.21.15.156:0/1985287990 >> 172.21.15.156:0/1985287990 conn(0x7f02e4072600 msgr2=0x7f02e415cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:38.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.695+0000 7f02ea1ff700 1 -- 172.21.15.156:0/1985287990 shutdown_connections 2023-11-28T13:14:38.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:38.695+0000 7f02ea1ff700 1 -- 172.21.15.156:0/1985287990 wait complete. 2023-11-28T13:14:38.715 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:14:43.716 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:14:43.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.921+0000 7f00855ff700 1 Processor -- start 2023-11-28T13:14:43.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.921+0000 7f00855ff700 1 -- start start 2023-11-28T13:14:43.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.922+0000 7f00855ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f008015c9b0 0x7f00800b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:43.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.922+0000 7f00855ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0080161340 0x7f00800d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:43.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.922+0000 7f00855ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00800d1950 con 0x7f00800d2320 2023-11-28T13:14:43.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.922+0000 7f00855ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00800b5800 con 0x7f0080161340 2023-11-28T13:14:43.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.922+0000 7f00855ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00800b5520 con 0x7f008015c9b0 2023-11-28T13:14:43.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.922+0000 7f007e7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0080161340 0x7f00800d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:43.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.922+0000 7f007e7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0080161340 0x7f00800d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37876/0 (socket says 172.21.15.156:37876) 2023-11-28T13:14:43.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.923+0000 7f007e7fc700 1 -- 172.21.15.156:0/3854487097 learned_addr learned my addr 172.21.15.156:0/3854487097 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:43.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.923+0000 7f007effd700 1 --2- 172.21.15.156:0/3854487097 >> v2:172.21.15.186:3300/0 conn(0x7f008015c9b0 0x7f00800b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:43.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.923+0000 7f007e7fc700 1 -- 172.21.15.156:0/3854487097 >> v2:172.21.15.186:3300/0 conn(0x7f008015c9b0 msgr2=0x7f00800b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:14:43.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.923+0000 7f007e7fc700 1 --2- 172.21.15.156:0/3854487097 >> v2:172.21.15.186:3300/0 conn(0x7f008015c9b0 0x7f00800b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:43.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.923+0000 7f007e7fc700 1 -- 172.21.15.156:0/3854487097 >> v1:172.21.15.156:6789/0 conn(0x7f00800d2320 legacy=0x7f00800cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:43.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.923+0000 7f007e7fc700 1 -- 172.21.15.156:0/3854487097 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00800d1300 con 0x7f0080161340 2023-11-28T13:14:43.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.923+0000 7f007e7fc700 1 --2- 172.21.15.156:0/3854487097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0080161340 0x7f00800d1de0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0080067460 tx=0x7f0074009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2587e879653fc7c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:43.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.923+0000 7f007dffb700 1 -- 172.21.15.156:0/3854487097 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007400b870 con 0x7f0080161340 2023-11-28T13:14:43.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.923+0000 7f007dffb700 1 -- 172.21.15.156:0/3854487097 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007400b9d0 con 0x7f0080161340 2023-11-28T13:14:43.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.923+0000 7f007dffb700 1 -- 172.21.15.156:0/3854487097 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007400bd60 con 0x7f0080161340 2023-11-28T13:14:43.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.924+0000 7f00855ff700 1 -- 172.21.15.156:0/3854487097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0080161340 msgr2=0x7f00800d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:43.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.924+0000 7f00855ff700 1 --2- 172.21.15.156:0/3854487097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0080161340 0x7f00800d1de0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0080067460 tx=0x7f0074009bd0 comp rx=0 tx=0).stop 2023-11-28T13:14:43.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.924+0000 7f00855ff700 1 -- 172.21.15.156:0/3854487097 shutdown_connections 2023-11-28T13:14:43.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.924+0000 7f00855ff700 1 --2- 172.21.15.156:0/3854487097 >> v2:172.21.15.186:3300/0 conn(0x7f008015c9b0 0x7f00800b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:43.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.924+0000 7f00855ff700 1 --2- 172.21.15.156:0/3854487097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0080161340 0x7f00800d1de0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:43.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.924+0000 7f00855ff700 1 -- 172.21.15.156:0/3854487097 >> 172.21.15.156:0/3854487097 conn(0x7f0080072600 msgr2=0x7f00800cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:43.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.924+0000 7f00855ff700 1 -- 172.21.15.156:0/3854487097 shutdown_connections 2023-11-28T13:14:43.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.924+0000 7f00855ff700 1 -- 172.21.15.156:0/3854487097 wait complete. 2023-11-28T13:14:43.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.926+0000 7f00855ff700 1 Processor -- start 2023-11-28T13:14:43.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.926+0000 7f00855ff700 1 -- start start 2023-11-28T13:14:43.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.927+0000 7f00855ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00800d2320 0x7f00800c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.927+0000 7f00855ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f008015c9b0 0x7f00801f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.927+0000 7f00855ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0080161340 0x7f008006a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.927+0000 7f00855ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00800c65a0 con 0x7f0080161340 2023-11-28T13:14:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.927+0000 7f00855ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0080154970 con 0x7f008015c9b0 2023-11-28T13:14:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.927+0000 7f00855ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00800c8830 con 0x7f00800d2320 2023-11-28T13:14:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.927+0000 7f007f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0080161340 0x7f008006a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.927+0000 7f007f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0080161340 0x7f008006a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37878/0 (socket says 172.21.15.156:37878) 2023-11-28T13:14:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.927+0000 7f007f7fe700 1 -- 172.21.15.156:0/3298633759 learned_addr learned my addr 172.21.15.156:0/3298633759 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.927+0000 7f007e7fc700 1 --2- 172.21.15.156:0/3298633759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f008015c9b0 0x7f00801f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.927+0000 7f007effd700 1 --2- 172.21.15.156:0/3298633759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00800d2320 0x7f00800c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.928+0000 7f007f7fe700 1 -- 172.21.15.156:0/3298633759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00800d2320 msgr2=0x7f00800c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.928+0000 7f007f7fe700 1 --2- 172.21.15.156:0/3298633759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00800d2320 0x7f00800c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:43.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.928+0000 7f007f7fe700 1 -- 172.21.15.156:0/3298633759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f008015c9b0 msgr2=0x7f00801f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:43.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.928+0000 7f007f7fe700 1 --2- 172.21.15.156:0/3298633759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f008015c9b0 0x7f00801f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:43.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.928+0000 7f007f7fe700 1 -- 172.21.15.156:0/3298633759 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0074009550 con 0x7f0080161340 2023-11-28T13:14:43.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.928+0000 7f007f7fe700 1 --2- 172.21.15.156:0/3298633759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0080161340 0x7f008006a900 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f007800cbc0 tx=0x7f007800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:43.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.928+0000 7f00677fe700 1 -- 172.21.15.156:0/3298633759 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078003c00 con 0x7f0080161340 2023-11-28T13:14:43.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.928+0000 7f00677fe700 1 -- 172.21.15.156:0/3298633759 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0078003d60 con 0x7f0080161340 2023-11-28T13:14:43.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.928+0000 7f00855ff700 1 -- 172.21.15.156:0/3298633759 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00800b5130 con 0x7f0080161340 2023-11-28T13:14:43.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.928+0000 7f00677fe700 1 -- 172.21.15.156:0/3298633759 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007800aae0 con 0x7f0080161340 2023-11-28T13:14:43.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.928+0000 7f00855ff700 1 -- 172.21.15.156:0/3298633759 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00800d15e0 con 0x7f0080161340 2023-11-28T13:14:43.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.929+0000 7f00855ff700 1 -- 172.21.15.156:0/3298633759 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00800b52c0 con 0x7f0080161340 2023-11-28T13:14:43.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.930+0000 7f00677fe700 1 -- 172.21.15.156:0/3298633759 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0078010050 con 0x7f0080161340 2023-11-28T13:14:43.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.931+0000 7f00677fe700 1 --2- 172.21.15.156:0/3298633759 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f006c0682f0 0x7f006c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:43.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.931+0000 7f00677fe700 1 -- 172.21.15.156:0/3298633759 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f0078085ca0 con 0x7f0080161340 2023-11-28T13:14:43.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.931+0000 7f007effd700 1 --2- 172.21.15.156:0/3298633759 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f006c0682f0 0x7f006c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:43.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.932+0000 7f007effd700 1 --2- 172.21.15.156:0/3298633759 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f006c0682f0 0x7f006c06a7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0070009930 tx=0x7f0070008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:43.936 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:43.935+0000 7f00677fe700 1 -- 172.21.15.156:0/3298633759 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0078052380 con 0x7f0080161340 2023-11-28T13:14:44.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:44.170+0000 7f00855ff700 1 -- 172.21.15.156:0/3298633759 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f00800745c0 con 0x7f0080161340 2023-11-28T13:14:44.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:44.170+0000 7f00677fe700 1 -- 172.21.15.156:0/3298633759 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f00780559a0 con 0x7f0080161340 2023-11-28T13:14:44.171 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:14:44.171 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:14:44.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:44.172+0000 7f00855ff700 1 -- 172.21.15.156:0/3298633759 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f006c0682f0 msgr2=0x7f006c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:44.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:44.172+0000 7f00855ff700 1 --2- 172.21.15.156:0/3298633759 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f006c0682f0 0x7f006c06a7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0070009930 tx=0x7f0070008040 comp rx=0 tx=0).stop 2023-11-28T13:14:44.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:44.172+0000 7f00855ff700 1 -- 172.21.15.156:0/3298633759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0080161340 msgr2=0x7f008006a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:44.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:44.172+0000 7f00855ff700 1 --2- 172.21.15.156:0/3298633759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0080161340 0x7f008006a900 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f007800cbc0 tx=0x7f007800ced0 comp rx=0 tx=0).stop 2023-11-28T13:14:44.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:44.172+0000 7f00855ff700 1 -- 172.21.15.156:0/3298633759 shutdown_connections 2023-11-28T13:14:44.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:44.172+0000 7f00855ff700 1 --2- 172.21.15.156:0/3298633759 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f006c0682f0 0x7f006c06a7a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:44.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:44.172+0000 7f00855ff700 1 --2- 172.21.15.156:0/3298633759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0080161340 0x7f008006a900 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:44.174 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:44.172+0000 7f00855ff700 1 --2- 172.21.15.156:0/3298633759 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f00800d2320 0x7f00800c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:44.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:44.172+0000 7f00855ff700 1 --2- 172.21.15.156:0/3298633759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f008015c9b0 0x7f00801f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:44.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:44.172+0000 7f00855ff700 1 -- 172.21.15.156:0/3298633759 >> 172.21.15.156:0/3298633759 conn(0x7f0080072600 msgr2=0x7f00800b8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:44.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:44.172+0000 7f00855ff700 1 -- 172.21.15.156:0/3298633759 shutdown_connections 2023-11-28T13:14:44.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:44.172+0000 7f00855ff700 1 -- 172.21.15.156:0/3298633759 wait complete. 2023-11-28T13:14:44.191 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:14:49.193 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:14:49.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.399+0000 7fbbc39ff700 1 Processor -- start 2023-11-28T13:14:49.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.399+0000 7fbbc39ff700 1 -- start start 2023-11-28T13:14:49.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.400+0000 7fbbc39ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbbbc161340 0x7fbbbc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:49.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.400+0000 7fbbc39ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbbbc0d2320 0x7fbbbc0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:49.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.400+0000 7fbbc39ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbbc0d1950 con 0x7fbbbc15c9b0 2023-11-28T13:14:49.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.400+0000 7fbbc39ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbbc0b5800 con 0x7fbbbc0d2320 2023-11-28T13:14:49.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.400+0000 7fbbc39ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbbc0b5300 con 0x7fbbbc161340 2023-11-28T13:14:49.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.400+0000 7fbbc1f9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbbbc0d2320 0x7fbbbc0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:49.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.400+0000 7fbbc1f9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbbbc0d2320 0x7fbbbc0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49822/0 (socket says 172.21.15.156:49822) 2023-11-28T13:14:49.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.400+0000 7fbbc1f9c700 1 -- 172.21.15.156:0/1871544926 learned_addr learned my addr 172.21.15.156:0/1871544926 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:49.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.401+0000 7fbbc0f9a700 1 --2- 172.21.15.156:0/1871544926 >> v2:172.21.15.186:3300/0 conn(0x7fbbbc161340 0x7fbbbc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:49.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.401+0000 7fbbc1f9c700 1 -- 172.21.15.156:0/1871544926 >> v2:172.21.15.186:3300/0 conn(0x7fbbbc161340 msgr2=0x7fbbbc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:14:49.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.401+0000 7fbbc1f9c700 1 --2- 172.21.15.156:0/1871544926 >> v2:172.21.15.186:3300/0 conn(0x7fbbbc161340 0x7fbbbc0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:49.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.401+0000 7fbbc1f9c700 1 -- 172.21.15.156:0/1871544926 >> v1:172.21.15.156:6789/0 conn(0x7fbbbc15c9b0 legacy=0x7fbbbc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:49.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.401+0000 7fbbc1f9c700 1 -- 172.21.15.156:0/1871544926 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbbc0b5520 con 0x7fbbbc0d2320 2023-11-28T13:14:49.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.401+0000 7fbbc0f9a700 1 --2- 172.21.15.156:0/1871544926 >> v2:172.21.15.186:3300/0 conn(0x7fbbbc161340 0x7fbbbc0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:14:49.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.401+0000 7fbbc1f9c700 1 --2- 172.21.15.156:0/1871544926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc0d2320 0x7fbbbc0cd7c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0685c0 tx=0x7fbbb400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ba6b2e7abf9001 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:49.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.401+0000 7fbbb3fff700 1 -- 172.21.15.156:0/1871544926 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb400e070 con 0x7fbbbc0d2320 2023-11-28T13:14:49.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.401+0000 7fbbb3fff700 1 -- 172.21.15.156:0/1871544926 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb400b980 con 0x7fbbbc0d2320 2023-11-28T13:14:49.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.401+0000 7fbbb3fff700 1 -- 172.21.15.156:0/1871544926 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb400bd30 con 0x7fbbbc0d2320 2023-11-28T13:14:49.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.402+0000 7fbbc39ff700 1 -- 172.21.15.156:0/1871544926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc0d2320 msgr2=0x7fbbbc0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:49.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.402+0000 7fbbc39ff700 1 --2- 172.21.15.156:0/1871544926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc0d2320 0x7fbbbc0cd7c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0685c0 tx=0x7fbbb400b330 comp rx=0 tx=0).stop 2023-11-28T13:14:49.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.402+0000 7fbbc39ff700 1 -- 172.21.15.156:0/1871544926 shutdown_connections 2023-11-28T13:14:49.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.402+0000 7fbbc39ff700 1 --2- 172.21.15.156:0/1871544926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc0d2320 0x7fbbbc0cd7c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:49.406 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.402+0000 7fbbc39ff700 1 --2- 172.21.15.156:0/1871544926 >> v2:172.21.15.186:3300/0 conn(0x7fbbbc161340 0x7fbbbc0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:49.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.402+0000 7fbbc39ff700 1 -- 172.21.15.156:0/1871544926 >> 172.21.15.156:0/1871544926 conn(0x7fbbbc072600 msgr2=0x7fbbbc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:49.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.402+0000 7fbbc39ff700 1 -- 172.21.15.156:0/1871544926 shutdown_connections 2023-11-28T13:14:49.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.402+0000 7fbbc39ff700 1 -- 172.21.15.156:0/1871544926 wait complete. 2023-11-28T13:14:49.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.404+0000 7fbbc39ff700 1 Processor -- start 2023-11-28T13:14:49.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.404+0000 7fbbc39ff700 1 -- start start 2023-11-28T13:14:49.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.405+0000 7fbbc39ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbbc0d2320 0x7fbbbc0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:49.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.405+0000 7fbbc39ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbbc15c9b0 0x7fbbbc0ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:49.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.405+0000 7fbbc39ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc161340 0x7fbbbc1494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:49.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.405+0000 7fbbc39ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc0cdff0 con 0x7fbbbc161340 2023-11-28T13:14:49.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.405+0000 7fbbc39ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc15a220 con 0x7fbbbc15c9b0 2023-11-28T13:14:49.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.405+0000 7fbbc39ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc0cd140 con 0x7fbbbc0d2320 2023-11-28T13:14:49.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.405+0000 7fbbc1f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc161340 0x7fbbbc1494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:49.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.405+0000 7fbbc1f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc161340 0x7fbbbc1494e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49828/0 (socket says 172.21.15.156:49828) 2023-11-28T13:14:49.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.405+0000 7fbbc1f9c700 1 -- 172.21.15.156:0/3271631989 learned_addr learned my addr 172.21.15.156:0/3271631989 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:49.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.405+0000 7fbbc0f9a700 1 --2- 172.21.15.156:0/3271631989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbbc15c9b0 0x7fbbbc0ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:49.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.405+0000 7fbbc179b700 1 --2- 172.21.15.156:0/3271631989 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbbc0d2320 0x7fbbbc0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:49.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.406+0000 7fbbc1f9c700 1 -- 172.21.15.156:0/3271631989 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbbc0d2320 msgr2=0x7fbbbc0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:49.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.406+0000 7fbbc1f9c700 1 --2- 172.21.15.156:0/3271631989 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbbc0d2320 0x7fbbbc0ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:49.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.406+0000 7fbbc1f9c700 1 -- 172.21.15.156:0/3271631989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbbc15c9b0 msgr2=0x7fbbbc0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:49.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.406+0000 7fbbc1f9c700 1 --2- 172.21.15.156:0/3271631989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbbc15c9b0 0x7fbbbc0ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:49.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.406+0000 7fbbc1f9c700 1 -- 172.21.15.156:0/3271631989 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb4009b30 con 0x7fbbbc161340 2023-11-28T13:14:49.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.406+0000 7fbbc179b700 1 --2- 172.21.15.156:0/3271631989 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbbc0d2320 0x7fbbbc0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:14:49.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.406+0000 7fbbc0f9a700 1 --2- 172.21.15.156:0/3271631989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbbc15c9b0 0x7fbbbc0ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:14:49.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.406+0000 7fbbc1f9c700 1 --2- 172.21.15.156:0/3271631989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc161340 0x7fbbbc1494e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4012000 tx=0x7fbbb4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:49.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.406+0000 7fbbb1ffb700 1 -- 172.21.15.156:0/3271631989 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb40092e0 con 0x7fbbbc161340 2023-11-28T13:14:49.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.406+0000 7fbbb1ffb700 1 -- 172.21.15.156:0/3271631989 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb4004960 con 0x7fbbbc161340 2023-11-28T13:14:49.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.406+0000 7fbbc39ff700 1 -- 172.21.15.156:0/3271631989 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbbc0d15e0 con 0x7fbbbc161340 2023-11-28T13:14:49.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.406+0000 7fbbb1ffb700 1 -- 172.21.15.156:0/3271631989 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb401b650 con 0x7fbbbc161340 2023-11-28T13:14:49.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.406+0000 7fbbc39ff700 1 -- 172.21.15.156:0/3271631989 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbbc0d1300 con 0x7fbbbc161340 2023-11-28T13:14:49.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.407+0000 7fbbc39ff700 1 -- 172.21.15.156:0/3271631989 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbbc15cd80 con 0x7fbbbc161340 2023-11-28T13:14:49.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.408+0000 7fbbb1ffb700 1 -- 172.21.15.156:0/3271631989 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbbb4021030 con 0x7fbbbc161340 2023-11-28T13:14:49.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.409+0000 7fbbb1ffb700 1 --2- 172.21.15.156:0/3271631989 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbba00682f0 0x7fbba006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:49.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.409+0000 7fbbb1ffb700 1 -- 172.21.15.156:0/3271631989 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fbbb4087440 con 0x7fbbbc161340 2023-11-28T13:14:49.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.409+0000 7fbbc179b700 1 --2- 172.21.15.156:0/3271631989 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbba00682f0 0x7fbba006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:49.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.410+0000 7fbbc179b700 1 --2- 172.21.15.156:0/3271631989 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbba00682f0 0x7fbba006a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0c69f0 tx=0x7fbbb8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:49.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.413+0000 7fbbb1ffb700 1 -- 172.21.15.156:0/3271631989 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbbb4053b20 con 0x7fbbbc161340 2023-11-28T13:14:49.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.644+0000 7fbbc39ff700 1 -- 172.21.15.156:0/3271631989 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbbbc0745c0 con 0x7fbbbc161340 2023-11-28T13:14:49.645 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.644+0000 7fbbb1ffb700 1 -- 172.21.15.156:0/3271631989 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fbbb4057140 con 0x7fbbbc161340 2023-11-28T13:14:49.645 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:14:49.645 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:14:49.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.646+0000 7fbbc39ff700 1 -- 172.21.15.156:0/3271631989 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbba00682f0 msgr2=0x7fbba006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:49.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.646+0000 7fbbc39ff700 1 --2- 172.21.15.156:0/3271631989 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbba00682f0 0x7fbba006a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0c69f0 tx=0x7fbbb8009040 comp rx=0 tx=0).stop 2023-11-28T13:14:49.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.646+0000 7fbbc39ff700 1 -- 172.21.15.156:0/3271631989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc161340 msgr2=0x7fbbbc1494e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:49.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.646+0000 7fbbc39ff700 1 --2- 172.21.15.156:0/3271631989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc161340 0x7fbbbc1494e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4012000 tx=0x7fbbb4004ad0 comp rx=0 tx=0).stop 2023-11-28T13:14:49.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.646+0000 7fbbc39ff700 1 -- 172.21.15.156:0/3271631989 shutdown_connections 2023-11-28T13:14:49.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.646+0000 7fbbc39ff700 1 --2- 172.21.15.156:0/3271631989 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbba00682f0 0x7fbba006a7a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:49.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.646+0000 7fbbc39ff700 1 --2- 172.21.15.156:0/3271631989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbbc161340 0x7fbbbc1494e0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:49.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.646+0000 7fbbc39ff700 1 --2- 172.21.15.156:0/3271631989 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbbc0d2320 0x7fbbbc0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:49.648 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.646+0000 7fbbc39ff700 1 --2- 172.21.15.156:0/3271631989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbbc15c9b0 0x7fbbbc0ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:49.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.646+0000 7fbbc39ff700 1 -- 172.21.15.156:0/3271631989 >> 172.21.15.156:0/3271631989 conn(0x7fbbbc072600 msgr2=0x7fbbbc15fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:49.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.647+0000 7fbbc39ff700 1 -- 172.21.15.156:0/3271631989 shutdown_connections 2023-11-28T13:14:49.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:49.647+0000 7fbbc39ff700 1 -- 172.21.15.156:0/3271631989 wait complete. 2023-11-28T13:14:49.665 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:14:54.666 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:14:54.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.872+0000 7f55ca1ff700 1 Processor -- start 2023-11-28T13:14:54.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.873+0000 7f55ca1ff700 1 -- start start 2023-11-28T13:14:54.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.874+0000 7f55ca1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55c415cbb0 0x7f55c40d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:54.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.874+0000 7f55ca1ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f55c40b47b0 0x7f55c40d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:54.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.874+0000 7f55ca1ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f55c40b1080 0x7f55c40b1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.874+0000 7f55ca1ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c40d3060 con 0x7f55c40b1080 2023-11-28T13:14:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.874+0000 7f55ca1ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c40afd90 con 0x7f55c40b47b0 2023-11-28T13:14:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.874+0000 7f55ca1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c40affe0 con 0x7f55c415cbb0 2023-11-28T13:14:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.874+0000 7f55c3fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f55c40b1080 0x7f55c40b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.874+0000 7f55c3fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f55c40b1080 0x7f55c40b1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49834/0 (socket says 172.21.15.156:49834) 2023-11-28T13:14:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.874+0000 7f55c3fff700 1 -- 172.21.15.156:0/1727167047 learned_addr learned my addr 172.21.15.156:0/1727167047 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.874+0000 7f55c37fe700 1 --2- 172.21.15.156:0/1727167047 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55c415cbb0 0x7f55c40d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.874+0000 7f55bbfff700 1 --2- 172.21.15.156:0/1727167047 >> v2:172.21.15.186:3300/0 conn(0x7f55c40b47b0 0x7f55c40d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.875+0000 7f55c3fff700 1 -- 172.21.15.156:0/1727167047 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55c415cbb0 msgr2=0x7f55c40d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:14:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.875+0000 7f55c3fff700 1 --2- 172.21.15.156:0/1727167047 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55c415cbb0 0x7f55c40d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.875+0000 7f55c3fff700 1 -- 172.21.15.156:0/1727167047 >> v2:172.21.15.186:3300/0 conn(0x7f55c40b47b0 msgr2=0x7f55c40d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:14:54.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.875+0000 7f55c3fff700 1 --2- 172.21.15.156:0/1727167047 >> v2:172.21.15.186:3300/0 conn(0x7f55c40b47b0 0x7f55c40d12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.875+0000 7f55c3fff700 1 -- 172.21.15.156:0/1727167047 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c40b02c0 con 0x7f55c40b1080 2023-11-28T13:14:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.875+0000 7f55c3fff700 1 --2- 172.21.15.156:0/1727167047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55c40b1080 0x7f55c40b1500 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f55c406a480 tx=0x7f55bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffaf5448a0d39082 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.875+0000 7f55c2ffd700 1 -- 172.21.15.156:0/1727167047 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00e070 con 0x7f55c40b1080 2023-11-28T13:14:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.875+0000 7f55c2ffd700 1 -- 172.21.15.156:0/1727167047 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc00b910 con 0x7f55c40b1080 2023-11-28T13:14:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.875+0000 7f55c2ffd700 1 -- 172.21.15.156:0/1727167047 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00bcc0 con 0x7f55c40b1080 2023-11-28T13:14:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.876+0000 7f55ca1ff700 1 -- 172.21.15.156:0/1727167047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55c40b1080 msgr2=0x7f55c40b1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.876+0000 7f55ca1ff700 1 --2- 172.21.15.156:0/1727167047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55c40b1080 0x7f55c40b1500 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f55c406a480 tx=0x7f55bc00b330 comp rx=0 tx=0).stop 2023-11-28T13:14:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.876+0000 7f55ca1ff700 1 -- 172.21.15.156:0/1727167047 shutdown_connections 2023-11-28T13:14:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.876+0000 7f55ca1ff700 1 --2- 172.21.15.156:0/1727167047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55c40b1080 0x7f55c40b1500 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.876+0000 7f55ca1ff700 1 --2- 172.21.15.156:0/1727167047 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55c415cbb0 0x7f55c40d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.876+0000 7f55ca1ff700 1 --2- 172.21.15.156:0/1727167047 >> v2:172.21.15.186:3300/0 conn(0x7f55c40b47b0 0x7f55c40d12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:54.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.876+0000 7f55ca1ff700 1 -- 172.21.15.156:0/1727167047 >> 172.21.15.156:0/1727167047 conn(0x7f55c409c430 msgr2=0x7f55c40d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:54.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.876+0000 7f55ca1ff700 1 -- 172.21.15.156:0/1727167047 shutdown_connections 2023-11-28T13:14:54.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.876+0000 7f55ca1ff700 1 -- 172.21.15.156:0/1727167047 wait complete. 2023-11-28T13:14:54.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.878+0000 7f55ca1ff700 1 Processor -- start 2023-11-28T13:14:54.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.878+0000 7f55ca1ff700 1 -- start start 2023-11-28T13:14:54.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.879+0000 7f55ca1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55c40b1080 0x7f55c4157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:54.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.879+0000 7f55ca1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55c40b47b0 0x7f55c41e3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:54.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.879+0000 7f55ca1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55c415cbb0 0x7f55c41e6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:54.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.879+0000 7f55ca1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c41e5a90 con 0x7f55c40b47b0 2023-11-28T13:14:54.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.879+0000 7f55ca1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c41e5600 con 0x7f55c415cbb0 2023-11-28T13:14:54.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.879+0000 7f55ca1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c4158030 con 0x7f55c40b1080 2023-11-28T13:14:54.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.879+0000 7f55bbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55c40b47b0 0x7f55c41e3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:54.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.879+0000 7f55bbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55c40b47b0 0x7f55c41e3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49846/0 (socket says 172.21.15.156:49846) 2023-11-28T13:14:54.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.879+0000 7f55bbfff700 1 -- 172.21.15.156:0/3268187630 learned_addr learned my addr 172.21.15.156:0/3268187630 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:14:54.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.879+0000 7f55c37fe700 1 --2- 172.21.15.156:0/3268187630 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55c40b1080 0x7f55c4157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:54.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.879+0000 7f55c3fff700 1 --2- 172.21.15.156:0/3268187630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55c415cbb0 0x7f55c41e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:54.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.880+0000 7f55bbfff700 1 -- 172.21.15.156:0/3268187630 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55c40b1080 msgr2=0x7f55c4157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:54.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.880+0000 7f55bbfff700 1 --2- 172.21.15.156:0/3268187630 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55c40b1080 0x7f55c4157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:54.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.880+0000 7f55bbfff700 1 -- 172.21.15.156:0/3268187630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55c415cbb0 msgr2=0x7f55c41e6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:54.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.880+0000 7f55bbfff700 1 --2- 172.21.15.156:0/3268187630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55c415cbb0 0x7f55c41e6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:54.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.880+0000 7f55bbfff700 1 -- 172.21.15.156:0/3268187630 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc009b30 con 0x7f55c40b47b0 2023-11-28T13:14:54.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.880+0000 7f55c3fff700 1 --2- 172.21.15.156:0/3268187630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55c415cbb0 0x7f55c41e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:14:54.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.880+0000 7f55bbfff700 1 --2- 172.21.15.156:0/3268187630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55c40b47b0 0x7f55c41e3fa0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00e930 tx=0x7f55ac00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:54.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.880+0000 7f55c0ff9700 1 -- 172.21.15.156:0/3268187630 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00ee80 con 0x7f55c40b47b0 2023-11-28T13:14:54.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.880+0000 7f55c0ff9700 1 -- 172.21.15.156:0/3268187630 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac00c850 con 0x7f55c40b47b0 2023-11-28T13:14:54.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.880+0000 7f55c0ff9700 1 -- 172.21.15.156:0/3268187630 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00cbe0 con 0x7f55c40b47b0 2023-11-28T13:14:54.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.880+0000 7f55ca1ff700 1 -- 172.21.15.156:0/3268187630 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c40b0880 con 0x7f55c40b47b0 2023-11-28T13:14:54.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.880+0000 7f55ca1ff700 1 -- 172.21.15.156:0/3268187630 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c40b05a0 con 0x7f55c40b47b0 2023-11-28T13:14:54.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.881+0000 7f55ca1ff700 1 -- 172.21.15.156:0/3268187630 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c40b0b60 con 0x7f55c40b47b0 2023-11-28T13:14:54.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.882+0000 7f55c0ff9700 1 -- 172.21.15.156:0/3268187630 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f55ac019030 con 0x7f55c40b47b0 2023-11-28T13:14:54.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.883+0000 7f55c0ff9700 1 --2- 172.21.15.156:0/3268187630 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f55a40682f0 0x7f55a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:14:54.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.883+0000 7f55c0ff9700 1 -- 172.21.15.156:0/3268187630 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f55ac085cc0 con 0x7f55c40b47b0 2023-11-28T13:14:54.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.883+0000 7f55c37fe700 1 --2- 172.21.15.156:0/3268187630 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f55a40682f0 0x7f55a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:14:54.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.884+0000 7f55c37fe700 1 --2- 172.21.15.156:0/3268187630 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f55a40682f0 0x7f55a406a7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009b20 tx=0x7f55b40091c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:14:54.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:54.887+0000 7f55c0ff9700 1 -- 172.21.15.156:0/3268187630 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f55ac08c080 con 0x7f55c40b47b0 2023-11-28T13:14:55.125 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:55.125+0000 7f55ca1ff700 1 -- 172.21.15.156:0/3268187630 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f55c40745c0 con 0x7f55c40b47b0 2023-11-28T13:14:55.126 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:55.126+0000 7f55c0ff9700 1 -- 172.21.15.156:0/3268187630 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f55ac0523a0 con 0x7f55c40b47b0 2023-11-28T13:14:55.126 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:14:55.126 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:14:55.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:55.128+0000 7f55ca1ff700 1 -- 172.21.15.156:0/3268187630 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f55a40682f0 msgr2=0x7f55a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:55.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:55.128+0000 7f55ca1ff700 1 --2- 172.21.15.156:0/3268187630 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f55a40682f0 0x7f55a406a7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009b20 tx=0x7f55b40091c0 comp rx=0 tx=0).stop 2023-11-28T13:14:55.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:55.128+0000 7f55ca1ff700 1 -- 172.21.15.156:0/3268187630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55c40b47b0 msgr2=0x7f55c41e3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:14:55.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:55.128+0000 7f55ca1ff700 1 --2- 172.21.15.156:0/3268187630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55c40b47b0 0x7f55c41e3fa0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00e930 tx=0x7f55ac00ec40 comp rx=0 tx=0).stop 2023-11-28T13:14:55.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:55.128+0000 7f55ca1ff700 1 -- 172.21.15.156:0/3268187630 shutdown_connections 2023-11-28T13:14:55.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:55.128+0000 7f55ca1ff700 1 --2- 172.21.15.156:0/3268187630 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f55a40682f0 0x7f55a406a7a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:55.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:55.128+0000 7f55ca1ff700 1 --2- 172.21.15.156:0/3268187630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55c415cbb0 0x7f55c41e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:55.129 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:55.128+0000 7f55ca1ff700 1 --2- 172.21.15.156:0/3268187630 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55c40b1080 0x7f55c4157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:55.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:55.128+0000 7f55ca1ff700 1 --2- 172.21.15.156:0/3268187630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55c40b47b0 0x7f55c41e3fa0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:14:55.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:55.128+0000 7f55ca1ff700 1 -- 172.21.15.156:0/3268187630 >> 172.21.15.156:0/3268187630 conn(0x7f55c409c430 msgr2=0x7f55c40b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:14:55.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:55.128+0000 7f55ca1ff700 1 -- 172.21.15.156:0/3268187630 shutdown_connections 2023-11-28T13:14:55.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:14:55.128+0000 7f55ca1ff700 1 -- 172.21.15.156:0/3268187630 wait complete. 2023-11-28T13:14:55.147 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:15:00.149 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:15:00.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.354+0000 7fb0c45ff700 1 Processor -- start 2023-11-28T13:15:00.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.354+0000 7fb0c45ff700 1 -- start start 2023-11-28T13:15:00.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.355+0000 7fb0c45ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb0bc15c9b0 0x7fb0bc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:00.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.355+0000 7fb0c45ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb0bc161340 0x7fb0bc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:00.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.356+0000 7fb0c45ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0bc0d1950 con 0x7fb0bc0d2320 2023-11-28T13:15:00.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.356+0000 7fb0c45ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0bc0b5800 con 0x7fb0bc15c9b0 2023-11-28T13:15:00.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.356+0000 7fb0c45ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0bc0b5520 con 0x7fb0bc161340 2023-11-28T13:15:00.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.356+0000 7fb0c239b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb0bc15c9b0 0x7fb0bc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:00.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.356+0000 7fb0c239b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb0bc15c9b0 0x7fb0bc0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49010/0 (socket says 172.21.15.156:49010) 2023-11-28T13:15:00.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.356+0000 7fb0c239b700 1 -- 172.21.15.156:0/3605390994 learned_addr learned my addr 172.21.15.156:0/3605390994 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:00.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.356+0000 7fb0c1b9a700 1 --2- 172.21.15.156:0/3605390994 >> v2:172.21.15.186:3300/0 conn(0x7fb0bc161340 0x7fb0bc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:00.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.356+0000 7fb0c239b700 1 -- 172.21.15.156:0/3605390994 >> v2:172.21.15.186:3300/0 conn(0x7fb0bc161340 msgr2=0x7fb0bc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:00.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.356+0000 7fb0c239b700 1 --2- 172.21.15.156:0/3605390994 >> v2:172.21.15.186:3300/0 conn(0x7fb0bc161340 0x7fb0bc0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:00.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.356+0000 7fb0c239b700 1 -- 172.21.15.156:0/3605390994 >> v1:172.21.15.156:6789/0 conn(0x7fb0bc0d2320 legacy=0x7fb0bc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:00.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.356+0000 7fb0c239b700 1 -- 172.21.15.156:0/3605390994 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc0d1300 con 0x7fb0bc15c9b0 2023-11-28T13:15:00.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.357+0000 7fb0c239b700 1 --2- 172.21.15.156:0/3605390994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0bc15c9b0 0x7fb0bc0b4bf0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0685c0 tx=0x7fb0b8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a54841adc4525 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:00.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.357+0000 7fb0c1399700 1 -- 172.21.15.156:0/3605390994 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b800b870 con 0x7fb0bc15c9b0 2023-11-28T13:15:00.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.357+0000 7fb0c1399700 1 -- 172.21.15.156:0/3605390994 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b800b9d0 con 0x7fb0bc15c9b0 2023-11-28T13:15:00.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.357+0000 7fb0c1399700 1 -- 172.21.15.156:0/3605390994 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b800bd60 con 0x7fb0bc15c9b0 2023-11-28T13:15:00.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.357+0000 7fb0c45ff700 1 -- 172.21.15.156:0/3605390994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0bc15c9b0 msgr2=0x7fb0bc0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:00.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.357+0000 7fb0c45ff700 1 --2- 172.21.15.156:0/3605390994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0bc15c9b0 0x7fb0bc0b4bf0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0685c0 tx=0x7fb0b8009bd0 comp rx=0 tx=0).stop 2023-11-28T13:15:00.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.357+0000 7fb0c45ff700 1 -- 172.21.15.156:0/3605390994 shutdown_connections 2023-11-28T13:15:00.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.357+0000 7fb0c45ff700 1 --2- 172.21.15.156:0/3605390994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0bc15c9b0 0x7fb0bc0b4bf0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:00.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.357+0000 7fb0c45ff700 1 --2- 172.21.15.156:0/3605390994 >> v2:172.21.15.186:3300/0 conn(0x7fb0bc161340 0x7fb0bc0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:00.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.357+0000 7fb0c45ff700 1 -- 172.21.15.156:0/3605390994 >> 172.21.15.156:0/3605390994 conn(0x7fb0bc072600 msgr2=0x7fb0bc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:00.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.358+0000 7fb0c45ff700 1 -- 172.21.15.156:0/3605390994 shutdown_connections 2023-11-28T13:15:00.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.358+0000 7fb0c45ff700 1 -- 172.21.15.156:0/3605390994 wait complete. 2023-11-28T13:15:00.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.359+0000 7fb0c45ff700 1 Processor -- start 2023-11-28T13:15:00.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.359+0000 7fb0c45ff700 1 -- start start 2023-11-28T13:15:00.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.360+0000 7fb0c45ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0bc0d2320 0x7fb0bc150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:00.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.360+0000 7fb0c45ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0bc15c9b0 0x7fb0bc14fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:00.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.360+0000 7fb0c45ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0bc161340 0x7fb0bc154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:00.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.360+0000 7fb0c45ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc150060 con 0x7fb0bc161340 2023-11-28T13:15:00.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.360+0000 7fb0c45ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc155780 con 0x7fb0bc15c9b0 2023-11-28T13:15:00.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.360+0000 7fb0c45ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc155a60 con 0x7fb0bc0d2320 2023-11-28T13:15:00.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.361+0000 7fb0c2b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0bc161340 0x7fb0bc154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:00.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.361+0000 7fb0c2b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0bc161340 0x7fb0bc154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49022/0 (socket says 172.21.15.156:49022) 2023-11-28T13:15:00.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.361+0000 7fb0c2b9c700 1 -- 172.21.15.156:0/4154788400 learned_addr learned my addr 172.21.15.156:0/4154788400 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:00.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.361+0000 7fb0c1b9a700 1 --2- 172.21.15.156:0/4154788400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0bc15c9b0 0x7fb0bc14fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:00.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.361+0000 7fb0c239b700 1 --2- 172.21.15.156:0/4154788400 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0bc0d2320 0x7fb0bc150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:00.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.361+0000 7fb0c2b9c700 1 -- 172.21.15.156:0/4154788400 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0bc0d2320 msgr2=0x7fb0bc150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:00.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.361+0000 7fb0c2b9c700 1 --2- 172.21.15.156:0/4154788400 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0bc0d2320 0x7fb0bc150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:00.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.361+0000 7fb0c2b9c700 1 -- 172.21.15.156:0/4154788400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0bc15c9b0 msgr2=0x7fb0bc14fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:00.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.361+0000 7fb0c2b9c700 1 --2- 172.21.15.156:0/4154788400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0bc15c9b0 0x7fb0bc14fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:00.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.361+0000 7fb0c2b9c700 1 -- 172.21.15.156:0/4154788400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b8009550 con 0x7fb0bc161340 2023-11-28T13:15:00.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.361+0000 7fb0c2b9c700 1 --2- 172.21.15.156:0/4154788400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0bc161340 0x7fb0bc154c20 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400cbc0 tx=0x7fb0b400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:00.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.361+0000 7fb0b2ffd700 1 -- 172.21.15.156:0/4154788400 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4003c00 con 0x7fb0bc161340 2023-11-28T13:15:00.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.362+0000 7fb0b2ffd700 1 -- 172.21.15.156:0/4154788400 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b4003d60 con 0x7fb0bc161340 2023-11-28T13:15:00.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.362+0000 7fb0c45ff700 1 -- 172.21.15.156:0/4154788400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0bc0b5130 con 0x7fb0bc161340 2023-11-28T13:15:00.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.362+0000 7fb0b2ffd700 1 -- 172.21.15.156:0/4154788400 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400aa20 con 0x7fb0bc161340 2023-11-28T13:15:00.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.362+0000 7fb0c45ff700 1 -- 172.21.15.156:0/4154788400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0bc0d15e0 con 0x7fb0bc161340 2023-11-28T13:15:00.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.363+0000 7fb0c45ff700 1 -- 172.21.15.156:0/4154788400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0bc0b52c0 con 0x7fb0bc161340 2023-11-28T13:15:00.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.364+0000 7fb0b2ffd700 1 -- 172.21.15.156:0/4154788400 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb0b4010050 con 0x7fb0bc161340 2023-11-28T13:15:00.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.364+0000 7fb0b2ffd700 1 --2- 172.21.15.156:0/4154788400 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0a00682f0 0x7fb0a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:00.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.364+0000 7fb0b2ffd700 1 -- 172.21.15.156:0/4154788400 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fb0b4085ba0 con 0x7fb0bc161340 2023-11-28T13:15:00.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.365+0000 7fb0c239b700 1 --2- 172.21.15.156:0/4154788400 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0a00682f0 0x7fb0a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:00.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.365+0000 7fb0c239b700 1 --2- 172.21.15.156:0/4154788400 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0a00682f0 0x7fb0a006a7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8000c00 tx=0x7fb0b8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:00.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.369+0000 7fb0b2ffd700 1 -- 172.21.15.156:0/4154788400 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb0b4052280 con 0x7fb0bc161340 2023-11-28T13:15:00.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.600+0000 7fb0c45ff700 1 -- 172.21.15.156:0/4154788400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb0bc152bc0 con 0x7fb0bc161340 2023-11-28T13:15:00.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.600+0000 7fb0b2ffd700 1 -- 172.21.15.156:0/4154788400 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fb0b40558a0 con 0x7fb0bc161340 2023-11-28T13:15:00.601 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:15:00.601 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:15:00.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.602+0000 7fb0c45ff700 1 -- 172.21.15.156:0/4154788400 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0a00682f0 msgr2=0x7fb0a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:00.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.602+0000 7fb0c45ff700 1 --2- 172.21.15.156:0/4154788400 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0a00682f0 0x7fb0a006a7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8000c00 tx=0x7fb0b8008030 comp rx=0 tx=0).stop 2023-11-28T13:15:00.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.602+0000 7fb0c45ff700 1 -- 172.21.15.156:0/4154788400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0bc161340 msgr2=0x7fb0bc154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:00.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.602+0000 7fb0c45ff700 1 --2- 172.21.15.156:0/4154788400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0bc161340 0x7fb0bc154c20 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400cbc0 tx=0x7fb0b400ced0 comp rx=0 tx=0).stop 2023-11-28T13:15:00.603 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.602+0000 7fb0c45ff700 1 -- 172.21.15.156:0/4154788400 shutdown_connections 2023-11-28T13:15:00.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.602+0000 7fb0c45ff700 1 --2- 172.21.15.156:0/4154788400 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb0a00682f0 0x7fb0a006a7a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:00.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.602+0000 7fb0c45ff700 1 --2- 172.21.15.156:0/4154788400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0bc161340 0x7fb0bc154c20 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:00.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.602+0000 7fb0c45ff700 1 --2- 172.21.15.156:0/4154788400 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0bc0d2320 0x7fb0bc150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:00.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.602+0000 7fb0c45ff700 1 --2- 172.21.15.156:0/4154788400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0bc15c9b0 0x7fb0bc14fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:00.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.602+0000 7fb0c45ff700 1 -- 172.21.15.156:0/4154788400 >> 172.21.15.156:0/4154788400 conn(0x7fb0bc072600 msgr2=0x7fb0bc15a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:00.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.603+0000 7fb0c45ff700 1 -- 172.21.15.156:0/4154788400 shutdown_connections 2023-11-28T13:15:00.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:00.603+0000 7fb0c45ff700 1 -- 172.21.15.156:0/4154788400 wait complete. 2023-11-28T13:15:00.621 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:15:04.419 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:15:04.425 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:15:05.622 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:15:05.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.828+0000 7f45743ff700 1 Processor -- start 2023-11-28T13:15:05.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.828+0000 7f45743ff700 1 -- start start 2023-11-28T13:15:05.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.829+0000 7f45743ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f456c161340 0x7f456c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:05.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.829+0000 7f45743ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f456c0d2320 0x7f456c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:05.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.829+0000 7f45743ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456c0d1950 con 0x7f456c15c9b0 2023-11-28T13:15:05.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.829+0000 7f45743ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f456c0b5800 con 0x7f456c161340 2023-11-28T13:15:05.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.830+0000 7f45743ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456c0b5300 con 0x7f456c0d2320 2023-11-28T13:15:05.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.830+0000 7f457199a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f456c161340 0x7f456c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:05.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.830+0000 7f457199a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f456c161340 0x7f456c0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38466/0 (socket says 172.21.15.156:38466) 2023-11-28T13:15:05.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.830+0000 7f457199a700 1 -- 172.21.15.156:0/884508291 learned_addr learned my addr 172.21.15.156:0/884508291 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:05.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.830+0000 7f457299c700 1 --2- 172.21.15.156:0/884508291 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f456c0d2320 0x7f456c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:05.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.830+0000 7f457199a700 1 -- 172.21.15.156:0/884508291 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f456c0d2320 msgr2=0x7f456c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:05.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.830+0000 7f457199a700 1 --2- 172.21.15.156:0/884508291 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f456c0d2320 0x7f456c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:05.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.830+0000 7f457199a700 1 -- 172.21.15.156:0/884508291 >> v1:172.21.15.186:6789/0 conn(0x7f456c15c9b0 legacy=0x7f456c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:05.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.830+0000 7f457199a700 1 -- 172.21.15.156:0/884508291 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c0b5520 con 0x7f456c161340 2023-11-28T13:15:05.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.830+0000 7f457199a700 1 --2- 172.21.15.156:0/884508291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f456c161340 0x7f456c0d1de0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f455c0098f0 tx=0x7f455c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ffb0cf6967fdc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:05.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.831+0000 7f4571199700 1 -- 172.21.15.156:0/884508291 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c00b8a0 con 0x7f456c161340 2023-11-28T13:15:05.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.831+0000 7f4571199700 1 -- 172.21.15.156:0/884508291 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455c00ba00 con 0x7f456c161340 2023-11-28T13:15:05.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.831+0000 7f4571199700 1 -- 172.21.15.156:0/884508291 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c00bd90 con 0x7f456c161340 2023-11-28T13:15:05.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.831+0000 7f45743ff700 1 -- 172.21.15.156:0/884508291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f456c161340 msgr2=0x7f456c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:05.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.831+0000 7f45743ff700 1 --2- 172.21.15.156:0/884508291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f456c161340 0x7f456c0d1de0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f455c0098f0 tx=0x7f455c009c00 comp rx=0 tx=0).stop 2023-11-28T13:15:05.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.831+0000 7f45743ff700 1 -- 172.21.15.156:0/884508291 shutdown_connections 2023-11-28T13:15:05.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.831+0000 7f45743ff700 1 --2- 172.21.15.156:0/884508291 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f456c0d2320 0x7f456c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:05.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.831+0000 7f45743ff700 1 --2- 172.21.15.156:0/884508291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f456c161340 0x7f456c0d1de0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:05.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.831+0000 7f45743ff700 1 -- 172.21.15.156:0/884508291 >> 172.21.15.156:0/884508291 conn(0x7f456c072600 msgr2=0x7f456c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:05.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.831+0000 7f45743ff700 1 -- 172.21.15.156:0/884508291 shutdown_connections 2023-11-28T13:15:05.841 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.831+0000 7f45743ff700 1 -- 172.21.15.156:0/884508291 wait complete. 2023-11-28T13:15:05.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.833+0000 7f45743ff700 1 Processor -- start 2023-11-28T13:15:05.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.833+0000 7f45743ff700 1 -- start start 2023-11-28T13:15:05.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.834+0000 7f45743ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f456c0d2320 0x7f456c1f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:05.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.834+0000 7f45743ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f456c15c9b0 0x7f456c0b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:05.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.834+0000 7f45743ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f456c161340 0x7f456c0b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:05.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.834+0000 7f45743ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c0c8690 con 0x7f456c0d2320 2023-11-28T13:15:05.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.834+0000 7f45743ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c154640 con 0x7f456c15c9b0 2023-11-28T13:15:05.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.834+0000 7f45743ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456c0c7880 con 0x7f456c161340 2023-11-28T13:15:05.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.834+0000 7f457219b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f456c0d2320 0x7f456c1f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:05.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.835+0000 7f457219b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f456c0d2320 0x7f456c1f5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38470/0 (socket says 172.21.15.156:38470) 2023-11-28T13:15:05.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.835+0000 7f457219b700 1 -- 172.21.15.156:0/3064902845 learned_addr learned my addr 172.21.15.156:0/3064902845 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:05.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.835+0000 7f457199a700 1 --2- 172.21.15.156:0/3064902845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f456c15c9b0 0x7f456c0b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:05.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.835+0000 7f457299c700 1 --2- 172.21.15.156:0/3064902845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f456c161340 0x7f456c0b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:05.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.835+0000 7f457219b700 1 -- 172.21.15.156:0/3064902845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f456c161340 msgr2=0x7f456c0b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:05.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.835+0000 7f457219b700 1 --2- 172.21.15.156:0/3064902845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f456c161340 0x7f456c0b64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:05.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.835+0000 7f457219b700 1 -- 172.21.15.156:0/3064902845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f456c15c9b0 msgr2=0x7f456c0b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:05.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.835+0000 7f457219b700 1 --2- 172.21.15.156:0/3064902845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f456c15c9b0 0x7f456c0b5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:05.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.835+0000 7f457219b700 1 -- 172.21.15.156:0/3064902845 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c009550 con 0x7f456c0d2320 2023-11-28T13:15:05.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.835+0000 7f457219b700 1 --2- 172.21.15.156:0/3064902845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f456c0d2320 0x7f456c1f5730 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f456800c450 tx=0x7f456800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:05.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.835+0000 7f4562ffd700 1 -- 172.21.15.156:0/3064902845 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680052d0 con 0x7f456c0d2320 2023-11-28T13:15:05.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.835+0000 7f4562ffd700 1 -- 172.21.15.156:0/3064902845 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4568005430 con 0x7f456c0d2320 2023-11-28T13:15:05.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.835+0000 7f45743ff700 1 -- 172.21.15.156:0/3064902845 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456c0d15e0 con 0x7f456c0d2320 2023-11-28T13:15:05.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.836+0000 7f4562ffd700 1 -- 172.21.15.156:0/3064902845 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680038c0 con 0x7f456c0d2320 2023-11-28T13:15:05.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.836+0000 7f45743ff700 1 -- 172.21.15.156:0/3064902845 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456c0d1300 con 0x7f456c0d2320 2023-11-28T13:15:05.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.837+0000 7f45743ff700 1 -- 172.21.15.156:0/3064902845 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f456c15cd80 con 0x7f456c0d2320 2023-11-28T13:15:05.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.837+0000 7f4562ffd700 1 -- 172.21.15.156:0/3064902845 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4568019030 con 0x7f456c0d2320 2023-11-28T13:15:05.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.838+0000 7f4562ffd700 1 --2- 172.21.15.156:0/3064902845 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f45500682f0 0x7f455006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:05.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.838+0000 7f4562ffd700 1 -- 172.21.15.156:0/3064902845 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f4568084fb0 con 0x7f456c0d2320 2023-11-28T13:15:05.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.839+0000 7f457199a700 1 --2- 172.21.15.156:0/3064902845 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f45500682f0 0x7f455006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:05.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.840+0000 7f457199a700 1 --2- 172.21.15.156:0/3064902845 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f45500682f0 0x7f455006a7a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f455c009c40 tx=0x7f455c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:05.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:05.842+0000 7f4562ffd700 1 -- 172.21.15.156:0/3064902845 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4568051690 con 0x7f456c0d2320 2023-11-28T13:15:06.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:06.073+0000 7f45743ff700 1 -- 172.21.15.156:0/3064902845 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f456c0745c0 con 0x7f456c0d2320 2023-11-28T13:15:06.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:06.073+0000 7f4562ffd700 1 -- 172.21.15.156:0/3064902845 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f45680170c0 con 0x7f456c0d2320 2023-11-28T13:15:06.075 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:15:06.075 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:15:06.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:06.076+0000 7f45743ff700 1 -- 172.21.15.156:0/3064902845 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f45500682f0 msgr2=0x7f455006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:06.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:06.076+0000 7f45743ff700 1 --2- 172.21.15.156:0/3064902845 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f45500682f0 0x7f455006a7a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f455c009c40 tx=0x7f455c008030 comp rx=0 tx=0).stop 2023-11-28T13:15:06.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:06.076+0000 7f45743ff700 1 -- 172.21.15.156:0/3064902845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f456c0d2320 msgr2=0x7f456c1f5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:06.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:06.076+0000 7f45743ff700 1 --2- 172.21.15.156:0/3064902845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f456c0d2320 0x7f456c1f5730 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f456800c450 tx=0x7f456800c760 comp rx=0 tx=0).stop 2023-11-28T13:15:06.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:06.076+0000 7f45743ff700 1 -- 172.21.15.156:0/3064902845 shutdown_connections 2023-11-28T13:15:06.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:06.076+0000 7f45743ff700 1 --2- 172.21.15.156:0/3064902845 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f45500682f0 0x7f455006a7a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:06.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:06.076+0000 7f45743ff700 1 --2- 172.21.15.156:0/3064902845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f456c161340 0x7f456c0b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:06.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:06.076+0000 7f45743ff700 1 --2- 172.21.15.156:0/3064902845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f456c0d2320 0x7f456c1f5730 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:06.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:06.076+0000 7f45743ff700 1 --2- 172.21.15.156:0/3064902845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f456c15c9b0 0x7f456c0b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:06.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:06.076+0000 7f45743ff700 1 -- 172.21.15.156:0/3064902845 >> 172.21.15.156:0/3064902845 conn(0x7f456c072600 msgr2=0x7f456c15f990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:06.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:06.076+0000 7f45743ff700 1 -- 172.21.15.156:0/3064902845 shutdown_connections 2023-11-28T13:15:06.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:06.076+0000 7f45743ff700 1 -- 172.21.15.156:0/3064902845 wait complete. 2023-11-28T13:15:06.095 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:15:11.096 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:15:11.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.300+0000 7fce67bff700 1 Processor -- start 2023-11-28T13:15:11.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.300+0000 7fce67bff700 1 -- start start 2023-11-28T13:15:11.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.301+0000 7fce67bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce60161340 0x7fce600d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:11.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.301+0000 7fce67bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fce600d2320 0x7fce600cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:11.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.302+0000 7fce67bff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce600d1950 con 0x7fce6015c9b0 2023-11-28T13:15:11.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.302+0000 7fce67bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce600b5800 con 0x7fce600d2320 2023-11-28T13:15:11.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.302+0000 7fce67bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce600b5300 con 0x7fce60161340 2023-11-28T13:15:11.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.302+0000 7fce6599b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fce6015c9b0 0x7fce600b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:35414/0 (socket says 172.21.15.156:35414) 2023-11-28T13:15:11.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.302+0000 7fce6599b700 1 -- 172.21.15.156:0/118962305 learned_addr learned my addr 172.21.15.156:0/118962305 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:11.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.302+0000 7fce6619c700 1 --2- 172.21.15.156:0/118962305 >> v2:172.21.15.156:3300/0 conn(0x7fce600d2320 0x7fce600cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:11.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.302+0000 7fce6519a700 1 --2- 172.21.15.156:0/118962305 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce60161340 0x7fce600d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:11.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.302+0000 7fce6619c700 1 -- 172.21.15.156:0/118962305 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce60161340 msgr2=0x7fce600d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:11.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.302+0000 7fce6619c700 1 --2- 172.21.15.156:0/118962305 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce60161340 0x7fce600d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:11.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.302+0000 7fce6619c700 1 -- 172.21.15.156:0/118962305 >> v1:172.21.15.156:6789/0 conn(0x7fce6015c9b0 legacy=0x7fce600b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:11.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.302+0000 7fce6619c700 1 -- 172.21.15.156:0/118962305 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce600b5520 con 0x7fce600d2320 2023-11-28T13:15:11.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.303+0000 7fce6619c700 1 --2- 172.21.15.156:0/118962305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce600d2320 0x7fce600cd7c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fce58009dc0 tx=0x7fce5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f100d39916996d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:11.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.303+0000 7fce64999700 1 -- 172.21.15.156:0/118962305 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5800e070 con 0x7fce600d2320 2023-11-28T13:15:11.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.303+0000 7fce64999700 1 -- 172.21.15.156:0/118962305 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5800bb90 con 0x7fce600d2320 2023-11-28T13:15:11.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.303+0000 7fce64999700 1 -- 172.21.15.156:0/118962305 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58012430 con 0x7fce600d2320 2023-11-28T13:15:11.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.303+0000 7fce67bff700 1 -- 172.21.15.156:0/118962305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce600d2320 msgr2=0x7fce600cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:11.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.303+0000 7fce67bff700 1 --2- 172.21.15.156:0/118962305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce600d2320 0x7fce600cd7c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fce58009dc0 tx=0x7fce5800b330 comp rx=0 tx=0).stop 2023-11-28T13:15:11.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.303+0000 7fce67bff700 1 -- 172.21.15.156:0/118962305 shutdown_connections 2023-11-28T13:15:11.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.303+0000 7fce67bff700 1 --2- 172.21.15.156:0/118962305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce600d2320 0x7fce600cd7c0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:11.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.303+0000 7fce67bff700 1 --2- 172.21.15.156:0/118962305 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce60161340 0x7fce600d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:11.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.303+0000 7fce67bff700 1 -- 172.21.15.156:0/118962305 >> 172.21.15.156:0/118962305 conn(0x7fce60072600 msgr2=0x7fce600cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:11.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.303+0000 7fce67bff700 1 -- 172.21.15.156:0/118962305 shutdown_connections 2023-11-28T13:15:11.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.304+0000 7fce67bff700 1 -- 172.21.15.156:0/118962305 wait complete. 2023-11-28T13:15:11.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.305+0000 7fce67bff700 1 Processor -- start 2023-11-28T13:15:11.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.305+0000 7fce67bff700 1 -- start start 2023-11-28T13:15:11.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.306+0000 7fce67bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce600d2320 0x7fce60151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:11.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.306+0000 7fce67bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6015c9b0 0x7fce600c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:11.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.306+0000 7fce67bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce60161340 0x7fce600c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:11.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.306+0000 7fce67bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce601531e0 con 0x7fce6015c9b0 2023-11-28T13:15:11.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.306+0000 7fce67bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce60152560 con 0x7fce600d2320 2023-11-28T13:15:11.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.306+0000 7fce67bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce60152840 con 0x7fce60161340 2023-11-28T13:15:11.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.306+0000 7fce6519a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6015c9b0 0x7fce600c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:11.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.306+0000 7fce6519a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6015c9b0 0x7fce600c8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38496/0 (socket says 172.21.15.156:38496) 2023-11-28T13:15:11.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.306+0000 7fce6519a700 1 -- 172.21.15.156:0/2713048197 learned_addr learned my addr 172.21.15.156:0/2713048197 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:11.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.307+0000 7fce6619c700 1 --2- 172.21.15.156:0/2713048197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce60161340 0x7fce600c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:11.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.307+0000 7fce6599b700 1 --2- 172.21.15.156:0/2713048197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce600d2320 0x7fce60151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:11.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.307+0000 7fce6519a700 1 -- 172.21.15.156:0/2713048197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce60161340 msgr2=0x7fce600c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:11.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.307+0000 7fce6519a700 1 --2- 172.21.15.156:0/2713048197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce60161340 0x7fce600c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:11.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.307+0000 7fce6519a700 1 -- 172.21.15.156:0/2713048197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce600d2320 msgr2=0x7fce60151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:11.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.307+0000 7fce6519a700 1 --2- 172.21.15.156:0/2713048197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce600d2320 0x7fce60151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:11.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.307+0000 7fce6519a700 1 -- 172.21.15.156:0/2713048197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce58009a70 con 0x7fce6015c9b0 2023-11-28T13:15:11.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.307+0000 7fce6519a700 1 --2- 172.21.15.156:0/2713048197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6015c9b0 0x7fce600c8e30 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fce5000e810 tx=0x7fce5000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:11.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.307+0000 7fce567fc700 1 -- 172.21.15.156:0/2713048197 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5000c070 con 0x7fce6015c9b0 2023-11-28T13:15:11.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.307+0000 7fce567fc700 1 -- 172.21.15.156:0/2713048197 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce50013610 con 0x7fce6015c9b0 2023-11-28T13:15:11.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.307+0000 7fce567fc700 1 -- 172.21.15.156:0/2713048197 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce500139c0 con 0x7fce6015c9b0 2023-11-28T13:15:11.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.307+0000 7fce67bff700 1 -- 172.21.15.156:0/2713048197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce600d15e0 con 0x7fce6015c9b0 2023-11-28T13:15:11.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.307+0000 7fce67bff700 1 -- 172.21.15.156:0/2713048197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce600d1300 con 0x7fce6015c9b0 2023-11-28T13:15:11.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.309+0000 7fce67bff700 1 -- 172.21.15.156:0/2713048197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6015cd80 con 0x7fce6015c9b0 2023-11-28T13:15:11.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.309+0000 7fce567fc700 1 -- 172.21.15.156:0/2713048197 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fce50023070 con 0x7fce6015c9b0 2023-11-28T13:15:11.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.310+0000 7fce567fc700 1 --2- 172.21.15.156:0/2713048197 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fce440682f0 0x7fce4406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:11.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.310+0000 7fce567fc700 1 -- 172.21.15.156:0/2713048197 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fce50085a40 con 0x7fce6015c9b0 2023-11-28T13:15:11.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.311+0000 7fce6599b700 1 --2- 172.21.15.156:0/2713048197 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fce440682f0 0x7fce4406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:11.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.311+0000 7fce6599b700 1 --2- 172.21.15.156:0/2713048197 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fce440682f0 0x7fce4406a7a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fce5c005910 tx=0x7fce5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:11.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.314+0000 7fce567fc700 1 -- 172.21.15.156:0/2713048197 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fce50052190 con 0x7fce6015c9b0 2023-11-28T13:15:11.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.547+0000 7fce67bff700 1 -- 172.21.15.156:0/2713048197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fce60153320 con 0x7fce6015c9b0 2023-11-28T13:15:11.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.547+0000 7fce567fc700 1 -- 172.21.15.156:0/2713048197 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fce5000f040 con 0x7fce6015c9b0 2023-11-28T13:15:11.549 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:15:11.549 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:15:11.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.549+0000 7fce67bff700 1 -- 172.21.15.156:0/2713048197 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fce440682f0 msgr2=0x7fce4406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:11.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.549+0000 7fce67bff700 1 --2- 172.21.15.156:0/2713048197 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fce440682f0 0x7fce4406a7a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fce5c005910 tx=0x7fce5c009040 comp rx=0 tx=0).stop 2023-11-28T13:15:11.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.549+0000 7fce67bff700 1 -- 172.21.15.156:0/2713048197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6015c9b0 msgr2=0x7fce600c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:11.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.549+0000 7fce67bff700 1 --2- 172.21.15.156:0/2713048197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6015c9b0 0x7fce600c8e30 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fce5000e810 tx=0x7fce5000eb20 comp rx=0 tx=0).stop 2023-11-28T13:15:11.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.549+0000 7fce67bff700 1 -- 172.21.15.156:0/2713048197 shutdown_connections 2023-11-28T13:15:11.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.549+0000 7fce67bff700 1 --2- 172.21.15.156:0/2713048197 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fce440682f0 0x7fce4406a7a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:11.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.549+0000 7fce67bff700 1 --2- 172.21.15.156:0/2713048197 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fce60161340 0x7fce600c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:11.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.549+0000 7fce67bff700 1 --2- 172.21.15.156:0/2713048197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce600d2320 0x7fce60151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:11.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.549+0000 7fce67bff700 1 --2- 172.21.15.156:0/2713048197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce6015c9b0 0x7fce600c8e30 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:11.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.549+0000 7fce67bff700 1 -- 172.21.15.156:0/2713048197 >> 172.21.15.156:0/2713048197 conn(0x7fce60072600 msgr2=0x7fce6015fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:11.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.549+0000 7fce67bff700 1 -- 172.21.15.156:0/2713048197 shutdown_connections 2023-11-28T13:15:11.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:11.550+0000 7fce67bff700 1 -- 172.21.15.156:0/2713048197 wait complete. 2023-11-28T13:15:11.569 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:15:16.570 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:15:16.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.774+0000 7f7ed2dff700 1 Processor -- start 2023-11-28T13:15:16.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.774+0000 7f7ed2dff700 1 -- start start 2023-11-28T13:15:16.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.775+0000 7f7ed2dff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7ecc0cd7c0 0x7f7ecc0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:16.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.775+0000 7f7ed2dff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ecc0d1950 con 0x7f7ecc161340 2023-11-28T13:15:16.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.775+0000 7f7ed2dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ecc0ce140 con 0x7f7ecc15c9b0 2023-11-28T13:15:16.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.775+0000 7f7ed2dff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ecc0b5800 con 0x7f7ecc0cd7c0 2023-11-28T13:15:16.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.775+0000 7f7ecbfff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f7ecc161340 0x7f7ecc0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50738/0 (socket says 172.21.15.156:50738) 2023-11-28T13:15:16.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.776+0000 7f7ecbfff700 1 -- 172.21.15.156:0/2200585739 learned_addr learned my addr 172.21.15.156:0/2200585739 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:16.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.776+0000 7f7ed139c700 1 --2- 172.21.15.156:0/2200585739 >> v2:172.21.15.156:3300/0 conn(0x7f7ecc0cd7c0 0x7f7ecc0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:16.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.776+0000 7f7ecb7fe700 1 -- 172.21.15.156:0/2200585739 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060334811 0 0) 0x7f7ecc0d1950 con 0x7f7ecc161340 2023-11-28T13:15:16.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.776+0000 7f7ecb7fe700 1 -- 172.21.15.156:0/2200585739 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ebc0035b0 con 0x7f7ecc161340 2023-11-28T13:15:16.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.776+0000 7f7ecb7fe700 1 -- 172.21.15.156:0/2200585739 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4086060396 0 0) 0x7f7ecc0ce140 con 0x7f7ecc15c9b0 2023-11-28T13:15:16.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.776+0000 7f7ecb7fe700 1 -- 172.21.15.156:0/2200585739 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ecc0d1950 con 0x7f7ecc15c9b0 2023-11-28T13:15:16.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.776+0000 7f7ed139c700 1 -- 172.21.15.156:0/2200585739 >> v1:172.21.15.186:6789/0 conn(0x7f7ecc15c9b0 legacy=0x7f7ecc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:16.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.776+0000 7f7ed139c700 1 -- 172.21.15.156:0/2200585739 >> v1:172.21.15.156:6789/0 conn(0x7f7ecc161340 legacy=0x7f7ecc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:16.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.776+0000 7f7ed139c700 1 -- 172.21.15.156:0/2200585739 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc0d24f0 con 0x7f7ecc0cd7c0 2023-11-28T13:15:16.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.776+0000 7f7ed139c700 1 --2- 172.21.15.156:0/2200585739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ecc0cd7c0 0x7f7ecc0cdc00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009d90 tx=0x7f7ec400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187e5fd43f91cbb4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:16.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.777+0000 7f7ecb7fe700 1 -- 172.21.15.156:0/2200585739 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec400e070 con 0x7f7ecc0cd7c0 2023-11-28T13:15:16.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.777+0000 7f7ecb7fe700 1 -- 172.21.15.156:0/2200585739 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec400bb20 con 0x7f7ecc0cd7c0 2023-11-28T13:15:16.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.777+0000 7f7ecb7fe700 1 -- 172.21.15.156:0/2200585739 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4012430 con 0x7f7ecc0cd7c0 2023-11-28T13:15:16.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.777+0000 7f7ed2dff700 1 -- 172.21.15.156:0/2200585739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ecc0cd7c0 msgr2=0x7f7ecc0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:16.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.777+0000 7f7ed2dff700 1 --2- 172.21.15.156:0/2200585739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ecc0cd7c0 0x7f7ecc0cdc00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009d90 tx=0x7f7ec400b330 comp rx=0 tx=0).stop 2023-11-28T13:15:16.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.777+0000 7f7ed2dff700 1 -- 172.21.15.156:0/2200585739 shutdown_connections 2023-11-28T13:15:16.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.777+0000 7f7ed2dff700 1 --2- 172.21.15.156:0/2200585739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ecc0cd7c0 0x7f7ecc0cdc00 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:16.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.777+0000 7f7ed2dff700 1 -- 172.21.15.156:0/2200585739 >> 172.21.15.156:0/2200585739 conn(0x7f7ecc072600 msgr2=0x7f7ecc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:16.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.777+0000 7f7ed2dff700 1 -- 172.21.15.156:0/2200585739 shutdown_connections 2023-11-28T13:15:16.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.777+0000 7f7ed2dff700 1 -- 172.21.15.156:0/2200585739 wait complete. 2023-11-28T13:15:16.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.779+0000 7f7ed2dff700 1 Processor -- start 2023-11-28T13:15:16.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.779+0000 7f7ed2dff700 1 -- start start 2023-11-28T13:15:16.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.780+0000 7f7ed2dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc0cd7c0 0x7f7ecc14a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:16.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.780+0000 7f7ed2dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ecc15c9b0 0x7f7ecc15e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:16.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.780+0000 7f7ed2dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ecc161340 0x7f7ecc15eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:16.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.780+0000 7f7ed2dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0ca240 con 0x7f7ecc161340 2023-11-28T13:15:16.785 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.780+0000 7f7ed2dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0c7ab0 con 0x7f7ecc15c9b0 2023-11-28T13:15:16.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.780+0000 7f7ed2dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0ca470 con 0x7f7ecc0cd7c0 2023-11-28T13:15:16.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.780+0000 7f7ed139c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ecc161340 0x7f7ecc15eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:16.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.780+0000 7f7ed139c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ecc161340 0x7f7ecc15eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50664/0 (socket says 172.21.15.156:50664) 2023-11-28T13:15:16.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.780+0000 7f7ed139c700 1 -- 172.21.15.156:0/3771874192 learned_addr learned my addr 172.21.15.156:0/3771874192 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:16.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.781+0000 7f7ecbfff700 1 --2- 172.21.15.156:0/3771874192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ecc15c9b0 0x7f7ecc15e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:16.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.781+0000 7f7ed0b9b700 1 --2- 172.21.15.156:0/3771874192 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc0cd7c0 0x7f7ecc14a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:16.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.781+0000 7f7ed139c700 1 -- 172.21.15.156:0/3771874192 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc0cd7c0 msgr2=0x7f7ecc14a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:16.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.781+0000 7f7ed139c700 1 --2- 172.21.15.156:0/3771874192 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc0cd7c0 0x7f7ecc14a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:16.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.781+0000 7f7ed139c700 1 -- 172.21.15.156:0/3771874192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ecc15c9b0 msgr2=0x7f7ecc15e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:16.786 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.781+0000 7f7ed139c700 1 --2- 172.21.15.156:0/3771874192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ecc15c9b0 0x7f7ecc15e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:16.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.781+0000 7f7ed139c700 1 -- 172.21.15.156:0/3771874192 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec4009a70 con 0x7f7ecc161340 2023-11-28T13:15:16.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.781+0000 7f7ed139c700 1 --2- 172.21.15.156:0/3771874192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ecc161340 0x7f7ecc15eba0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0675d0 tx=0x7f7ec4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:16.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.781+0000 7f7ec97fa700 1 -- 172.21.15.156:0/3771874192 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec401b810 con 0x7f7ecc161340 2023-11-28T13:15:16.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.781+0000 7f7ec97fa700 1 -- 172.21.15.156:0/3771874192 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec4020070 con 0x7f7ecc161340 2023-11-28T13:15:16.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.781+0000 7f7ed2dff700 1 -- 172.21.15.156:0/3771874192 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ecc0d1300 con 0x7f7ecc161340 2023-11-28T13:15:16.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.781+0000 7f7ec97fa700 1 -- 172.21.15.156:0/3771874192 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec401bbb0 con 0x7f7ecc161340 2023-11-28T13:15:16.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.782+0000 7f7ed2dff700 1 -- 172.21.15.156:0/3771874192 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ecc0b5520 con 0x7f7ecc161340 2023-11-28T13:15:16.787 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.783+0000 7f7ed2dff700 1 -- 172.21.15.156:0/3771874192 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ecc0d15e0 con 0x7f7ecc161340 2023-11-28T13:15:16.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.783+0000 7f7ec97fa700 1 -- 172.21.15.156:0/3771874192 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7ec401bd10 con 0x7f7ecc161340 2023-11-28T13:15:16.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.785+0000 7f7ec97fa700 1 --2- 172.21.15.156:0/3771874192 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7eb00682f0 0x7f7eb006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:16.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.785+0000 7f7ec97fa700 1 -- 172.21.15.156:0/3771874192 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f7ec4022070 con 0x7f7ecc161340 2023-11-28T13:15:16.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.786+0000 7f7ed0b9b700 1 --2- 172.21.15.156:0/3771874192 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7eb00682f0 0x7f7eb006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:16.788 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.786+0000 7f7ed0b9b700 1 --2- 172.21.15.156:0/3771874192 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7eb00682f0 0x7f7eb006a7a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00011e0 tx=0x7f7ec0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:16.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:16.789+0000 7f7ec97fa700 1 -- 172.21.15.156:0/3771874192 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7ec4054d50 con 0x7f7ecc161340 2023-11-28T13:15:17.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.024+0000 7f7ed2dff700 1 -- 172.21.15.156:0/3771874192 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7ecc0745c0 con 0x7f7ecc161340 2023-11-28T13:15:17.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.025+0000 7f7ec97fa700 1 -- 172.21.15.156:0/3771874192 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f7ec4058370 con 0x7f7ecc161340 2023-11-28T13:15:17.026 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:15:17.026 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:15:17.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.027+0000 7f7ed2dff700 1 -- 172.21.15.156:0/3771874192 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7eb00682f0 msgr2=0x7f7eb006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:17.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.027+0000 7f7ed2dff700 1 --2- 172.21.15.156:0/3771874192 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7eb00682f0 0x7f7eb006a7a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00011e0 tx=0x7f7ec0009040 comp rx=0 tx=0).stop 2023-11-28T13:15:17.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.027+0000 7f7ed2dff700 1 -- 172.21.15.156:0/3771874192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ecc161340 msgr2=0x7f7ecc15eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:17.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.027+0000 7f7ed2dff700 1 --2- 172.21.15.156:0/3771874192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ecc161340 0x7f7ecc15eba0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0675d0 tx=0x7f7ec4004ad0 comp rx=0 tx=0).stop 2023-11-28T13:15:17.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.027+0000 7f7ed2dff700 1 -- 172.21.15.156:0/3771874192 shutdown_connections 2023-11-28T13:15:17.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.027+0000 7f7ed2dff700 1 --2- 172.21.15.156:0/3771874192 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f7eb00682f0 0x7f7eb006a7a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:17.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.027+0000 7f7ed2dff700 1 --2- 172.21.15.156:0/3771874192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ecc161340 0x7f7ecc15eba0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:17.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.027+0000 7f7ed2dff700 1 --2- 172.21.15.156:0/3771874192 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ecc0cd7c0 0x7f7ecc14a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:17.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.027+0000 7f7ed2dff700 1 --2- 172.21.15.156:0/3771874192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ecc15c9b0 0x7f7ecc15e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:17.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.027+0000 7f7ed2dff700 1 -- 172.21.15.156:0/3771874192 >> 172.21.15.156:0/3771874192 conn(0x7f7ecc072600 msgr2=0x7f7ecc15a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:17.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.027+0000 7f7ed2dff700 1 -- 172.21.15.156:0/3771874192 shutdown_connections 2023-11-28T13:15:17.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.027+0000 7f7ed2dff700 1 -- 172.21.15.156:0/3771874192 wait complete. 2023-11-28T13:15:17.047 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:15:17.047 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:15:17.133 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:15:17.236 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:15:17.237 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:15:17.237 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:15:17.281 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:15:17.282 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:15:17.367 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:15:17.606 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.605+0000 7f1ab01ff700 1 Processor -- start 2023-11-28T13:15:17.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.605+0000 7f1ab01ff700 1 -- start start 2023-11-28T13:15:17.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.606+0000 7f1ab01ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1aa80d0ff0 0x7f1aa80cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:17.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.606+0000 7f1ab01ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1aa80b37b0 0x7f1aa80b6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:17.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.606+0000 7f1ab01ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa80cf220 con 0x7f1aa80b4750 2023-11-28T13:15:17.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.606+0000 7f1ab01ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa80b3e50 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.606+0000 7f1ab01ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa80cf470 con 0x7f1aa80b37b0 2023-11-28T13:15:17.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.606+0000 7f1aadf9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1aa80d0ff0 0x7f1aa80cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:17.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.606+0000 7f1aadf9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1aa80d0ff0 0x7f1aa80cff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50676/0 (socket says 172.21.15.156:50676) 2023-11-28T13:15:17.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.606+0000 7f1aadf9b700 1 -- 172.21.15.156:0/3231465276 learned_addr learned my addr 172.21.15.156:0/3231465276 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:17.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.606+0000 7f1aad79a700 1 --1- 172.21.15.156:0/3231465276 >> v1:172.21.15.156:6789/0 conn(0x7f1aa80b4750 0x7f1aa80b30a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50744/0 (socket says 172.21.15.156:50744) 2023-11-28T13:15:17.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.606+0000 7f1aae79c700 1 --2- 172.21.15.156:0/3231465276 >> v2:172.21.15.186:3300/0 conn(0x7f1aa80b37b0 0x7f1aa80b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:17.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.607+0000 7f1aadf9b700 1 -- 172.21.15.156:0/3231465276 >> v2:172.21.15.186:3300/0 conn(0x7f1aa80b37b0 msgr2=0x7f1aa80b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:17.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.607+0000 7f1aadf9b700 1 --2- 172.21.15.156:0/3231465276 >> v2:172.21.15.186:3300/0 conn(0x7f1aa80b37b0 0x7f1aa80b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:17.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.607+0000 7f1aadf9b700 1 -- 172.21.15.156:0/3231465276 >> v1:172.21.15.156:6789/0 conn(0x7f1aa80b4750 legacy=0x7f1aa80b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:17.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.607+0000 7f1aadf9b700 1 -- 172.21.15.156:0/3231465276 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa80cf750 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.607+0000 7f1aadf9b700 1 --2- 172.21.15.156:0/3231465276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1aa80d0ff0 0x7f1aa80cff50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80d2cc0 tx=0x7f1aa4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5547a0e54e0b50ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:17.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.607+0000 7f1aacf99700 1 -- 172.21.15.156:0/3231465276 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4005e30 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.607+0000 7f1aacf99700 1 -- 172.21.15.156:0/3231465276 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa4004030 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.607+0000 7f1aacf99700 1 -- 172.21.15.156:0/3231465276 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa40043e0 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.608+0000 7f1ab01ff700 1 -- 172.21.15.156:0/3231465276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1aa80d0ff0 msgr2=0x7f1aa80cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:17.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.608+0000 7f1ab01ff700 1 --2- 172.21.15.156:0/3231465276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1aa80d0ff0 0x7f1aa80cff50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80d2cc0 tx=0x7f1aa4009bd0 comp rx=0 tx=0).stop 2023-11-28T13:15:17.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.608+0000 7f1ab01ff700 1 -- 172.21.15.156:0/3231465276 shutdown_connections 2023-11-28T13:15:17.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.608+0000 7f1ab01ff700 1 --2- 172.21.15.156:0/3231465276 >> v2:172.21.15.186:3300/0 conn(0x7f1aa80b37b0 0x7f1aa80b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:17.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.608+0000 7f1ab01ff700 1 --2- 172.21.15.156:0/3231465276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1aa80d0ff0 0x7f1aa80cff50 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:17.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.608+0000 7f1ab01ff700 1 -- 172.21.15.156:0/3231465276 >> 172.21.15.156:0/3231465276 conn(0x7f1aa809c3c0 msgr2=0x7f1aa80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:17.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.608+0000 7f1ab01ff700 1 -- 172.21.15.156:0/3231465276 shutdown_connections 2023-11-28T13:15:17.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.608+0000 7f1ab01ff700 1 -- 172.21.15.156:0/3231465276 wait complete. 2023-11-28T13:15:17.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.610+0000 7f1ab01ff700 1 Processor -- start 2023-11-28T13:15:17.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.610+0000 7f1ab01ff700 1 -- start start 2023-11-28T13:15:17.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.611+0000 7f1ab01ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1aa80b37b0 0x7f1aa81495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:17.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.611+0000 7f1ab01ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1aa80b4750 0x7f1aa81e1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:17.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.611+0000 7f1ab01ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1aa80d0ff0 0x7f1aa80c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:17.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.611+0000 7f1ab01ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa814a650 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.611+0000 7f1ab01ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa8149af0 con 0x7f1aa80b37b0 2023-11-28T13:15:17.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.611+0000 7f1ab01ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa81e2950 con 0x7f1aa80b4750 2023-11-28T13:15:17.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.611+0000 7f1aae79c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1aa80d0ff0 0x7f1aa80c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:17.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.611+0000 7f1aae79c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1aa80d0ff0 0x7f1aa80c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50678/0 (socket says 172.21.15.156:50678) 2023-11-28T13:15:17.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.611+0000 7f1aae79c700 1 -- 172.21.15.156:0/2769333121 learned_addr learned my addr 172.21.15.156:0/2769333121 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:17.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.611+0000 7f1aadf9b700 1 --2- 172.21.15.156:0/2769333121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1aa80b37b0 0x7f1aa81495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:17.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.612+0000 7f1aad79a700 1 --2- 172.21.15.156:0/2769333121 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1aa80b4750 0x7f1aa81e1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:17.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.612+0000 7f1aae79c700 1 -- 172.21.15.156:0/2769333121 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1aa80b4750 msgr2=0x7f1aa81e1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:17.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.612+0000 7f1aae79c700 1 --2- 172.21.15.156:0/2769333121 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1aa80b4750 0x7f1aa81e1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:17.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.612+0000 7f1aae79c700 1 -- 172.21.15.156:0/2769333121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1aa80b37b0 msgr2=0x7f1aa81495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:17.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.612+0000 7f1aae79c700 1 --2- 172.21.15.156:0/2769333121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1aa80b37b0 0x7f1aa81495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:17.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.612+0000 7f1aae79c700 1 -- 172.21.15.156:0/2769333121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa4009500 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.612+0000 7f1aad79a700 1 --2- 172.21.15.156:0/2769333121 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1aa80b4750 0x7f1aa81e1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:15:17.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.612+0000 7f1aae79c700 1 --2- 172.21.15.156:0/2769333121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1aa80d0ff0 0x7f1aa80c6cd0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009dd0 tx=0x7f1aa000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:17.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.612+0000 7f1a9e7fc700 1 -- 172.21.15.156:0/2769333121 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa000cd40 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.612+0000 7f1a9e7fc700 1 -- 172.21.15.156:0/2769333121 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa000f040 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.612+0000 7f1a9e7fc700 1 -- 172.21.15.156:0/2769333121 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa00146c0 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.612+0000 7f1ab01ff700 1 -- 172.21.15.156:0/2769333121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa80d0a50 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.612+0000 7f1ab01ff700 1 -- 172.21.15.156:0/2769333121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa80cfa30 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.613+0000 7f1ab01ff700 1 -- 172.21.15.156:0/2769333121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa80d0490 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.614+0000 7f1a9e7fc700 1 -- 172.21.15.156:0/2769333121 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1aa0014820 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.615+0000 7f1a9e7fc700 1 --2- 172.21.15.156:0/2769333121 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a8c0682f0 0x7f1a8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:17.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.615+0000 7f1a9e7fc700 1 -- 172.21.15.156:0/2769333121 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f1aa0086b70 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.615+0000 7f1aadf9b700 1 --2- 172.21.15.156:0/2769333121 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a8c0682f0 0x7f1a8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:17.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.616+0000 7f1aadf9b700 1 --2- 172.21.15.156:0/2769333121 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a8c0682f0 0x7f1a8c06a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009c10 tx=0x7f1aa4008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:17.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.619+0000 7f1a9e7fc700 1 -- 172.21.15.156:0/2769333121 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1aa0053250 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.811+0000 7f1ab01ff700 1 -- 172.21.15.156:0/2769333121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1aa80745c0 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.812+0000 7f1a9e7fc700 1 -- 172.21.15.156:0/2769333121 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+18152 (secure 0 0 0) 0x7f1aa0018090 con 0x7f1aa80d0ff0 2023-11-28T13:15:17.814 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:15:17.814 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":61,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:11:52.553240+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:10:52.732399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T13:10:53.912753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.11","mappings":[{"from":7,"to":2}]},{"pgid":"10.19","mappings":[{"from":1,"to":3}]},{"pgid":"10.1b","mappings":[{"from":4,"to":5}]},{"pgid":"10.1f","mappings":[{"from":7,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:15:17.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.814+0000 7f1ab01ff700 1 -- 172.21.15.156:0/2769333121 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a8c0682f0 msgr2=0x7f1a8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:17.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.814+0000 7f1ab01ff700 1 --2- 172.21.15.156:0/2769333121 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a8c0682f0 0x7f1a8c06a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009c10 tx=0x7f1aa4008070 comp rx=0 tx=0).stop 2023-11-28T13:15:17.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.814+0000 7f1ab01ff700 1 -- 172.21.15.156:0/2769333121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1aa80d0ff0 msgr2=0x7f1aa80c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:17.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.814+0000 7f1ab01ff700 1 --2- 172.21.15.156:0/2769333121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1aa80d0ff0 0x7f1aa80c6cd0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009dd0 tx=0x7f1aa000c3a0 comp rx=0 tx=0).stop 2023-11-28T13:15:17.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.814+0000 7f1ab01ff700 1 -- 172.21.15.156:0/2769333121 shutdown_connections 2023-11-28T13:15:17.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.814+0000 7f1ab01ff700 1 --2- 172.21.15.156:0/2769333121 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1a8c0682f0 0x7f1a8c06a7a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:17.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.814+0000 7f1ab01ff700 1 --2- 172.21.15.156:0/2769333121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1aa80d0ff0 0x7f1aa80c6cd0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:17.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.814+0000 7f1ab01ff700 1 --2- 172.21.15.156:0/2769333121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1aa80b37b0 0x7f1aa81495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:17.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.814+0000 7f1ab01ff700 1 --2- 172.21.15.156:0/2769333121 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1aa80b4750 0x7f1aa81e1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:17.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.814+0000 7f1ab01ff700 1 -- 172.21.15.156:0/2769333121 >> 172.21.15.156:0/2769333121 conn(0x7f1aa809c3c0 msgr2=0x7f1aa806ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:17.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.815+0000 7f1ab01ff700 1 -- 172.21.15.156:0/2769333121 shutdown_connections 2023-11-28T13:15:17.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:17.815+0000 7f1ab01ff700 1 -- 172.21.15.156:0/2769333121 wait complete. 2023-11-28T13:15:17.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:10:52.732399+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:10:53.912753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:15:17.835 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:15:18.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.033+0000 7f51123ff700 1 Processor -- start 2023-11-28T13:15:18.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.033+0000 7f51123ff700 1 -- start start 2023-11-28T13:15:18.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.034+0000 7f51123ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f510c15c9b0 0x7f510c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.034+0000 7f51123ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f510c161340 0x7f510c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.034+0000 7f51123ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f510c0d2320 0x7f510c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.034+0000 7f51123ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510c0b5800 con 0x7f510c161340 2023-11-28T13:15:18.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.034+0000 7f51123ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510c0d1950 con 0x7f510c0d2320 2023-11-28T13:15:18.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.034+0000 7f51123ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510c0b5520 con 0x7f510c15c9b0 2023-11-28T13:15:18.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.035+0000 7f510b7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f510c161340 0x7f510c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.035+0000 7f510b7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f510c161340 0x7f510c0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50686/0 (socket says 172.21.15.156:50686) 2023-11-28T13:15:18.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.035+0000 7f510b7fe700 1 -- 172.21.15.156:0/2599927390 learned_addr learned my addr 172.21.15.156:0/2599927390 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:18.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.035+0000 7f510bfff700 1 --2- 172.21.15.156:0/2599927390 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f510c15c9b0 0x7f510c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.035+0000 7f511099c700 1 --2- 172.21.15.156:0/2599927390 >> v2:172.21.15.186:3300/0 conn(0x7f510c0d2320 0x7f510c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.035+0000 7f510b7fe700 1 -- 172.21.15.156:0/2599927390 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f510c15c9b0 msgr2=0x7f510c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:18.038 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.035+0000 7f510b7fe700 1 --2- 172.21.15.156:0/2599927390 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f510c15c9b0 0x7f510c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.035+0000 7f510b7fe700 1 -- 172.21.15.156:0/2599927390 >> v2:172.21.15.186:3300/0 conn(0x7f510c0d2320 msgr2=0x7f510c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.035+0000 7f510b7fe700 1 --2- 172.21.15.156:0/2599927390 >> v2:172.21.15.186:3300/0 conn(0x7f510c0d2320 0x7f510c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.035+0000 7f510b7fe700 1 -- 172.21.15.156:0/2599927390 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c0d1300 con 0x7f510c161340 2023-11-28T13:15:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.035+0000 7f511099c700 1 --2- 172.21.15.156:0/2599927390 >> v2:172.21.15.186:3300/0 conn(0x7f510c0d2320 0x7f510c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:15:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.036+0000 7f510b7fe700 1 --2- 172.21.15.156:0/2599927390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f510c161340 0x7f510c0d1de0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f510c08c4d0 tx=0x7f5100009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc435ba0035ebe43 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.036+0000 7f510affd700 1 -- 172.21.15.156:0/2599927390 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510000b870 con 0x7f510c161340 2023-11-28T13:15:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.036+0000 7f510affd700 1 -- 172.21.15.156:0/2599927390 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510000b9d0 con 0x7f510c161340 2023-11-28T13:15:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.036+0000 7f510affd700 1 -- 172.21.15.156:0/2599927390 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510000bd60 con 0x7f510c161340 2023-11-28T13:15:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.036+0000 7f51123ff700 1 -- 172.21.15.156:0/2599927390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f510c161340 msgr2=0x7f510c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.036+0000 7f51123ff700 1 --2- 172.21.15.156:0/2599927390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f510c161340 0x7f510c0d1de0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f510c08c4d0 tx=0x7f5100009bd0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.036+0000 7f51123ff700 1 -- 172.21.15.156:0/2599927390 shutdown_connections 2023-11-28T13:15:18.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.036+0000 7f51123ff700 1 --2- 172.21.15.156:0/2599927390 >> v2:172.21.15.186:3300/0 conn(0x7f510c0d2320 0x7f510c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.036+0000 7f51123ff700 1 --2- 172.21.15.156:0/2599927390 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f510c15c9b0 0x7f510c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.036+0000 7f51123ff700 1 --2- 172.21.15.156:0/2599927390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f510c161340 0x7f510c0d1de0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.036+0000 7f51123ff700 1 -- 172.21.15.156:0/2599927390 >> 172.21.15.156:0/2599927390 conn(0x7f510c072600 msgr2=0x7f510c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:18.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.037+0000 7f51123ff700 1 -- 172.21.15.156:0/2599927390 shutdown_connections 2023-11-28T13:15:18.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.037+0000 7f51123ff700 1 -- 172.21.15.156:0/2599927390 wait complete. 2023-11-28T13:15:18.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.038+0000 7f51123ff700 1 Processor -- start 2023-11-28T13:15:18.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.038+0000 7f51123ff700 1 -- start start 2023-11-28T13:15:18.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.039+0000 7f51123ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f510c0d2320 0x7f510c14e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.039+0000 7f51123ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f510c15c9b0 0x7f510c1e73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.039+0000 7f51123ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f510c161340 0x7f510c1501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.039+0000 7f51123ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c1e88d0 con 0x7f510c15c9b0 2023-11-28T13:15:18.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.039+0000 7f51123ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c1e7cc0 con 0x7f510c161340 2023-11-28T13:15:18.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.039+0000 7f51123ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510c1e7fa0 con 0x7f510c0d2320 2023-11-28T13:15:18.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.040+0000 7f510b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f510c15c9b0 0x7f510c1e73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.040+0000 7f510b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f510c15c9b0 0x7f510c1e73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50700/0 (socket says 172.21.15.156:50700) 2023-11-28T13:15:18.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.040+0000 7f510b7fe700 1 -- 172.21.15.156:0/1489132089 learned_addr learned my addr 172.21.15.156:0/1489132089 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:18.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.040+0000 7f510bfff700 1 --2- 172.21.15.156:0/1489132089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f510c0d2320 0x7f510c14e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.040+0000 7f511099c700 1 --2- 172.21.15.156:0/1489132089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f510c161340 0x7f510c1501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.040+0000 7f510b7fe700 1 -- 172.21.15.156:0/1489132089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f510c0d2320 msgr2=0x7f510c14e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:18.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.040+0000 7f510b7fe700 1 --2- 172.21.15.156:0/1489132089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f510c0d2320 0x7f510c14e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.040+0000 7f510b7fe700 1 -- 172.21.15.156:0/1489132089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f510c161340 msgr2=0x7f510c1501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:18.043 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.040+0000 7f510b7fe700 1 --2- 172.21.15.156:0/1489132089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f510c161340 0x7f510c1501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.040+0000 7f510b7fe700 1 -- 172.21.15.156:0/1489132089 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5100009550 con 0x7f510c15c9b0 2023-11-28T13:15:18.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.040+0000 7f510b7fe700 1 --2- 172.21.15.156:0/1489132089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f510c15c9b0 0x7f510c1e73d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5100000c00 tx=0x7f5100004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:18.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.041+0000 7f5108ff9700 1 -- 172.21.15.156:0/1489132089 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100004480 con 0x7f510c15c9b0 2023-11-28T13:15:18.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.041+0000 7f5108ff9700 1 -- 172.21.15.156:0/1489132089 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51000045e0 con 0x7f510c15c9b0 2023-11-28T13:15:18.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.041+0000 7f51123ff700 1 -- 172.21.15.156:0/1489132089 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f510c0b5130 con 0x7f510c15c9b0 2023-11-28T13:15:18.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.041+0000 7f5108ff9700 1 -- 172.21.15.156:0/1489132089 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51000038e0 con 0x7f510c15c9b0 2023-11-28T13:15:18.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.041+0000 7f51123ff700 1 -- 172.21.15.156:0/1489132089 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f510c0d15e0 con 0x7f510c15c9b0 2023-11-28T13:15:18.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.042+0000 7f51123ff700 1 -- 172.21.15.156:0/1489132089 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f510c0b52c0 con 0x7f510c15c9b0 2023-11-28T13:15:18.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.043+0000 7f5108ff9700 1 -- 172.21.15.156:0/1489132089 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f510000f050 con 0x7f510c15c9b0 2023-11-28T13:15:18.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.043+0000 7f5108ff9700 1 --2- 172.21.15.156:0/1489132089 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f50f80682f0 0x7f50f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.044+0000 7f5108ff9700 1 -- 172.21.15.156:0/1489132089 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f5100087580 con 0x7f510c15c9b0 2023-11-28T13:15:18.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.044+0000 7f510bfff700 1 --2- 172.21.15.156:0/1489132089 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f50f80682f0 0x7f50f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.045+0000 7f510bfff700 1 --2- 172.21.15.156:0/1489132089 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f50f80682f0 0x7f50f806a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f50fc006b20 tx=0x7f50fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:18.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.048+0000 7f5108ff9700 1 -- 172.21.15.156:0/1489132089 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5100053c00 con 0x7f510c15c9b0 2023-11-28T13:15:18.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.240+0000 7f51123ff700 1 -- 172.21.15.156:0/1489132089 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f510c0745c0 con 0x7f510c15c9b0 2023-11-28T13:15:18.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.240+0000 7f5108ff9700 1 -- 172.21.15.156:0/1489132089 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v61) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5100057220 con 0x7f510c15c9b0 2023-11-28T13:15:18.242 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:15:18.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.243+0000 7f51123ff700 1 -- 172.21.15.156:0/1489132089 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f50f80682f0 msgr2=0x7f50f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:18.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.243+0000 7f51123ff700 1 --2- 172.21.15.156:0/1489132089 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f50f80682f0 0x7f50f806a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f50fc006b20 tx=0x7f50fc008040 comp rx=0 tx=0).stop 2023-11-28T13:15:18.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.243+0000 7f51123ff700 1 -- 172.21.15.156:0/1489132089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f510c15c9b0 msgr2=0x7f510c1e73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:18.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.243+0000 7f51123ff700 1 --2- 172.21.15.156:0/1489132089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f510c15c9b0 0x7f510c1e73d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5100000c00 tx=0x7f5100004750 comp rx=0 tx=0).stop 2023-11-28T13:15:18.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.243+0000 7f51123ff700 1 -- 172.21.15.156:0/1489132089 shutdown_connections 2023-11-28T13:15:18.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.243+0000 7f51123ff700 1 --2- 172.21.15.156:0/1489132089 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f50f80682f0 0x7f50f806a7a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.243+0000 7f51123ff700 1 --2- 172.21.15.156:0/1489132089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f510c161340 0x7f510c1501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.243+0000 7f51123ff700 1 --2- 172.21.15.156:0/1489132089 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f510c0d2320 0x7f510c14e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.243+0000 7f51123ff700 1 --2- 172.21.15.156:0/1489132089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f510c15c9b0 0x7f510c1e73d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.243+0000 7f51123ff700 1 -- 172.21.15.156:0/1489132089 >> 172.21.15.156:0/1489132089 conn(0x7f510c072600 msgr2=0x7f510c15cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:18.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.243+0000 7f51123ff700 1 -- 172.21.15.156:0/1489132089 shutdown_connections 2023-11-28T13:15:18.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.243+0000 7f51123ff700 1 -- 172.21.15.156:0/1489132089 wait complete. 2023-11-28T13:15:18.262 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:15:18.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.463+0000 7fdb9ffff700 1 Processor -- start 2023-11-28T13:15:18.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.463+0000 7fdb9ffff700 1 -- start start 2023-11-28T13:15:18.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.464+0000 7fdb9ffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb98070c50 0x7fdb980b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.464+0000 7fdb9ffff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb980b1010 con 0x7fdb98158200 2023-11-28T13:15:18.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.464+0000 7fdb9ffff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9815e1a0 con 0x7fdb980b0bb0 2023-11-28T13:15:18.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.464+0000 7fdb9ffff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9815f070 con 0x7fdb98070c50 2023-11-28T13:15:18.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.464+0000 7fdb9dd9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fdb98158200 0x7fdb980caa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50752/0 (socket says 172.21.15.156:50752) 2023-11-28T13:15:18.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.464+0000 7fdb9dd9b700 1 -- 172.21.15.156:0/2730007880 learned_addr learned my addr 172.21.15.156:0/2730007880 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:18.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.464+0000 7fdb9d59a700 1 --2- 172.21.15.156:0/2730007880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb98070c50 0x7fdb980b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.465+0000 7fdb9cd99700 1 -- 172.21.15.156:0/2730007880 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527321392 0 0) 0x7fdb980b1010 con 0x7fdb98158200 2023-11-28T13:15:18.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.465+0000 7fdb9cd99700 1 -- 172.21.15.156:0/2730007880 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb840035b0 con 0x7fdb98158200 2023-11-28T13:15:18.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.465+0000 7fdb9d59a700 1 -- 172.21.15.156:0/2730007880 >> v1:172.21.15.186:6789/0 conn(0x7fdb980b0bb0 legacy=0x7fdb9815add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:18.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.465+0000 7fdb9d59a700 1 -- 172.21.15.156:0/2730007880 >> v1:172.21.15.156:6789/0 conn(0x7fdb98158200 legacy=0x7fdb980caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:18.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.465+0000 7fdb9d59a700 1 -- 172.21.15.156:0/2730007880 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb980cb180 con 0x7fdb98070c50 2023-11-28T13:15:18.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.465+0000 7fdb9d59a700 1 --2- 172.21.15.156:0/2730007880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb98070c50 0x7fdb980b0670 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009870 tx=0x7fdb88009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37a9abe36a01d6a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:18.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.465+0000 7fdb9cd99700 1 -- 172.21.15.156:0/2730007880 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88011830 con 0x7fdb98070c50 2023-11-28T13:15:18.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.465+0000 7fdb9cd99700 1 -- 172.21.15.156:0/2730007880 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb88011990 con 0x7fdb98070c50 2023-11-28T13:15:18.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.465+0000 7fdb9cd99700 1 -- 172.21.15.156:0/2730007880 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88011d20 con 0x7fdb98070c50 2023-11-28T13:15:18.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.466+0000 7fdb9ffff700 1 -- 172.21.15.156:0/2730007880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb98070c50 msgr2=0x7fdb980b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:18.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.466+0000 7fdb9ffff700 1 --2- 172.21.15.156:0/2730007880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb98070c50 0x7fdb980b0670 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009870 tx=0x7fdb88009b80 comp rx=0 tx=0).stop 2023-11-28T13:15:18.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.466+0000 7fdb9ffff700 1 -- 172.21.15.156:0/2730007880 shutdown_connections 2023-11-28T13:15:18.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.466+0000 7fdb9ffff700 1 --2- 172.21.15.156:0/2730007880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb98070c50 0x7fdb980b0670 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.466+0000 7fdb9ffff700 1 -- 172.21.15.156:0/2730007880 >> 172.21.15.156:0/2730007880 conn(0x7fdb980cb670 msgr2=0x7fdb980cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:18.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.466+0000 7fdb9ffff700 1 -- 172.21.15.156:0/2730007880 shutdown_connections 2023-11-28T13:15:18.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.466+0000 7fdb9ffff700 1 -- 172.21.15.156:0/2730007880 wait complete. 2023-11-28T13:15:18.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.468+0000 7fdb9ffff700 1 Processor -- start 2023-11-28T13:15:18.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.468+0000 7fdb9ffff700 1 -- start start 2023-11-28T13:15:18.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.469+0000 7fdb9ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb98070c50 0x7fdb980c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.469+0000 7fdb9ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb980b0bb0 0x7fdb980b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.469+0000 7fdb9ffff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb98158200 0x7fdb980ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.469+0000 7fdb9ffff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb980b3900 con 0x7fdb98070c50 2023-11-28T13:15:18.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.469+0000 7fdb9ffff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb980b3d20 con 0x7fdb980b0bb0 2023-11-28T13:15:18.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.469+0000 7fdb9ffff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb980b2c80 con 0x7fdb98158200 2023-11-28T13:15:18.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.469+0000 7fdb9dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb98070c50 0x7fdb980c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.469+0000 7fdb9dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb98070c50 0x7fdb980c4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50710/0 (socket says 172.21.15.156:50710) 2023-11-28T13:15:18.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.469+0000 7fdb9dd9b700 1 -- 172.21.15.156:0/539381253 learned_addr learned my addr 172.21.15.156:0/539381253 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:18.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.469+0000 7fdb9d59a700 1 --2- 172.21.15.156:0/539381253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb980b0bb0 0x7fdb980b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.469+0000 7fdb9e59c700 1 --2- 172.21.15.156:0/539381253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb98158200 0x7fdb980ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.470+0000 7fdb9dd9b700 1 -- 172.21.15.156:0/539381253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb98158200 msgr2=0x7fdb980ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:18.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.470+0000 7fdb9dd9b700 1 --2- 172.21.15.156:0/539381253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb98158200 0x7fdb980ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.470+0000 7fdb9dd9b700 1 -- 172.21.15.156:0/539381253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb980b0bb0 msgr2=0x7fdb980b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:18.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.470+0000 7fdb9dd9b700 1 --2- 172.21.15.156:0/539381253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb980b0bb0 0x7fdb980b2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.470+0000 7fdb9dd9b700 1 -- 172.21.15.156:0/539381253 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb880094a0 con 0x7fdb98070c50 2023-11-28T13:15:18.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.470+0000 7fdb9e59c700 1 --2- 172.21.15.156:0/539381253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb98158200 0x7fdb980ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:15:18.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.470+0000 7fdb9dd9b700 1 --2- 172.21.15.156:0/539381253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb98070c50 0x7fdb980c4ed0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fdb94001eb0 tx=0x7fdb9400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:18.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.470+0000 7fdb8e7fc700 1 -- 172.21.15.156:0/539381253 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94005d70 con 0x7fdb98070c50 2023-11-28T13:15:18.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.470+0000 7fdb8e7fc700 1 -- 172.21.15.156:0/539381253 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb940070d0 con 0x7fdb98070c50 2023-11-28T13:15:18.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.470+0000 7fdb9ffff700 1 -- 172.21.15.156:0/539381253 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb981585d0 con 0x7fdb98070c50 2023-11-28T13:15:18.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.470+0000 7fdb8e7fc700 1 -- 172.21.15.156:0/539381253 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94007460 con 0x7fdb98070c50 2023-11-28T13:15:18.474 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.470+0000 7fdb9ffff700 1 -- 172.21.15.156:0/539381253 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb980710f0 con 0x7fdb98070c50 2023-11-28T13:15:18.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.471+0000 7fdb9ffff700 1 -- 172.21.15.156:0/539381253 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb9815e3f0 con 0x7fdb98070c50 2023-11-28T13:15:18.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.472+0000 7fdb8e7fc700 1 -- 172.21.15.156:0/539381253 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdb94019030 con 0x7fdb98070c50 2023-11-28T13:15:18.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.473+0000 7fdb8e7fc700 1 --2- 172.21.15.156:0/539381253 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdb7c0682f0 0x7fdb7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.473+0000 7fdb8e7fc700 1 -- 172.21.15.156:0/539381253 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fdb94085170 con 0x7fdb98070c50 2023-11-28T13:15:18.475 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.474+0000 7fdb9d59a700 1 --2- 172.21.15.156:0/539381253 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdb7c0682f0 0x7fdb7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.476 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.474+0000 7fdb9d59a700 1 --2- 172.21.15.156:0/539381253 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdb7c0682f0 0x7fdb7c06a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdb880090b0 tx=0x7fdb88008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:18.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.478+0000 7fdb8e7fc700 1 -- 172.21.15.156:0/539381253 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdb94051850 con 0x7fdb98070c50 2023-11-28T13:15:18.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.672+0000 7fdb9ffff700 1 -- 172.21.15.156:0/539381253 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdb9806fd60 con 0x7fdb98070c50 2023-11-28T13:15:18.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.672+0000 7fdb8e7fc700 1 -- 172.21.15.156:0/539381253 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb94054e70 con 0x7fdb98070c50 2023-11-28T13:15:18.674 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:15:18.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.675+0000 7fdb9ffff700 1 -- 172.21.15.156:0/539381253 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdb7c0682f0 msgr2=0x7fdb7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:18.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.675+0000 7fdb9ffff700 1 --2- 172.21.15.156:0/539381253 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdb7c0682f0 0x7fdb7c06a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdb880090b0 tx=0x7fdb88008030 comp rx=0 tx=0).stop 2023-11-28T13:15:18.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.675+0000 7fdb9ffff700 1 -- 172.21.15.156:0/539381253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb98070c50 msgr2=0x7fdb980c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:18.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.675+0000 7fdb9ffff700 1 --2- 172.21.15.156:0/539381253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb98070c50 0x7fdb980c4ed0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fdb94001eb0 tx=0x7fdb9400c7b0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.675+0000 7fdb9ffff700 1 -- 172.21.15.156:0/539381253 shutdown_connections 2023-11-28T13:15:18.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.675+0000 7fdb9ffff700 1 --2- 172.21.15.156:0/539381253 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fdb7c0682f0 0x7fdb7c06a7a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.675+0000 7fdb9ffff700 1 --2- 172.21.15.156:0/539381253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb98158200 0x7fdb980ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.675+0000 7fdb9ffff700 1 --2- 172.21.15.156:0/539381253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb98070c50 0x7fdb980c4ed0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.675+0000 7fdb9ffff700 1 --2- 172.21.15.156:0/539381253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb980b0bb0 0x7fdb980b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.675+0000 7fdb9ffff700 1 -- 172.21.15.156:0/539381253 >> 172.21.15.156:0/539381253 conn(0x7fdb980cb670 msgr2=0x7fdb980c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:18.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.675+0000 7fdb9ffff700 1 -- 172.21.15.156:0/539381253 shutdown_connections 2023-11-28T13:15:18.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.675+0000 7fdb9ffff700 1 -- 172.21.15.156:0/539381253 wait complete. 2023-11-28T13:15:18.695 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:15:18.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.895+0000 7f6a46dff700 1 Processor -- start 2023-11-28T13:15:18.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.895+0000 7f6a46dff700 1 -- start start 2023-11-28T13:15:18.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.896+0000 7f6a46dff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6a4015c9b0 0x7f6a400b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.896+0000 7f6a46dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a40161340 0x7f6a400d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.896+0000 7f6a46dff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6a400d2320 0x7f6a400cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.896+0000 7f6a46dff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a400b5800 con 0x7f6a400d2320 2023-11-28T13:15:18.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.896+0000 7f6a46dff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a400d1950 con 0x7f6a4015c9b0 2023-11-28T13:15:18.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.896+0000 7f6a46dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a400b5520 con 0x7f6a40161340 2023-11-28T13:15:18.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.897+0000 7f6a4539c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6a400d2320 0x7f6a400cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.897+0000 7f6a4539c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6a400d2320 0x7f6a400cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50714/0 (socket says 172.21.15.156:50714) 2023-11-28T13:15:18.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.897+0000 7f6a4539c700 1 -- 172.21.15.156:0/2697341684 learned_addr learned my addr 172.21.15.156:0/2697341684 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:18.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.897+0000 7f6a3ffff700 1 --2- 172.21.15.156:0/2697341684 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a40161340 0x7f6a400d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.897+0000 7f6a44b9b700 1 --2- 172.21.15.156:0/2697341684 >> v2:172.21.15.186:3300/0 conn(0x7f6a4015c9b0 0x7f6a400b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.897+0000 7f6a4539c700 1 -- 172.21.15.156:0/2697341684 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a40161340 msgr2=0x7f6a400d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:18.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.897+0000 7f6a4539c700 1 --2- 172.21.15.156:0/2697341684 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a40161340 0x7f6a400d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.897+0000 7f6a4539c700 1 -- 172.21.15.156:0/2697341684 >> v2:172.21.15.186:3300/0 conn(0x7f6a4015c9b0 msgr2=0x7f6a400b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:18.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.897+0000 7f6a4539c700 1 --2- 172.21.15.156:0/2697341684 >> v2:172.21.15.186:3300/0 conn(0x7f6a4015c9b0 0x7f6a400b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.897+0000 7f6a4539c700 1 -- 172.21.15.156:0/2697341684 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a400d1300 con 0x7f6a400d2320 2023-11-28T13:15:18.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.897+0000 7f6a44b9b700 1 --2- 172.21.15.156:0/2697341684 >> v2:172.21.15.186:3300/0 conn(0x7f6a4015c9b0 0x7f6a400b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:15:18.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.897+0000 7f6a4539c700 1 --2- 172.21.15.156:0/2697341684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a400d2320 0x7f6a400cd7c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6a40067460 tx=0x7f6a3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95736ccb440bfa2b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:18.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.897+0000 7f6a3f7fe700 1 -- 172.21.15.156:0/2697341684 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800e070 con 0x7f6a400d2320 2023-11-28T13:15:18.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.898+0000 7f6a3f7fe700 1 -- 172.21.15.156:0/2697341684 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3800b910 con 0x7f6a400d2320 2023-11-28T13:15:18.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.898+0000 7f6a3f7fe700 1 -- 172.21.15.156:0/2697341684 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800bcc0 con 0x7f6a400d2320 2023-11-28T13:15:18.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.898+0000 7f6a46dff700 1 -- 172.21.15.156:0/2697341684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a400d2320 msgr2=0x7f6a400cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:18.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.898+0000 7f6a46dff700 1 --2- 172.21.15.156:0/2697341684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a400d2320 0x7f6a400cd7c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6a40067460 tx=0x7f6a3800b330 comp rx=0 tx=0).stop 2023-11-28T13:15:18.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.898+0000 7f6a46dff700 1 -- 172.21.15.156:0/2697341684 shutdown_connections 2023-11-28T13:15:18.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.898+0000 7f6a46dff700 1 --2- 172.21.15.156:0/2697341684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a400d2320 0x7f6a400cd7c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.898+0000 7f6a46dff700 1 --2- 172.21.15.156:0/2697341684 >> v2:172.21.15.186:3300/0 conn(0x7f6a4015c9b0 0x7f6a400b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.898+0000 7f6a46dff700 1 --2- 172.21.15.156:0/2697341684 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a40161340 0x7f6a400d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.898+0000 7f6a46dff700 1 -- 172.21.15.156:0/2697341684 >> 172.21.15.156:0/2697341684 conn(0x7f6a40072600 msgr2=0x7f6a400cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:18.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.898+0000 7f6a46dff700 1 -- 172.21.15.156:0/2697341684 shutdown_connections 2023-11-28T13:15:18.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.898+0000 7f6a46dff700 1 -- 172.21.15.156:0/2697341684 wait complete. 2023-11-28T13:15:18.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.900+0000 7f6a46dff700 1 Processor -- start 2023-11-28T13:15:18.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.900+0000 7f6a46dff700 1 -- start start 2023-11-28T13:15:18.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.901+0000 7f6a46dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a400d2320 0x7f6a400c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.901+0000 7f6a46dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a4015c9b0 0x7f6a40152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.901+0000 7f6a46dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a40161340 0x7f6a40159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.901+0000 7f6a46dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a40154340 con 0x7f6a4015c9b0 2023-11-28T13:15:18.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.901+0000 7f6a46dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a400c6410 con 0x7f6a400d2320 2023-11-28T13:15:18.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.901+0000 7f6a46dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a401525d0 con 0x7f6a40161340 2023-11-28T13:15:18.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.901+0000 7f6a3ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a4015c9b0 0x7f6a40152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.901+0000 7f6a3ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a4015c9b0 0x7f6a40152090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50724/0 (socket says 172.21.15.156:50724) 2023-11-28T13:15:18.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.901+0000 7f6a3ffff700 1 -- 172.21.15.156:0/2023139053 learned_addr learned my addr 172.21.15.156:0/2023139053 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:18.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.902+0000 7f6a44b9b700 1 --2- 172.21.15.156:0/2023139053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a400d2320 0x7f6a400c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.902+0000 7f6a4539c700 1 --2- 172.21.15.156:0/2023139053 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a40161340 0x7f6a40159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.902+0000 7f6a3ffff700 1 -- 172.21.15.156:0/2023139053 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a40161340 msgr2=0x7f6a40159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:18.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.902+0000 7f6a3ffff700 1 --2- 172.21.15.156:0/2023139053 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a40161340 0x7f6a40159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.902+0000 7f6a3ffff700 1 -- 172.21.15.156:0/2023139053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a400d2320 msgr2=0x7f6a400c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:18.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.902+0000 7f6a3ffff700 1 --2- 172.21.15.156:0/2023139053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a400d2320 0x7f6a400c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:18.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.902+0000 7f6a3ffff700 1 -- 172.21.15.156:0/2023139053 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a38009b30 con 0x7f6a4015c9b0 2023-11-28T13:15:18.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.902+0000 7f6a3ffff700 1 --2- 172.21.15.156:0/2023139053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a4015c9b0 0x7f6a40152090 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00e9e0 tx=0x7f6a2c00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:18.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.902+0000 7f6a3d7fa700 1 -- 172.21.15.156:0/2023139053 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c009710 con 0x7f6a4015c9b0 2023-11-28T13:15:18.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.902+0000 7f6a3d7fa700 1 -- 172.21.15.156:0/2023139053 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c00ee80 con 0x7f6a4015c9b0 2023-11-28T13:15:18.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.902+0000 7f6a3d7fa700 1 -- 172.21.15.156:0/2023139053 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00ca50 con 0x7f6a4015c9b0 2023-11-28T13:15:18.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.902+0000 7f6a46dff700 1 -- 172.21.15.156:0/2023139053 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a400b5130 con 0x7f6a4015c9b0 2023-11-28T13:15:18.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.902+0000 7f6a46dff700 1 -- 172.21.15.156:0/2023139053 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a400d15e0 con 0x7f6a4015c9b0 2023-11-28T13:15:18.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.903+0000 7f6a46dff700 1 -- 172.21.15.156:0/2023139053 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a400b52c0 con 0x7f6a4015c9b0 2023-11-28T13:15:18.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.904+0000 7f6a3d7fa700 1 -- 172.21.15.156:0/2023139053 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6a2c019030 con 0x7f6a4015c9b0 2023-11-28T13:15:18.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.905+0000 7f6a3d7fa700 1 --2- 172.21.15.156:0/2023139053 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6a240682f0 0x7f6a2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:18.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.906+0000 7f6a3d7fa700 1 -- 172.21.15.156:0/2023139053 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f6a2c085c00 con 0x7f6a4015c9b0 2023-11-28T13:15:18.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.906+0000 7f6a44b9b700 1 --2- 172.21.15.156:0/2023139053 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6a240682f0 0x7f6a2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:18.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.907+0000 7f6a44b9b700 1 --2- 172.21.15.156:0/2023139053 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6a240682f0 0x7f6a2406a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6a34006f10 tx=0x7f6a34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:18.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:18.909+0000 7f6a3d7fa700 1 -- 172.21.15.156:0/2023139053 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6a2c0522e0 con 0x7f6a4015c9b0 2023-11-28T13:15:19.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.108+0000 7f6a46dff700 1 -- 172.21.15.156:0/2023139053 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6a40153790 con 0x7f6a4015c9b0 2023-11-28T13:15:19.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.109+0000 7f6a3d7fa700 1 -- 172.21.15.156:0/2023139053 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v61) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6a2c055900 con 0x7f6a4015c9b0 2023-11-28T13:15:19.110 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T13:15:19.113 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.111+0000 7f6a46dff700 1 -- 172.21.15.156:0/2023139053 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6a240682f0 msgr2=0x7f6a2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:19.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.111+0000 7f6a46dff700 1 --2- 172.21.15.156:0/2023139053 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6a240682f0 0x7f6a2406a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6a34006f10 tx=0x7f6a34008040 comp rx=0 tx=0).stop 2023-11-28T13:15:19.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.111+0000 7f6a46dff700 1 -- 172.21.15.156:0/2023139053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a4015c9b0 msgr2=0x7f6a40152090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:19.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.111+0000 7f6a46dff700 1 --2- 172.21.15.156:0/2023139053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a4015c9b0 0x7f6a40152090 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00e9e0 tx=0x7f6a2c00ecf0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.111+0000 7f6a46dff700 1 -- 172.21.15.156:0/2023139053 shutdown_connections 2023-11-28T13:15:19.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.111+0000 7f6a46dff700 1 --2- 172.21.15.156:0/2023139053 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6a240682f0 0x7f6a2406a7a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.111+0000 7f6a46dff700 1 --2- 172.21.15.156:0/2023139053 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6a40161340 0x7f6a40159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.111+0000 7f6a46dff700 1 --2- 172.21.15.156:0/2023139053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a400d2320 0x7f6a400c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.111+0000 7f6a46dff700 1 --2- 172.21.15.156:0/2023139053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a4015c9b0 0x7f6a40152090 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.115 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.111+0000 7f6a46dff700 1 -- 172.21.15.156:0/2023139053 >> 172.21.15.156:0/2023139053 conn(0x7f6a40072600 msgr2=0x7f6a4015ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:19.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.111+0000 7f6a46dff700 1 -- 172.21.15.156:0/2023139053 shutdown_connections 2023-11-28T13:15:19.116 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.111+0000 7f6a46dff700 1 -- 172.21.15.156:0/2023139053 wait complete. 2023-11-28T13:15:19.131 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:15:19.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.327+0000 7f46247ff700 1 Processor -- start 2023-11-28T13:15:19.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.328+0000 7f46247ff700 1 -- start start 2023-11-28T13:15:19.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.329+0000 7f46247ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f461c15ca50 0x7f461c06b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:19.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.329+0000 7f46247ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f461c06b8c0 0x7f461c0cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:19.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.329+0000 7f46247ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461c0cf5e0 con 0x7f461c0ccc40 2023-11-28T13:15:19.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.329+0000 7f46247ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461c0cd470 con 0x7f461c06b8c0 2023-11-28T13:15:19.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.329+0000 7f46247ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c06bd30 con 0x7f461c15ca50 2023-11-28T13:15:19.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.329+0000 7f4621d9a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f461c06b8c0 0x7f461c0cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:19.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.329+0000 7f4621d9a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f461c06b8c0 0x7f461c0cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50728/0 (socket says 172.21.15.156:50728) 2023-11-28T13:15:19.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.329+0000 7f4621d9a700 1 -- 172.21.15.156:0/133333188 learned_addr learned my addr 172.21.15.156:0/133333188 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:19.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.329+0000 7f4622d9c700 1 --1- 172.21.15.156:0/133333188 >> v1:172.21.15.156:6789/0 conn(0x7f461c0ccc40 0x7f461c0cd7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50768/0 (socket says 172.21.15.156:50768) 2023-11-28T13:15:19.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.329+0000 7f462259b700 1 --2- 172.21.15.156:0/133333188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f461c15ca50 0x7f461c06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:19.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.329+0000 7f4621d9a700 1 -- 172.21.15.156:0/133333188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f461c15ca50 msgr2=0x7f461c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:19.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.329+0000 7f4621d9a700 1 --2- 172.21.15.156:0/133333188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f461c15ca50 0x7f461c06b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.329+0000 7f4621d9a700 1 -- 172.21.15.156:0/133333188 >> v1:172.21.15.156:6789/0 conn(0x7f461c0ccc40 legacy=0x7f461c0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:19.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.329+0000 7f4621d9a700 1 -- 172.21.15.156:0/133333188 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c0cecb0 con 0x7f461c06b8c0 2023-11-28T13:15:19.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.329+0000 7f462259b700 1 --2- 172.21.15.156:0/133333188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f461c15ca50 0x7f461c06b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:15:19.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.330+0000 7f4621d9a700 1 --2- 172.21.15.156:0/133333188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f461c06b8c0 0x7f461c0cc700 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f460c009920 tx=0x7f460c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ee287ecb96197 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:19.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.330+0000 7f4621599700 1 -- 172.21.15.156:0/133333188 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c00ba20 con 0x7f461c06b8c0 2023-11-28T13:15:19.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.330+0000 7f4621599700 1 -- 172.21.15.156:0/133333188 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460c00bb80 con 0x7f461c06b8c0 2023-11-28T13:15:19.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.330+0000 7f4621599700 1 -- 172.21.15.156:0/133333188 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c009e70 con 0x7f461c06b8c0 2023-11-28T13:15:19.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.331+0000 7f46247ff700 1 -- 172.21.15.156:0/133333188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f461c06b8c0 msgr2=0x7f461c0cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:19.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.331+0000 7f46247ff700 1 --2- 172.21.15.156:0/133333188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f461c06b8c0 0x7f461c0cc700 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f460c009920 tx=0x7f460c009c30 comp rx=0 tx=0).stop 2023-11-28T13:15:19.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.331+0000 7f46247ff700 1 -- 172.21.15.156:0/133333188 shutdown_connections 2023-11-28T13:15:19.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.331+0000 7f46247ff700 1 --2- 172.21.15.156:0/133333188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f461c15ca50 0x7f461c06b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.331+0000 7f46247ff700 1 --2- 172.21.15.156:0/133333188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f461c06b8c0 0x7f461c0cc700 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.331+0000 7f46247ff700 1 -- 172.21.15.156:0/133333188 >> 172.21.15.156:0/133333188 conn(0x7f461c09a230 msgr2=0x7f461c08afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:19.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.331+0000 7f46247ff700 1 -- 172.21.15.156:0/133333188 shutdown_connections 2023-11-28T13:15:19.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.331+0000 7f46247ff700 1 -- 172.21.15.156:0/133333188 wait complete. 2023-11-28T13:15:19.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.332+0000 7f46247ff700 1 Processor -- start 2023-11-28T13:15:19.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.333+0000 7f46247ff700 1 -- start start 2023-11-28T13:15:19.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.333+0000 7f46247ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f461c06b8c0 0x7f461c0c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:19.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.333+0000 7f46247ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f461c0ccc40 0x7f461c0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:19.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.333+0000 7f46247ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f461c15ca50 0x7f461c0b6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:19.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.333+0000 7f46247ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0b5a70 con 0x7f461c15ca50 2023-11-28T13:15:19.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.333+0000 7f46247ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0b4100 con 0x7f461c0ccc40 2023-11-28T13:15:19.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.333+0000 7f46247ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0c59e0 con 0x7f461c06b8c0 2023-11-28T13:15:19.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.334+0000 7f4622d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f461c15ca50 0x7f461c0b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:19.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.334+0000 7f4622d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f461c15ca50 0x7f461c0b6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50730/0 (socket says 172.21.15.156:50730) 2023-11-28T13:15:19.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.334+0000 7f4622d9c700 1 -- 172.21.15.156:0/1455098509 learned_addr learned my addr 172.21.15.156:0/1455098509 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:19.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.334+0000 7f4621d9a700 1 --2- 172.21.15.156:0/1455098509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f461c0ccc40 0x7f461c0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:19.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.334+0000 7f462259b700 1 --2- 172.21.15.156:0/1455098509 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f461c06b8c0 0x7f461c0c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:19.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.334+0000 7f4622d9c700 1 -- 172.21.15.156:0/1455098509 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f461c06b8c0 msgr2=0x7f461c0c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:19.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.334+0000 7f4622d9c700 1 --2- 172.21.15.156:0/1455098509 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f461c06b8c0 0x7f461c0c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.334+0000 7f4622d9c700 1 -- 172.21.15.156:0/1455098509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f461c0ccc40 msgr2=0x7f461c0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:19.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.334+0000 7f4622d9c700 1 --2- 172.21.15.156:0/1455098509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f461c0ccc40 0x7f461c0b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.334+0000 7f4622d9c700 1 -- 172.21.15.156:0/1455098509 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c009550 con 0x7f461c15ca50 2023-11-28T13:15:19.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.334+0000 7f4622d9c700 1 --2- 172.21.15.156:0/1455098509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f461c15ca50 0x7f461c0b6720 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f461400cbd0 tx=0x7f461400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:19.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.335+0000 7f4612ffd700 1 -- 172.21.15.156:0/1455098509 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461400aa40 con 0x7f461c15ca50 2023-11-28T13:15:19.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.335+0000 7f4612ffd700 1 -- 172.21.15.156:0/1455098509 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461400aba0 con 0x7f461c15ca50 2023-11-28T13:15:19.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.335+0000 7f4612ffd700 1 -- 172.21.15.156:0/1455098509 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614007070 con 0x7f461c15ca50 2023-11-28T13:15:19.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.335+0000 7f46247ff700 1 -- 172.21.15.156:0/1455098509 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461c0cf270 con 0x7f461c15ca50 2023-11-28T13:15:19.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.335+0000 7f46247ff700 1 -- 172.21.15.156:0/1455098509 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461c0cef90 con 0x7f461c15ca50 2023-11-28T13:15:19.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.336+0000 7f46247ff700 1 -- 172.21.15.156:0/1455098509 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461c0cd0c0 con 0x7f461c15ca50 2023-11-28T13:15:19.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.337+0000 7f4612ffd700 1 -- 172.21.15.156:0/1455098509 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4614019030 con 0x7f461c15ca50 2023-11-28T13:15:19.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.337+0000 7f4612ffd700 1 --2- 172.21.15.156:0/1455098509 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f46000682f0 0x7f460006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:19.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.338+0000 7f4612ffd700 1 -- 172.21.15.156:0/1455098509 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f4614085890 con 0x7f461c15ca50 2023-11-28T13:15:19.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.338+0000 7f462259b700 1 --2- 172.21.15.156:0/1455098509 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f46000682f0 0x7f460006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:19.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.339+0000 7f462259b700 1 --2- 172.21.15.156:0/1455098509 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f46000682f0 0x7f460006a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4618006fd0 tx=0x7f4618008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:19.343 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.342+0000 7f4612ffd700 1 -- 172.21.15.156:0/1455098509 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f461404b5e0 con 0x7f461c15ca50 2023-11-28T13:15:19.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.531+0000 7f46247ff700 1 -- 172.21.15.156:0/1455098509 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f461c0745c0 con 0x7f461c15ca50 2023-11-28T13:15:19.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.531+0000 7f4612ffd700 1 -- 172.21.15.156:0/1455098509 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) v1 ==== 100+0+11 (secure 0 0 0) 0x7f46140170a0 con 0x7f461c15ca50 2023-11-28T13:15:19.533 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T13:15:19.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.534+0000 7f46247ff700 1 -- 172.21.15.156:0/1455098509 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f46000682f0 msgr2=0x7f460006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:19.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.534+0000 7f46247ff700 1 --2- 172.21.15.156:0/1455098509 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f46000682f0 0x7f460006a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4618006fd0 tx=0x7f4618008040 comp rx=0 tx=0).stop 2023-11-28T13:15:19.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.534+0000 7f46247ff700 1 -- 172.21.15.156:0/1455098509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f461c15ca50 msgr2=0x7f461c0b6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:19.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.534+0000 7f46247ff700 1 --2- 172.21.15.156:0/1455098509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f461c15ca50 0x7f461c0b6720 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f461400cbd0 tx=0x7f461400cee0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.534+0000 7f46247ff700 1 -- 172.21.15.156:0/1455098509 shutdown_connections 2023-11-28T13:15:19.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.534+0000 7f46247ff700 1 --2- 172.21.15.156:0/1455098509 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f46000682f0 0x7f460006a7a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.534+0000 7f46247ff700 1 --2- 172.21.15.156:0/1455098509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f461c15ca50 0x7f461c0b6720 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.534+0000 7f46247ff700 1 --2- 172.21.15.156:0/1455098509 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f461c06b8c0 0x7f461c0c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.534+0000 7f46247ff700 1 --2- 172.21.15.156:0/1455098509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f461c0ccc40 0x7f461c0b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.534+0000 7f46247ff700 1 -- 172.21.15.156:0/1455098509 >> 172.21.15.156:0/1455098509 conn(0x7f461c09a230 msgr2=0x7f461c0d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:19.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.534+0000 7f46247ff700 1 -- 172.21.15.156:0/1455098509 shutdown_connections 2023-11-28T13:15:19.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.534+0000 7f46247ff700 1 -- 172.21.15.156:0/1455098509 wait complete. 2023-11-28T13:15:19.554 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:15:19.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.752+0000 7fd4bfdff700 1 Processor -- start 2023-11-28T13:15:19.754 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.753+0000 7fd4bfdff700 1 -- start start 2023-11-28T13:15:19.755 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.754+0000 7fd4bfdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b8161340 0x7fd4b80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:19.755 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.754+0000 7fd4bfdff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd4b80d2320 0x7fd4b80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:19.755 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.754+0000 7fd4bfdff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b80d1950 con 0x7fd4b815c9b0 2023-11-28T13:15:19.755 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.754+0000 7fd4bfdff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b80b5800 con 0x7fd4b80d2320 2023-11-28T13:15:19.756 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.754+0000 7fd4bfdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b80b5300 con 0x7fd4b8161340 2023-11-28T13:15:19.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.754+0000 7fd4bd39a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b8161340 0x7fd4b80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:19.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.754+0000 7fd4be39c700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd4b80d2320 0x7fd4b80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:19.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.754+0000 7fd4bd39a700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b8161340 0x7fd4b80d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:37314/0 (socket says 172.21.15.156:37314) 2023-11-28T13:15:19.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.754+0000 7fd4bd39a700 1 -- 172.21.15.156:0/1392544655 learned_addr learned my addr 172.21.15.156:0/1392544655 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:19.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.754+0000 7fd4bdb9b700 1 --1- 172.21.15.156:0/1392544655 >> v1:172.21.15.186:6789/0 conn(0x7fd4b815c9b0 0x7fd4b80b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:59962/0 (socket says 172.21.15.156:59962) 2023-11-28T13:15:19.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.755+0000 7fd4be39c700 1 -- 172.21.15.156:0/1392544655 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b8161340 msgr2=0x7fd4b80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:19.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.755+0000 7fd4be39c700 1 --2- 172.21.15.156:0/1392544655 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b8161340 0x7fd4b80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.757 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.755+0000 7fd4be39c700 1 -- 172.21.15.156:0/1392544655 >> v1:172.21.15.186:6789/0 conn(0x7fd4b815c9b0 legacy=0x7fd4b80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:19.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.755+0000 7fd4be39c700 1 -- 172.21.15.156:0/1392544655 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b80b5520 con 0x7fd4b80d2320 2023-11-28T13:15:19.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.755+0000 7fd4bd39a700 1 --2- 172.21.15.156:0/1392544655 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b8161340 0x7fd4b80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:15:19.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.755+0000 7fd4be39c700 1 --2- 172.21.15.156:0/1392544655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4b80d2320 0x7fd4b80cd7c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009dc0 tx=0x7fd4b000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5afaf3eeb75f3402 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:19.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.755+0000 7fd4bcb99700 1 -- 172.21.15.156:0/1392544655 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b000e070 con 0x7fd4b80d2320 2023-11-28T13:15:19.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.755+0000 7fd4bcb99700 1 -- 172.21.15.156:0/1392544655 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b000bae0 con 0x7fd4b80d2320 2023-11-28T13:15:19.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.755+0000 7fd4bcb99700 1 -- 172.21.15.156:0/1392544655 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b000be90 con 0x7fd4b80d2320 2023-11-28T13:15:19.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.756+0000 7fd4bfdff700 1 -- 172.21.15.156:0/1392544655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4b80d2320 msgr2=0x7fd4b80cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:19.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.756+0000 7fd4bfdff700 1 --2- 172.21.15.156:0/1392544655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4b80d2320 0x7fd4b80cd7c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009dc0 tx=0x7fd4b000b330 comp rx=0 tx=0).stop 2023-11-28T13:15:19.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.756+0000 7fd4bfdff700 1 -- 172.21.15.156:0/1392544655 shutdown_connections 2023-11-28T13:15:19.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.756+0000 7fd4bfdff700 1 --2- 172.21.15.156:0/1392544655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4b80d2320 0x7fd4b80cd7c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.756+0000 7fd4bfdff700 1 --2- 172.21.15.156:0/1392544655 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b8161340 0x7fd4b80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.756+0000 7fd4bfdff700 1 -- 172.21.15.156:0/1392544655 >> 172.21.15.156:0/1392544655 conn(0x7fd4b8072600 msgr2=0x7fd4b80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:19.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.756+0000 7fd4bfdff700 1 -- 172.21.15.156:0/1392544655 shutdown_connections 2023-11-28T13:15:19.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.756+0000 7fd4bfdff700 1 -- 172.21.15.156:0/1392544655 wait complete. 2023-11-28T13:15:19.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.758+0000 7fd4bfdff700 1 Processor -- start 2023-11-28T13:15:19.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.758+0000 7fd4bfdff700 1 -- start start 2023-11-28T13:15:19.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.759+0000 7fd4bfdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4b80d2320 0x7fd4b81f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:19.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.759+0000 7fd4bfdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4b815c9b0 0x7fd4b8157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:19.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.759+0000 7fd4bfdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b8161340 0x7fd4b8160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:19.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.759+0000 7fd4bfdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b8152880 con 0x7fd4b815c9b0 2023-11-28T13:15:19.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.759+0000 7fd4bfdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b8152b60 con 0x7fd4b80d2320 2023-11-28T13:15:19.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.759+0000 7fd4bfdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b8152e40 con 0x7fd4b8161340 2023-11-28T13:15:19.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.759+0000 7fd4bd39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4b815c9b0 0x7fd4b8157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:19.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.759+0000 7fd4bd39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4b815c9b0 0x7fd4b8157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50742/0 (socket says 172.21.15.156:50742) 2023-11-28T13:15:19.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.759+0000 7fd4bd39a700 1 -- 172.21.15.156:0/4105556001 learned_addr learned my addr 172.21.15.156:0/4105556001 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:19.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.759+0000 7fd4bdb9b700 1 --2- 172.21.15.156:0/4105556001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4b80d2320 0x7fd4b81f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:19.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.760+0000 7fd4be39c700 1 --2- 172.21.15.156:0/4105556001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b8161340 0x7fd4b8160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:19.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.760+0000 7fd4bd39a700 1 -- 172.21.15.156:0/4105556001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b8161340 msgr2=0x7fd4b8160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:19.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.760+0000 7fd4bd39a700 1 --2- 172.21.15.156:0/4105556001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b8161340 0x7fd4b8160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.760+0000 7fd4bd39a700 1 -- 172.21.15.156:0/4105556001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4b80d2320 msgr2=0x7fd4b81f2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:19.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.760+0000 7fd4bd39a700 1 --2- 172.21.15.156:0/4105556001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4b80d2320 0x7fd4b81f2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.760+0000 7fd4bd39a700 1 -- 172.21.15.156:0/4105556001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a8009550 con 0x7fd4b815c9b0 2023-11-28T13:15:19.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.760+0000 7fd4bd39a700 1 --2- 172.21.15.156:0/4105556001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4b815c9b0 0x7fd4b8157620 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800ebe0 tx=0x7fd4a800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:19.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.760+0000 7fd4ae7fc700 1 -- 172.21.15.156:0/4105556001 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a800c850 con 0x7fd4b815c9b0 2023-11-28T13:15:19.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.760+0000 7fd4ae7fc700 1 -- 172.21.15.156:0/4105556001 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a800c9b0 con 0x7fd4b815c9b0 2023-11-28T13:15:19.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.760+0000 7fd4ae7fc700 1 -- 172.21.15.156:0/4105556001 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a800cd40 con 0x7fd4b815c9b0 2023-11-28T13:15:19.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.760+0000 7fd4bfdff700 1 -- 172.21.15.156:0/4105556001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b0009a70 con 0x7fd4b815c9b0 2023-11-28T13:15:19.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.761+0000 7fd4bfdff700 1 -- 172.21.15.156:0/4105556001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b80d15e0 con 0x7fd4b815c9b0 2023-11-28T13:15:19.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.762+0000 7fd4bfdff700 1 -- 172.21.15.156:0/4105556001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b80d1300 con 0x7fd4b815c9b0 2023-11-28T13:15:19.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.762+0000 7fd4ae7fc700 1 -- 172.21.15.156:0/4105556001 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd4a8019030 con 0x7fd4b815c9b0 2023-11-28T13:15:19.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.764+0000 7fd4ae7fc700 1 --2- 172.21.15.156:0/4105556001 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd49c0682f0 0x7fd49c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:19.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.764+0000 7fd4ae7fc700 1 -- 172.21.15.156:0/4105556001 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fd4a8085df0 con 0x7fd4b815c9b0 2023-11-28T13:15:19.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.764+0000 7fd4bdb9b700 1 --2- 172.21.15.156:0/4105556001 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd49c0682f0 0x7fd49c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:19.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.765+0000 7fd4bdb9b700 1 --2- 172.21.15.156:0/4105556001 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd49c0682f0 0x7fd49c06a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4000f60 tx=0x7fd4b4007ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:19.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.768+0000 7fd4ae7fc700 1 -- 172.21.15.156:0/4105556001 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd4a80524d0 con 0x7fd4b815c9b0 2023-11-28T13:15:19.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.976+0000 7fd4bfdff700 1 -- 172.21.15.156:0/4105556001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd4b8153770 con 0x7fd4b815c9b0 2023-11-28T13:15:19.978 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.976+0000 7fd4ae7fc700 1 -- 172.21.15.156:0/4105556001 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v61) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd4a8017080 con 0x7fd4b815c9b0 2023-11-28T13:15:19.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.979+0000 7fd4bfdff700 1 -- 172.21.15.156:0/4105556001 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd49c0682f0 msgr2=0x7fd49c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:19.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.979+0000 7fd4bfdff700 1 --2- 172.21.15.156:0/4105556001 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd49c0682f0 0x7fd49c06a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4000f60 tx=0x7fd4b4007ed0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.979+0000 7fd4bfdff700 1 -- 172.21.15.156:0/4105556001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4b815c9b0 msgr2=0x7fd4b8157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:19.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.979+0000 7fd4bfdff700 1 --2- 172.21.15.156:0/4105556001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4b815c9b0 0x7fd4b8157620 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800ebe0 tx=0x7fd4a800eef0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.979+0000 7fd4bfdff700 1 -- 172.21.15.156:0/4105556001 shutdown_connections 2023-11-28T13:15:19.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.979+0000 7fd4bfdff700 1 --2- 172.21.15.156:0/4105556001 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fd49c0682f0 0x7fd49c06a7a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.979+0000 7fd4bfdff700 1 --2- 172.21.15.156:0/4105556001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4b8161340 0x7fd4b8160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.979+0000 7fd4bfdff700 1 --2- 172.21.15.156:0/4105556001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4b80d2320 0x7fd4b81f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.979+0000 7fd4bfdff700 1 --2- 172.21.15.156:0/4105556001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4b815c9b0 0x7fd4b8157620 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:19.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.979+0000 7fd4bfdff700 1 -- 172.21.15.156:0/4105556001 >> 172.21.15.156:0/4105556001 conn(0x7fd4b8072600 msgr2=0x7fd4b815a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:19.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.979+0000 7fd4bfdff700 1 -- 172.21.15.156:0/4105556001 shutdown_connections 2023-11-28T13:15:19.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:19.979+0000 7fd4bfdff700 1 -- 172.21.15.156:0/4105556001 wait complete. 2023-11-28T13:15:19.981 INFO:teuthology.orchestra.run.smithi156.stderr:listed 0 entries 2023-11-28T13:15:19.998 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T13:15:19.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:15:19.999 DEBUG:teuthology.orchestra.run.smithi156:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T13:15:20.099 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T13:15:20.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:15:20.100 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T13:15:20.154 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:15:20.154 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:15:20.200 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:15:20.200 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:15:20.200 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:15:20.244 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:15:20.244 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:15:20.244 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:15:20.288 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:15:20.288 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:15:20.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.483+0000 7f8c227ff700 1 Processor -- start 2023-11-28T13:15:20.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.484+0000 7f8c227ff700 1 -- start start 2023-11-28T13:15:20.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.485+0000 7f8c227ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c1c070c50 0x7f8c1c0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:20.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.485+0000 7f8c227ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1c0b1010 con 0x7f8c1c0b0bb0 2023-11-28T13:15:20.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.485+0000 7f8c227ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1c15e1a0 con 0x7f8c1c158200 2023-11-28T13:15:20.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.485+0000 7f8c227ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c15f070 con 0x7f8c1c070c50 2023-11-28T13:15:20.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.485+0000 7f8c20d9c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f8c1c0b0bb0 0x7f8c1c15add0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50776/0 (socket says 172.21.15.156:50776) 2023-11-28T13:15:20.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.485+0000 7f8c20d9c700 1 -- 172.21.15.156:0/2349093892 learned_addr learned my addr 172.21.15.156:0/2349093892 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:20.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.485+0000 7f8c1b7fe700 1 --2- 172.21.15.156:0/2349093892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c1c070c50 0x7f8c1c0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:20.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.485+0000 7f8c1affd700 1 -- 172.21.15.156:0/2349093892 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366127244 0 0) 0x7f8c1c0b1010 con 0x7f8c1c0b0bb0 2023-11-28T13:15:20.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1affd700 1 -- 172.21.15.156:0/2349093892 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c040035b0 con 0x7f8c1c0b0bb0 2023-11-28T13:15:20.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1affd700 1 -- 172.21.15.156:0/2349093892 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541139733 0 0) 0x7f8c1c15e1a0 con 0x7f8c1c158200 2023-11-28T13:15:20.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1affd700 1 -- 172.21.15.156:0/2349093892 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c1c0b1010 con 0x7f8c1c158200 2023-11-28T13:15:20.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1affd700 1 -- 172.21.15.156:0/2349093892 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2326587150 0 0) 0x7f8c040035b0 con 0x7f8c1c0b0bb0 2023-11-28T13:15:20.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1affd700 1 -- 172.21.15.156:0/2349093892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c1c070c50 msgr2=0x7f8c1c0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:20.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1affd700 1 --2- 172.21.15.156:0/2349093892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c1c070c50 0x7f8c1c0b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:20.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1affd700 1 -- 172.21.15.156:0/2349093892 >> v1:172.21.15.186:6789/0 conn(0x7f8c1c158200 legacy=0x7f8c1c0caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:20.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1affd700 1 -- 172.21.15.156:0/2349093892 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c0cb180 con 0x7f8c1c0b0bb0 2023-11-28T13:15:20.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1affd700 1 -- 172.21.15.156:0/2349093892 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f8c140019f0 con 0x7f8c1c0b0bb0 2023-11-28T13:15:20.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1affd700 1 -- 172.21.15.156:0/2349093892 >> v1:172.21.15.156:6789/0 conn(0x7f8c1c0b0bb0 legacy=0x7f8c1c15add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:20.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1affd700 1 --2- 172.21.15.156:0/2349093892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c04003bc0 0x7f8c04003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:20.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1affd700 1 --2- 172.21.15.156:0/2349093892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c04004850 0x7f8c04004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:20.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1affd700 1 --2- 172.21.15.156:0/2349093892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c04005200 0x7f8c0400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:20.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1affd700 1 -- 172.21.15.156:0/2349093892 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c15f070 con 0x7f8c04003bc0 2023-11-28T13:15:20.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1affd700 1 -- 172.21.15.156:0/2349093892 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0400c030 con 0x7f8c04004850 2023-11-28T13:15:20.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1affd700 1 -- 172.21.15.156:0/2349093892 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0400c1f0 con 0x7f8c04005200 2023-11-28T13:15:20.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1b7fe700 1 --2- 172.21.15.156:0/2349093892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c1c070c50 0x7f8c1c0b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:15:20.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1bfff700 1 --2- 172.21.15.156:0/2349093892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c04003bc0 0x7f8c04003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:20.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c1b7fe700 1 --2- 172.21.15.156:0/2349093892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c04004850 0x7f8c04004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:20.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.486+0000 7f8c20d9c700 1 --2- 172.21.15.156:0/2349093892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c04005200 0x7f8c0400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:20.492 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.487+0000 7f8c1bfff700 1 -- 172.21.15.156:0/2349093892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c04005200 msgr2=0x7f8c0400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.487+0000 7f8c1bfff700 1 --2- 172.21.15.156:0/2349093892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c04005200 0x7f8c0400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.487+0000 7f8c1bfff700 1 -- 172.21.15.156:0/2349093892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c04004850 msgr2=0x7f8c04004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.487+0000 7f8c1bfff700 1 --2- 172.21.15.156:0/2349093892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c04004850 0x7f8c04004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.487+0000 7f8c1bfff700 1 -- 172.21.15.156:0/2349093892 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c040035b0 con 0x7f8c04003bc0 2023-11-28T13:15:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.487+0000 7f8c1bfff700 1 --2- 172.21.15.156:0/2349093892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c04003bc0 0x7f8c04003fb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c001940 tx=0x7f8c04005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd6333d32251093 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.487+0000 7f8c1affd700 1 -- 172.21.15.156:0/2349093892 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c001430 con 0x7f8c04003bc0 2023-11-28T13:15:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.487+0000 7f8c1affd700 1 -- 172.21.15.156:0/2349093892 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c001d70 con 0x7f8c04003bc0 2023-11-28T13:15:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.487+0000 7f8c1affd700 1 -- 172.21.15.156:0/2349093892 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c005510 con 0x7f8c04003bc0 2023-11-28T13:15:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.488+0000 7f8c227ff700 1 -- 172.21.15.156:0/2349093892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c04003bc0 msgr2=0x7f8c04003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.488+0000 7f8c227ff700 1 --2- 172.21.15.156:0/2349093892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c04003bc0 0x7f8c04003fb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c001940 tx=0x7f8c04005880 comp rx=0 tx=0).stop 2023-11-28T13:15:20.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.488+0000 7f8c20d9c700 1 -- 172.21.15.156:0/2349093892 reap_dead start 2023-11-28T13:15:20.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.488+0000 7f8c227ff700 1 -- 172.21.15.156:0/2349093892 shutdown_connections 2023-11-28T13:15:20.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.488+0000 7f8c227ff700 1 -- 172.21.15.156:0/2349093892 >> 172.21.15.156:0/2349093892 conn(0x7f8c1c0cb670 msgr2=0x7f8c1c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:20.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.488+0000 7f8c227ff700 1 -- 172.21.15.156:0/2349093892 shutdown_connections 2023-11-28T13:15:20.494 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.488+0000 7f8c227ff700 1 -- 172.21.15.156:0/2349093892 wait complete. 2023-11-28T13:15:20.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.490+0000 7f8c227ff700 1 Processor -- start 2023-11-28T13:15:20.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.490+0000 7f8c227ff700 1 -- start start 2023-11-28T13:15:20.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.491+0000 7f8c227ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c04003bc0 0x7f8c1c1ef6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:20.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.491+0000 7f8c227ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c1c0b2430 0x7f8c1c0b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:20.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.491+0000 7f8c227ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c0b3f00 0x7f8c1c0b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:20.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.491+0000 7f8c227ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0b2de0 con 0x7f8c1c0b3f00 2023-11-28T13:15:20.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.491+0000 7f8c227ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0b48b0 con 0x7f8c1c0b2430 2023-11-28T13:15:20.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.491+0000 7f8c227ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0c51a0 con 0x7f8c04003bc0 2023-11-28T13:15:20.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.491+0000 7f8c20d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c0b3f00 0x7f8c1c0b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:20.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.491+0000 7f8c20d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c0b3f00 0x7f8c1c0b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50750/0 (socket says 172.21.15.156:50750) 2023-11-28T13:15:20.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.491+0000 7f8c20d9c700 1 -- 172.21.15.156:0/2380750405 learned_addr learned my addr 172.21.15.156:0/2380750405 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:20.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.491+0000 7f8c1bfff700 1 --2- 172.21.15.156:0/2380750405 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c04003bc0 0x7f8c1c1ef6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:20.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.491+0000 7f8c1b7fe700 1 --2- 172.21.15.156:0/2380750405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c1c0b2430 0x7f8c1c0b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:20.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.491+0000 7f8c20d9c700 1 -- 172.21.15.156:0/2380750405 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c04003bc0 msgr2=0x7f8c1c1ef6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:20.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.491+0000 7f8c20d9c700 1 --2- 172.21.15.156:0/2380750405 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c04003bc0 0x7f8c1c1ef6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:20.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.491+0000 7f8c20d9c700 1 -- 172.21.15.156:0/2380750405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c1c0b2430 msgr2=0x7f8c1c0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:20.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.491+0000 7f8c20d9c700 1 --2- 172.21.15.156:0/2380750405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c1c0b2430 0x7f8c1c0b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:20.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.491+0000 7f8c20d9c700 1 -- 172.21.15.156:0/2380750405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c1585d0 con 0x7f8c1c0b3f00 2023-11-28T13:15:20.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.492+0000 7f8c20d9c700 1 --2- 172.21.15.156:0/2380750405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c0b3f00 0x7f8c1c0b4370 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8c14001aa0 tx=0x7f8c1400a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:20.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.492+0000 7f8c18ff9700 1 -- 172.21.15.156:0/2380750405 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1400ac40 con 0x7f8c1c0b3f00 2023-11-28T13:15:20.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.492+0000 7f8c18ff9700 1 -- 172.21.15.156:0/2380750405 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1400adc0 con 0x7f8c1c0b3f00 2023-11-28T13:15:20.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.492+0000 7f8c227ff700 1 -- 172.21.15.156:0/2380750405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c0710f0 con 0x7f8c1c0b3f00 2023-11-28T13:15:20.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.492+0000 7f8c18ff9700 1 -- 172.21.15.156:0/2380750405 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c14013630 con 0x7f8c1c0b3f00 2023-11-28T13:15:20.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.492+0000 7f8c227ff700 1 -- 172.21.15.156:0/2380750405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1c15e3f0 con 0x7f8c1c0b3f00 2023-11-28T13:15:20.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.493+0000 7f8c227ff700 1 -- 172.21.15.156:0/2380750405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1c06fd60 con 0x7f8c1c0b3f00 2023-11-28T13:15:20.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.494+0000 7f8c18ff9700 1 -- 172.21.15.156:0/2380750405 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8c14013790 con 0x7f8c1c0b3f00 2023-11-28T13:15:20.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.495+0000 7f8c18ff9700 1 --2- 172.21.15.156:0/2380750405 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8c080682f0 0x7f8c0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:20.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.495+0000 7f8c18ff9700 1 -- 172.21.15.156:0/2380750405 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f8c14086700 con 0x7f8c1c0b3f00 2023-11-28T13:15:20.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.495+0000 7f8c1bfff700 1 --2- 172.21.15.156:0/2380750405 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8c080682f0 0x7f8c0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:20.499 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.496+0000 7f8c1bfff700 1 --2- 172.21.15.156:0/2380750405 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8c080682f0 0x7f8c0806a7a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c015040 tx=0x7f8c0c00a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:20.500 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.499+0000 7f8c18ff9700 1 -- 172.21.15.156:0/2380750405 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8c14052de0 con 0x7f8c1c0b3f00 2023-11-28T13:15:20.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.734+0000 7f8c227ff700 1 -- 172.21.15.156:0/2380750405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c1c0745c0 con 0x7f8c1c0b3f00 2023-11-28T13:15:20.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.735+0000 7f8c18ff9700 1 -- 172.21.15.156:0/2380750405 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4658 (secure 0 0 0) 0x7f8c14056400 con 0x7f8c1c0b3f00 2023-11-28T13:15:20.737 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:15:20.737 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3942362276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3942362276},{"type":"v1","addr":"172.21.15.186:6835","nonce":3942362276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/722198303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":722198303},{"type":"v1","addr":"172.21.15.186:6837","nonce":722198303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5916,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3601975324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3601975324},{"type":"v1","addr":"172.21.15.156:6835","nonce":3601975324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:10:54.928601+0000","modified":"2023-11-28T13:11:11.455488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5919":{"gid":5919,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6834/3314379816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3314379816},{"type":"v1","addr":"172.21.15.156:6834","nonce":3314379816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:15:20.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.737+0000 7f8c227ff700 1 -- 172.21.15.156:0/2380750405 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8c080682f0 msgr2=0x7f8c0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:20.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.737+0000 7f8c227ff700 1 --2- 172.21.15.156:0/2380750405 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8c080682f0 0x7f8c0806a7a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c015040 tx=0x7f8c0c00a000 comp rx=0 tx=0).stop 2023-11-28T13:15:20.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.737+0000 7f8c227ff700 1 -- 172.21.15.156:0/2380750405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c0b3f00 msgr2=0x7f8c1c0b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:20.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.737+0000 7f8c227ff700 1 --2- 172.21.15.156:0/2380750405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c0b3f00 0x7f8c1c0b4370 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8c14001aa0 tx=0x7f8c1400a510 comp rx=0 tx=0).stop 2023-11-28T13:15:20.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.737+0000 7f8c227ff700 1 -- 172.21.15.156:0/2380750405 shutdown_connections 2023-11-28T13:15:20.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.737+0000 7f8c227ff700 1 --2- 172.21.15.156:0/2380750405 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f8c080682f0 0x7f8c0806a7a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:20.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.737+0000 7f8c227ff700 1 --2- 172.21.15.156:0/2380750405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c0b3f00 0x7f8c1c0b4370 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:20.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.737+0000 7f8c227ff700 1 --2- 172.21.15.156:0/2380750405 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c04003bc0 0x7f8c1c1ef6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:20.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.737+0000 7f8c227ff700 1 --2- 172.21.15.156:0/2380750405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c1c0b2430 0x7f8c1c0b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:20.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.737+0000 7f8c227ff700 1 -- 172.21.15.156:0/2380750405 >> 172.21.15.156:0/2380750405 conn(0x7f8c1c0cb670 msgr2=0x7f8c1c071280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:20.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.738+0000 7f8c227ff700 1 -- 172.21.15.156:0/2380750405 shutdown_connections 2023-11-28T13:15:20.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.738+0000 7f8c227ff700 1 -- 172.21.15.156:0/2380750405 wait complete. 2023-11-28T13:15:20.741 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 32 2023-11-28T13:15:20.758 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:15:20.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.951+0000 7f5c137ff700 1 Processor -- start 2023-11-28T13:15:20.952 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.951+0000 7f5c137ff700 1 -- start start 2023-11-28T13:15:20.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.952+0000 7f5c137ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5c0c15c9b0 0x7f5c0c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:20.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.952+0000 7f5c137ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0c161340 0x7f5c0c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:20.954 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.952+0000 7f5c137ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5c0c0d2320 0x7f5c0c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:20.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.952+0000 7f5c137ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0c0b5800 con 0x7f5c0c15c9b0 2023-11-28T13:15:20.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.952+0000 7f5c137ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0c0d1950 con 0x7f5c0c0d2320 2023-11-28T13:15:20.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.952+0000 7f5c137ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0b5520 con 0x7f5c0c161340 2023-11-28T13:15:20.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.952+0000 7f5c1159b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5c0c15c9b0 0x7f5c0c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:20.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.952+0000 7f5c1159b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5c0c15c9b0 0x7f5c0c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50764/0 (socket says 172.21.15.156:50764) 2023-11-28T13:15:20.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.952+0000 7f5c1159b700 1 -- 172.21.15.156:0/958540034 learned_addr learned my addr 172.21.15.156:0/958540034 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:20.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.952+0000 7f5c10d9a700 1 --2- 172.21.15.156:0/958540034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0c161340 0x7f5c0c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:20.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.952+0000 7f5c11d9c700 1 --2- 172.21.15.156:0/958540034 >> v2:172.21.15.186:3300/0 conn(0x7f5c0c0d2320 0x7f5c0c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:20.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.953+0000 7f5c1159b700 1 -- 172.21.15.156:0/958540034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0c161340 msgr2=0x7f5c0c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:20.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.953+0000 7f5c1159b700 1 --2- 172.21.15.156:0/958540034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0c161340 0x7f5c0c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:20.955 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.953+0000 7f5c1159b700 1 -- 172.21.15.156:0/958540034 >> v2:172.21.15.186:3300/0 conn(0x7f5c0c0d2320 msgr2=0x7f5c0c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:20.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.953+0000 7f5c1159b700 1 --2- 172.21.15.156:0/958540034 >> v2:172.21.15.186:3300/0 conn(0x7f5c0c0d2320 0x7f5c0c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:20.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.953+0000 7f5c1159b700 1 -- 172.21.15.156:0/958540034 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c0d1300 con 0x7f5c0c15c9b0 2023-11-28T13:15:20.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.953+0000 7f5c1159b700 1 --2- 172.21.15.156:0/958540034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c15c9b0 0x7f5c0c0b4bf0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c08c4d0 tx=0x7f5c08009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1507f7d4281f7681 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:20.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.953+0000 7f5c03fff700 1 -- 172.21.15.156:0/958540034 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0800b870 con 0x7f5c0c15c9b0 2023-11-28T13:15:20.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.953+0000 7f5c03fff700 1 -- 172.21.15.156:0/958540034 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0800b9d0 con 0x7f5c0c15c9b0 2023-11-28T13:15:20.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.953+0000 7f5c03fff700 1 -- 172.21.15.156:0/958540034 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0800bd60 con 0x7f5c0c15c9b0 2023-11-28T13:15:20.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.954+0000 7f5c137ff700 1 -- 172.21.15.156:0/958540034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c15c9b0 msgr2=0x7f5c0c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:20.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.954+0000 7f5c137ff700 1 --2- 172.21.15.156:0/958540034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c15c9b0 0x7f5c0c0b4bf0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c08c4d0 tx=0x7f5c08009bd0 comp rx=0 tx=0).stop 2023-11-28T13:15:20.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.954+0000 7f5c137ff700 1 -- 172.21.15.156:0/958540034 shutdown_connections 2023-11-28T13:15:20.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.954+0000 7f5c137ff700 1 --2- 172.21.15.156:0/958540034 >> v2:172.21.15.186:3300/0 conn(0x7f5c0c0d2320 0x7f5c0c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:20.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.954+0000 7f5c137ff700 1 --2- 172.21.15.156:0/958540034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c15c9b0 0x7f5c0c0b4bf0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:20.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.954+0000 7f5c137ff700 1 --2- 172.21.15.156:0/958540034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0c161340 0x7f5c0c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:20.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.954+0000 7f5c137ff700 1 -- 172.21.15.156:0/958540034 >> 172.21.15.156:0/958540034 conn(0x7f5c0c072600 msgr2=0x7f5c0c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:20.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.954+0000 7f5c137ff700 1 -- 172.21.15.156:0/958540034 shutdown_connections 2023-11-28T13:15:20.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.954+0000 7f5c137ff700 1 -- 172.21.15.156:0/958540034 wait complete. 2023-11-28T13:15:20.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.956+0000 7f5c137ff700 1 Processor -- start 2023-11-28T13:15:20.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.956+0000 7f5c137ff700 1 -- start start 2023-11-28T13:15:20.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.957+0000 7f5c137ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c0c0d2320 0x7f5c0c0c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:20.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.957+0000 7f5c137ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c15c9b0 0x7f5c0c1e8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:20.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.957+0000 7f5c137ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0c161340 0x7f5c0c1e8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:20.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.957+0000 7f5c137ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c151e00 con 0x7f5c0c15c9b0 2023-11-28T13:15:20.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.957+0000 7f5c137ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c1f7040 con 0x7f5c0c0d2320 2023-11-28T13:15:20.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.957+0000 7f5c137ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c1f6d60 con 0x7f5c0c161340 2023-11-28T13:15:20.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.957+0000 7f5c10d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c15c9b0 0x7f5c0c1e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:20.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.957+0000 7f5c10d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c15c9b0 0x7f5c0c1e8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50774/0 (socket says 172.21.15.156:50774) 2023-11-28T13:15:20.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.957+0000 7f5c10d9a700 1 -- 172.21.15.156:0/868740716 learned_addr learned my addr 172.21.15.156:0/868740716 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:20.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.957+0000 7f5c1159b700 1 --2- 172.21.15.156:0/868740716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c0c0d2320 0x7f5c0c0c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:20.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.957+0000 7f5c11d9c700 1 --2- 172.21.15.156:0/868740716 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0c161340 0x7f5c0c1e8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:20.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.957+0000 7f5c10d9a700 1 -- 172.21.15.156:0/868740716 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0c161340 msgr2=0x7f5c0c1e8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:20.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.957+0000 7f5c10d9a700 1 --2- 172.21.15.156:0/868740716 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0c161340 0x7f5c0c1e8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:20.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.957+0000 7f5c10d9a700 1 -- 172.21.15.156:0/868740716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c0c0d2320 msgr2=0x7f5c0c0c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:20.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.957+0000 7f5c10d9a700 1 --2- 172.21.15.156:0/868740716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c0c0d2320 0x7f5c0c0c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:20.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.957+0000 7f5c10d9a700 1 -- 172.21.15.156:0/868740716 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c08009550 con 0x7f5c0c15c9b0 2023-11-28T13:15:20.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.957+0000 7f5c10d9a700 1 --2- 172.21.15.156:0/868740716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c15c9b0 0x7f5c0c1e8690 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00e810 tx=0x7f5bfc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:20.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.958+0000 7f5c01ffb700 1 -- 172.21.15.156:0/868740716 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc00c070 con 0x7f5c0c15c9b0 2023-11-28T13:15:20.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.958+0000 7f5c01ffb700 1 -- 172.21.15.156:0/868740716 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bfc013610 con 0x7f5c0c15c9b0 2023-11-28T13:15:20.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.958+0000 7f5c137ff700 1 -- 172.21.15.156:0/868740716 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c0c0b5130 con 0x7f5c0c15c9b0 2023-11-28T13:15:20.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.958+0000 7f5c01ffb700 1 -- 172.21.15.156:0/868740716 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0139c0 con 0x7f5c0c15c9b0 2023-11-28T13:15:20.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.958+0000 7f5c137ff700 1 -- 172.21.15.156:0/868740716 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0c0d15e0 con 0x7f5c0c15c9b0 2023-11-28T13:15:20.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.959+0000 7f5c137ff700 1 -- 172.21.15.156:0/868740716 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0c0b52c0 con 0x7f5c0c15c9b0 2023-11-28T13:15:20.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.960+0000 7f5c01ffb700 1 -- 172.21.15.156:0/868740716 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5bfc023070 con 0x7f5c0c15c9b0 2023-11-28T13:15:20.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.961+0000 7f5c01ffb700 1 --2- 172.21.15.156:0/868740716 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5bf00682f0 0x7f5bf006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:20.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.961+0000 7f5c01ffb700 1 -- 172.21.15.156:0/868740716 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f5bfc085aa0 con 0x7f5c0c15c9b0 2023-11-28T13:15:20.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.961+0000 7f5c1159b700 1 --2- 172.21.15.156:0/868740716 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5bf00682f0 0x7f5bf006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:20.964 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.962+0000 7f5c1159b700 1 --2- 172.21.15.156:0/868740716 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5bf00682f0 0x7f5bf006a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5c08014040 tx=0x7f5c08008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:20.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:20.964+0000 7f5c01ffb700 1 -- 172.21.15.156:0/868740716 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5bfc052100 con 0x7f5c0c15c9b0 2023-11-28T13:15:21.157 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.156+0000 7f5c137ff700 1 -- 172.21.15.156:0/868740716 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c0c0745c0 con 0x7f5c0c15c9b0 2023-11-28T13:15:21.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.157+0000 7f5c01ffb700 1 -- 172.21.15.156:0/868740716 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+18152 (secure 0 0 0) 0x7f5bfc055720 con 0x7f5c0c15c9b0 2023-11-28T13:15:21.159 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:15:21.159 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":61,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:11:52.553240+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:10:52.732399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T13:10:53.912753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.11","mappings":[{"from":7,"to":2}]},{"pgid":"10.19","mappings":[{"from":1,"to":3}]},{"pgid":"10.1b","mappings":[{"from":4,"to":5}]},{"pgid":"10.1f","mappings":[{"from":7,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:15:21.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.159+0000 7f5c137ff700 1 -- 172.21.15.156:0/868740716 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5bf00682f0 msgr2=0x7f5bf006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:21.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.159+0000 7f5c137ff700 1 --2- 172.21.15.156:0/868740716 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5bf00682f0 0x7f5bf006a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5c08014040 tx=0x7f5c08008030 comp rx=0 tx=0).stop 2023-11-28T13:15:21.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.159+0000 7f5c137ff700 1 -- 172.21.15.156:0/868740716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c15c9b0 msgr2=0x7f5c0c1e8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:21.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.159+0000 7f5c137ff700 1 --2- 172.21.15.156:0/868740716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c15c9b0 0x7f5c0c1e8690 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00e810 tx=0x7f5bfc00eb20 comp rx=0 tx=0).stop 2023-11-28T13:15:21.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.160+0000 7f5c137ff700 1 -- 172.21.15.156:0/868740716 shutdown_connections 2023-11-28T13:15:21.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.160+0000 7f5c137ff700 1 --2- 172.21.15.156:0/868740716 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5bf00682f0 0x7f5bf006a7a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:21.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.160+0000 7f5c137ff700 1 --2- 172.21.15.156:0/868740716 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0c161340 0x7f5c0c1e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:21.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.160+0000 7f5c137ff700 1 --2- 172.21.15.156:0/868740716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c0c0d2320 0x7f5c0c0c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:21.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.160+0000 7f5c137ff700 1 --2- 172.21.15.156:0/868740716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c0c15c9b0 0x7f5c0c1e8690 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:21.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.160+0000 7f5c137ff700 1 -- 172.21.15.156:0/868740716 >> 172.21.15.156:0/868740716 conn(0x7f5c0c072600 msgr2=0x7f5c0c0b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:21.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.160+0000 7f5c137ff700 1 -- 172.21.15.156:0/868740716 shutdown_connections 2023-11-28T13:15:21.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.160+0000 7f5c137ff700 1 -- 172.21.15.156:0/868740716 wait complete. 2023-11-28T13:15:21.180 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:07:55.600013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:07:56.083469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:10:52.732399+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:10:53.912753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:15:21.180 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:15:21.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.379+0000 7f3d359ff700 1 Processor -- start 2023-11-28T13:15:21.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.380+0000 7f3d359ff700 1 -- start start 2023-11-28T13:15:21.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.381+0000 7f3d359ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3d30161340 0x7f3d300d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:21.383 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.381+0000 7f3d359ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d300d1950 con 0x7f3d3015c9b0 2023-11-28T13:15:21.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.381+0000 7f3d359ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d300b5520 con 0x7f3d300d2320 2023-11-28T13:15:21.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.381+0000 7f3d359ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d300b5800 con 0x7f3d30161340 2023-11-28T13:15:21.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.382+0000 7f3d27fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3d30161340 0x7f3d300d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:21.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.382+0000 7f3d27fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3d30161340 0x7f3d300d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50780/0 (socket says 172.21.15.156:50780) 2023-11-28T13:15:21.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.382+0000 7f3d27fff700 1 -- 172.21.15.156:0/979137355 learned_addr learned my addr 172.21.15.156:0/979137355 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:21.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.382+0000 7f3d27fff700 1 -- 172.21.15.156:0/979137355 >> v1:172.21.15.186:6789/0 conn(0x7f3d300d2320 legacy=0x7f3d300cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:21.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.382+0000 7f3d27fff700 1 -- 172.21.15.156:0/979137355 >> v1:172.21.15.156:6789/0 conn(0x7f3d3015c9b0 legacy=0x7f3d300b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:21.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.382+0000 7f3d27fff700 1 -- 172.21.15.156:0/979137355 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d300d1300 con 0x7f3d30161340 2023-11-28T13:15:21.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.383+0000 7f3d27fff700 1 --2- 172.21.15.156:0/979137355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d30161340 0x7f3d300d1de0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3d180098f0 tx=0x7f3d18009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0876c0593adc2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:21.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.383+0000 7f3d2e7fc700 1 -- 172.21.15.156:0/979137355 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1800b8a0 con 0x7f3d30161340 2023-11-28T13:15:21.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.383+0000 7f3d2e7fc700 1 -- 172.21.15.156:0/979137355 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1800ba00 con 0x7f3d30161340 2023-11-28T13:15:21.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.383+0000 7f3d2e7fc700 1 -- 172.21.15.156:0/979137355 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1800bd90 con 0x7f3d30161340 2023-11-28T13:15:21.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.383+0000 7f3d359ff700 1 -- 172.21.15.156:0/979137355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d30161340 msgr2=0x7f3d300d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:21.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.383+0000 7f3d359ff700 1 --2- 172.21.15.156:0/979137355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d30161340 0x7f3d300d1de0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3d180098f0 tx=0x7f3d18009c00 comp rx=0 tx=0).stop 2023-11-28T13:15:21.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.383+0000 7f3d359ff700 1 -- 172.21.15.156:0/979137355 shutdown_connections 2023-11-28T13:15:21.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.383+0000 7f3d359ff700 1 --2- 172.21.15.156:0/979137355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d30161340 0x7f3d300d1de0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:21.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.383+0000 7f3d359ff700 1 -- 172.21.15.156:0/979137355 >> 172.21.15.156:0/979137355 conn(0x7f3d30072600 msgr2=0x7f3d300cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:21.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.383+0000 7f3d359ff700 1 -- 172.21.15.156:0/979137355 shutdown_connections 2023-11-28T13:15:21.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.383+0000 7f3d359ff700 1 -- 172.21.15.156:0/979137355 wait complete. 2023-11-28T13:15:21.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.386+0000 7f3d359ff700 1 Processor -- start 2023-11-28T13:15:21.387 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.386+0000 7f3d359ff700 1 -- start start 2023-11-28T13:15:21.389 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.387+0000 7f3d359ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d300d2320 0x7f3d3014a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:21.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.387+0000 7f3d359ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d3015c9b0 0x7f3d301594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:21.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.387+0000 7f3d359ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d30161340 0x7f3d30159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:21.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.387+0000 7f3d359ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d301e6460 con 0x7f3d30161340 2023-11-28T13:15:21.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.387+0000 7f3d359ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3014a920 con 0x7f3d300d2320 2023-11-28T13:15:21.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.387+0000 7f3d359ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d301e66b0 con 0x7f3d3015c9b0 2023-11-28T13:15:21.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.387+0000 7f3d2f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d30161340 0x7f3d30159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:21.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.388+0000 7f3d2f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d30161340 0x7f3d30159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50788/0 (socket says 172.21.15.156:50788) 2023-11-28T13:15:21.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.388+0000 7f3d2f7fe700 1 -- 172.21.15.156:0/90010515 learned_addr learned my addr 172.21.15.156:0/90010515 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:21.390 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.388+0000 7f3d2effd700 1 --2- 172.21.15.156:0/90010515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d300d2320 0x7f3d3014a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:21.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.388+0000 7f3d27fff700 1 --2- 172.21.15.156:0/90010515 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d3015c9b0 0x7f3d301594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:21.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.388+0000 7f3d2f7fe700 1 -- 172.21.15.156:0/90010515 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d3015c9b0 msgr2=0x7f3d301594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:21.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.388+0000 7f3d2f7fe700 1 --2- 172.21.15.156:0/90010515 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d3015c9b0 0x7f3d301594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:21.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.388+0000 7f3d2f7fe700 1 -- 172.21.15.156:0/90010515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d300d2320 msgr2=0x7f3d3014a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:21.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.388+0000 7f3d2f7fe700 1 --2- 172.21.15.156:0/90010515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d300d2320 0x7f3d3014a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:21.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.388+0000 7f3d2f7fe700 1 -- 172.21.15.156:0/90010515 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d18009550 con 0x7f3d30161340 2023-11-28T13:15:21.391 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.388+0000 7f3d2f7fe700 1 --2- 172.21.15.156:0/90010515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d30161340 0x7f3d30159a30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800cbc0 tx=0x7f3d2800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:21.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.388+0000 7f3d2cff9700 1 -- 172.21.15.156:0/90010515 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28003c00 con 0x7f3d30161340 2023-11-28T13:15:21.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.389+0000 7f3d2cff9700 1 -- 172.21.15.156:0/90010515 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d28003d60 con 0x7f3d30161340 2023-11-28T13:15:21.392 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.389+0000 7f3d359ff700 1 -- 172.21.15.156:0/90010515 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3015cd80 con 0x7f3d30161340 2023-11-28T13:15:21.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.389+0000 7f3d2cff9700 1 -- 172.21.15.156:0/90010515 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2800aae0 con 0x7f3d30161340 2023-11-28T13:15:21.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.389+0000 7f3d359ff700 1 -- 172.21.15.156:0/90010515 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d300d15e0 con 0x7f3d30161340 2023-11-28T13:15:21.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.390+0000 7f3d359ff700 1 -- 172.21.15.156:0/90010515 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d3015cf10 con 0x7f3d30161340 2023-11-28T13:15:21.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.391+0000 7f3d2cff9700 1 -- 172.21.15.156:0/90010515 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3d28010050 con 0x7f3d30161340 2023-11-28T13:15:21.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.391+0000 7f3d2cff9700 1 --2- 172.21.15.156:0/90010515 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d100682f0 0x7f3d1006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:21.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.392+0000 7f3d2cff9700 1 -- 172.21.15.156:0/90010515 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f3d28085b70 con 0x7f3d30161340 2023-11-28T13:15:21.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.392+0000 7f3d2effd700 1 --2- 172.21.15.156:0/90010515 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d100682f0 0x7f3d1006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:21.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.393+0000 7f3d2effd700 1 --2- 172.21.15.156:0/90010515 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d100682f0 0x7f3d1006a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3d20007fd0 tx=0x7f3d20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:21.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.395+0000 7f3d2cff9700 1 -- 172.21.15.156:0/90010515 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3d28052340 con 0x7f3d30161340 2023-11-28T13:15:21.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.588+0000 7f3d359ff700 1 -- 172.21.15.156:0/90010515 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3d300745c0 con 0x7f3d30161340 2023-11-28T13:15:21.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.588+0000 7f3d2cff9700 1 -- 172.21.15.156:0/90010515 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v61) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3d28055960 con 0x7f3d30161340 2023-11-28T13:15:21.590 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 8 2023-11-28T13:15:21.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.591+0000 7f3d359ff700 1 -- 172.21.15.156:0/90010515 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d100682f0 msgr2=0x7f3d1006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:21.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.591+0000 7f3d359ff700 1 --2- 172.21.15.156:0/90010515 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d100682f0 0x7f3d1006a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3d20007fd0 tx=0x7f3d20009040 comp rx=0 tx=0).stop 2023-11-28T13:15:21.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.591+0000 7f3d359ff700 1 -- 172.21.15.156:0/90010515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d30161340 msgr2=0x7f3d30159a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:21.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.591+0000 7f3d359ff700 1 --2- 172.21.15.156:0/90010515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d30161340 0x7f3d30159a30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800cbc0 tx=0x7f3d2800ced0 comp rx=0 tx=0).stop 2023-11-28T13:15:21.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.591+0000 7f3d359ff700 1 -- 172.21.15.156:0/90010515 shutdown_connections 2023-11-28T13:15:21.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.591+0000 7f3d359ff700 1 --2- 172.21.15.156:0/90010515 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3d100682f0 0x7f3d1006a7a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:21.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.591+0000 7f3d359ff700 1 --2- 172.21.15.156:0/90010515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d30161340 0x7f3d30159a30 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:21.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.591+0000 7f3d359ff700 1 --2- 172.21.15.156:0/90010515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d300d2320 0x7f3d3014a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:21.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.591+0000 7f3d359ff700 1 --2- 172.21.15.156:0/90010515 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d3015c9b0 0x7f3d301594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:21.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.591+0000 7f3d359ff700 1 -- 172.21.15.156:0/90010515 >> 172.21.15.156:0/90010515 conn(0x7f3d30072600 msgr2=0x7f3d3015a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:21.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.591+0000 7f3d359ff700 1 -- 172.21.15.156:0/90010515 shutdown_connections 2023-11-28T13:15:21.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.591+0000 7f3d359ff700 1 -- 172.21.15.156:0/90010515 wait complete. 2023-11-28T13:15:21.611 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:15:21.811 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.810+0000 7ff98c3ff700 1 Processor -- start 2023-11-28T13:15:21.812 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.810+0000 7ff98c3ff700 1 -- start start 2023-11-28T13:15:21.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.811+0000 7ff98c3ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff98415c9b0 0x7ff9840b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:21.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.811+0000 7ff98c3ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9840d1950 con 0x7ff9840cd7c0 2023-11-28T13:15:21.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.811+0000 7ff98c3ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9840d24f0 con 0x7ff984161340 2023-11-28T13:15:21.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.811+0000 7ff98c3ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9840b5800 con 0x7ff98415c9b0 2023-11-28T13:15:21.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.811+0000 7ff98a19b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff98415c9b0 0x7ff9840b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:21.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.811+0000 7ff98a19b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff98415c9b0 0x7ff9840b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50792/0 (socket says 172.21.15.156:50792) 2023-11-28T13:15:21.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.811+0000 7ff98a19b700 1 -- 172.21.15.156:0/2650368634 learned_addr learned my addr 172.21.15.156:0/2650368634 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:21.813 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.811+0000 7ff98a99c700 1 --1- 172.21.15.156:0/2650368634 >> v1:172.21.15.156:6789/0 conn(0x7ff9840cd7c0 0x7ff9840b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50792/0 (socket says 172.21.15.156:50792) 2023-11-28T13:15:21.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.812+0000 7ff98a19b700 1 -- 172.21.15.156:0/2650368634 >> v1:172.21.15.186:6789/0 conn(0x7ff984161340 legacy=0x7ff9840d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:21.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.812+0000 7ff98a19b700 1 -- 172.21.15.156:0/2650368634 >> v1:172.21.15.156:6789/0 conn(0x7ff9840cd7c0 legacy=0x7ff9840b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:21.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.812+0000 7ff98a19b700 1 -- 172.21.15.156:0/2650368634 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9840b5520 con 0x7ff98415c9b0 2023-11-28T13:15:21.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.812+0000 7ff98a19b700 1 --2- 172.21.15.156:0/2650368634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98415c9b0 0x7ff9840b4bf0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff9800098f0 tx=0x7ff980009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab13592c5a3c4909 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:21.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.812+0000 7ff989199700 1 -- 172.21.15.156:0/2650368634 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98000b8a0 con 0x7ff98415c9b0 2023-11-28T13:15:21.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.812+0000 7ff989199700 1 -- 172.21.15.156:0/2650368634 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98000ba00 con 0x7ff98415c9b0 2023-11-28T13:15:21.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.812+0000 7ff989199700 1 -- 172.21.15.156:0/2650368634 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98000bd90 con 0x7ff98415c9b0 2023-11-28T13:15:21.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.813+0000 7ff98c3ff700 1 -- 172.21.15.156:0/2650368634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98415c9b0 msgr2=0x7ff9840b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:21.815 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.813+0000 7ff98c3ff700 1 --2- 172.21.15.156:0/2650368634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98415c9b0 0x7ff9840b4bf0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff9800098f0 tx=0x7ff980009c00 comp rx=0 tx=0).stop 2023-11-28T13:15:21.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.813+0000 7ff98c3ff700 1 -- 172.21.15.156:0/2650368634 shutdown_connections 2023-11-28T13:15:21.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.813+0000 7ff98c3ff700 1 --2- 172.21.15.156:0/2650368634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98415c9b0 0x7ff9840b4bf0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:21.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.813+0000 7ff98c3ff700 1 -- 172.21.15.156:0/2650368634 >> 172.21.15.156:0/2650368634 conn(0x7ff984072600 msgr2=0x7ff9840cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:21.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.813+0000 7ff98c3ff700 1 -- 172.21.15.156:0/2650368634 shutdown_connections 2023-11-28T13:15:21.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.813+0000 7ff98c3ff700 1 -- 172.21.15.156:0/2650368634 wait complete. 2023-11-28T13:15:21.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.815+0000 7ff98c3ff700 1 Processor -- start 2023-11-28T13:15:21.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.816+0000 7ff98c3ff700 1 -- start start 2023-11-28T13:15:21.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.817+0000 7ff98c3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9840cd7c0 0x7ff9841531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:21.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.817+0000 7ff98c3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff98415c9b0 0x7ff9840c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:21.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.817+0000 7ff98c3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff984161340 0x7ff9840c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:21.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.817+0000 7ff98c3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9840c9e20 con 0x7ff984161340 2023-11-28T13:15:21.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.817+0000 7ff98c3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff984153700 con 0x7ff9840cd7c0 2023-11-28T13:15:21.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.817+0000 7ff98c3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98415f940 con 0x7ff98415c9b0 2023-11-28T13:15:21.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.817+0000 7ff98a99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff984161340 0x7ff9840c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:21.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.817+0000 7ff98a99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff984161340 0x7ff9840c74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50804/0 (socket says 172.21.15.156:50804) 2023-11-28T13:15:21.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.817+0000 7ff98a99c700 1 -- 172.21.15.156:0/253337481 learned_addr learned my addr 172.21.15.156:0/253337481 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:21.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.817+0000 7ff98a19b700 1 --2- 172.21.15.156:0/253337481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9840cd7c0 0x7ff9841531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:21.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.817+0000 7ff98999a700 1 --2- 172.21.15.156:0/253337481 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff98415c9b0 0x7ff9840c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:21.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.817+0000 7ff98a99c700 1 -- 172.21.15.156:0/253337481 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff98415c9b0 msgr2=0x7ff9840c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:21.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.817+0000 7ff98a99c700 1 --2- 172.21.15.156:0/253337481 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff98415c9b0 0x7ff9840c6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:21.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.817+0000 7ff98a99c700 1 -- 172.21.15.156:0/253337481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9840cd7c0 msgr2=0x7ff9841531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:21.821 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.817+0000 7ff98a99c700 1 --2- 172.21.15.156:0/253337481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9840cd7c0 0x7ff9841531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:21.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.817+0000 7ff98a99c700 1 -- 172.21.15.156:0/253337481 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff980009550 con 0x7ff984161340 2023-11-28T13:15:21.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.818+0000 7ff98a19b700 1 --2- 172.21.15.156:0/253337481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9840cd7c0 0x7ff9841531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:15:21.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.818+0000 7ff98a99c700 1 --2- 172.21.15.156:0/253337481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff984161340 0x7ff9840c74d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00cbc0 tx=0x7ff97c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:21.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.818+0000 7ff97affd700 1 -- 172.21.15.156:0/253337481 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c00ab50 con 0x7ff984161340 2023-11-28T13:15:21.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.818+0000 7ff97affd700 1 -- 172.21.15.156:0/253337481 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97c00acb0 con 0x7ff984161340 2023-11-28T13:15:21.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.818+0000 7ff97affd700 1 -- 172.21.15.156:0/253337481 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c007270 con 0x7ff984161340 2023-11-28T13:15:21.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.818+0000 7ff98c3ff700 1 -- 172.21.15.156:0/253337481 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9840d15e0 con 0x7ff984161340 2023-11-28T13:15:21.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.818+0000 7ff98c3ff700 1 -- 172.21.15.156:0/253337481 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9840d1300 con 0x7ff984161340 2023-11-28T13:15:21.823 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.819+0000 7ff98c3ff700 1 -- 172.21.15.156:0/253337481 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9840b5130 con 0x7ff984161340 2023-11-28T13:15:21.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.820+0000 7ff97affd700 1 -- 172.21.15.156:0/253337481 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff97c019030 con 0x7ff984161340 2023-11-28T13:15:21.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.821+0000 7ff97affd700 1 --2- 172.21.15.156:0/253337481 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff9680682f0 0x7ff96806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:21.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.821+0000 7ff97affd700 1 -- 172.21.15.156:0/253337481 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ff97c085950 con 0x7ff984161340 2023-11-28T13:15:21.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.821+0000 7ff98a19b700 1 --2- 172.21.15.156:0/253337481 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff9680682f0 0x7ff96806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:21.824 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.822+0000 7ff98a19b700 1 --2- 172.21.15.156:0/253337481 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff9680682f0 0x7ff96806a7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff980014040 tx=0x7ff980008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:21.826 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:21.825+0000 7ff97affd700 1 -- 172.21.15.156:0/253337481 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff97c04b5e0 con 0x7ff984161340 2023-11-28T13:15:22.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.016+0000 7ff98c3ff700 1 -- 172.21.15.156:0/253337481 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9840745c0 con 0x7ff984161340 2023-11-28T13:15:22.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.017+0000 7ff97affd700 1 -- 172.21.15.156:0/253337481 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff97c017020 con 0x7ff984161340 2023-11-28T13:15:22.019 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2023-11-28T13:15:22.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.019+0000 7ff98c3ff700 1 -- 172.21.15.156:0/253337481 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff9680682f0 msgr2=0x7ff96806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:22.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.019+0000 7ff98c3ff700 1 --2- 172.21.15.156:0/253337481 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff9680682f0 0x7ff96806a7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff980014040 tx=0x7ff980008030 comp rx=0 tx=0).stop 2023-11-28T13:15:22.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.019+0000 7ff98c3ff700 1 -- 172.21.15.156:0/253337481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff984161340 msgr2=0x7ff9840c74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:22.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.019+0000 7ff98c3ff700 1 --2- 172.21.15.156:0/253337481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff984161340 0x7ff9840c74d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00cbc0 tx=0x7ff97c00cf80 comp rx=0 tx=0).stop 2023-11-28T13:15:22.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.019+0000 7ff98c3ff700 1 -- 172.21.15.156:0/253337481 shutdown_connections 2023-11-28T13:15:22.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.019+0000 7ff98c3ff700 1 --2- 172.21.15.156:0/253337481 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff9680682f0 0x7ff96806a7a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.019+0000 7ff98c3ff700 1 --2- 172.21.15.156:0/253337481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff984161340 0x7ff9840c74d0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.019+0000 7ff98c3ff700 1 --2- 172.21.15.156:0/253337481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9840cd7c0 0x7ff9841531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.019+0000 7ff98c3ff700 1 --2- 172.21.15.156:0/253337481 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff98415c9b0 0x7ff9840c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.019+0000 7ff98c3ff700 1 -- 172.21.15.156:0/253337481 >> 172.21.15.156:0/253337481 conn(0x7ff984072600 msgr2=0x7ff9840d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:22.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.019+0000 7ff98c3ff700 1 -- 172.21.15.156:0/253337481 shutdown_connections 2023-11-28T13:15:22.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.019+0000 7ff98c3ff700 1 -- 172.21.15.156:0/253337481 wait complete. 2023-11-28T13:15:22.039 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:15:22.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.238+0000 7f1c2b1ff700 1 Processor -- start 2023-11-28T13:15:22.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.238+0000 7f1c2b1ff700 1 -- start start 2023-11-28T13:15:22.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.239+0000 7f1c2b1ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1c2415c9b0 0x7f1c240b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:22.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.239+0000 7f1c2b1ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1c240cd7c0 0x7f1c240cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:22.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.239+0000 7f1c2b1ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c240d1950 con 0x7f1c24161340 2023-11-28T13:15:22.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.239+0000 7f1c2b1ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c240b5800 con 0x7f1c2415c9b0 2023-11-28T13:15:22.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.239+0000 7f1c2b1ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c240ce140 con 0x7f1c240cd7c0 2023-11-28T13:15:22.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.239+0000 7f1c28f9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1c2415c9b0 0x7f1c240b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:22.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.239+0000 7f1c28f9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1c2415c9b0 0x7f1c240b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50812/0 (socket says 172.21.15.156:50812) 2023-11-28T13:15:22.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.239+0000 7f1c28f9b700 1 -- 172.21.15.156:0/570745077 learned_addr learned my addr 172.21.15.156:0/570745077 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:22.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.240+0000 7f1c2979c700 1 --2- 172.21.15.156:0/570745077 >> v2:172.21.15.186:3300/0 conn(0x7f1c240cd7c0 0x7f1c240cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:22.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.240+0000 7f1c28f9b700 1 -- 172.21.15.156:0/570745077 >> v2:172.21.15.186:3300/0 conn(0x7f1c240cd7c0 msgr2=0x7f1c240cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:22.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.240+0000 7f1c28f9b700 1 --2- 172.21.15.156:0/570745077 >> v2:172.21.15.186:3300/0 conn(0x7f1c240cd7c0 0x7f1c240cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.240+0000 7f1c28f9b700 1 -- 172.21.15.156:0/570745077 >> v1:172.21.15.186:6789/0 conn(0x7f1c24161340 legacy=0x7f1c240d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:22.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.240+0000 7f1c28f9b700 1 -- 172.21.15.156:0/570745077 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c240d24f0 con 0x7f1c2415c9b0 2023-11-28T13:15:22.244 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.240+0000 7f1c28f9b700 1 --2- 172.21.15.156:0/570745077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c2415c9b0 0x7f1c240b4bf0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1c2406c170 tx=0x7f1c18009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2716ba8df1439 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:22.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.240+0000 7f1c237fe700 1 -- 172.21.15.156:0/570745077 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c18004230 con 0x7f1c2415c9b0 2023-11-28T13:15:22.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.240+0000 7f1c237fe700 1 -- 172.21.15.156:0/570745077 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c18004390 con 0x7f1c2415c9b0 2023-11-28T13:15:22.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.240+0000 7f1c237fe700 1 -- 172.21.15.156:0/570745077 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c180036a0 con 0x7f1c2415c9b0 2023-11-28T13:15:22.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.241+0000 7f1c2b1ff700 1 -- 172.21.15.156:0/570745077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c2415c9b0 msgr2=0x7f1c240b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:22.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.241+0000 7f1c2b1ff700 1 --2- 172.21.15.156:0/570745077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c2415c9b0 0x7f1c240b4bf0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1c2406c170 tx=0x7f1c18009bd0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.241+0000 7f1c2b1ff700 1 -- 172.21.15.156:0/570745077 shutdown_connections 2023-11-28T13:15:22.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.241+0000 7f1c2b1ff700 1 --2- 172.21.15.156:0/570745077 >> v2:172.21.15.186:3300/0 conn(0x7f1c240cd7c0 0x7f1c240cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.241+0000 7f1c2b1ff700 1 --2- 172.21.15.156:0/570745077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c2415c9b0 0x7f1c240b4bf0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.241+0000 7f1c2b1ff700 1 -- 172.21.15.156:0/570745077 >> 172.21.15.156:0/570745077 conn(0x7f1c24072600 msgr2=0x7f1c240cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:22.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.241+0000 7f1c2b1ff700 1 -- 172.21.15.156:0/570745077 shutdown_connections 2023-11-28T13:15:22.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.241+0000 7f1c2b1ff700 1 -- 172.21.15.156:0/570745077 wait complete. 2023-11-28T13:15:22.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.243+0000 7f1c2b1ff700 1 Processor -- start 2023-11-28T13:15:22.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.243+0000 7f1c2b1ff700 1 -- start start 2023-11-28T13:15:22.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c2b1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c240cd7c0 0x7f1c241f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:22.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c2b1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c2415c9b0 0x7f1c241547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:22.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c2b1ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c24161340 0x7f1c240caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:22.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c2b1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c240cbae0 con 0x7f1c240cd7c0 2023-11-28T13:15:22.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c2b1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c24154cf0 con 0x7f1c2415c9b0 2023-11-28T13:15:22.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c2b1ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c24156660 con 0x7f1c24161340 2023-11-28T13:15:22.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c28f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c240cd7c0 0x7f1c241f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:22.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c28f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c240cd7c0 0x7f1c241f3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50816/0 (socket says 172.21.15.156:50816) 2023-11-28T13:15:22.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c28f9b700 1 -- 172.21.15.156:0/2618511732 learned_addr learned my addr 172.21.15.156:0/2618511732 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:22.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c23fff700 1 --2- 172.21.15.156:0/2618511732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c2415c9b0 0x7f1c241547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:22.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c2979c700 1 --2- 172.21.15.156:0/2618511732 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c24161340 0x7f1c240caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:22.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c28f9b700 1 -- 172.21.15.156:0/2618511732 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c24161340 msgr2=0x7f1c240caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:22.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c28f9b700 1 --2- 172.21.15.156:0/2618511732 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c24161340 0x7f1c240caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c28f9b700 1 -- 172.21.15.156:0/2618511732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c2415c9b0 msgr2=0x7f1c241547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:22.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c28f9b700 1 --2- 172.21.15.156:0/2618511732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c2415c9b0 0x7f1c241547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c28f9b700 1 -- 172.21.15.156:0/2618511732 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c18009550 con 0x7f1c240cd7c0 2023-11-28T13:15:22.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c23fff700 1 --2- 172.21.15.156:0/2618511732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c2415c9b0 0x7f1c241547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:15:22.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.244+0000 7f1c2979c700 1 --2- 172.21.15.156:0/2618511732 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c24161340 0x7f1c240caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:15:22.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.245+0000 7f1c28f9b700 1 --2- 172.21.15.156:0/2618511732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c240cd7c0 0x7f1c241f3160 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009130 tx=0x7f1c18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:22.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.245+0000 7f1c217fa700 1 -- 172.21.15.156:0/2618511732 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c18011630 con 0x7f1c240cd7c0 2023-11-28T13:15:22.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.245+0000 7f1c217fa700 1 -- 172.21.15.156:0/2618511732 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c18011790 con 0x7f1c240cd7c0 2023-11-28T13:15:22.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.245+0000 7f1c2b1ff700 1 -- 172.21.15.156:0/2618511732 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c240d1300 con 0x7f1c240cd7c0 2023-11-28T13:15:22.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.245+0000 7f1c217fa700 1 -- 172.21.15.156:0/2618511732 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c18011b40 con 0x7f1c240cd7c0 2023-11-28T13:15:22.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.245+0000 7f1c2b1ff700 1 -- 172.21.15.156:0/2618511732 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c240b5520 con 0x7f1c240cd7c0 2023-11-28T13:15:22.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.246+0000 7f1c2b1ff700 1 -- 172.21.15.156:0/2618511732 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c240d15e0 con 0x7f1c240cd7c0 2023-11-28T13:15:22.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.247+0000 7f1c217fa700 1 -- 172.21.15.156:0/2618511732 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1c18017020 con 0x7f1c240cd7c0 2023-11-28T13:15:22.256 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.248+0000 7f1c217fa700 1 --2- 172.21.15.156:0/2618511732 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1c080682f0 0x7f1c0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:22.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.248+0000 7f1c217fa700 1 -- 172.21.15.156:0/2618511732 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f1c18087250 con 0x7f1c240cd7c0 2023-11-28T13:15:22.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.248+0000 7f1c23fff700 1 --2- 172.21.15.156:0/2618511732 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1c080682f0 0x7f1c0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:22.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.249+0000 7f1c23fff700 1 --2- 172.21.15.156:0/2618511732 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1c080682f0 0x7f1c0806a7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1c24160e10 tx=0x7f1c10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:22.257 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.252+0000 7f1c217fa700 1 -- 172.21.15.156:0/2618511732 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1c18053930 con 0x7f1c240cd7c0 2023-11-28T13:15:22.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.438+0000 7f1c2b1ff700 1 -- 172.21.15.156:0/2618511732 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c240745c0 con 0x7f1c240cd7c0 2023-11-28T13:15:22.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.439+0000 7f1c217fa700 1 -- 172.21.15.156:0/2618511732 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v61) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c18056f50 con 0x7f1c240cd7c0 2023-11-28T13:15:22.440 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 32 2023-11-28T13:15:22.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.441+0000 7f1c2b1ff700 1 -- 172.21.15.156:0/2618511732 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1c080682f0 msgr2=0x7f1c0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:22.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.441+0000 7f1c2b1ff700 1 --2- 172.21.15.156:0/2618511732 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1c080682f0 0x7f1c0806a7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1c24160e10 tx=0x7f1c10008040 comp rx=0 tx=0).stop 2023-11-28T13:15:22.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.441+0000 7f1c2b1ff700 1 -- 172.21.15.156:0/2618511732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c240cd7c0 msgr2=0x7f1c241f3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:22.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.441+0000 7f1c2b1ff700 1 --2- 172.21.15.156:0/2618511732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c240cd7c0 0x7f1c241f3160 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009130 tx=0x7f1c18004750 comp rx=0 tx=0).stop 2023-11-28T13:15:22.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.441+0000 7f1c2b1ff700 1 -- 172.21.15.156:0/2618511732 shutdown_connections 2023-11-28T13:15:22.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.441+0000 7f1c2b1ff700 1 --2- 172.21.15.156:0/2618511732 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1c080682f0 0x7f1c0806a7a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.441+0000 7f1c2b1ff700 1 --2- 172.21.15.156:0/2618511732 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c24161340 0x7f1c240caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.441+0000 7f1c2b1ff700 1 --2- 172.21.15.156:0/2618511732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c240cd7c0 0x7f1c241f3160 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.441+0000 7f1c2b1ff700 1 --2- 172.21.15.156:0/2618511732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c2415c9b0 0x7f1c241547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.443 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.441+0000 7f1c2b1ff700 1 -- 172.21.15.156:0/2618511732 >> 172.21.15.156:0/2618511732 conn(0x7f1c24072600 msgr2=0x7f1c24159c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:22.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.441+0000 7f1c2b1ff700 1 -- 172.21.15.156:0/2618511732 shutdown_connections 2023-11-28T13:15:22.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.441+0000 7f1c2b1ff700 1 -- 172.21.15.156:0/2618511732 wait complete. 2023-11-28T13:15:22.461 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:15:22.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.663+0000 7f5c5cfff700 1 Processor -- start 2023-11-28T13:15:22.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.663+0000 7f5c5cfff700 1 -- start start 2023-11-28T13:15:22.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.665+0000 7f5c5cfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c5815c9b0 0x7f5c580b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:22.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.665+0000 7f5c5cfff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5c580cd7c0 0x7f5c580cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:22.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.665+0000 7f5c5cfff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c580ce160 con 0x7f5c58161340 2023-11-28T13:15:22.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.665+0000 7f5c5cfff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c580b5800 con 0x7f5c580cd7c0 2023-11-28T13:15:22.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.665+0000 7f5c5cfff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580d1950 con 0x7f5c5815c9b0 2023-11-28T13:15:22.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.665+0000 7f5c4ffff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f5c58161340 0x7f5c580d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50800/0 (socket says 172.21.15.156:50800) 2023-11-28T13:15:22.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.665+0000 7f5c4ffff700 1 -- 172.21.15.156:0/4256198178 learned_addr learned my addr 172.21.15.156:0/4256198178 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:22.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.665+0000 7f5c5659c700 1 --2- 172.21.15.156:0/4256198178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c5815c9b0 0x7f5c580b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:22.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.665+0000 7f5c56d9d700 1 --2- 172.21.15.156:0/4256198178 >> v2:172.21.15.186:3300/0 conn(0x7f5c580cd7c0 0x7f5c580cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:22.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.665+0000 7f5c55d9b700 1 -- 172.21.15.156:0/4256198178 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506019067 0 0) 0x7f5c580ce160 con 0x7f5c58161340 2023-11-28T13:15:22.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.665+0000 7f5c55d9b700 1 -- 172.21.15.156:0/4256198178 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c440035b0 con 0x7f5c58161340 2023-11-28T13:15:22.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c55d9b700 1 -- 172.21.15.156:0/4256198178 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 710244182 0 0) 0x7f5c440035b0 con 0x7f5c58161340 2023-11-28T13:15:22.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c55d9b700 1 -- 172.21.15.156:0/4256198178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c5815c9b0 msgr2=0x7f5c580b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:22.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c55d9b700 1 --2- 172.21.15.156:0/4256198178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c5815c9b0 0x7f5c580b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c55d9b700 1 -- 172.21.15.156:0/4256198178 >> v2:172.21.15.186:3300/0 conn(0x7f5c580cd7c0 msgr2=0x7f5c580cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:22.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c55d9b700 1 --2- 172.21.15.156:0/4256198178 >> v2:172.21.15.186:3300/0 conn(0x7f5c580cd7c0 0x7f5c580cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c55d9b700 1 -- 172.21.15.156:0/4256198178 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c580d24f0 con 0x7f5c58161340 2023-11-28T13:15:22.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c55d9b700 1 -- 172.21.15.156:0/4256198178 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f5c40001670 con 0x7f5c58161340 2023-11-28T13:15:22.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c5659c700 1 --2- 172.21.15.156:0/4256198178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c5815c9b0 0x7f5c580b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:15:22.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c55d9b700 1 -- 172.21.15.156:0/4256198178 >> v1:172.21.15.156:6789/0 conn(0x7f5c58161340 legacy=0x7f5c580d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:22.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c55d9b700 1 --2- 172.21.15.156:0/4256198178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c44003a00 0x7f5c44003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:22.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c55d9b700 1 --2- 172.21.15.156:0/4256198178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c440043e0 0x7f5c440048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:22.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c55d9b700 1 --2- 172.21.15.156:0/4256198178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c44004fc0 0x7f5c4400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:22.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c55d9b700 1 -- 172.21.15.156:0/4256198178 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580b5800 con 0x7f5c44004fc0 2023-11-28T13:15:22.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c55d9b700 1 -- 172.21.15.156:0/4256198178 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580d1950 con 0x7f5c440043e0 2023-11-28T13:15:22.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c55d9b700 1 -- 172.21.15.156:0/4256198178 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4400bf50 con 0x7f5c44003a00 2023-11-28T13:15:22.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c56d9d700 1 --2- 172.21.15.156:0/4256198178 >> v2:172.21.15.186:3300/0 conn(0x7f5c580cd7c0 0x7f5c580cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:15:22.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c5659c700 1 --2- 172.21.15.156:0/4256198178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c44003a00 0x7f5c44003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:22.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c4ffff700 1 --2- 172.21.15.156:0/4256198178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c440043e0 0x7f5c440048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:22.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.666+0000 7f5c56d9d700 1 --2- 172.21.15.156:0/4256198178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c44004fc0 0x7f5c4400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:22.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.667+0000 7f5c5659c700 1 -- 172.21.15.156:0/4256198178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c440043e0 msgr2=0x7f5c440048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:22.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.667+0000 7f5c5659c700 1 --2- 172.21.15.156:0/4256198178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c440043e0 0x7f5c440048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.667+0000 7f5c5659c700 1 -- 172.21.15.156:0/4256198178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c44004fc0 msgr2=0x7f5c4400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:22.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.667+0000 7f5c5659c700 1 --2- 172.21.15.156:0/4256198178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c44004fc0 0x7f5c4400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.667+0000 7f5c5659c700 1 -- 172.21.15.156:0/4256198178 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c440035b0 con 0x7f5c44003a00 2023-11-28T13:15:22.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.667+0000 7f5c4ffff700 1 --2- 172.21.15.156:0/4256198178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c440043e0 0x7f5c440048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:15:22.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.667+0000 7f5c5659c700 1 --2- 172.21.15.156:0/4256198178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c44003a00 0x7f5c44003e10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009bc0 tx=0x7f5c44005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aa217a344f3ed25 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:22.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.667+0000 7f5c55d9b700 1 -- 172.21.15.156:0/4256198178 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48004030 con 0x7f5c44003a00 2023-11-28T13:15:22.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.667+0000 7f5c55d9b700 1 -- 172.21.15.156:0/4256198178 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c48004190 con 0x7f5c44003a00 2023-11-28T13:15:22.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.667+0000 7f5c55d9b700 1 -- 172.21.15.156:0/4256198178 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48004520 con 0x7f5c44003a00 2023-11-28T13:15:22.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.668+0000 7f5c5cfff700 1 -- 172.21.15.156:0/4256198178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c44003a00 msgr2=0x7f5c44003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:22.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.668+0000 7f5c5cfff700 1 --2- 172.21.15.156:0/4256198178 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c44003a00 0x7f5c44003e10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009bc0 tx=0x7f5c44005880 comp rx=0 tx=0).stop 2023-11-28T13:15:22.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.668+0000 7f5c56d9d700 1 -- 172.21.15.156:0/4256198178 reap_dead start 2023-11-28T13:15:22.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.668+0000 7f5c5cfff700 1 -- 172.21.15.156:0/4256198178 shutdown_connections 2023-11-28T13:15:22.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.668+0000 7f5c5cfff700 1 -- 172.21.15.156:0/4256198178 >> 172.21.15.156:0/4256198178 conn(0x7f5c58072600 msgr2=0x7f5c580cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:22.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.668+0000 7f5c5cfff700 1 -- 172.21.15.156:0/4256198178 shutdown_connections 2023-11-28T13:15:22.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.668+0000 7f5c5cfff700 1 -- 172.21.15.156:0/4256198178 wait complete. 2023-11-28T13:15:22.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.670+0000 7f5c5cfff700 1 Processor -- start 2023-11-28T13:15:22.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.670+0000 7f5c5cfff700 1 -- start start 2023-11-28T13:15:22.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.671+0000 7f5c5cfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c44003a00 0x7f5c580c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:22.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.671+0000 7f5c5cfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c581e78d0 0x7f5c5815e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:22.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.671+0000 7f5c5cfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c58149fe0 0x7f5c5814a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:22.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.671+0000 7f5c5cfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c581e8d10 con 0x7f5c44003a00 2023-11-28T13:15:22.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.671+0000 7f5c5cfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5814a990 con 0x7f5c58149fe0 2023-11-28T13:15:22.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.671+0000 7f5c5cfff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c581e8090 con 0x7f5c581e78d0 2023-11-28T13:15:22.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.671+0000 7f5c5659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c44003a00 0x7f5c580c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:22.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.671+0000 7f5c5659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c44003a00 0x7f5c580c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50830/0 (socket says 172.21.15.156:50830) 2023-11-28T13:15:22.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.671+0000 7f5c5659c700 1 -- 172.21.15.156:0/1173255181 learned_addr learned my addr 172.21.15.156:0/1173255181 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:22.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.671+0000 7f5c56d9d700 1 --2- 172.21.15.156:0/1173255181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c58149fe0 0x7f5c5814a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:22.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.671+0000 7f5c4ffff700 1 --2- 172.21.15.156:0/1173255181 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c581e78d0 0x7f5c5815e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:22.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.671+0000 7f5c5659c700 1 -- 172.21.15.156:0/1173255181 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c581e78d0 msgr2=0x7f5c5815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:22.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.671+0000 7f5c5659c700 1 --2- 172.21.15.156:0/1173255181 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c581e78d0 0x7f5c5815e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.671+0000 7f5c5659c700 1 -- 172.21.15.156:0/1173255181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c58149fe0 msgr2=0x7f5c5814a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:22.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.671+0000 7f5c5659c700 1 --2- 172.21.15.156:0/1173255181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c58149fe0 0x7f5c5814a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.671+0000 7f5c5659c700 1 -- 172.21.15.156:0/1173255181 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c580d1300 con 0x7f5c44003a00 2023-11-28T13:15:22.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.671+0000 7f5c56d9d700 1 --2- 172.21.15.156:0/1173255181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c58149fe0 0x7f5c5814a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:15:22.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.672+0000 7f5c5659c700 1 --2- 172.21.15.156:0/1173255181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c44003a00 0x7f5c580c8250 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400c0e0 tx=0x7f5c480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:22.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.672+0000 7f5c4effd700 1 -- 172.21.15.156:0/1173255181 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48020070 con 0x7f5c44003a00 2023-11-28T13:15:22.677 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.672+0000 7f5c4effd700 1 -- 172.21.15.156:0/1173255181 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c480038b0 con 0x7f5c44003a00 2023-11-28T13:15:22.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.672+0000 7f5c4effd700 1 -- 172.21.15.156:0/1173255181 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48004030 con 0x7f5c44003a00 2023-11-28T13:15:22.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.672+0000 7f5c5cfff700 1 -- 172.21.15.156:0/1173255181 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c580b5520 con 0x7f5c44003a00 2023-11-28T13:15:22.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.672+0000 7f5c5cfff700 1 -- 172.21.15.156:0/1173255181 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c580d15e0 con 0x7f5c44003a00 2023-11-28T13:15:22.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.673+0000 7f5c5cfff700 1 -- 172.21.15.156:0/1173255181 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c580745c0 con 0x7f5c44003a00 2023-11-28T13:15:22.678 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.674+0000 7f5c4effd700 1 -- 172.21.15.156:0/1173255181 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5c4802c030 con 0x7f5c44003a00 2023-11-28T13:15:22.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.675+0000 7f5c4effd700 1 --2- 172.21.15.156:0/1173255181 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5c380682f0 0x7f5c3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:22.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.675+0000 7f5c4effd700 1 -- 172.21.15.156:0/1173255181 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f5c48087350 con 0x7f5c44003a00 2023-11-28T13:15:22.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.675+0000 7f5c4ffff700 1 --2- 172.21.15.156:0/1173255181 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5c380682f0 0x7f5c3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:22.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.676+0000 7f5c4ffff700 1 --2- 172.21.15.156:0/1173255181 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5c380682f0 0x7f5c3806a7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5c580b8400 tx=0x7f5c40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:22.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.680+0000 7f5c4effd700 1 -- 172.21.15.156:0/1173255181 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5c48053a30 con 0x7f5c44003a00 2023-11-28T13:15:22.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.870+0000 7f5c5cfff700 1 -- 172.21.15.156:0/1173255181 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c580d08a0 con 0x7f5c44003a00 2023-11-28T13:15:22.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.871+0000 7f5c4effd700 1 -- 172.21.15.156:0/1173255181 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c48057050 con 0x7f5c44003a00 2023-11-28T13:15:22.872 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 64 2023-11-28T13:15:22.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.873+0000 7f5c5cfff700 1 -- 172.21.15.156:0/1173255181 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5c380682f0 msgr2=0x7f5c3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:22.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.873+0000 7f5c5cfff700 1 --2- 172.21.15.156:0/1173255181 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5c380682f0 0x7f5c3806a7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5c580b8400 tx=0x7f5c40009040 comp rx=0 tx=0).stop 2023-11-28T13:15:22.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.873+0000 7f5c5cfff700 1 -- 172.21.15.156:0/1173255181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c44003a00 msgr2=0x7f5c580c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:22.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.873+0000 7f5c5cfff700 1 --2- 172.21.15.156:0/1173255181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c44003a00 0x7f5c580c8250 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400c0e0 tx=0x7f5c480047c0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.873+0000 7f5c5cfff700 1 -- 172.21.15.156:0/1173255181 shutdown_connections 2023-11-28T13:15:22.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.873+0000 7f5c5cfff700 1 --2- 172.21.15.156:0/1173255181 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5c380682f0 0x7f5c3806a7a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.873+0000 7f5c5cfff700 1 --2- 172.21.15.156:0/1173255181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c58149fe0 0x7f5c5814a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.873+0000 7f5c5cfff700 1 --2- 172.21.15.156:0/1173255181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c44003a00 0x7f5c580c8250 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.873+0000 7f5c5cfff700 1 --2- 172.21.15.156:0/1173255181 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c581e78d0 0x7f5c5815e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:22.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.873+0000 7f5c5cfff700 1 -- 172.21.15.156:0/1173255181 >> 172.21.15.156:0/1173255181 conn(0x7f5c58072600 msgr2=0x7f5c5815bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:22.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.873+0000 7f5c5cfff700 1 -- 172.21.15.156:0/1173255181 shutdown_connections 2023-11-28T13:15:22.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:22.873+0000 7f5c5cfff700 1 -- 172.21.15.156:0/1173255181 wait complete. 2023-11-28T13:15:22.893 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:15:23.091 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.090+0000 7f1bb09ff700 1 Processor -- start 2023-11-28T13:15:23.092 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.090+0000 7f1bb09ff700 1 -- start start 2023-11-28T13:15:23.093 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.091+0000 7f1bb09ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1bac15c9b0 0x7f1bac0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:23.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.091+0000 7f1bb09ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bac161340 0x7f1bac0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:23.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.091+0000 7f1bb09ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bac0d1950 con 0x7f1bac0d2320 2023-11-28T13:15:23.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.091+0000 7f1bb09ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bac0b5800 con 0x7f1bac15c9b0 2023-11-28T13:15:23.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.091+0000 7f1bb09ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac0b5520 con 0x7f1bac161340 2023-11-28T13:15:23.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.091+0000 7f1baa59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1bac15c9b0 0x7f1bac0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:23.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.091+0000 7f1baa59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1bac15c9b0 0x7f1bac0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50832/0 (socket says 172.21.15.156:50832) 2023-11-28T13:15:23.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.092+0000 7f1baa59c700 1 -- 172.21.15.156:0/2590935855 learned_addr learned my addr 172.21.15.156:0/2590935855 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:23.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.092+0000 7f1ba9d9b700 1 --2- 172.21.15.156:0/2590935855 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bac161340 0x7f1bac0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:23.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.092+0000 7f1baa59c700 1 -- 172.21.15.156:0/2590935855 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bac161340 msgr2=0x7f1bac0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:23.094 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.092+0000 7f1baa59c700 1 --2- 172.21.15.156:0/2590935855 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bac161340 0x7f1bac0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.092+0000 7f1baa59c700 1 -- 172.21.15.156:0/2590935855 >> v1:172.21.15.186:6789/0 conn(0x7f1bac0d2320 legacy=0x7f1bac0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.092+0000 7f1baa59c700 1 -- 172.21.15.156:0/2590935855 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac0d1300 con 0x7f1bac15c9b0 2023-11-28T13:15:23.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.092+0000 7f1baa59c700 1 --2- 172.21.15.156:0/2590935855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bac15c9b0 0x7f1bac0b4bf0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0098f0 tx=0x7f1b9c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0fd1f7b5070550 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:23.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.092+0000 7f1ba959a700 1 -- 172.21.15.156:0/2590935855 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c00b8a0 con 0x7f1bac15c9b0 2023-11-28T13:15:23.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.092+0000 7f1ba959a700 1 -- 172.21.15.156:0/2590935855 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9c00ba00 con 0x7f1bac15c9b0 2023-11-28T13:15:23.095 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.092+0000 7f1ba959a700 1 -- 172.21.15.156:0/2590935855 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c00bd90 con 0x7f1bac15c9b0 2023-11-28T13:15:23.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.093+0000 7f1bb09ff700 1 -- 172.21.15.156:0/2590935855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bac15c9b0 msgr2=0x7f1bac0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.093+0000 7f1bb09ff700 1 --2- 172.21.15.156:0/2590935855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bac15c9b0 0x7f1bac0b4bf0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0098f0 tx=0x7f1b9c009c00 comp rx=0 tx=0).stop 2023-11-28T13:15:23.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.093+0000 7f1bb09ff700 1 -- 172.21.15.156:0/2590935855 shutdown_connections 2023-11-28T13:15:23.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.093+0000 7f1bb09ff700 1 --2- 172.21.15.156:0/2590935855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bac15c9b0 0x7f1bac0b4bf0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.093+0000 7f1bb09ff700 1 --2- 172.21.15.156:0/2590935855 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bac161340 0x7f1bac0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.093+0000 7f1bb09ff700 1 -- 172.21.15.156:0/2590935855 >> 172.21.15.156:0/2590935855 conn(0x7f1bac072600 msgr2=0x7f1bac0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:23.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.093+0000 7f1bb09ff700 1 -- 172.21.15.156:0/2590935855 shutdown_connections 2023-11-28T13:15:23.096 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.093+0000 7f1bb09ff700 1 -- 172.21.15.156:0/2590935855 wait complete. 2023-11-28T13:15:23.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.095+0000 7f1bb09ff700 1 Processor -- start 2023-11-28T13:15:23.097 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.095+0000 7f1bb09ff700 1 -- start start 2023-11-28T13:15:23.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.096+0000 7f1bb09ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bac0d2320 0x7f1bac150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:23.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.096+0000 7f1bb09ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bac15c9b0 0x7f1bac15f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:23.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.096+0000 7f1bb09ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bac161340 0x7f1bac15f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:23.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.096+0000 7f1bb09ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac1e8ac0 con 0x7f1bac161340 2023-11-28T13:15:23.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.096+0000 7f1bb09ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac1f6d80 con 0x7f1bac0d2320 2023-11-28T13:15:23.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.096+0000 7f1bb09ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac1f71a0 con 0x7f1bac15c9b0 2023-11-28T13:15:23.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.096+0000 7f1baad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bac161340 0x7f1bac15f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:23.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.096+0000 7f1baad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bac161340 0x7f1bac15f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50844/0 (socket says 172.21.15.156:50844) 2023-11-28T13:15:23.099 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.096+0000 7f1baad9d700 1 -- 172.21.15.156:0/1987419624 learned_addr learned my addr 172.21.15.156:0/1987419624 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:23.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.096+0000 7f1ba9d9b700 1 --2- 172.21.15.156:0/1987419624 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bac15c9b0 0x7f1bac15f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:23.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.096+0000 7f1baa59c700 1 --2- 172.21.15.156:0/1987419624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bac0d2320 0x7f1bac150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:23.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.097+0000 7f1baad9d700 1 -- 172.21.15.156:0/1987419624 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bac15c9b0 msgr2=0x7f1bac15f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.097+0000 7f1baad9d700 1 --2- 172.21.15.156:0/1987419624 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bac15c9b0 0x7f1bac15f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.097+0000 7f1baad9d700 1 -- 172.21.15.156:0/1987419624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bac0d2320 msgr2=0x7f1bac150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.097+0000 7f1baad9d700 1 --2- 172.21.15.156:0/1987419624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bac0d2320 0x7f1bac150320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.097+0000 7f1baad9d700 1 -- 172.21.15.156:0/1987419624 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c009550 con 0x7f1bac161340 2023-11-28T13:15:23.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.097+0000 7f1baad9d700 1 --2- 172.21.15.156:0/1987419624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bac161340 0x7f1bac15f820 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400cb50 tx=0x7f1ba400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:23.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.097+0000 7f1b9affd700 1 -- 172.21.15.156:0/1987419624 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4003c00 con 0x7f1bac161340 2023-11-28T13:15:23.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.097+0000 7f1b9affd700 1 -- 172.21.15.156:0/1987419624 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba4003d60 con 0x7f1bac161340 2023-11-28T13:15:23.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.097+0000 7f1bb09ff700 1 -- 172.21.15.156:0/1987419624 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bac0b5130 con 0x7f1bac161340 2023-11-28T13:15:23.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.097+0000 7f1b9affd700 1 -- 172.21.15.156:0/1987419624 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba400aa20 con 0x7f1bac161340 2023-11-28T13:15:23.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.097+0000 7f1bb09ff700 1 -- 172.21.15.156:0/1987419624 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bac0d15e0 con 0x7f1bac161340 2023-11-28T13:15:23.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.098+0000 7f1bb09ff700 1 -- 172.21.15.156:0/1987419624 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bac0b52c0 con 0x7f1bac161340 2023-11-28T13:15:23.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.099+0000 7f1b9affd700 1 -- 172.21.15.156:0/1987419624 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1ba4010050 con 0x7f1bac161340 2023-11-28T13:15:23.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.100+0000 7f1b9affd700 1 --2- 172.21.15.156:0/1987419624 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1b8c0682f0 0x7f1b8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:23.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.100+0000 7f1b9affd700 1 -- 172.21.15.156:0/1987419624 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f1ba4085ca0 con 0x7f1bac161340 2023-11-28T13:15:23.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.101+0000 7f1baa59c700 1 --2- 172.21.15.156:0/1987419624 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1b8c0682f0 0x7f1b8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:23.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.101+0000 7f1baa59c700 1 --2- 172.21.15.156:0/1987419624 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1b8c0682f0 0x7f1b8c06a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009c40 tx=0x7f1b9c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.104+0000 7f1b9affd700 1 -- 172.21.15.156:0/1987419624 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1ba4052380 con 0x7f1bac161340 2023-11-28T13:15:23.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.337+0000 7f1bb09ff700 1 -- 172.21.15.156:0/1987419624 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1bac150860 con 0x7f1bac161340 2023-11-28T13:15:23.339 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.338+0000 7f1b9affd700 1 -- 172.21.15.156:0/1987419624 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4658 (secure 0 0 0) 0x7f1bac150860 con 0x7f1bac161340 2023-11-28T13:15:23.339 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:15:23.339 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3942362276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3942362276},{"type":"v1","addr":"172.21.15.186:6835","nonce":3942362276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/722198303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":722198303},{"type":"v1","addr":"172.21.15.186:6837","nonce":722198303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5916,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3601975324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3601975324},{"type":"v1","addr":"172.21.15.156:6835","nonce":3601975324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:10:54.928601+0000","modified":"2023-11-28T13:11:11.455488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5919":{"gid":5919,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6834/3314379816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3314379816},{"type":"v1","addr":"172.21.15.156:6834","nonce":3314379816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:15:23.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.340+0000 7f1bb09ff700 1 -- 172.21.15.156:0/1987419624 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1b8c0682f0 msgr2=0x7f1b8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.340+0000 7f1bb09ff700 1 --2- 172.21.15.156:0/1987419624 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1b8c0682f0 0x7f1b8c06a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009c40 tx=0x7f1b9c008030 comp rx=0 tx=0).stop 2023-11-28T13:15:23.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.340+0000 7f1bb09ff700 1 -- 172.21.15.156:0/1987419624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bac161340 msgr2=0x7f1bac15f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.340+0000 7f1bb09ff700 1 --2- 172.21.15.156:0/1987419624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bac161340 0x7f1bac15f820 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400cb50 tx=0x7f1ba400ce60 comp rx=0 tx=0).stop 2023-11-28T13:15:23.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.340+0000 7f1bb09ff700 1 -- 172.21.15.156:0/1987419624 shutdown_connections 2023-11-28T13:15:23.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.340+0000 7f1bb09ff700 1 --2- 172.21.15.156:0/1987419624 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f1b8c0682f0 0x7f1b8c06a7a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.340+0000 7f1bb09ff700 1 --2- 172.21.15.156:0/1987419624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bac161340 0x7f1bac15f820 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.340+0000 7f1bb09ff700 1 --2- 172.21.15.156:0/1987419624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bac0d2320 0x7f1bac150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.340+0000 7f1bb09ff700 1 --2- 172.21.15.156:0/1987419624 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bac15c9b0 0x7f1bac15f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.340+0000 7f1bb09ff700 1 -- 172.21.15.156:0/1987419624 >> 172.21.15.156:0/1987419624 conn(0x7f1bac072600 msgr2=0x7f1bac06a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:23.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.340+0000 7f1bb09ff700 1 -- 172.21.15.156:0/1987419624 shutdown_connections 2023-11-28T13:15:23.342 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.340+0000 7f1bb09ff700 1 -- 172.21.15.156:0/1987419624 wait complete. 2023-11-28T13:15:23.343 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 32 2023-11-28T13:15:23.360 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:15:23.558 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.556+0000 7f9e01fff700 1 Processor -- start 2023-11-28T13:15:23.558 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.556+0000 7f9e01fff700 1 -- start start 2023-11-28T13:15:23.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.557+0000 7f9e01fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9dfc161340 0x7f9dfc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:23.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.557+0000 7f9e01fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc0d1950 con 0x7f9dfc15c9b0 2023-11-28T13:15:23.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.557+0000 7f9e01fff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc0b5520 con 0x7f9dfc0d2320 2023-11-28T13:15:23.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.557+0000 7f9e01fff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dfc0b5800 con 0x7f9dfc161340 2023-11-28T13:15:23.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.557+0000 7f9dfb7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f9dfc15c9b0 0x7f9dfc0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50804/0 (socket says 172.21.15.156:50804) 2023-11-28T13:15:23.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.557+0000 7f9dfb7fe700 1 -- 172.21.15.156:0/4082318695 learned_addr learned my addr 172.21.15.156:0/4082318695 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:23.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.558+0000 7f9dfaffd700 1 --2- 172.21.15.156:0/4082318695 >> v2:172.21.15.186:3300/0 conn(0x7f9dfc161340 0x7f9dfc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:23.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.558+0000 7f9dfa7fc700 1 -- 172.21.15.156:0/4082318695 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965882477 0 0) 0x7f9dfc0d1950 con 0x7f9dfc15c9b0 2023-11-28T13:15:23.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.558+0000 7f9dfa7fc700 1 -- 172.21.15.156:0/4082318695 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9de40035b0 con 0x7f9dfc15c9b0 2023-11-28T13:15:23.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.558+0000 7f9dfa7fc700 1 -- 172.21.15.156:0/4082318695 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3261616697 0 0) 0x7f9de40035b0 con 0x7f9dfc15c9b0 2023-11-28T13:15:23.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.558+0000 7f9dfaffd700 1 -- 172.21.15.156:0/4082318695 >> v1:172.21.15.186:6789/0 conn(0x7f9dfc0d2320 legacy=0x7f9dfc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.558+0000 7f9dfaffd700 1 -- 172.21.15.156:0/4082318695 >> v1:172.21.15.156:6789/0 conn(0x7f9dfc15c9b0 legacy=0x7f9dfc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.558+0000 7f9dfaffd700 1 -- 172.21.15.156:0/4082318695 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc0d1300 con 0x7f9dfc161340 2023-11-28T13:15:23.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.558+0000 7f9dfaffd700 1 --2- 172.21.15.156:0/4082318695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9dfc161340 0x7f9dfc0d1de0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009840 tx=0x7f9df0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b15daf1e35e80d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:23.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.559+0000 7f9dfa7fc700 1 -- 172.21.15.156:0/4082318695 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec001670 con 0x7f9dfc161340 2023-11-28T13:15:23.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.559+0000 7f9dfa7fc700 1 -- 172.21.15.156:0/4082318695 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df0004030 con 0x7f9dfc161340 2023-11-28T13:15:23.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.559+0000 7f9dfa7fc700 1 -- 172.21.15.156:0/4082318695 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df00043c0 con 0x7f9dfc161340 2023-11-28T13:15:23.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.559+0000 7f9e01fff700 1 -- 172.21.15.156:0/4082318695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9dfc161340 msgr2=0x7f9dfc0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.559+0000 7f9e01fff700 1 --2- 172.21.15.156:0/4082318695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9dfc161340 0x7f9dfc0d1de0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009840 tx=0x7f9df0009b50 comp rx=0 tx=0).stop 2023-11-28T13:15:23.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.559+0000 7f9e01fff700 1 -- 172.21.15.156:0/4082318695 shutdown_connections 2023-11-28T13:15:23.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.559+0000 7f9e01fff700 1 --2- 172.21.15.156:0/4082318695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9dfc161340 0x7f9dfc0d1de0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.559+0000 7f9e01fff700 1 -- 172.21.15.156:0/4082318695 >> 172.21.15.156:0/4082318695 conn(0x7f9dfc072600 msgr2=0x7f9dfc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:23.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.559+0000 7f9e01fff700 1 -- 172.21.15.156:0/4082318695 shutdown_connections 2023-11-28T13:15:23.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.559+0000 7f9e01fff700 1 -- 172.21.15.156:0/4082318695 wait complete. 2023-11-28T13:15:23.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.561+0000 7f9e01fff700 1 Processor -- start 2023-11-28T13:15:23.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.561+0000 7f9e01fff700 1 -- start start 2023-11-28T13:15:23.570 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.562+0000 7f9e01fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dfc0d2320 0x7f9dfc0b83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:23.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.562+0000 7f9e01fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9dfc15c9b0 0x7f9dfc14d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:23.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.562+0000 7f9e01fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9dfc161340 0x7f9dfc15a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:23.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.562+0000 7f9e01fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0cab70 con 0x7f9dfc0d2320 2023-11-28T13:15:23.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.562+0000 7f9e01fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc1f2af0 con 0x7f9dfc161340 2023-11-28T13:15:23.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.562+0000 7f9e01fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0c9270 con 0x7f9dfc15c9b0 2023-11-28T13:15:23.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.562+0000 7f9dfb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dfc0d2320 0x7f9dfc0b83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:23.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.562+0000 7f9dfb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dfc0d2320 0x7f9dfc0b83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50858/0 (socket says 172.21.15.156:50858) 2023-11-28T13:15:23.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.562+0000 7f9dfb7fe700 1 -- 172.21.15.156:0/3966243456 learned_addr learned my addr 172.21.15.156:0/3966243456 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:23.571 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.562+0000 7f9dfaffd700 1 --2- 172.21.15.156:0/3966243456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9dfc15c9b0 0x7f9dfc14d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:23.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.563+0000 7f9dfbfff700 1 --2- 172.21.15.156:0/3966243456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9dfc161340 0x7f9dfc15a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:23.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.563+0000 7f9dfb7fe700 1 -- 172.21.15.156:0/3966243456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9dfc15c9b0 msgr2=0x7f9dfc14d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.563+0000 7f9dfb7fe700 1 --2- 172.21.15.156:0/3966243456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9dfc15c9b0 0x7f9dfc14d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.563+0000 7f9dfb7fe700 1 -- 172.21.15.156:0/3966243456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9dfc161340 msgr2=0x7f9dfc15a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.563+0000 7f9dfb7fe700 1 --2- 172.21.15.156:0/3966243456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9dfc161340 0x7f9dfc15a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.563+0000 7f9dfb7fe700 1 -- 172.21.15.156:0/3966243456 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df00094a0 con 0x7f9dfc0d2320 2023-11-28T13:15:23.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.563+0000 7f9dfbfff700 1 --2- 172.21.15.156:0/3966243456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9dfc161340 0x7f9dfc15a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:15:23.572 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.563+0000 7f9dfb7fe700 1 --2- 172.21.15.156:0/3966243456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dfc0d2320 0x7f9dfc0b83e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00c700 tx=0x7f9dec00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:23.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.563+0000 7f9de3fff700 1 -- 172.21.15.156:0/3966243456 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec004c80 con 0x7f9dfc0d2320 2023-11-28T13:15:23.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.563+0000 7f9de3fff700 1 -- 172.21.15.156:0/3966243456 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec004e00 con 0x7f9dfc0d2320 2023-11-28T13:15:23.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.563+0000 7f9e01fff700 1 -- 172.21.15.156:0/3966243456 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc15cd80 con 0x7f9dfc0d2320 2023-11-28T13:15:23.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.563+0000 7f9de3fff700 1 -- 172.21.15.156:0/3966243456 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec007310 con 0x7f9dfc0d2320 2023-11-28T13:15:23.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.563+0000 7f9e01fff700 1 -- 172.21.15.156:0/3966243456 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc0d15e0 con 0x7f9dfc0d2320 2023-11-28T13:15:23.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.564+0000 7f9e01fff700 1 -- 172.21.15.156:0/3966243456 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dfc15cf10 con 0x7f9dfc0d2320 2023-11-28T13:15:23.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.565+0000 7f9de3fff700 1 -- 172.21.15.156:0/3966243456 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9dec01a020 con 0x7f9dfc0d2320 2023-11-28T13:15:23.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.566+0000 7f9de3fff700 1 --2- 172.21.15.156:0/3966243456 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9de80682f0 0x7f9de806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:23.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.566+0000 7f9de3fff700 1 -- 172.21.15.156:0/3966243456 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f9dec085b30 con 0x7f9dfc0d2320 2023-11-28T13:15:23.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.567+0000 7f9dfaffd700 1 --2- 172.21.15.156:0/3966243456 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9de80682f0 0x7f9de806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:23.573 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.567+0000 7f9dfaffd700 1 --2- 172.21.15.156:0/3966243456 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9de80682f0 0x7f9de806a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009b90 tx=0x7f9df0008000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:23.574 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.570+0000 7f9de3fff700 1 -- 172.21.15.156:0/3966243456 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9dec052210 con 0x7f9dfc0d2320 2023-11-28T13:15:23.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.764+0000 7f9e01fff700 1 -- 172.21.15.156:0/3966243456 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9dfc0745c0 con 0x7f9dfc0d2320 2023-11-28T13:15:23.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.765+0000 7f9de3fff700 1 -- 172.21.15.156:0/3966243456 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+18152 (secure 0 0 0) 0x7f9dec055830 con 0x7f9dfc0d2320 2023-11-28T13:15:23.767 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:15:23.767 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":61,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:11:52.553240+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:10:52.732399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T13:10:53.912753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.11","mappings":[{"from":7,"to":2}]},{"pgid":"10.19","mappings":[{"from":1,"to":3}]},{"pgid":"10.1b","mappings":[{"from":4,"to":5}]},{"pgid":"10.1f","mappings":[{"from":7,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:15:23.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.767+0000 7f9e01fff700 1 -- 172.21.15.156:0/3966243456 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9de80682f0 msgr2=0x7f9de806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.767+0000 7f9e01fff700 1 --2- 172.21.15.156:0/3966243456 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9de80682f0 0x7f9de806a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009b90 tx=0x7f9df0008000 comp rx=0 tx=0).stop 2023-11-28T13:15:23.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.767+0000 7f9e01fff700 1 -- 172.21.15.156:0/3966243456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dfc0d2320 msgr2=0x7f9dfc0b83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.767+0000 7f9e01fff700 1 --2- 172.21.15.156:0/3966243456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dfc0d2320 0x7f9dfc0b83e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00c700 tx=0x7f9dec00ca10 comp rx=0 tx=0).stop 2023-11-28T13:15:23.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.767+0000 7f9e01fff700 1 -- 172.21.15.156:0/3966243456 shutdown_connections 2023-11-28T13:15:23.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.767+0000 7f9e01fff700 1 --2- 172.21.15.156:0/3966243456 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9de80682f0 0x7f9de806a7a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.767+0000 7f9e01fff700 1 --2- 172.21.15.156:0/3966243456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9dfc161340 0x7f9dfc15a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.767+0000 7f9e01fff700 1 --2- 172.21.15.156:0/3966243456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dfc0d2320 0x7f9dfc0b83e0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.767+0000 7f9e01fff700 1 --2- 172.21.15.156:0/3966243456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9dfc15c9b0 0x7f9dfc14d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.767+0000 7f9e01fff700 1 -- 172.21.15.156:0/3966243456 >> 172.21.15.156:0/3966243456 conn(0x7f9dfc072600 msgr2=0x7f9dfc15f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:23.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.768+0000 7f9e01fff700 1 -- 172.21.15.156:0/3966243456 shutdown_connections 2023-11-28T13:15:23.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.768+0000 7f9e01fff700 1 -- 172.21.15.156:0/3966243456 wait complete. 2023-11-28T13:15:23.788 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T13:15:23.788 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:15:23.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.982+0000 7fbd1cdff700 1 Processor -- start 2023-11-28T13:15:23.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.983+0000 7fbd1cdff700 1 -- start start 2023-11-28T13:15:23.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.984+0000 7fbd1cdff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbd18161340 0x7fbd180d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:23.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.984+0000 7fbd1cdff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd180d1950 con 0x7fbd180d2320 2023-11-28T13:15:23.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.984+0000 7fbd1cdff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd180b5520 con 0x7fbd1815c9b0 2023-11-28T13:15:23.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.984+0000 7fbd1cdff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd180b5800 con 0x7fbd18161340 2023-11-28T13:15:23.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.984+0000 7fbd16d9d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fbd180d2320 0x7fbd180cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50814/0 (socket says 172.21.15.156:50814) 2023-11-28T13:15:23.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.984+0000 7fbd16d9d700 1 -- 172.21.15.156:0/1251296250 learned_addr learned my addr 172.21.15.156:0/1251296250 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:23.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.984+0000 7fbd0ffff700 1 --2- 172.21.15.156:0/1251296250 >> v2:172.21.15.186:3300/0 conn(0x7fbd18161340 0x7fbd180d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:23.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.985+0000 7fbd15d9b700 1 -- 172.21.15.156:0/1251296250 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977737664 0 0) 0x7fbd180d1950 con 0x7fbd180d2320 2023-11-28T13:15:23.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.985+0000 7fbd15d9b700 1 -- 172.21.15.156:0/1251296250 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd040035b0 con 0x7fbd180d2320 2023-11-28T13:15:23.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.985+0000 7fbd15d9b700 1 -- 172.21.15.156:0/1251296250 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105374155 0 0) 0x7fbd180b5520 con 0x7fbd1815c9b0 2023-11-28T13:15:23.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.985+0000 7fbd15d9b700 1 -- 172.21.15.156:0/1251296250 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd180d1950 con 0x7fbd1815c9b0 2023-11-28T13:15:23.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.985+0000 7fbd0ffff700 1 -- 172.21.15.156:0/1251296250 >> v1:172.21.15.186:6789/0 conn(0x7fbd1815c9b0 legacy=0x7fbd180b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.985+0000 7fbd15d9b700 1 -- 172.21.15.156:0/1251296250 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2134065521 0 0) 0x7fbd040035b0 con 0x7fbd180d2320 2023-11-28T13:15:23.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.985+0000 7fbd0ffff700 1 -- 172.21.15.156:0/1251296250 >> v1:172.21.15.156:6789/0 conn(0x7fbd180d2320 legacy=0x7fbd180cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.985+0000 7fbd0ffff700 1 -- 172.21.15.156:0/1251296250 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd180d1300 con 0x7fbd18161340 2023-11-28T13:15:23.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.985+0000 7fbd0ffff700 1 --2- 172.21.15.156:0/1251296250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd18161340 0x7fbd180d1de0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009840 tx=0x7fbd00009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea84fa1b56702125 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:23.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.986+0000 7fbd15d9b700 1 -- 172.21.15.156:0/1251296250 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd100019f0 con 0x7fbd18161340 2023-11-28T13:15:23.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.986+0000 7fbd15d9b700 1 -- 172.21.15.156:0/1251296250 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd00004030 con 0x7fbd18161340 2023-11-28T13:15:23.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.986+0000 7fbd15d9b700 1 -- 172.21.15.156:0/1251296250 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd000043c0 con 0x7fbd18161340 2023-11-28T13:15:23.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.986+0000 7fbd1cdff700 1 -- 172.21.15.156:0/1251296250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd18161340 msgr2=0x7fbd180d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.986+0000 7fbd1cdff700 1 --2- 172.21.15.156:0/1251296250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd18161340 0x7fbd180d1de0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009840 tx=0x7fbd00009b50 comp rx=0 tx=0).stop 2023-11-28T13:15:23.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.986+0000 7fbd1cdff700 1 -- 172.21.15.156:0/1251296250 shutdown_connections 2023-11-28T13:15:23.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.986+0000 7fbd1cdff700 1 --2- 172.21.15.156:0/1251296250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd18161340 0x7fbd180d1de0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.986+0000 7fbd1cdff700 1 -- 172.21.15.156:0/1251296250 >> 172.21.15.156:0/1251296250 conn(0x7fbd18072600 msgr2=0x7fbd180cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:23.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.986+0000 7fbd1cdff700 1 -- 172.21.15.156:0/1251296250 shutdown_connections 2023-11-28T13:15:23.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.986+0000 7fbd1cdff700 1 -- 172.21.15.156:0/1251296250 wait complete. 2023-11-28T13:15:23.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.988+0000 7fbd1cdff700 1 Processor -- start 2023-11-28T13:15:23.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.988+0000 7fbd1cdff700 1 -- start start 2023-11-28T13:15:23.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.989+0000 7fbd1cdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd180d2320 0x7fbd181e8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:23.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.989+0000 7fbd1cdff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd1815c9b0 0x7fbd181f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:23.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.989+0000 7fbd1cdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd18161340 0x7fbd1806ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:23.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.989+0000 7fbd1cdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180c8000 con 0x7fbd18161340 2023-11-28T13:15:23.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.989+0000 7fbd1cdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd180c7cf0 con 0x7fbd180d2320 2023-11-28T13:15:23.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.989+0000 7fbd1cdff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd181f6720 con 0x7fbd1815c9b0 2023-11-28T13:15:23.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.989+0000 7fbd16d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd18161340 0x7fbd1806ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:23.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.989+0000 7fbd16d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd18161340 0x7fbd1806ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50866/0 (socket says 172.21.15.156:50866) 2023-11-28T13:15:23.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.989+0000 7fbd16d9d700 1 -- 172.21.15.156:0/718086125 learned_addr learned my addr 172.21.15.156:0/718086125 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:23.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.990+0000 7fbd1659c700 1 --2- 172.21.15.156:0/718086125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd180d2320 0x7fbd181e8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:23.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.990+0000 7fbd0ffff700 1 --2- 172.21.15.156:0/718086125 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd1815c9b0 0x7fbd181f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:23.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.990+0000 7fbd16d9d700 1 -- 172.21.15.156:0/718086125 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd1815c9b0 msgr2=0x7fbd181f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.990+0000 7fbd16d9d700 1 --2- 172.21.15.156:0/718086125 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd1815c9b0 0x7fbd181f61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.990+0000 7fbd16d9d700 1 -- 172.21.15.156:0/718086125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd180d2320 msgr2=0x7fbd181e8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:23.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.990+0000 7fbd16d9d700 1 --2- 172.21.15.156:0/718086125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd180d2320 0x7fbd181e8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:23.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.990+0000 7fbd16d9d700 1 -- 172.21.15.156:0/718086125 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd000094a0 con 0x7fbd18161340 2023-11-28T13:15:23.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.990+0000 7fbd16d9d700 1 --2- 172.21.15.156:0/718086125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd18161340 0x7fbd1806ab70 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000cc10 tx=0x7fbd1000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:23.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.990+0000 7fbd0effd700 1 -- 172.21.15.156:0/718086125 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1000abf0 con 0x7fbd18161340 2023-11-28T13:15:23.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.990+0000 7fbd0effd700 1 -- 172.21.15.156:0/718086125 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1000ad50 con 0x7fbd18161340 2023-11-28T13:15:23.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.990+0000 7fbd0effd700 1 -- 172.21.15.156:0/718086125 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd10007310 con 0x7fbd18161340 2023-11-28T13:15:23.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.990+0000 7fbd1cdff700 1 -- 172.21.15.156:0/718086125 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1815cd80 con 0x7fbd18161340 2023-11-28T13:15:23.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.990+0000 7fbd1cdff700 1 -- 172.21.15.156:0/718086125 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd180d15e0 con 0x7fbd18161340 2023-11-28T13:15:23.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.992+0000 7fbd1cdff700 1 -- 172.21.15.156:0/718086125 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd1815cf10 con 0x7fbd18161340 2023-11-28T13:15:23.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.993+0000 7fbd0effd700 1 -- 172.21.15.156:0/718086125 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbd10010070 con 0x7fbd18161340 2023-11-28T13:15:23.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.993+0000 7fbd0effd700 1 --2- 172.21.15.156:0/718086125 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcf80682f0 0x7fbcf806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:23.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.993+0000 7fbd0effd700 1 -- 172.21.15.156:0/718086125 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fbd10085860 con 0x7fbd18161340 2023-11-28T13:15:23.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.994+0000 7fbd1659c700 1 --2- 172.21.15.156:0/718086125 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcf80682f0 0x7fbcf806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:23.997 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.994+0000 7fbd1659c700 1 --2- 172.21.15.156:0/718086125 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcf80682f0 0x7fbcf806a7a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbd080011e0 tx=0x7fbd08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:23.998 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:23.997+0000 7fbd0effd700 1 -- 172.21.15.156:0/718086125 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbd10051f40 con 0x7fbd18161340 2023-11-28T13:15:24.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.232+0000 7fbd1cdff700 1 -- 172.21.15.156:0/718086125 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd180c8140 con 0x7fbd18161340 2023-11-28T13:15:24.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.233+0000 7fbd0effd700 1 -- 172.21.15.156:0/718086125 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4658 (secure 0 0 0) 0x7fbd10055560 con 0x7fbd18161340 2023-11-28T13:15:24.234 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:15:24.234 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3942362276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3942362276},{"type":"v1","addr":"172.21.15.186:6835","nonce":3942362276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/722198303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":722198303},{"type":"v1","addr":"172.21.15.186:6837","nonce":722198303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5916,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3601975324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3601975324},{"type":"v1","addr":"172.21.15.156:6835","nonce":3601975324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:10:54.928601+0000","modified":"2023-11-28T13:11:11.455488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5919":{"gid":5919,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6834/3314379816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3314379816},{"type":"v1","addr":"172.21.15.156:6834","nonce":3314379816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:15:24.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.235+0000 7fbd1cdff700 1 -- 172.21.15.156:0/718086125 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcf80682f0 msgr2=0x7fbcf806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:24.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.235+0000 7fbd1cdff700 1 --2- 172.21.15.156:0/718086125 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcf80682f0 0x7fbcf806a7a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbd080011e0 tx=0x7fbd08009040 comp rx=0 tx=0).stop 2023-11-28T13:15:24.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.235+0000 7fbd1cdff700 1 -- 172.21.15.156:0/718086125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd18161340 msgr2=0x7fbd1806ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:24.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.235+0000 7fbd1cdff700 1 --2- 172.21.15.156:0/718086125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd18161340 0x7fbd1806ab70 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000cc10 tx=0x7fbd1000a060 comp rx=0 tx=0).stop 2023-11-28T13:15:24.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.235+0000 7fbd1cdff700 1 -- 172.21.15.156:0/718086125 shutdown_connections 2023-11-28T13:15:24.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.235+0000 7fbd1cdff700 1 --2- 172.21.15.156:0/718086125 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbcf80682f0 0x7fbcf806a7a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.235+0000 7fbd1cdff700 1 --2- 172.21.15.156:0/718086125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd18161340 0x7fbd1806ab70 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.235+0000 7fbd1cdff700 1 --2- 172.21.15.156:0/718086125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd180d2320 0x7fbd181e8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.235+0000 7fbd1cdff700 1 --2- 172.21.15.156:0/718086125 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd1815c9b0 0x7fbd181f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.235+0000 7fbd1cdff700 1 -- 172.21.15.156:0/718086125 >> 172.21.15.156:0/718086125 conn(0x7fbd18072600 msgr2=0x7fbd1815fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:24.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.235+0000 7fbd1cdff700 1 -- 172.21.15.156:0/718086125 shutdown_connections 2023-11-28T13:15:24.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.235+0000 7fbd1cdff700 1 -- 172.21.15.156:0/718086125 wait complete. 2023-11-28T13:15:24.238 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 32 2023-11-28T13:15:24.255 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:15:24.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.449+0000 7f9f467ff700 1 Processor -- start 2023-11-28T13:15:24.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.449+0000 7f9f467ff700 1 -- start start 2023-11-28T13:15:24.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.450+0000 7f9f467ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f4015c9b0 0x7f9f400b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:24.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.450+0000 7f9f467ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9f40161340 0x7f9f400d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:24.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.450+0000 7f9f467ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9f400d2320 0x7f9f400cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:24.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.450+0000 7f9f467ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f400b5800 con 0x7f9f400d2320 2023-11-28T13:15:24.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.450+0000 7f9f467ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f400d1950 con 0x7f9f40161340 2023-11-28T13:15:24.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.450+0000 7f9f467ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400b5520 con 0x7f9f4015c9b0 2023-11-28T13:15:24.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.451+0000 7f9f44d9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9f400d2320 0x7f9f400cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:24.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.451+0000 7f9f44d9c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9f400d2320 0x7f9f400cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50880/0 (socket says 172.21.15.156:50880) 2023-11-28T13:15:24.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.451+0000 7f9f44d9c700 1 -- 172.21.15.156:0/3030211826 learned_addr learned my addr 172.21.15.156:0/3030211826 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:24.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.451+0000 7f9f3ffff700 1 --2- 172.21.15.156:0/3030211826 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f4015c9b0 0x7f9f400b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:24.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.451+0000 7f9f3f7fe700 1 --2- 172.21.15.156:0/3030211826 >> v2:172.21.15.186:3300/0 conn(0x7f9f40161340 0x7f9f400d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:24.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.451+0000 7f9f44d9c700 1 -- 172.21.15.156:0/3030211826 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f4015c9b0 msgr2=0x7f9f400b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:24.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.451+0000 7f9f44d9c700 1 --2- 172.21.15.156:0/3030211826 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f4015c9b0 0x7f9f400b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.451+0000 7f9f44d9c700 1 -- 172.21.15.156:0/3030211826 >> v2:172.21.15.186:3300/0 conn(0x7f9f40161340 msgr2=0x7f9f400d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:24.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.451+0000 7f9f44d9c700 1 --2- 172.21.15.156:0/3030211826 >> v2:172.21.15.186:3300/0 conn(0x7f9f40161340 0x7f9f400d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.451+0000 7f9f44d9c700 1 -- 172.21.15.156:0/3030211826 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f400d1300 con 0x7f9f400d2320 2023-11-28T13:15:24.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.451+0000 7f9f3f7fe700 1 --2- 172.21.15.156:0/3030211826 >> v2:172.21.15.186:3300/0 conn(0x7f9f40161340 0x7f9f400d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:15:24.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.451+0000 7f9f44d9c700 1 --2- 172.21.15.156:0/3030211826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f400d2320 0x7f9f400cd7c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9f4006bc00 tx=0x7f9f3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f7880e8630182b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:24.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.452+0000 7f9f3effd700 1 -- 172.21.15.156:0/3030211826 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3800e070 con 0x7f9f400d2320 2023-11-28T13:15:24.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.452+0000 7f9f3effd700 1 -- 172.21.15.156:0/3030211826 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3800b9c0 con 0x7f9f400d2320 2023-11-28T13:15:24.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.452+0000 7f9f3effd700 1 -- 172.21.15.156:0/3030211826 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3800bd70 con 0x7f9f400d2320 2023-11-28T13:15:24.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.452+0000 7f9f467ff700 1 -- 172.21.15.156:0/3030211826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f400d2320 msgr2=0x7f9f400cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:24.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.452+0000 7f9f467ff700 1 --2- 172.21.15.156:0/3030211826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f400d2320 0x7f9f400cd7c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9f4006bc00 tx=0x7f9f3800b330 comp rx=0 tx=0).stop 2023-11-28T13:15:24.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.452+0000 7f9f467ff700 1 -- 172.21.15.156:0/3030211826 shutdown_connections 2023-11-28T13:15:24.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.452+0000 7f9f467ff700 1 --2- 172.21.15.156:0/3030211826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f400d2320 0x7f9f400cd7c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.452+0000 7f9f467ff700 1 --2- 172.21.15.156:0/3030211826 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f4015c9b0 0x7f9f400b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.452+0000 7f9f467ff700 1 --2- 172.21.15.156:0/3030211826 >> v2:172.21.15.186:3300/0 conn(0x7f9f40161340 0x7f9f400d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.452+0000 7f9f467ff700 1 -- 172.21.15.156:0/3030211826 >> 172.21.15.156:0/3030211826 conn(0x7f9f40072600 msgr2=0x7f9f400cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:24.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.452+0000 7f9f467ff700 1 -- 172.21.15.156:0/3030211826 shutdown_connections 2023-11-28T13:15:24.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.452+0000 7f9f467ff700 1 -- 172.21.15.156:0/3030211826 wait complete. 2023-11-28T13:15:24.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.454+0000 7f9f467ff700 1 Processor -- start 2023-11-28T13:15:24.463 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.454+0000 7f9f467ff700 1 -- start start 2023-11-28T13:15:24.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.455+0000 7f9f467ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f400d2320 0x7f9f401efa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:24.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.455+0000 7f9f467ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f4015c9b0 0x7f9f401f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:24.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.455+0000 7f9f467ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f40161340 0x7f9f400c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:24.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.455+0000 7f9f467ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400c69b0 con 0x7f9f40161340 2023-11-28T13:15:24.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.455+0000 7f9f467ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400c7ac0 con 0x7f9f400d2320 2023-11-28T13:15:24.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.455+0000 7f9f467ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f40158a60 con 0x7f9f4015c9b0 2023-11-28T13:15:24.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.455+0000 7f9f44d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f40161340 0x7f9f400c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:24.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.455+0000 7f9f44d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f40161340 0x7f9f400c7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50886/0 (socket says 172.21.15.156:50886) 2023-11-28T13:15:24.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.455+0000 7f9f44d9c700 1 -- 172.21.15.156:0/175038548 learned_addr learned my addr 172.21.15.156:0/175038548 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:24.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.455+0000 7f9f3ffff700 1 --2- 172.21.15.156:0/175038548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f400d2320 0x7f9f401efa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:24.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.455+0000 7f9f3f7fe700 1 --2- 172.21.15.156:0/175038548 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f4015c9b0 0x7f9f401f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:24.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.456+0000 7f9f44d9c700 1 -- 172.21.15.156:0/175038548 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f4015c9b0 msgr2=0x7f9f401f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:24.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.456+0000 7f9f44d9c700 1 --2- 172.21.15.156:0/175038548 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f4015c9b0 0x7f9f401f2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.456+0000 7f9f44d9c700 1 -- 172.21.15.156:0/175038548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f400d2320 msgr2=0x7f9f401efa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:24.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.456+0000 7f9f44d9c700 1 --2- 172.21.15.156:0/175038548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f400d2320 0x7f9f401efa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.456+0000 7f9f44d9c700 1 -- 172.21.15.156:0/175038548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f38009b30 con 0x7f9f40161340 2023-11-28T13:15:24.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.456+0000 7f9f44d9c700 1 --2- 172.21.15.156:0/175038548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f40161340 0x7f9f400c7ee0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9f400675d0 tx=0x7f9f38004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:24.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.456+0000 7f9f3cff9700 1 -- 172.21.15.156:0/175038548 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f380099b0 con 0x7f9f40161340 2023-11-28T13:15:24.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.456+0000 7f9f3cff9700 1 -- 172.21.15.156:0/175038548 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f38004250 con 0x7f9f40161340 2023-11-28T13:15:24.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.456+0000 7f9f467ff700 1 -- 172.21.15.156:0/175038548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f400b5130 con 0x7f9f40161340 2023-11-28T13:15:24.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.456+0000 7f9f3cff9700 1 -- 172.21.15.156:0/175038548 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f38004600 con 0x7f9f40161340 2023-11-28T13:15:24.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.456+0000 7f9f467ff700 1 -- 172.21.15.156:0/175038548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f400d15e0 con 0x7f9f40161340 2023-11-28T13:15:24.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.457+0000 7f9f467ff700 1 -- 172.21.15.156:0/175038548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f400b52c0 con 0x7f9f40161340 2023-11-28T13:15:24.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.458+0000 7f9f3cff9700 1 -- 172.21.15.156:0/175038548 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9f3802b070 con 0x7f9f40161340 2023-11-28T13:15:24.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.459+0000 7f9f3cff9700 1 --2- 172.21.15.156:0/175038548 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9f2c0682f0 0x7f9f2c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:24.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.459+0000 7f9f3cff9700 1 -- 172.21.15.156:0/175038548 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f9f38086fe0 con 0x7f9f40161340 2023-11-28T13:15:24.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.460+0000 7f9f3ffff700 1 --2- 172.21.15.156:0/175038548 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9f2c0682f0 0x7f9f2c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:24.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.460+0000 7f9f3ffff700 1 --2- 172.21.15.156:0/175038548 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9f2c0682f0 0x7f9f2c06a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9f30006b20 tx=0x7f9f30008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:24.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.463+0000 7f9f3cff9700 1 -- 172.21.15.156:0/175038548 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9f380537b0 con 0x7f9f40161340 2023-11-28T13:15:24.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.701+0000 7f9f467ff700 1 -- 172.21.15.156:0/175038548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f400c7c00 con 0x7f9f40161340 2023-11-28T13:15:24.703 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.702+0000 7f9f3cff9700 1 -- 172.21.15.156:0/175038548 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4658 (secure 0 0 0) 0x7f9f400c7c00 con 0x7f9f40161340 2023-11-28T13:15:24.704 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:15:24.704 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3942362276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3942362276},{"type":"v1","addr":"172.21.15.186:6835","nonce":3942362276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/722198303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":722198303},{"type":"v1","addr":"172.21.15.186:6837","nonce":722198303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5916,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3601975324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3601975324},{"type":"v1","addr":"172.21.15.156:6835","nonce":3601975324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:10:54.928601+0000","modified":"2023-11-28T13:11:11.455488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5919":{"gid":5919,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6834/3314379816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3314379816},{"type":"v1","addr":"172.21.15.156:6834","nonce":3314379816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:15:24.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.704+0000 7f9f467ff700 1 -- 172.21.15.156:0/175038548 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9f2c0682f0 msgr2=0x7f9f2c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:24.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.704+0000 7f9f467ff700 1 --2- 172.21.15.156:0/175038548 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9f2c0682f0 0x7f9f2c06a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9f30006b20 tx=0x7f9f30008040 comp rx=0 tx=0).stop 2023-11-28T13:15:24.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.704+0000 7f9f467ff700 1 -- 172.21.15.156:0/175038548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f40161340 msgr2=0x7f9f400c7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:24.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.704+0000 7f9f467ff700 1 --2- 172.21.15.156:0/175038548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f40161340 0x7f9f400c7ee0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9f400675d0 tx=0x7f9f38004ad0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.704+0000 7f9f467ff700 1 -- 172.21.15.156:0/175038548 shutdown_connections 2023-11-28T13:15:24.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.704+0000 7f9f467ff700 1 --2- 172.21.15.156:0/175038548 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9f2c0682f0 0x7f9f2c06a7a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.704+0000 7f9f467ff700 1 --2- 172.21.15.156:0/175038548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f40161340 0x7f9f400c7ee0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.704+0000 7f9f467ff700 1 --2- 172.21.15.156:0/175038548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f400d2320 0x7f9f401efa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.704+0000 7f9f467ff700 1 --2- 172.21.15.156:0/175038548 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9f4015c9b0 0x7f9f401f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.704+0000 7f9f467ff700 1 -- 172.21.15.156:0/175038548 >> 172.21.15.156:0/175038548 conn(0x7f9f40072600 msgr2=0x7f9f400b8900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:24.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.705+0000 7f9f467ff700 1 -- 172.21.15.156:0/175038548 shutdown_connections 2023-11-28T13:15:24.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.705+0000 7f9f467ff700 1 -- 172.21.15.156:0/175038548 wait complete. 2023-11-28T13:15:24.708 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 32 2023-11-28T13:15:24.725 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:15:24.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.922+0000 7fadd47ff700 1 Processor -- start 2023-11-28T13:15:24.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.922+0000 7fadd47ff700 1 -- start start 2023-11-28T13:15:24.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.924+0000 7fadd47ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fadcc0cd7c0 0x7fadcc0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:24.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.924+0000 7fadd47ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadcc0d1950 con 0x7fadcc15c9b0 2023-11-28T13:15:24.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.924+0000 7fadd47ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadcc0ce140 con 0x7fadcc161340 2023-11-28T13:15:24.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.924+0000 7fadd47ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadcc0b5800 con 0x7fadcc0cd7c0 2023-11-28T13:15:24.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.924+0000 7fadd259b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fadcc15c9b0 0x7fadcc0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:50824/0 (socket says 172.21.15.156:50824) 2023-11-28T13:15:24.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.924+0000 7fadd259b700 1 -- 172.21.15.156:0/3260092978 learned_addr learned my addr 172.21.15.156:0/3260092978 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:24.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.924+0000 7fadd2d9c700 1 --2- 172.21.15.156:0/3260092978 >> v2:172.21.15.156:3300/0 conn(0x7fadcc0cd7c0 0x7fadcc0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:24.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.924+0000 7fadd2d9c700 1 -- 172.21.15.156:0/3260092978 >> v1:172.21.15.186:6789/0 conn(0x7fadcc161340 legacy=0x7fadcc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:24.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.924+0000 7fadd2d9c700 1 -- 172.21.15.156:0/3260092978 >> v1:172.21.15.156:6789/0 conn(0x7fadcc15c9b0 legacy=0x7fadcc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:24.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.924+0000 7fadd2d9c700 1 -- 172.21.15.156:0/3260092978 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc0d24f0 con 0x7fadcc0cd7c0 2023-11-28T13:15:24.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.925+0000 7fadd2d9c700 1 --2- 172.21.15.156:0/3260092978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadcc0cd7c0 0x7fadcc0cdc00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009d90 tx=0x7fadc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb59538ebdc0de0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:24.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.925+0000 7fadd1599700 1 -- 172.21.15.156:0/3260092978 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc400e070 con 0x7fadcc0cd7c0 2023-11-28T13:15:24.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.925+0000 7fadd1599700 1 -- 172.21.15.156:0/3260092978 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc400bbd0 con 0x7fadcc0cd7c0 2023-11-28T13:15:24.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.925+0000 7fadd1599700 1 -- 172.21.15.156:0/3260092978 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc4012430 con 0x7fadcc0cd7c0 2023-11-28T13:15:24.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.925+0000 7fadd47ff700 1 -- 172.21.15.156:0/3260092978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadcc0cd7c0 msgr2=0x7fadcc0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:24.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.925+0000 7fadd47ff700 1 --2- 172.21.15.156:0/3260092978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadcc0cd7c0 0x7fadcc0cdc00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009d90 tx=0x7fadc400b330 comp rx=0 tx=0).stop 2023-11-28T13:15:24.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.926+0000 7fadd47ff700 1 -- 172.21.15.156:0/3260092978 shutdown_connections 2023-11-28T13:15:24.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.926+0000 7fadd47ff700 1 --2- 172.21.15.156:0/3260092978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadcc0cd7c0 0x7fadcc0cdc00 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.926+0000 7fadd47ff700 1 -- 172.21.15.156:0/3260092978 >> 172.21.15.156:0/3260092978 conn(0x7fadcc072600 msgr2=0x7fadcc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:24.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.926+0000 7fadd47ff700 1 -- 172.21.15.156:0/3260092978 shutdown_connections 2023-11-28T13:15:24.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.926+0000 7fadd47ff700 1 -- 172.21.15.156:0/3260092978 wait complete. 2023-11-28T13:15:24.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.927+0000 7fadd47ff700 1 Processor -- start 2023-11-28T13:15:24.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.927+0000 7fadd47ff700 1 -- start start 2023-11-28T13:15:24.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.928+0000 7fadd47ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadcc0cd7c0 0x7fadcc153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:24.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.928+0000 7fadd47ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadcc15c9b0 0x7fadcc15ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:24.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.928+0000 7fadd47ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadcc161340 0x7fadcc15f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:24.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.928+0000 7fadd47ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0c9fb0 con 0x7fadcc161340 2023-11-28T13:15:24.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.928+0000 7fadd47ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0cb100 con 0x7fadcc0cd7c0 2023-11-28T13:15:24.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.928+0000 7fadd47ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0cb3e0 con 0x7fadcc15c9b0 2023-11-28T13:15:24.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.929+0000 7fadd2d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadcc161340 0x7fadcc15f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:24.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.929+0000 7fadd2d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadcc161340 0x7fadcc15f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50900/0 (socket says 172.21.15.156:50900) 2023-11-28T13:15:24.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.929+0000 7fadd2d9c700 1 -- 172.21.15.156:0/4257628562 learned_addr learned my addr 172.21.15.156:0/4257628562 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:24.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.929+0000 7fadd259b700 1 --2- 172.21.15.156:0/4257628562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadcc0cd7c0 0x7fadcc153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:24.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.929+0000 7fadd1d9a700 1 --2- 172.21.15.156:0/4257628562 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadcc15c9b0 0x7fadcc15ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:24.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.929+0000 7fadd2d9c700 1 -- 172.21.15.156:0/4257628562 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadcc15c9b0 msgr2=0x7fadcc15ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:24.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.929+0000 7fadd2d9c700 1 --2- 172.21.15.156:0/4257628562 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadcc15c9b0 0x7fadcc15ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.929+0000 7fadd2d9c700 1 -- 172.21.15.156:0/4257628562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadcc0cd7c0 msgr2=0x7fadcc153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:24.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.929+0000 7fadd2d9c700 1 --2- 172.21.15.156:0/4257628562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadcc0cd7c0 0x7fadcc153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:24.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.929+0000 7fadd2d9c700 1 -- 172.21.15.156:0/4257628562 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc4009a70 con 0x7fadcc161340 2023-11-28T13:15:24.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.929+0000 7fadd259b700 1 --2- 172.21.15.156:0/4257628562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadcc0cd7c0 0x7fadcc153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:15:24.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.929+0000 7fadd2d9c700 1 --2- 172.21.15.156:0/4257628562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadcc161340 0x7fadcc15f480 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fadc4014040 tx=0x7fadc4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:24.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.930+0000 7fadc2ffd700 1 -- 172.21.15.156:0/4257628562 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc40098f0 con 0x7fadcc161340 2023-11-28T13:15:24.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.930+0000 7fadc2ffd700 1 -- 172.21.15.156:0/4257628562 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc4020070 con 0x7fadcc161340 2023-11-28T13:15:24.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.930+0000 7fadd47ff700 1 -- 172.21.15.156:0/4257628562 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadcc0d1300 con 0x7fadcc161340 2023-11-28T13:15:24.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.930+0000 7fadc2ffd700 1 -- 172.21.15.156:0/4257628562 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc4004280 con 0x7fadcc161340 2023-11-28T13:15:24.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.930+0000 7fadd47ff700 1 -- 172.21.15.156:0/4257628562 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadcc0b5520 con 0x7fadcc161340 2023-11-28T13:15:24.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.931+0000 7fadd47ff700 1 -- 172.21.15.156:0/4257628562 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadcc0d15e0 con 0x7fadcc161340 2023-11-28T13:15:24.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.932+0000 7fadc2ffd700 1 -- 172.21.15.156:0/4257628562 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fadc40043e0 con 0x7fadcc161340 2023-11-28T13:15:24.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.932+0000 7fadc2ffd700 1 --2- 172.21.15.156:0/4257628562 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fadb00682f0 0x7fadb006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:24.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.933+0000 7fadc2ffd700 1 -- 172.21.15.156:0/4257628562 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fadc4022070 con 0x7fadcc161340 2023-11-28T13:15:24.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.933+0000 7fadd259b700 1 --2- 172.21.15.156:0/4257628562 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fadb00682f0 0x7fadb006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:24.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.934+0000 7fadd259b700 1 --2- 172.21.15.156:0/4257628562 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fadb00682f0 0x7fadb006a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fadc8005910 tx=0x7fadc8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:24.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:24.937+0000 7fadc2ffd700 1 -- 172.21.15.156:0/4257628562 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fadc4054ab0 con 0x7fadcc161340 2023-11-28T13:15:25.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.130+0000 7fadd47ff700 1 -- 172.21.15.156:0/4257628562 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadcc0745c0 con 0x7fadcc161340 2023-11-28T13:15:25.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.132+0000 7fadc2ffd700 1 -- 172.21.15.156:0/4257628562 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+18152 (secure 0 0 0) 0x7fadc40580d0 con 0x7fadcc161340 2023-11-28T13:15:25.133 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:15:25.133 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":61,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:11:52.553240+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:10:52.732399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T13:10:53.912753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.11","mappings":[{"from":7,"to":2}]},{"pgid":"10.19","mappings":[{"from":1,"to":3}]},{"pgid":"10.1b","mappings":[{"from":4,"to":5}]},{"pgid":"10.1f","mappings":[{"from":7,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:15:25.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.134+0000 7fadd47ff700 1 -- 172.21.15.156:0/4257628562 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fadb00682f0 msgr2=0x7fadb006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:25.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.134+0000 7fadd47ff700 1 --2- 172.21.15.156:0/4257628562 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fadb00682f0 0x7fadb006a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fadc8005910 tx=0x7fadc8009040 comp rx=0 tx=0).stop 2023-11-28T13:15:25.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.134+0000 7fadd47ff700 1 -- 172.21.15.156:0/4257628562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadcc161340 msgr2=0x7fadcc15f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:25.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.134+0000 7fadd47ff700 1 --2- 172.21.15.156:0/4257628562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadcc161340 0x7fadcc15f480 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fadc4014040 tx=0x7fadc4004ad0 comp rx=0 tx=0).stop 2023-11-28T13:15:25.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.134+0000 7fadd47ff700 1 -- 172.21.15.156:0/4257628562 shutdown_connections 2023-11-28T13:15:25.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.134+0000 7fadd47ff700 1 --2- 172.21.15.156:0/4257628562 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fadb00682f0 0x7fadb006a7a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:25.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.134+0000 7fadd47ff700 1 --2- 172.21.15.156:0/4257628562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadcc161340 0x7fadcc15f480 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:25.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.134+0000 7fadd47ff700 1 --2- 172.21.15.156:0/4257628562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadcc0cd7c0 0x7fadcc153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:25.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.134+0000 7fadd47ff700 1 --2- 172.21.15.156:0/4257628562 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fadcc15c9b0 0x7fadcc15ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:25.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.134+0000 7fadd47ff700 1 -- 172.21.15.156:0/4257628562 >> 172.21.15.156:0/4257628562 conn(0x7fadcc072600 msgr2=0x7fadcc158350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:25.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.134+0000 7fadd47ff700 1 -- 172.21.15.156:0/4257628562 shutdown_connections 2023-11-28T13:15:25.137 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.135+0000 7fadd47ff700 1 -- 172.21.15.156:0/4257628562 wait complete. 2023-11-28T13:15:25.155 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T13:15:25.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.351+0000 7f58011ff700 1 Processor -- start 2023-11-28T13:15:25.353 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.351+0000 7f58011ff700 1 -- start start 2023-11-28T13:15:25.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.353+0000 7f58011ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57fc15c9b0 0x7f57fc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:25.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.353+0000 7f58011ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f57fc0cd7c0 0x7f57fc0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:25.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.353+0000 7f58011ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57fc0ce160 con 0x7f57fc161340 2023-11-28T13:15:25.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.353+0000 7f58011ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57fc0b5800 con 0x7f57fc0cd7c0 2023-11-28T13:15:25.354 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.353+0000 7f58011ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0d1950 con 0x7f57fc15c9b0 2023-11-28T13:15:25.355 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.353+0000 7f57fb59e700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f57fc0cd7c0 0x7f57fc0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:25.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.353+0000 7f57fad9d700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57fc15c9b0 0x7f57fc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:25.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.353+0000 7f57fb59e700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f57fc0cd7c0 0x7f57fc0cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:33322/0 (socket says 172.21.15.156:33322) 2023-11-28T13:15:25.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.353+0000 7f57fb59e700 1 -- 172.21.15.156:0/761393271 learned_addr learned my addr 172.21.15.156:0/761393271 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:25.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.354+0000 7f57fa59c700 1 -- 172.21.15.156:0/761393271 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658938135 0 0) 0x7f57fc0ce160 con 0x7f57fc161340 2023-11-28T13:15:25.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.354+0000 7f57fad9d700 1 -- 172.21.15.156:0/761393271 >> v2:172.21.15.186:3300/0 conn(0x7f57fc0cd7c0 msgr2=0x7f57fc0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:25.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.354+0000 7f57fad9d700 1 --2- 172.21.15.156:0/761393271 >> v2:172.21.15.186:3300/0 conn(0x7f57fc0cd7c0 0x7f57fc0cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:25.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.354+0000 7f57fad9d700 1 -- 172.21.15.156:0/761393271 >> v1:172.21.15.186:6789/0 conn(0x7f57fc161340 legacy=0x7f57fc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:25.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.354+0000 7f57fad9d700 1 -- 172.21.15.156:0/761393271 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57fc0d24f0 con 0x7f57fc15c9b0 2023-11-28T13:15:25.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.354+0000 7f57fb59e700 1 --2- 172.21.15.156:0/761393271 >> v2:172.21.15.186:3300/0 conn(0x7f57fc0cd7c0 0x7f57fc0cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:15:25.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.354+0000 7f57fad9d700 1 --2- 172.21.15.156:0/761393271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57fc15c9b0 0x7f57fc0b4bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009840 tx=0x7f57ec009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=760db6a0246c6b96 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:25.356 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.354+0000 7f57fa59c700 1 -- 172.21.15.156:0/761393271 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec011830 con 0x7f57fc15c9b0 2023-11-28T13:15:25.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.354+0000 7f57fa59c700 1 -- 172.21.15.156:0/761393271 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec011990 con 0x7f57fc15c9b0 2023-11-28T13:15:25.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.354+0000 7f57fa59c700 1 -- 172.21.15.156:0/761393271 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec011d20 con 0x7f57fc15c9b0 2023-11-28T13:15:25.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.355+0000 7f58011ff700 1 -- 172.21.15.156:0/761393271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57fc15c9b0 msgr2=0x7f57fc0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:25.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.355+0000 7f58011ff700 1 --2- 172.21.15.156:0/761393271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57fc15c9b0 0x7f57fc0b4bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009840 tx=0x7f57ec009b50 comp rx=0 tx=0).stop 2023-11-28T13:15:25.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.355+0000 7f58011ff700 1 -- 172.21.15.156:0/761393271 shutdown_connections 2023-11-28T13:15:25.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.355+0000 7f58011ff700 1 --2- 172.21.15.156:0/761393271 >> v2:172.21.15.186:3300/0 conn(0x7f57fc0cd7c0 0x7f57fc0cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:25.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.355+0000 7f58011ff700 1 --2- 172.21.15.156:0/761393271 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57fc15c9b0 0x7f57fc0b4bf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:25.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.355+0000 7f58011ff700 1 -- 172.21.15.156:0/761393271 >> 172.21.15.156:0/761393271 conn(0x7f57fc072600 msgr2=0x7f57fc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:25.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.355+0000 7f58011ff700 1 -- 172.21.15.156:0/761393271 shutdown_connections 2023-11-28T13:15:25.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.355+0000 7f58011ff700 1 -- 172.21.15.156:0/761393271 wait complete. 2023-11-28T13:15:25.358 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.357+0000 7f58011ff700 1 Processor -- start 2023-11-28T13:15:25.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.357+0000 7f58011ff700 1 -- start start 2023-11-28T13:15:25.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.358+0000 7f58011ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57fc0cd7c0 0x7f57fc157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:25.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.358+0000 7f58011ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57fc15c9b0 0x7f57fc156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:25.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.358+0000 7f58011ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57fc161340 0x7f57fc1538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:25.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.358+0000 7f58011ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc1f2780 con 0x7f57fc0cd7c0 2023-11-28T13:15:25.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.358+0000 7f58011ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc1f3720 con 0x7f57fc15c9b0 2023-11-28T13:15:25.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.358+0000 7f58011ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc158130 con 0x7f57fc161340 2023-11-28T13:15:25.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.358+0000 7f57fad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57fc0cd7c0 0x7f57fc157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:25.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.358+0000 7f57fad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57fc0cd7c0 0x7f57fc157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50914/0 (socket says 172.21.15.156:50914) 2023-11-28T13:15:25.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.358+0000 7f57fad9d700 1 -- 172.21.15.156:0/681725805 learned_addr learned my addr 172.21.15.156:0/681725805 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:25.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.358+0000 7f57fb59e700 1 --2- 172.21.15.156:0/681725805 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57fc161340 0x7f57fc1538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:25.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.358+0000 7f57f3fff700 1 --2- 172.21.15.156:0/681725805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57fc15c9b0 0x7f57fc156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:25.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.359+0000 7f57fad9d700 1 -- 172.21.15.156:0/681725805 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57fc161340 msgr2=0x7f57fc1538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:25.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.359+0000 7f57fad9d700 1 --2- 172.21.15.156:0/681725805 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57fc161340 0x7f57fc1538a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:25.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.359+0000 7f57fad9d700 1 -- 172.21.15.156:0/681725805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57fc15c9b0 msgr2=0x7f57fc156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:25.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.359+0000 7f57fad9d700 1 --2- 172.21.15.156:0/681725805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57fc15c9b0 0x7f57fc156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:25.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.359+0000 7f57fad9d700 1 -- 172.21.15.156:0/681725805 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f4009b30 con 0x7f57fc0cd7c0 2023-11-28T13:15:25.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.359+0000 7f57f3fff700 1 --2- 172.21.15.156:0/681725805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57fc15c9b0 0x7f57fc156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:15:25.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.359+0000 7f57fad9d700 1 --2- 172.21.15.156:0/681725805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57fc0cd7c0 0x7f57fc157ae0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009fd0 tx=0x7f57ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:25.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.359+0000 7f57f37fe700 1 -- 172.21.15.156:0/681725805 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0036a0 con 0x7f57fc0cd7c0 2023-11-28T13:15:25.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.359+0000 7f57f37fe700 1 -- 172.21.15.156:0/681725805 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec01e070 con 0x7f57fc0cd7c0 2023-11-28T13:15:25.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.359+0000 7f58011ff700 1 -- 172.21.15.156:0/681725805 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ec0094a0 con 0x7f57fc0cd7c0 2023-11-28T13:15:25.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.359+0000 7f57f37fe700 1 -- 172.21.15.156:0/681725805 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec003a60 con 0x7f57fc0cd7c0 2023-11-28T13:15:25.365 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.359+0000 7f58011ff700 1 -- 172.21.15.156:0/681725805 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc0d1300 con 0x7f57fc0cd7c0 2023-11-28T13:15:25.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.360+0000 7f58011ff700 1 -- 172.21.15.156:0/681725805 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57fc0b5520 con 0x7f57fc0cd7c0 2023-11-28T13:15:25.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.361+0000 7f57f37fe700 1 -- 172.21.15.156:0/681725805 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f57ec00f050 con 0x7f57fc0cd7c0 2023-11-28T13:15:25.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.362+0000 7f57f37fe700 1 --2- 172.21.15.156:0/681725805 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57dc0682f0 0x7f57dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:25.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.362+0000 7f57f37fe700 1 -- 172.21.15.156:0/681725805 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f57ec086ea0 con 0x7f57fc0cd7c0 2023-11-28T13:15:25.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.362+0000 7f57f3fff700 1 --2- 172.21.15.156:0/681725805 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57dc0682f0 0x7f57dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:25.366 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.363+0000 7f57f3fff700 1 --2- 172.21.15.156:0/681725805 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57dc0682f0 0x7f57dc06a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f57fc15fce0 tx=0x7f57e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:25.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.366+0000 7f57f37fe700 1 -- 172.21.15.156:0/681725805 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f57ec053580 con 0x7f57fc0cd7c0 2023-11-28T13:15:25.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:25.599+0000 7f58011ff700 1 -- 172.21.15.156:0/681725805 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f57fc0d15e0 con 0x7f57fc0cd7c0 2023-11-28T13:15:26.556 INFO:tasks.ceph.mon.a.smithi156.stderr:2023-11-28T13:15:26.555+0000 7f61ad1f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T13:15:26.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.562+0000 7f57f37fe700 1 -- 172.21.15.156:0/681725805 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7f57ec015090 con 0x7f57fc0cd7c0 2023-11-28T13:15:26.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.565+0000 7f58011ff700 1 -- 172.21.15.156:0/681725805 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57dc0682f0 msgr2=0x7f57dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:26.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.565+0000 7f58011ff700 1 --2- 172.21.15.156:0/681725805 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57dc0682f0 0x7f57dc06a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f57fc15fce0 tx=0x7f57e4009040 comp rx=0 tx=0).stop 2023-11-28T13:15:26.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.565+0000 7f58011ff700 1 -- 172.21.15.156:0/681725805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57fc0cd7c0 msgr2=0x7f57fc157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:26.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.565+0000 7f58011ff700 1 --2- 172.21.15.156:0/681725805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57fc0cd7c0 0x7f57fc157ae0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009fd0 tx=0x7f57ec004750 comp rx=0 tx=0).stop 2023-11-28T13:15:26.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.565+0000 7f58011ff700 1 -- 172.21.15.156:0/681725805 shutdown_connections 2023-11-28T13:15:26.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.565+0000 7f58011ff700 1 --2- 172.21.15.156:0/681725805 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f57dc0682f0 0x7f57dc06a7a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:26.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.565+0000 7f58011ff700 1 --2- 172.21.15.156:0/681725805 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57fc161340 0x7f57fc1538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:26.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.565+0000 7f58011ff700 1 --2- 172.21.15.156:0/681725805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57fc0cd7c0 0x7f57fc157ae0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:26.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.565+0000 7f58011ff700 1 --2- 172.21.15.156:0/681725805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57fc15c9b0 0x7f57fc156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:26.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.565+0000 7f58011ff700 1 -- 172.21.15.156:0/681725805 >> 172.21.15.156:0/681725805 conn(0x7f57fc072600 msgr2=0x7f57fc06af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:26.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.566+0000 7f58011ff700 1 -- 172.21.15.156:0/681725805 shutdown_connections 2023-11-28T13:15:26.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.566+0000 7f58011ff700 1 -- 172.21.15.156:0/681725805 wait complete. 2023-11-28T13:15:26.568 INFO:teuthology.orchestra.run.smithi156.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T13:15:26.588 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T13:15:26.632 INFO:tasks.ceph.mds.c.smithi156.stdout:starting mds.c at 2023-11-28T13:15:26.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.789+0000 7f7872fff700 1 Processor -- start 2023-11-28T13:15:26.791 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.789+0000 7f7872fff700 1 -- start start 2023-11-28T13:15:26.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.791+0000 7f7872fff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f786c161340 0x7f786c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:26.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.791+0000 7f7872fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f786c0d2320 0x7f786c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:26.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.791+0000 7f7872fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786c0d1950 con 0x7f786c15c9b0 2023-11-28T13:15:26.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.791+0000 7f7872fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786c0b5800 con 0x7f786c0d2320 2023-11-28T13:15:26.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.791+0000 7f7872fff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786c0b5300 con 0x7f786c161340 2023-11-28T13:15:26.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.791+0000 7f7870d9b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f786c15c9b0 0x7f786c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39398/0 (socket says 172.21.15.156:39398) 2023-11-28T13:15:26.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.791+0000 7f7870d9b700 1 -- 172.21.15.156:0/1390977156 learned_addr learned my addr 172.21.15.156:0/1390977156 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:26.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.791+0000 7f787159c700 1 --2- 172.21.15.156:0/1390977156 >> v2:172.21.15.156:3300/0 conn(0x7f786c0d2320 0x7f786c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:26.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.791+0000 7f786bfff700 1 --2- 172.21.15.156:0/1390977156 >> v2:172.21.15.186:3300/0 conn(0x7f786c161340 0x7f786c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:26.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.791+0000 7f786b7fe700 1 -- 172.21.15.156:0/1390977156 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155227279 0 0) 0x7f786c0d1950 con 0x7f786c15c9b0 2023-11-28T13:15:26.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.791+0000 7f786b7fe700 1 -- 172.21.15.156:0/1390977156 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f785c0035b0 con 0x7f786c15c9b0 2023-11-28T13:15:26.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.791+0000 7f787159c700 1 -- 172.21.15.156:0/1390977156 >> v2:172.21.15.186:3300/0 conn(0x7f786c161340 msgr2=0x7f786c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:26.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.791+0000 7f787159c700 1 --2- 172.21.15.156:0/1390977156 >> v2:172.21.15.186:3300/0 conn(0x7f786c161340 0x7f786c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:26.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.791+0000 7f787159c700 1 -- 172.21.15.156:0/1390977156 >> v1:172.21.15.156:6789/0 conn(0x7f786c15c9b0 legacy=0x7f786c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:26.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.791+0000 7f787159c700 1 -- 172.21.15.156:0/1390977156 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c0b5520 con 0x7f786c0d2320 2023-11-28T13:15:26.794 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.792+0000 7f787159c700 1 --2- 172.21.15.156:0/1390977156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c0d2320 0x7f786c0cd7c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f786c0685c0 tx=0x7f786400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab591c2ea130ade4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:26.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.792+0000 7f786b7fe700 1 -- 172.21.15.156:0/1390977156 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786400e070 con 0x7f786c0d2320 2023-11-28T13:15:26.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.792+0000 7f786b7fe700 1 -- 172.21.15.156:0/1390977156 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786400bae0 con 0x7f786c0d2320 2023-11-28T13:15:26.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.792+0000 7f786b7fe700 1 -- 172.21.15.156:0/1390977156 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786400be90 con 0x7f786c0d2320 2023-11-28T13:15:26.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.792+0000 7f7872fff700 1 -- 172.21.15.156:0/1390977156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c0d2320 msgr2=0x7f786c0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:26.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.792+0000 7f7872fff700 1 --2- 172.21.15.156:0/1390977156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c0d2320 0x7f786c0cd7c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f786c0685c0 tx=0x7f786400b330 comp rx=0 tx=0).stop 2023-11-28T13:15:26.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.792+0000 7f7872fff700 1 -- 172.21.15.156:0/1390977156 shutdown_connections 2023-11-28T13:15:26.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.792+0000 7f7872fff700 1 --2- 172.21.15.156:0/1390977156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c0d2320 0x7f786c0cd7c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:26.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.792+0000 7f7872fff700 1 --2- 172.21.15.156:0/1390977156 >> v2:172.21.15.186:3300/0 conn(0x7f786c161340 0x7f786c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:26.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.792+0000 7f7872fff700 1 -- 172.21.15.156:0/1390977156 >> 172.21.15.156:0/1390977156 conn(0x7f786c072600 msgr2=0x7f786c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:26.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.793+0000 7f7872fff700 1 -- 172.21.15.156:0/1390977156 shutdown_connections 2023-11-28T13:15:26.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.793+0000 7f7872fff700 1 -- 172.21.15.156:0/1390977156 wait complete. 2023-11-28T13:15:26.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.794+0000 7f7872fff700 1 Processor -- start 2023-11-28T13:15:26.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.794+0000 7f7872fff700 1 -- start start 2023-11-28T13:15:26.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.795+0000 7f7872fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f786c0d2320 0x7f786c14f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:26.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.795+0000 7f7872fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f786c15c9b0 0x7f786c0b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:26.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.795+0000 7f7872fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c161340 0x7f786c0b81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:26.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.795+0000 7f7872fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c154af0 con 0x7f786c161340 2023-11-28T13:15:26.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.795+0000 7f7872fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c1503a0 con 0x7f786c0d2320 2023-11-28T13:15:26.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.795+0000 7f7872fff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c154d00 con 0x7f786c15c9b0 2023-11-28T13:15:26.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.796+0000 7f787159c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c161340 0x7f786c0b81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:26.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.796+0000 7f787159c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c161340 0x7f786c0b81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39402/0 (socket says 172.21.15.156:39402) 2023-11-28T13:15:26.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.796+0000 7f787159c700 1 -- 172.21.15.156:0/1374435642 learned_addr learned my addr 172.21.15.156:0/1374435642 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:26.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.796+0000 7f7870d9b700 1 --2- 172.21.15.156:0/1374435642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f786c0d2320 0x7f786c14f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:26.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.796+0000 7f786bfff700 1 --2- 172.21.15.156:0/1374435642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f786c15c9b0 0x7f786c0b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:26.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.796+0000 7f787159c700 1 -- 172.21.15.156:0/1374435642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f786c15c9b0 msgr2=0x7f786c0b7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:26.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.796+0000 7f787159c700 1 --2- 172.21.15.156:0/1374435642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f786c15c9b0 0x7f786c0b7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:26.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.796+0000 7f787159c700 1 -- 172.21.15.156:0/1374435642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f786c0d2320 msgr2=0x7f786c14f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:26.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.796+0000 7f787159c700 1 --2- 172.21.15.156:0/1374435642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f786c0d2320 0x7f786c14f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:26.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.796+0000 7f787159c700 1 -- 172.21.15.156:0/1374435642 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7864009a70 con 0x7f786c161340 2023-11-28T13:15:26.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.796+0000 7f787159c700 1 --2- 172.21.15.156:0/1374435642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c161340 0x7f786c0b81b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7864014040 tx=0x7f7864004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:26.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.796+0000 7f78697fa700 1 -- 172.21.15.156:0/1374435642 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78640046b0 con 0x7f786c161340 2023-11-28T13:15:26.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.797+0000 7f78697fa700 1 -- 172.21.15.156:0/1374435642 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7864020070 con 0x7f786c161340 2023-11-28T13:15:26.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.797+0000 7f7872fff700 1 -- 172.21.15.156:0/1374435642 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c0d15e0 con 0x7f786c161340 2023-11-28T13:15:26.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.797+0000 7f78697fa700 1 -- 172.21.15.156:0/1374435642 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864027480 con 0x7f786c161340 2023-11-28T13:15:26.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.797+0000 7f7872fff700 1 -- 172.21.15.156:0/1374435642 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c0d1300 con 0x7f786c161340 2023-11-28T13:15:26.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.798+0000 7f7872fff700 1 -- 172.21.15.156:0/1374435642 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c15cd80 con 0x7f786c161340 2023-11-28T13:15:26.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.799+0000 7f78697fa700 1 -- 172.21.15.156:0/1374435642 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7864012430 con 0x7f786c161340 2023-11-28T13:15:26.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.800+0000 7f78697fa700 1 --2- 172.21.15.156:0/1374435642 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78500682f0 0x7f785006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:26.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.800+0000 7f78697fa700 1 -- 172.21.15.156:0/1374435642 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f7864022070 con 0x7f786c161340 2023-11-28T13:15:26.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.800+0000 7f7870d9b700 1 --2- 172.21.15.156:0/1374435642 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78500682f0 0x7f785006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:26.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.801+0000 7f7870d9b700 1 --2- 172.21.15.156:0/1374435642 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78500682f0 0x7f785006a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f78600011e0 tx=0x7f7860009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:26.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:26.804+0000 7f78697fa700 1 -- 172.21.15.156:0/1374435642 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7864054790 con 0x7f786c161340 2023-11-28T13:15:27.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.042+0000 7f7872fff700 1 -- 172.21.15.156:0/1374435642 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f786c0745c0 con 0x7f786c161340 2023-11-28T13:15:27.612 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.610+0000 7f78697fa700 1 -- 172.21.15.156:0/1374435642 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7864057db0 con 0x7f786c161340 2023-11-28T13:15:27.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.613+0000 7f7872fff700 1 -- 172.21.15.156:0/1374435642 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78500682f0 msgr2=0x7f785006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:27.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.613+0000 7f7872fff700 1 --2- 172.21.15.156:0/1374435642 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78500682f0 0x7f785006a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f78600011e0 tx=0x7f7860009040 comp rx=0 tx=0).stop 2023-11-28T13:15:27.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.613+0000 7f7872fff700 1 -- 172.21.15.156:0/1374435642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c161340 msgr2=0x7f786c0b81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:27.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.613+0000 7f7872fff700 1 --2- 172.21.15.156:0/1374435642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c161340 0x7f786c0b81b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7864014040 tx=0x7f7864004ad0 comp rx=0 tx=0).stop 2023-11-28T13:15:27.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.613+0000 7f7872fff700 1 -- 172.21.15.156:0/1374435642 shutdown_connections 2023-11-28T13:15:27.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.613+0000 7f7872fff700 1 --2- 172.21.15.156:0/1374435642 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78500682f0 0x7f785006a7a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:27.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.613+0000 7f7872fff700 1 --2- 172.21.15.156:0/1374435642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f786c161340 0x7f786c0b81b0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:27.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.613+0000 7f7872fff700 1 --2- 172.21.15.156:0/1374435642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f786c0d2320 0x7f786c14f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:27.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.613+0000 7f7872fff700 1 --2- 172.21.15.156:0/1374435642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f786c15c9b0 0x7f786c0b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:27.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.613+0000 7f7872fff700 1 -- 172.21.15.156:0/1374435642 >> 172.21.15.156:0/1374435642 conn(0x7f786c072600 msgr2=0x7f786c0b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:27.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.614+0000 7f7872fff700 1 -- 172.21.15.156:0/1374435642 shutdown_connections 2023-11-28T13:15:27.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.614+0000 7f7872fff700 1 -- 172.21.15.156:0/1374435642 wait complete. 2023-11-28T13:15:27.638 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T13:15:27.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.878+0000 7fe14afff700 1 Processor -- start 2023-11-28T13:15:27.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.878+0000 7fe14afff700 1 -- start start 2023-11-28T13:15:27.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.879+0000 7fe14afff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe144157160 0x7fe14415a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:27.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.879+0000 7fe14afff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe1440afd60 0x7fe14415d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:27.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.879+0000 7fe14afff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe144157530 con 0x7fe1440af8e0 2023-11-28T13:15:27.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.879+0000 7fe14afff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe14415aa60 con 0x7fe1440afd60 2023-11-28T13:15:27.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.879+0000 7fe14afff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1440c9820 con 0x7fe144157160 2023-11-28T13:15:27.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.879+0000 7fe14959c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe1440afd60 0x7fe14415d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:27.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.879+0000 7fe14959c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe1440afd60 0x7fe14415d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39406/0 (socket says 172.21.15.156:39406) 2023-11-28T13:15:27.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.879+0000 7fe14959c700 1 -- 172.21.15.156:0/3285316979 learned_addr learned my addr 172.21.15.156:0/3285316979 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:27.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.879+0000 7fe143fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fe1440af8e0 0x7fe14415d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39412/0 (socket says 172.21.15.156:39412) 2023-11-28T13:15:27.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.879+0000 7fe148d9b700 1 --2- 172.21.15.156:0/3285316979 >> v2:172.21.15.186:3300/0 conn(0x7fe144157160 0x7fe14415a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:27.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.880+0000 7fe14959c700 1 -- 172.21.15.156:0/3285316979 >> v2:172.21.15.186:3300/0 conn(0x7fe144157160 msgr2=0x7fe14415a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:27.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.880+0000 7fe14959c700 1 --2- 172.21.15.156:0/3285316979 >> v2:172.21.15.186:3300/0 conn(0x7fe144157160 0x7fe14415a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:27.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.880+0000 7fe14959c700 1 -- 172.21.15.156:0/3285316979 >> v1:172.21.15.156:6789/0 conn(0x7fe1440af8e0 legacy=0x7fe14415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:27.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.880+0000 7fe14959c700 1 -- 172.21.15.156:0/3285316979 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1440c9b00 con 0x7fe1440afd60 2023-11-28T13:15:27.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.880+0000 7fe148d9b700 1 --2- 172.21.15.156:0/3285316979 >> v2:172.21.15.186:3300/0 conn(0x7fe144157160 0x7fe14415a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:15:27.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.880+0000 7fe14959c700 1 --2- 172.21.15.156:0/3285316979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1440afd60 0x7fe14415d7a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe1440724b0 tx=0x7fe13c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e374cc488456098 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:27.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.880+0000 7fe1437fe700 1 -- 172.21.15.156:0/3285316979 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c00e070 con 0x7fe1440afd60 2023-11-28T13:15:27.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.880+0000 7fe1437fe700 1 -- 172.21.15.156:0/3285316979 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c00b9c0 con 0x7fe1440afd60 2023-11-28T13:15:27.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.880+0000 7fe1437fe700 1 -- 172.21.15.156:0/3285316979 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c00bd70 con 0x7fe1440afd60 2023-11-28T13:15:27.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.881+0000 7fe14afff700 1 -- 172.21.15.156:0/3285316979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1440afd60 msgr2=0x7fe14415d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:27.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.881+0000 7fe14afff700 1 --2- 172.21.15.156:0/3285316979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1440afd60 0x7fe14415d7a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe1440724b0 tx=0x7fe13c00b330 comp rx=0 tx=0).stop 2023-11-28T13:15:27.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.881+0000 7fe14afff700 1 -- 172.21.15.156:0/3285316979 shutdown_connections 2023-11-28T13:15:27.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.881+0000 7fe14afff700 1 --2- 172.21.15.156:0/3285316979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1440afd60 0x7fe14415d7a0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:27.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.881+0000 7fe14afff700 1 --2- 172.21.15.156:0/3285316979 >> v2:172.21.15.186:3300/0 conn(0x7fe144157160 0x7fe14415a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:27.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.881+0000 7fe14afff700 1 -- 172.21.15.156:0/3285316979 >> 172.21.15.156:0/3285316979 conn(0x7fe1440b0860 msgr2=0x7fe1440b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:27.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.881+0000 7fe14afff700 1 -- 172.21.15.156:0/3285316979 shutdown_connections 2023-11-28T13:15:27.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.881+0000 7fe14afff700 1 -- 172.21.15.156:0/3285316979 wait complete. 2023-11-28T13:15:27.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.883+0000 7fe14afff700 1 Processor -- start 2023-11-28T13:15:27.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.883+0000 7fe14afff700 1 -- start start 2023-11-28T13:15:27.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.884+0000 7fe14afff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1440af8e0 0x7fe1441465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:27.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.884+0000 7fe14afff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1440afd60 0x7fe14406d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:27.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.884+0000 7fe14afff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe144157160 0x7fe1440cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:27.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.884+0000 7fe14afff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440c3ad0 con 0x7fe1440afd60 2023-11-28T13:15:27.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.884+0000 7fe14afff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440c4090 con 0x7fe144157160 2023-11-28T13:15:27.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.884+0000 7fe14afff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14406db80 con 0x7fe1440af8e0 2023-11-28T13:15:27.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.884+0000 7fe143fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1440afd60 0x7fe14406d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:27.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.884+0000 7fe143fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1440afd60 0x7fe14406d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39418/0 (socket says 172.21.15.156:39418) 2023-11-28T13:15:27.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.884+0000 7fe143fff700 1 -- 172.21.15.156:0/3976353099 learned_addr learned my addr 172.21.15.156:0/3976353099 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:27.888 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.884+0000 7fe148d9b700 1 --2- 172.21.15.156:0/3976353099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1440af8e0 0x7fe1441465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:27.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.884+0000 7fe14959c700 1 --2- 172.21.15.156:0/3976353099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe144157160 0x7fe1440cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:27.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.885+0000 7fe143fff700 1 -- 172.21.15.156:0/3976353099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1440af8e0 msgr2=0x7fe1441465b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:27.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.885+0000 7fe143fff700 1 --2- 172.21.15.156:0/3976353099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1440af8e0 0x7fe1441465b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:27.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.885+0000 7fe143fff700 1 -- 172.21.15.156:0/3976353099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe144157160 msgr2=0x7fe1440cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:27.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.885+0000 7fe143fff700 1 --2- 172.21.15.156:0/3976353099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe144157160 0x7fe1440cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:27.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.885+0000 7fe143fff700 1 -- 172.21.15.156:0/3976353099 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13c009b30 con 0x7fe1440afd60 2023-11-28T13:15:27.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.885+0000 7fe143fff700 1 --2- 172.21.15.156:0/3976353099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1440afd60 0x7fe14406d1a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe13000c4e0 tx=0x7fe13000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:27.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.885+0000 7fe1417fa700 1 -- 172.21.15.156:0/3976353099 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1300074e0 con 0x7fe1440afd60 2023-11-28T13:15:27.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.885+0000 7fe1417fa700 1 -- 172.21.15.156:0/3976353099 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe130007640 con 0x7fe1440afd60 2023-11-28T13:15:27.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.885+0000 7fe14afff700 1 -- 172.21.15.156:0/3976353099 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1440ca0c0 con 0x7fe1440afd60 2023-11-28T13:15:27.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.885+0000 7fe1417fa700 1 -- 172.21.15.156:0/3976353099 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130005a70 con 0x7fe1440afd60 2023-11-28T13:15:27.890 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.885+0000 7fe14afff700 1 -- 172.21.15.156:0/3976353099 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1440c9de0 con 0x7fe1440afd60 2023-11-28T13:15:27.890 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.886+0000 7fe14afff700 1 -- 172.21.15.156:0/3976353099 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1440ca3a0 con 0x7fe1440afd60 2023-11-28T13:15:27.890 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.887+0000 7fe1417fa700 1 -- 172.21.15.156:0/3976353099 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe130019030 con 0x7fe1440afd60 2023-11-28T13:15:27.890 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.888+0000 7fe1417fa700 1 --2- 172.21.15.156:0/3976353099 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe1280682f0 0x7fe12806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:27.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.888+0000 7fe1417fa700 1 -- 172.21.15.156:0/3976353099 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fe130085060 con 0x7fe1440afd60 2023-11-28T13:15:27.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.889+0000 7fe148d9b700 1 --2- 172.21.15.156:0/3976353099 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe1280682f0 0x7fe12806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:27.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.890+0000 7fe148d9b700 1 --2- 172.21.15.156:0/3976353099 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe1280682f0 0x7fe12806a7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe1440c9160 tx=0x7fe138008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:27.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:27.892+0000 7fe1417fa700 1 -- 172.21.15.156:0/3976353099 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe1300516e0 con 0x7fe1440afd60 2023-11-28T13:15:28.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.083+0000 7fe14afff700 1 -- 172.21.15.156:0/3976353099 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe14404e620 con 0x7fe1440afd60 2023-11-28T13:15:28.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.615+0000 7fe1417fa700 1 -- 172.21.15.156:0/3976353099 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v63) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe130017020 con 0x7fe1440afd60 2023-11-28T13:15:28.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.618+0000 7fe14afff700 1 -- 172.21.15.156:0/3976353099 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe1280682f0 msgr2=0x7fe12806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:28.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.618+0000 7fe14afff700 1 --2- 172.21.15.156:0/3976353099 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe1280682f0 0x7fe12806a7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe1440c9160 tx=0x7fe138008040 comp rx=0 tx=0).stop 2023-11-28T13:15:28.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.618+0000 7fe14afff700 1 -- 172.21.15.156:0/3976353099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1440afd60 msgr2=0x7fe14406d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:28.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.618+0000 7fe14afff700 1 --2- 172.21.15.156:0/3976353099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1440afd60 0x7fe14406d1a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe13000c4e0 tx=0x7fe13000c7f0 comp rx=0 tx=0).stop 2023-11-28T13:15:28.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.618+0000 7fe14afff700 1 -- 172.21.15.156:0/3976353099 shutdown_connections 2023-11-28T13:15:28.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.618+0000 7fe14afff700 1 --2- 172.21.15.156:0/3976353099 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe1280682f0 0x7fe12806a7a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:28.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.618+0000 7fe14afff700 1 --2- 172.21.15.156:0/3976353099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe144157160 0x7fe1440cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:28.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.618+0000 7fe14afff700 1 --2- 172.21.15.156:0/3976353099 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1440af8e0 0x7fe1441465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:28.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.618+0000 7fe14afff700 1 --2- 172.21.15.156:0/3976353099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1440afd60 0x7fe14406d1a0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:28.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.618+0000 7fe14afff700 1 -- 172.21.15.156:0/3976353099 >> 172.21.15.156:0/3976353099 conn(0x7fe1440b0860 msgr2=0x7fe1440ae7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:28.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.618+0000 7fe14afff700 1 -- 172.21.15.156:0/3976353099 shutdown_connections 2023-11-28T13:15:28.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.618+0000 7fe14afff700 1 -- 172.21.15.156:0/3976353099 wait complete. 2023-11-28T13:15:28.621 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_metadata' removed 2023-11-28T13:15:28.640 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T13:15:28.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.842+0000 7f53489ff700 1 Processor -- start 2023-11-28T13:15:28.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.843+0000 7f53489ff700 1 -- start start 2023-11-28T13:15:28.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.844+0000 7f53489ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5344161200 0x7f534415dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:28.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.844+0000 7f53489ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5344160d70 con 0x7f534415a760 2023-11-28T13:15:28.846 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.844+0000 7f53489ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53440b5390 con 0x7f53440cded0 2023-11-28T13:15:28.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.844+0000 7f53489ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534415ab30 con 0x7f5344161200 2023-11-28T13:15:28.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.845+0000 7f534259c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f534415a760 0x7f53440cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39416/0 (socket says 172.21.15.156:39416) 2023-11-28T13:15:28.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.845+0000 7f534259c700 1 -- 172.21.15.156:0/3452566659 learned_addr learned my addr 172.21.15.156:0/3452566659 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:28.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.845+0000 7f5342d9d700 1 --2- 172.21.15.156:0/3452566659 >> v2:172.21.15.186:3300/0 conn(0x7f5344161200 0x7f534415dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:28.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.845+0000 7f534159a700 1 -- 172.21.15.156:0/3452566659 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345287136 0 0) 0x7f5344160d70 con 0x7f534415a760 2023-11-28T13:15:28.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.845+0000 7f534159a700 1 -- 172.21.15.156:0/3452566659 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f532c0035b0 con 0x7f534415a760 2023-11-28T13:15:28.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.845+0000 7f534159a700 1 -- 172.21.15.156:0/3452566659 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3784828903 0 0) 0x7f532c0035b0 con 0x7f534415a760 2023-11-28T13:15:28.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.845+0000 7f534159a700 1 -- 172.21.15.156:0/3452566659 >> v2:172.21.15.186:3300/0 conn(0x7f5344161200 msgr2=0x7f534415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:28.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.845+0000 7f534159a700 1 --2- 172.21.15.156:0/3452566659 >> v2:172.21.15.186:3300/0 conn(0x7f5344161200 0x7f534415dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:28.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.845+0000 7f534159a700 1 -- 172.21.15.156:0/3452566659 >> v1:172.21.15.186:6789/0 conn(0x7f53440cded0 legacy=0x7f53440d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:28.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.845+0000 7f534159a700 1 -- 172.21.15.156:0/3452566659 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53440b5670 con 0x7f534415a760 2023-11-28T13:15:28.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.845+0000 7f534159a700 1 -- 172.21.15.156:0/3452566659 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f5334001670 con 0x7f534415a760 2023-11-28T13:15:28.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.845+0000 7f5342d9d700 1 --2- 172.21.15.156:0/3452566659 >> v2:172.21.15.186:3300/0 conn(0x7f5344161200 0x7f534415dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:15:28.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.845+0000 7f534159a700 1 -- 172.21.15.156:0/3452566659 >> v1:172.21.15.156:6789/0 conn(0x7f534415a760 legacy=0x7f53440cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:28.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f534159a700 1 --2- 172.21.15.156:0/3452566659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f532c003a00 0x7f532c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:28.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f534159a700 1 --2- 172.21.15.156:0/3452566659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f532c004850 0x7f532c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:28.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f534159a700 1 --2- 172.21.15.156:0/3452566659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f532c005200 0x7f532c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:28.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f534159a700 1 -- 172.21.15.156:0/3452566659 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534415ab30 con 0x7f532c003a00 2023-11-28T13:15:28.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f534159a700 1 -- 172.21.15.156:0/3452566659 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c00c080 con 0x7f532c005200 2023-11-28T13:15:28.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f534159a700 1 -- 172.21.15.156:0/3452566659 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c00c240 con 0x7f532c004850 2023-11-28T13:15:28.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f534259c700 1 --2- 172.21.15.156:0/3452566659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f532c003a00 0x7f532c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:28.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f5342d9d700 1 --2- 172.21.15.156:0/3452566659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f532c005200 0x7f532c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:28.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f5341d9b700 1 --2- 172.21.15.156:0/3452566659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f532c004850 0x7f532c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:28.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f534259c700 1 -- 172.21.15.156:0/3452566659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f532c004850 msgr2=0x7f532c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:28.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f534259c700 1 --2- 172.21.15.156:0/3452566659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f532c004850 0x7f532c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:28.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f534259c700 1 -- 172.21.15.156:0/3452566659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f532c005200 msgr2=0x7f532c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:28.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f534259c700 1 --2- 172.21.15.156:0/3452566659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f532c005200 0x7f532c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:28.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f534259c700 1 -- 172.21.15.156:0/3452566659 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c0035b0 con 0x7f532c003a00 2023-11-28T13:15:28.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f5342d9d700 1 --2- 172.21.15.156:0/3452566659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f532c005200 0x7f532c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:15:28.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f5341d9b700 1 --2- 172.21.15.156:0/3452566659 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f532c004850 0x7f532c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:15:28.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f5342d9d700 1 -- 172.21.15.156:0/3452566659 reap_dead start 2023-11-28T13:15:28.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.846+0000 7f534259c700 1 --2- 172.21.15.156:0/3452566659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f532c003a00 0x7f532c003e10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f533400c240 tx=0x7f532c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34148cf89214a20d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:28.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.847+0000 7f534159a700 1 -- 172.21.15.156:0/3452566659 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5334007c10 con 0x7f532c003a00 2023-11-28T13:15:28.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.847+0000 7f534159a700 1 -- 172.21.15.156:0/3452566659 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533400ce00 con 0x7f532c003a00 2023-11-28T13:15:28.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.847+0000 7f534159a700 1 -- 172.21.15.156:0/3452566659 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5334005ba0 con 0x7f532c003a00 2023-11-28T13:15:28.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.847+0000 7f53489ff700 1 -- 172.21.15.156:0/3452566659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f532c003a00 msgr2=0x7f532c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:28.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.847+0000 7f53489ff700 1 --2- 172.21.15.156:0/3452566659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f532c003a00 0x7f532c003e10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f533400c240 tx=0x7f532c005880 comp rx=0 tx=0).stop 2023-11-28T13:15:28.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.847+0000 7f53489ff700 1 -- 172.21.15.156:0/3452566659 shutdown_connections 2023-11-28T13:15:28.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.847+0000 7f53489ff700 1 --2- 172.21.15.156:0/3452566659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f532c003a00 0x7f532c003e10 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:28.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.847+0000 7f53489ff700 1 -- 172.21.15.156:0/3452566659 >> 172.21.15.156:0/3452566659 conn(0x7f5344072620 msgr2=0x7f53440cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:28.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.847+0000 7f53489ff700 1 -- 172.21.15.156:0/3452566659 shutdown_connections 2023-11-28T13:15:28.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.847+0000 7f53489ff700 1 -- 172.21.15.156:0/3452566659 wait complete. 2023-11-28T13:15:28.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.849+0000 7f53489ff700 1 Processor -- start 2023-11-28T13:15:28.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.849+0000 7f53489ff700 1 -- start start 2023-11-28T13:15:28.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.850+0000 7f53489ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f532c003a00 0x7f53441f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:28.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.850+0000 7f53489ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53440ca240 0x7f53441f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:28.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.850+0000 7f53489ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53441f2c00 0x7f53441f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:28.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.850+0000 7f53489ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53441f35c0 con 0x7f53441f2c00 2023-11-28T13:15:28.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.850+0000 7f53489ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53440b6f10 con 0x7f53440ca240 2023-11-28T13:15:28.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.850+0000 7f53489ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53440b71f0 con 0x7f532c003a00 2023-11-28T13:15:28.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.850+0000 7f5342d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53441f2c00 0x7f53441f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:28.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.850+0000 7f5342d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53441f2c00 0x7f53441f3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39438/0 (socket says 172.21.15.156:39438) 2023-11-28T13:15:28.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.850+0000 7f5342d9d700 1 -- 172.21.15.156:0/3187482411 learned_addr learned my addr 172.21.15.156:0/3187482411 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:28.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.851+0000 7f5341d9b700 1 --2- 172.21.15.156:0/3187482411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53440ca240 0x7f53441f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:28.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.851+0000 7f534259c700 1 --2- 172.21.15.156:0/3187482411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f532c003a00 0x7f53441f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:28.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.851+0000 7f5342d9d700 1 -- 172.21.15.156:0/3187482411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f532c003a00 msgr2=0x7f53441f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:28.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.851+0000 7f5342d9d700 1 --2- 172.21.15.156:0/3187482411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f532c003a00 0x7f53441f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:28.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.851+0000 7f5342d9d700 1 -- 172.21.15.156:0/3187482411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53440ca240 msgr2=0x7f53441f26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:28.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.851+0000 7f5342d9d700 1 --2- 172.21.15.156:0/3187482411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53440ca240 0x7f53441f26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:28.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.851+0000 7f5342d9d700 1 -- 172.21.15.156:0/3187482411 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5344160a00 con 0x7f53441f2c00 2023-11-28T13:15:28.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.851+0000 7f534259c700 1 --2- 172.21.15.156:0/3187482411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f532c003a00 0x7f53441f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:15:28.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.851+0000 7f5342d9d700 1 --2- 172.21.15.156:0/3187482411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53441f2c00 0x7f53441f3080 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f53441605c0 tx=0x7f533c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:28.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.851+0000 7f5332ffd700 1 -- 172.21.15.156:0/3187482411 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c00e070 con 0x7f53441f2c00 2023-11-28T13:15:28.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.851+0000 7f5332ffd700 1 -- 172.21.15.156:0/3187482411 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533c011cd0 con 0x7f53441f2c00 2023-11-28T13:15:28.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.851+0000 7f53489ff700 1 -- 172.21.15.156:0/3187482411 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5344160720 con 0x7f53441f2c00 2023-11-28T13:15:28.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.851+0000 7f5332ffd700 1 -- 172.21.15.156:0/3187482411 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c011e80 con 0x7f53441f2c00 2023-11-28T13:15:28.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.851+0000 7f53489ff700 1 -- 172.21.15.156:0/3187482411 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534415dc30 con 0x7f53441f2c00 2023-11-28T13:15:28.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.852+0000 7f53489ff700 1 -- 172.21.15.156:0/3187482411 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f534404e620 con 0x7f53441f2c00 2023-11-28T13:15:28.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.853+0000 7f5332ffd700 1 -- 172.21.15.156:0/3187482411 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f533c003a20 con 0x7f53441f2c00 2023-11-28T13:15:28.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.854+0000 7f5332ffd700 1 --2- 172.21.15.156:0/3187482411 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f53240682f0 0x7f532406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:28.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.854+0000 7f5332ffd700 1 -- 172.21.15.156:0/3187482411 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5826+0+0 (secure 0 0 0) 0x7f533c086580 con 0x7f53441f2c00 2023-11-28T13:15:28.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.855+0000 7f534259c700 1 --2- 172.21.15.156:0/3187482411 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f53240682f0 0x7f532406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:28.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.856+0000 7f534259c700 1 --2- 172.21.15.156:0/3187482411 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f53240682f0 0x7f532406a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f53440b8fa0 tx=0x7f5334007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:28.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:28.860+0000 7f5332ffd700 1 -- 172.21.15.156:0/3187482411 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f533c052dd0 con 0x7f53441f2c00 2023-11-28T13:15:29.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.053+0000 7f53489ff700 1 -- 172.21.15.156:0/3187482411 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f53440d08a0 con 0x7f53441f2c00 2023-11-28T13:15:29.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.618+0000 7f5332ffd700 1 -- 172.21.15.156:0/3187482411 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v64) v1 ==== 169+0+0 (secure 0 0 0) 0x7f533c003cb0 con 0x7f53441f2c00 2023-11-28T13:15:29.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.622+0000 7f53489ff700 1 -- 172.21.15.156:0/3187482411 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f53240682f0 msgr2=0x7f532406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:29.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.622+0000 7f53489ff700 1 --2- 172.21.15.156:0/3187482411 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f53240682f0 0x7f532406a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f53440b8fa0 tx=0x7f5334007000 comp rx=0 tx=0).stop 2023-11-28T13:15:29.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.623+0000 7f53489ff700 1 -- 172.21.15.156:0/3187482411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53441f2c00 msgr2=0x7f53441f3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:29.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.623+0000 7f53489ff700 1 --2- 172.21.15.156:0/3187482411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53441f2c00 0x7f53441f3080 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f53441605c0 tx=0x7f533c009f90 comp rx=0 tx=0).stop 2023-11-28T13:15:29.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.623+0000 7f53489ff700 1 -- 172.21.15.156:0/3187482411 shutdown_connections 2023-11-28T13:15:29.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.623+0000 7f53489ff700 1 --2- 172.21.15.156:0/3187482411 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f53240682f0 0x7f532406a7a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:29.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.623+0000 7f53489ff700 1 --2- 172.21.15.156:0/3187482411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53441f2c00 0x7f53441f3080 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:29.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.623+0000 7f53489ff700 1 --2- 172.21.15.156:0/3187482411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f532c003a00 0x7f53441f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:29.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.623+0000 7f53489ff700 1 --2- 172.21.15.156:0/3187482411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53440ca240 0x7f53441f26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:29.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.623+0000 7f53489ff700 1 -- 172.21.15.156:0/3187482411 >> 172.21.15.156:0/3187482411 conn(0x7f5344072620 msgr2=0x7f534415ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:29.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.623+0000 7f53489ff700 1 -- 172.21.15.156:0/3187482411 shutdown_connections 2023-11-28T13:15:29.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.623+0000 7f53489ff700 1 -- 172.21.15.156:0/3187482411 wait complete. 2023-11-28T13:15:29.626 INFO:teuthology.orchestra.run.smithi156.stderr:pool 'cephfs_data' removed 2023-11-28T13:15:29.648 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T13:15:29.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.849+0000 7fc7f65ff700 1 Processor -- start 2023-11-28T13:15:29.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.849+0000 7fc7f65ff700 1 -- start start 2023-11-28T13:15:29.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.850+0000 7fc7f65ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc7f00d01b0 0x7fc7f00d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:29.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.850+0000 7fc7f65ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7f00b20b0 0x7fc7f00d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:29.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.850+0000 7fc7f65ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f00b0230 con 0x7fc7f00b29b0 2023-11-28T13:15:29.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.850+0000 7fc7f65ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f015cec0 con 0x7fc7f00d01b0 2023-11-28T13:15:29.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.850+0000 7fc7f65ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00b4570 con 0x7fc7f00b20b0 2023-11-28T13:15:29.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.850+0000 7fc7f4b9c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fc7f00b29b0 0x7fc7f00b2e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39424/0 (socket says 172.21.15.156:39424) 2023-11-28T13:15:29.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.850+0000 7fc7f4b9c700 1 -- 172.21.15.156:0/36247368 learned_addr learned my addr 172.21.15.156:0/36247368 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:29.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.851+0000 7fc7effff700 1 --2- 172.21.15.156:0/36247368 >> v2:172.21.15.186:3300/0 conn(0x7fc7f00d01b0 0x7fc7f00d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:29.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.851+0000 7fc7ef7fe700 1 --2- 172.21.15.156:0/36247368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7f00b20b0 0x7fc7f00d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:29.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.851+0000 7fc7eeffd700 1 -- 172.21.15.156:0/36247368 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965981915 0 0) 0x7fc7f00b0230 con 0x7fc7f00b29b0 2023-11-28T13:15:29.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.851+0000 7fc7eeffd700 1 -- 172.21.15.156:0/36247368 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7d80035b0 con 0x7fc7f00b29b0 2023-11-28T13:15:29.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.851+0000 7fc7effff700 1 -- 172.21.15.156:0/36247368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7f00b20b0 msgr2=0x7fc7f00d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:29.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.851+0000 7fc7effff700 1 --2- 172.21.15.156:0/36247368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7f00b20b0 0x7fc7f00d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:29.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.851+0000 7fc7eeffd700 1 -- 172.21.15.156:0/36247368 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1081232956 0 0) 0x7fc7d80035b0 con 0x7fc7f00b29b0 2023-11-28T13:15:29.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.851+0000 7fc7effff700 1 -- 172.21.15.156:0/36247368 >> v1:172.21.15.156:6789/0 conn(0x7fc7f00b29b0 legacy=0x7fc7f00b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:29.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.851+0000 7fc7effff700 1 -- 172.21.15.156:0/36247368 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f00b4850 con 0x7fc7f00d01b0 2023-11-28T13:15:29.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.851+0000 7fc7effff700 1 --2- 172.21.15.156:0/36247368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7f00d01b0 0x7fc7f00d0c20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009840 tx=0x7fc7e0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b80a34c86c1208a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:29.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.852+0000 7fc7eeffd700 1 -- 172.21.15.156:0/36247368 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e80019f0 con 0x7fc7f00d01b0 2023-11-28T13:15:29.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.852+0000 7fc7eeffd700 1 -- 172.21.15.156:0/36247368 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e0004030 con 0x7fc7f00d01b0 2023-11-28T13:15:29.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.852+0000 7fc7eeffd700 1 -- 172.21.15.156:0/36247368 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e00043c0 con 0x7fc7f00d01b0 2023-11-28T13:15:29.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.852+0000 7fc7f65ff700 1 -- 172.21.15.156:0/36247368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7f00d01b0 msgr2=0x7fc7f00d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:29.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.852+0000 7fc7f65ff700 1 --2- 172.21.15.156:0/36247368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7f00d01b0 0x7fc7f00d0c20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009840 tx=0x7fc7e0009b50 comp rx=0 tx=0).stop 2023-11-28T13:15:29.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.852+0000 7fc7f65ff700 1 -- 172.21.15.156:0/36247368 shutdown_connections 2023-11-28T13:15:29.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.852+0000 7fc7f65ff700 1 --2- 172.21.15.156:0/36247368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7f00d01b0 0x7fc7f00d0c20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:29.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.852+0000 7fc7f65ff700 1 --2- 172.21.15.156:0/36247368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7f00b20b0 0x7fc7f00d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:29.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.852+0000 7fc7f65ff700 1 -- 172.21.15.156:0/36247368 >> 172.21.15.156:0/36247368 conn(0x7fc7f009c3f0 msgr2=0x7fc7f00d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:29.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.852+0000 7fc7f65ff700 1 -- 172.21.15.156:0/36247368 shutdown_connections 2023-11-28T13:15:29.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.852+0000 7fc7f65ff700 1 -- 172.21.15.156:0/36247368 wait complete. 2023-11-28T13:15:29.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.854+0000 7fc7f65ff700 1 Processor -- start 2023-11-28T13:15:29.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.854+0000 7fc7f65ff700 1 -- start start 2023-11-28T13:15:29.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.855+0000 7fc7f65ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f00b20b0 0x7fc7f00cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:29.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.855+0000 7fc7f65ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7f00b29b0 0x7fc7f00b8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:29.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.855+0000 7fc7f65ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7f00d01b0 0x7fc7f015f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:29.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.855+0000 7fc7f65ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00cc570 con 0x7fc7f00b20b0 2023-11-28T13:15:29.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.855+0000 7fc7f65ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00cc850 con 0x7fc7f00b29b0 2023-11-28T13:15:29.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.855+0000 7fc7f65ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00b6020 con 0x7fc7f00d01b0 2023-11-28T13:15:29.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.855+0000 7fc7effff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f00b20b0 0x7fc7f00cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:29.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.855+0000 7fc7effff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f00b20b0 0x7fc7f00cbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39440/0 (socket says 172.21.15.156:39440) 2023-11-28T13:15:29.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.855+0000 7fc7effff700 1 -- 172.21.15.156:0/2906603556 learned_addr learned my addr 172.21.15.156:0/2906603556 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:29.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.855+0000 7fc7ef7fe700 1 --2- 172.21.15.156:0/2906603556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7f00b29b0 0x7fc7f00b8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:29.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.855+0000 7fc7f4b9c700 1 --2- 172.21.15.156:0/2906603556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7f00d01b0 0x7fc7f015f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:29.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.856+0000 7fc7effff700 1 -- 172.21.15.156:0/2906603556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7f00d01b0 msgr2=0x7fc7f015f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:29.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.856+0000 7fc7effff700 1 --2- 172.21.15.156:0/2906603556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7f00d01b0 0x7fc7f015f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:29.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.856+0000 7fc7effff700 1 -- 172.21.15.156:0/2906603556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7f00b29b0 msgr2=0x7fc7f00b8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:29.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.856+0000 7fc7effff700 1 --2- 172.21.15.156:0/2906603556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7f00b29b0 0x7fc7f00b8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:29.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.856+0000 7fc7effff700 1 -- 172.21.15.156:0/2906603556 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e00094a0 con 0x7fc7f00b20b0 2023-11-28T13:15:29.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.856+0000 7fc7effff700 1 --2- 172.21.15.156:0/2906603556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f00b20b0 0x7fc7f00cbe80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000a010 tx=0x7fc7e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:29.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.856+0000 7fc7ecff9700 1 -- 172.21.15.156:0/2906603556 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0003ae0 con 0x7fc7f00b20b0 2023-11-28T13:15:29.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.856+0000 7fc7ecff9700 1 -- 172.21.15.156:0/2906603556 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e001f070 con 0x7fc7f00b20b0 2023-11-28T13:15:29.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.856+0000 7fc7f65ff700 1 -- 172.21.15.156:0/2906603556 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f00afec0 con 0x7fc7f00b20b0 2023-11-28T13:15:29.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.856+0000 7fc7ecff9700 1 -- 172.21.15.156:0/2906603556 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0003ea0 con 0x7fc7f00b20b0 2023-11-28T13:15:29.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.856+0000 7fc7f65ff700 1 -- 172.21.15.156:0/2906603556 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f00afbe0 con 0x7fc7f00b20b0 2023-11-28T13:15:29.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.857+0000 7fc7f65ff700 1 -- 172.21.15.156:0/2906603556 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f00b2600 con 0x7fc7f00b20b0 2023-11-28T13:15:29.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.858+0000 7fc7ecff9700 1 -- 172.21.15.156:0/2906603556 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc7e002a020 con 0x7fc7f00b20b0 2023-11-28T13:15:29.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.859+0000 7fc7ecff9700 1 --2- 172.21.15.156:0/2906603556 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc7dc0682f0 0x7fc7dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:29.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.859+0000 7fc7ecff9700 1 -- 172.21.15.156:0/2906603556 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc7e0086db0 con 0x7fc7f00b20b0 2023-11-28T13:15:29.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.859+0000 7fc7ef7fe700 1 --2- 172.21.15.156:0/2906603556 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc7dc0682f0 0x7fc7dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:29.863 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.860+0000 7fc7ef7fe700 1 --2- 172.21.15.156:0/2906603556 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc7dc0682f0 0x7fc7dc06a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009be0 tx=0x7fc7e4009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:29.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:29.863+0000 7fc7ecff9700 1 -- 172.21.15.156:0/2906603556 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc7e008b070 con 0x7fc7f00b20b0 2023-11-28T13:15:30.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.064+0000 7fc7f65ff700 1 -- 172.21.15.156:0/2906603556 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc7f00b5e80 con 0x7fc7f00b20b0 2023-11-28T13:15:30.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.070+0000 7fc7ecff9700 1 -- 172.21.15.156:0/2906603556 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc7f00b5e80 con 0x7fc7f00b20b0 2023-11-28T13:15:30.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.072+0000 7fc7f65ff700 1 -- 172.21.15.156:0/2906603556 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc7dc0682f0 msgr2=0x7fc7dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:30.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.072+0000 7fc7f65ff700 1 --2- 172.21.15.156:0/2906603556 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc7dc0682f0 0x7fc7dc06a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009be0 tx=0x7fc7e4009210 comp rx=0 tx=0).stop 2023-11-28T13:15:30.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.072+0000 7fc7f65ff700 1 -- 172.21.15.156:0/2906603556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f00b20b0 msgr2=0x7fc7f00cbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:30.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.072+0000 7fc7f65ff700 1 --2- 172.21.15.156:0/2906603556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f00b20b0 0x7fc7f00cbe80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000a010 tx=0x7fc7e0004750 comp rx=0 tx=0).stop 2023-11-28T13:15:30.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.072+0000 7fc7f65ff700 1 -- 172.21.15.156:0/2906603556 shutdown_connections 2023-11-28T13:15:30.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.072+0000 7fc7f65ff700 1 --2- 172.21.15.156:0/2906603556 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc7dc0682f0 0x7fc7dc06a7a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:30.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.072+0000 7fc7f65ff700 1 --2- 172.21.15.156:0/2906603556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7f00d01b0 0x7fc7f015f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:30.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.072+0000 7fc7f65ff700 1 --2- 172.21.15.156:0/2906603556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f00b20b0 0x7fc7f00cbe80 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:30.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.072+0000 7fc7f65ff700 1 --2- 172.21.15.156:0/2906603556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7f00b29b0 0x7fc7f00b8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:30.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.072+0000 7fc7f65ff700 1 -- 172.21.15.156:0/2906603556 >> 172.21.15.156:0/2906603556 conn(0x7fc7f009c3f0 msgr2=0x7fc7f006b840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:30.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.072+0000 7fc7f65ff700 1 -- 172.21.15.156:0/2906603556 shutdown_connections 2023-11-28T13:15:30.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.072+0000 7fc7f65ff700 1 -- 172.21.15.156:0/2906603556 wait complete. 2023-11-28T13:15:30.092 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T13:15:30.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.293+0000 7f34ed5ff700 1 Processor -- start 2023-11-28T13:15:30.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.293+0000 7f34ed5ff700 1 -- start start 2023-11-28T13:15:30.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.294+0000 7f34ed5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34e8161340 0x7f34e80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:30.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.294+0000 7f34ed5ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f34e80d2320 0x7f34e80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:30.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.294+0000 7f34ed5ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e80d1950 con 0x7f34e815c9b0 2023-11-28T13:15:30.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.294+0000 7f34ed5ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e80b5800 con 0x7f34e80d2320 2023-11-28T13:15:30.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.294+0000 7f34ed5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e80b5300 con 0x7f34e8161340 2023-11-28T13:15:30.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.294+0000 7f34e77fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f34e80d2320 0x7f34e80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:30.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.294+0000 7f34e77fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f34e80d2320 0x7f34e80cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39452/0 (socket says 172.21.15.156:39452) 2023-11-28T13:15:30.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.294+0000 7f34e77fe700 1 -- 172.21.15.156:0/1064968710 learned_addr learned my addr 172.21.15.156:0/1064968710 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:30.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.295+0000 7f34e67fc700 1 --2- 172.21.15.156:0/1064968710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34e8161340 0x7f34e80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:30.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.295+0000 7f34e77fe700 1 -- 172.21.15.156:0/1064968710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34e8161340 msgr2=0x7f34e80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:30.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.295+0000 7f34e77fe700 1 --2- 172.21.15.156:0/1064968710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34e8161340 0x7f34e80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:30.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.295+0000 7f34e77fe700 1 -- 172.21.15.156:0/1064968710 >> v1:172.21.15.186:6789/0 conn(0x7f34e815c9b0 legacy=0x7f34e80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:30.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.295+0000 7f34e77fe700 1 -- 172.21.15.156:0/1064968710 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e80b5520 con 0x7f34e80d2320 2023-11-28T13:15:30.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.295+0000 7f34e77fe700 1 --2- 172.21.15.156:0/1064968710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34e80d2320 0x7f34e80cd7c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009e80 tx=0x7f34e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c071dd03e50ae2c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:30.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.295+0000 7f34e5ffb700 1 -- 172.21.15.156:0/1064968710 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e000e070 con 0x7f34e80d2320 2023-11-28T13:15:30.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.295+0000 7f34e5ffb700 1 -- 172.21.15.156:0/1064968710 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e000b8d0 con 0x7f34e80d2320 2023-11-28T13:15:30.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.295+0000 7f34e5ffb700 1 -- 172.21.15.156:0/1064968710 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e000bc80 con 0x7f34e80d2320 2023-11-28T13:15:30.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.296+0000 7f34ed5ff700 1 -- 172.21.15.156:0/1064968710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34e80d2320 msgr2=0x7f34e80cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:30.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.296+0000 7f34ed5ff700 1 --2- 172.21.15.156:0/1064968710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34e80d2320 0x7f34e80cd7c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009e80 tx=0x7f34e000b330 comp rx=0 tx=0).stop 2023-11-28T13:15:30.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.296+0000 7f34ed5ff700 1 -- 172.21.15.156:0/1064968710 shutdown_connections 2023-11-28T13:15:30.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.296+0000 7f34ed5ff700 1 --2- 172.21.15.156:0/1064968710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34e80d2320 0x7f34e80cd7c0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:30.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.296+0000 7f34ed5ff700 1 --2- 172.21.15.156:0/1064968710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34e8161340 0x7f34e80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:30.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.296+0000 7f34ed5ff700 1 -- 172.21.15.156:0/1064968710 >> 172.21.15.156:0/1064968710 conn(0x7f34e8072600 msgr2=0x7f34e80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:30.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.296+0000 7f34ed5ff700 1 -- 172.21.15.156:0/1064968710 shutdown_connections 2023-11-28T13:15:30.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.296+0000 7f34ed5ff700 1 -- 172.21.15.156:0/1064968710 wait complete. 2023-11-28T13:15:30.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.298+0000 7f34ed5ff700 1 Processor -- start 2023-11-28T13:15:30.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.298+0000 7f34ed5ff700 1 -- start start 2023-11-28T13:15:30.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.299+0000 7f34ed5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34e80d2320 0x7f34e8150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:30.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.299+0000 7f34ed5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e815c9b0 0x7f34e814fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:30.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.299+0000 7f34ed5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34e8161340 0x7f34e81e7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:30.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.299+0000 7f34ed5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e81e7e80 con 0x7f34e80d2320 2023-11-28T13:15:30.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.299+0000 7f34ed5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e80c85b0 con 0x7f34e815c9b0 2023-11-28T13:15:30.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.299+0000 7f34ed5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e80c8890 con 0x7f34e8161340 2023-11-28T13:15:30.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.299+0000 7f34e6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34e80d2320 0x7f34e8150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:30.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.299+0000 7f34e6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34e80d2320 0x7f34e8150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39468/0 (socket says 172.21.15.156:39468) 2023-11-28T13:15:30.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.299+0000 7f34e6ffd700 1 -- 172.21.15.156:0/3583515401 learned_addr learned my addr 172.21.15.156:0/3583515401 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:30.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.299+0000 7f34e67fc700 1 --2- 172.21.15.156:0/3583515401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e815c9b0 0x7f34e814fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:30.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.299+0000 7f34e77fe700 1 --2- 172.21.15.156:0/3583515401 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34e8161340 0x7f34e81e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:30.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.300+0000 7f34e6ffd700 1 -- 172.21.15.156:0/3583515401 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34e8161340 msgr2=0x7f34e81e7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:30.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.300+0000 7f34e6ffd700 1 --2- 172.21.15.156:0/3583515401 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34e8161340 0x7f34e81e7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:30.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.300+0000 7f34e6ffd700 1 -- 172.21.15.156:0/3583515401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e815c9b0 msgr2=0x7f34e814fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:30.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.300+0000 7f34e6ffd700 1 --2- 172.21.15.156:0/3583515401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e815c9b0 0x7f34e814fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:30.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.300+0000 7f34e6ffd700 1 -- 172.21.15.156:0/3583515401 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e0009b30 con 0x7f34e80d2320 2023-11-28T13:15:30.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.300+0000 7f34e67fc700 1 --2- 172.21.15.156:0/3583515401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e815c9b0 0x7f34e814fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:15:30.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.300+0000 7f34e6ffd700 1 --2- 172.21.15.156:0/3583515401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34e80d2320 0x7f34e8150690 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f34d800c450 tx=0x7f34d800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:30.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.300+0000 7f34cf7fe700 1 -- 172.21.15.156:0/3583515401 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d80052d0 con 0x7f34e80d2320 2023-11-28T13:15:30.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.300+0000 7f34cf7fe700 1 -- 172.21.15.156:0/3583515401 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d8005430 con 0x7f34e80d2320 2023-11-28T13:15:30.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.300+0000 7f34ed5ff700 1 -- 172.21.15.156:0/3583515401 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e80d15e0 con 0x7f34e80d2320 2023-11-28T13:15:30.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.300+0000 7f34cf7fe700 1 -- 172.21.15.156:0/3583515401 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d80038c0 con 0x7f34e80d2320 2023-11-28T13:15:30.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.300+0000 7f34ed5ff700 1 -- 172.21.15.156:0/3583515401 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e80d1300 con 0x7f34e80d2320 2023-11-28T13:15:30.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.301+0000 7f34ed5ff700 1 -- 172.21.15.156:0/3583515401 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e815cd80 con 0x7f34e80d2320 2023-11-28T13:15:30.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.302+0000 7f34cf7fe700 1 -- 172.21.15.156:0/3583515401 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f34d8019030 con 0x7f34e80d2320 2023-11-28T13:15:30.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.303+0000 7f34cf7fe700 1 --2- 172.21.15.156:0/3583515401 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f34d40682f0 0x7f34d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:30.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.303+0000 7f34cf7fe700 1 -- 172.21.15.156:0/3583515401 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f34d8084c10 con 0x7f34e80d2320 2023-11-28T13:15:30.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.304+0000 7f34e67fc700 1 --2- 172.21.15.156:0/3583515401 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f34d40682f0 0x7f34d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:30.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.305+0000 7f34e67fc700 1 --2- 172.21.15.156:0/3583515401 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f34d40682f0 0x7f34d406a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f34e81f5ed0 tx=0x7f34dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:30.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.307+0000 7f34cf7fe700 1 -- 172.21.15.156:0/3583515401 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f34d8051770 con 0x7f34e80d2320 2023-11-28T13:15:30.575 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.573+0000 7f34ed5ff700 1 -- 172.21.15.156:0/3583515401 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f34e80745c0 con 0x7f34e80d2320 2023-11-28T13:15:30.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.627+0000 7f34cf7fe700 1 -- 172.21.15.156:0/3583515401 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v333) v1 ==== 169+0+0 (secure 0 0 0) 0x7f34d80170f0 con 0x7f34e80d2320 2023-11-28T13:15:30.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.630+0000 7f34ed5ff700 1 -- 172.21.15.156:0/3583515401 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f34d40682f0 msgr2=0x7f34d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:30.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.630+0000 7f34ed5ff700 1 --2- 172.21.15.156:0/3583515401 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f34d40682f0 0x7f34d406a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f34e81f5ed0 tx=0x7f34dc008040 comp rx=0 tx=0).stop 2023-11-28T13:15:30.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.630+0000 7f34ed5ff700 1 -- 172.21.15.156:0/3583515401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34e80d2320 msgr2=0x7f34e8150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:30.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.630+0000 7f34ed5ff700 1 --2- 172.21.15.156:0/3583515401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34e80d2320 0x7f34e8150690 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f34d800c450 tx=0x7f34d800c760 comp rx=0 tx=0).stop 2023-11-28T13:15:30.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.630+0000 7f34ed5ff700 1 -- 172.21.15.156:0/3583515401 shutdown_connections 2023-11-28T13:15:30.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.630+0000 7f34ed5ff700 1 --2- 172.21.15.156:0/3583515401 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f34d40682f0 0x7f34d406a7a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:30.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.630+0000 7f34ed5ff700 1 --2- 172.21.15.156:0/3583515401 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34e8161340 0x7f34e81e7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:30.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.630+0000 7f34ed5ff700 1 --2- 172.21.15.156:0/3583515401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34e80d2320 0x7f34e8150690 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:30.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.630+0000 7f34ed5ff700 1 --2- 172.21.15.156:0/3583515401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34e815c9b0 0x7f34e814fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:30.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.630+0000 7f34ed5ff700 1 -- 172.21.15.156:0/3583515401 >> 172.21.15.156:0/3583515401 conn(0x7f34e8072600 msgr2=0x7f34e80b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:30.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.630+0000 7f34ed5ff700 1 -- 172.21.15.156:0/3583515401 shutdown_connections 2023-11-28T13:15:30.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:30.630+0000 7f34ed5ff700 1 -- 172.21.15.156:0/3583515401 wait complete. 2023-11-28T13:15:30.650 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ERROR 2023-11-28T13:15:30.651 INFO:tasks.cephfs_test_runner: 2023-11-28T13:15:30.651 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T13:15:30.651 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:15:30.651 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T13:15:30.651 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T13:15:30.651 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T13:15:30.651 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T13:15:30.651 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=200) 2023-11-28T13:15:30.651 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T13:15:30.651 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T13:15:30.651 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 200s and 0 retries 2023-11-28T13:15:30.652 INFO:tasks.cephfs_test_runner: 2023-11-28T13:15:30.652 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T13:15:30.652 INFO:tasks.cephfs_test_runner:Ran 3 tests in 412.658s 2023-11-28T13:15:30.652 INFO:tasks.cephfs_test_runner: 2023-11-28T13:15:30.652 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-28T13:15:30.652 INFO:tasks.cephfs_test_runner: 2023-11-28T13:15:30.652 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T13:15:30.652 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:15:30.652 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T13:15:30.652 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T13:15:30.652 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T13:15:30.652 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T13:15:30.653 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=200) 2023-11-28T13:15:30.653 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T13:15:30.653 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T13:15:30.653 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 200s and 0 retries 2023-11-28T13:15:30.653 INFO:tasks.cephfs_test_runner: 2023-11-28T13:15:30.653 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:15:30.725 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=7ecd282109e64400a18f20efba006070 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:15:30.727 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-28T13:15:30.736 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-28T13:15:30.758 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-28T13:15:30.759 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:15:30.759 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:15:30.806 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:15:30.807 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-28T13:15:30.807 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:15:30.890 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:15:30.890 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:15:30.890 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:15:30.939 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:15:30.940 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-28T13:15:30.940 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:15:31.027 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348340k,nr_inodes=4087085,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:15:31.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:15:31.028 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T13:15:31.073 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T13:15:31.073 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T13:15:31.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:15:31.074 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:15:31.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:15:31.183 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T13:15:31.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:15:31.254 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link delete ceph-brx 2023-11-28T13:15:31.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:15:31.382 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-28T13:15:31.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:15:31.429 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link delete ceph-brx 2023-11-28T13:15:31.503 INFO:teuthology.orchestra.run.smithi156.stderr:Cannot find device "ceph-brx" 2023-11-28T13:15:31.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:15:31.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:15:31.506 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2023-11-28T13:15:31.555 INFO:teuthology.orchestra.run.smithi186.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T13:15:31.555 INFO:teuthology.orchestra.run.smithi186.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-28T13:15:31.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:15:31.557 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T13:15:31.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:15:31.629 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T13:15:31.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:15:31.703 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link delete ceph-brx 2023-11-28T13:15:31.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:15:31.825 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2023-11-28T13:15:31.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:15:31.872 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link delete ceph-brx 2023-11-28T13:15:31.945 INFO:teuthology.orchestra.run.smithi186.stderr:Cannot find device "ceph-brx" 2023-11-28T13:15:31.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:15:31.950 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-28T13:15:31.959 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T13:15:31.960 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:15:32.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.157+0000 7fabe1bff700 1 Processor -- start 2023-11-28T13:15:32.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.158+0000 7fabe1bff700 1 -- start start 2023-11-28T13:15:32.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.159+0000 7fabe1bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fabdc158200 0x7fabdc070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:32.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.159+0000 7fabe1bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fabdc0caa70 0x7fabdc0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:32.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.159+0000 7fabe1bff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabdc15e1a0 con 0x7fabdc0caf10 2023-11-28T13:15:32.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.159+0000 7fabe1bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabdc15f070 con 0x7fabdc158200 2023-11-28T13:15:32.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.159+0000 7fabe1bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc071190 con 0x7fabdc0caa70 2023-11-28T13:15:32.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.159+0000 7fabdb7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fabdc158200 0x7fabdc070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:32.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.159+0000 7fabdb7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fabdc158200 0x7fabdc070c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39480/0 (socket says 172.21.15.156:39480) 2023-11-28T13:15:32.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.159+0000 7fabdb7fe700 1 -- 172.21.15.156:0/3566704023 learned_addr learned my addr 172.21.15.156:0/3566704023 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:32.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.159+0000 7fabd3fff700 1 --2- 172.21.15.156:0/3566704023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fabdc0caa70 0x7fabdc0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:32.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.160+0000 7fabdb7fe700 1 -- 172.21.15.156:0/3566704023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fabdc0caa70 msgr2=0x7fabdc0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:32.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.160+0000 7fabdb7fe700 1 --2- 172.21.15.156:0/3566704023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fabdc0caa70 0x7fabdc0b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.160+0000 7fabdb7fe700 1 -- 172.21.15.156:0/3566704023 >> v1:172.21.15.186:6789/0 conn(0x7fabdc0caf10 legacy=0x7fabdc15add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:32.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.160+0000 7fabdb7fe700 1 -- 172.21.15.156:0/3566704023 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc1585d0 con 0x7fabdc158200 2023-11-28T13:15:32.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.160+0000 7fabdaffd700 1 -- 172.21.15.156:0/3566704023 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161842067 0 0) 0x7fabdc15e1a0 con 0x7fabdc0caf10 2023-11-28T13:15:32.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.160+0000 7fabdb7fe700 1 --2- 172.21.15.156:0/3566704023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc158200 0x7fabdc070c50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fabcc009920 tx=0x7fabcc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765cdeb83cd28c7d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:32.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.160+0000 7fabdaffd700 1 -- 172.21.15.156:0/3566704023 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc011830 con 0x7fabdc158200 2023-11-28T13:15:32.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.160+0000 7fabdaffd700 1 -- 172.21.15.156:0/3566704023 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc011990 con 0x7fabdc158200 2023-11-28T13:15:32.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.160+0000 7fabdaffd700 1 -- 172.21.15.156:0/3566704023 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc011d20 con 0x7fabdc158200 2023-11-28T13:15:32.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.161+0000 7fabe1bff700 1 -- 172.21.15.156:0/3566704023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc158200 msgr2=0x7fabdc070c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:32.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.161+0000 7fabe1bff700 1 --2- 172.21.15.156:0/3566704023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc158200 0x7fabdc070c50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fabcc009920 tx=0x7fabcc009c30 comp rx=0 tx=0).stop 2023-11-28T13:15:32.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.161+0000 7fabe1bff700 1 -- 172.21.15.156:0/3566704023 shutdown_connections 2023-11-28T13:15:32.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.161+0000 7fabe1bff700 1 --2- 172.21.15.156:0/3566704023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc158200 0x7fabdc070c50 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.161+0000 7fabe1bff700 1 --2- 172.21.15.156:0/3566704023 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fabdc0caa70 0x7fabdc0b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.161+0000 7fabe1bff700 1 -- 172.21.15.156:0/3566704023 >> 172.21.15.156:0/3566704023 conn(0x7fabdc0cb670 msgr2=0x7fabdc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:32.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.161+0000 7fabe1bff700 1 -- 172.21.15.156:0/3566704023 shutdown_connections 2023-11-28T13:15:32.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.161+0000 7fabe1bff700 1 -- 172.21.15.156:0/3566704023 wait complete. 2023-11-28T13:15:32.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.163+0000 7fabe1bff700 1 Processor -- start 2023-11-28T13:15:32.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.163+0000 7fabe1bff700 1 -- start start 2023-11-28T13:15:32.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.164+0000 7fabe1bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabdc0caa70 0x7fabdc155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:32.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.164+0000 7fabe1bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc0caf10 0x7fabdc0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:32.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.164+0000 7fabe1bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fabdc158200 0x7fabdc0c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:32.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.164+0000 7fabe1bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0acf30 con 0x7fabdc0caf10 2023-11-28T13:15:32.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.164+0000 7fabe1bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0ac900 con 0x7fabdc0caa70 2023-11-28T13:15:32.168 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.164+0000 7fabe1bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0af8b0 con 0x7fabdc158200 2023-11-28T13:15:32.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.164+0000 7fabd3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc0caf10 0x7fabdc0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:32.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.164+0000 7fabdb7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabdc0caa70 0x7fabdc155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:32.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.164+0000 7fabdb7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabdc0caa70 0x7fabdc155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:40514/0 (socket says 172.21.15.156:40514) 2023-11-28T13:15:32.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.164+0000 7fabdb7fe700 1 -- 172.21.15.156:0/2542618383 learned_addr learned my addr 172.21.15.156:0/2542618383 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:32.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.164+0000 7fabdbfff700 1 --2- 172.21.15.156:0/2542618383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fabdc158200 0x7fabdc0c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:32.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.164+0000 7fabd3fff700 1 -- 172.21.15.156:0/2542618383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fabdc158200 msgr2=0x7fabdc0c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:32.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.164+0000 7fabd3fff700 1 --2- 172.21.15.156:0/2542618383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fabdc158200 0x7fabdc0c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.164+0000 7fabd3fff700 1 -- 172.21.15.156:0/2542618383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabdc0caa70 msgr2=0x7fabdc155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:32.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.164+0000 7fabd3fff700 1 --2- 172.21.15.156:0/2542618383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabdc0caa70 0x7fabdc155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.164+0000 7fabd3fff700 1 -- 172.21.15.156:0/2542618383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabcc009550 con 0x7fabdc0caf10 2023-11-28T13:15:32.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.164+0000 7fabdb7fe700 1 --2- 172.21.15.156:0/2542618383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabdc0caa70 0x7fabdc155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:15:32.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.165+0000 7fabd3fff700 1 --2- 172.21.15.156:0/2542618383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc0caf10 0x7fabdc0aef40 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fabc400e900 tx=0x7fabc400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:32.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.165+0000 7fabd8ff9700 1 -- 172.21.15.156:0/2542618383 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc4013680 con 0x7fabdc0caf10 2023-11-28T13:15:32.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.165+0000 7fabd8ff9700 1 -- 172.21.15.156:0/2542618383 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc40137e0 con 0x7fabdc0caf10 2023-11-28T13:15:32.170 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.165+0000 7fabd8ff9700 1 -- 172.21.15.156:0/2542618383 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc4013b50 con 0x7fabdc0caf10 2023-11-28T13:15:32.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.165+0000 7fabe1bff700 1 -- 172.21.15.156:0/2542618383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabdc15e6d0 con 0x7fabdc0caf10 2023-11-28T13:15:32.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.165+0000 7fabe1bff700 1 -- 172.21.15.156:0/2542618383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabdc15e3f0 con 0x7fabdc0caf10 2023-11-28T13:15:32.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.166+0000 7fabe1bff700 1 -- 172.21.15.156:0/2542618383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabdc15e9b0 con 0x7fabdc0caf10 2023-11-28T13:15:32.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.167+0000 7fabd8ff9700 1 -- 172.21.15.156:0/2542618383 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fabc4019030 con 0x7fabdc0caf10 2023-11-28T13:15:32.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.168+0000 7fabd8ff9700 1 --2- 172.21.15.156:0/2542618383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fabbc0682f0 0x7fabbc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:32.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.168+0000 7fabd8ff9700 1 -- 172.21.15.156:0/2542618383 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fabc4085b20 con 0x7fabdc0caf10 2023-11-28T13:15:32.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.168+0000 7fabdb7fe700 1 --2- 172.21.15.156:0/2542618383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fabbc0682f0 0x7fabbc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:32.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.169+0000 7fabdb7fe700 1 --2- 172.21.15.156:0/2542618383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fabbc0682f0 0x7fabbc06a7a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00a010 tx=0x7fabcc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:32.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.171+0000 7fabd8ff9700 1 -- 172.21.15.156:0/2542618383 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fabc40525b0 con 0x7fabdc0caf10 2023-11-28T13:15:32.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.359+0000 7fabe1bff700 1 -- 172.21.15.156:0/2542618383 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fabdc06fd60 con 0x7fabbc0682f0 2023-11-28T13:15:32.364 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.363+0000 7fabd8ff9700 1 -- 172.21.15.156:0/2542618383 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73022 (secure 0 0 0) 0x7fabdc06fd60 con 0x7fabbc0682f0 2023-11-28T13:15:32.365 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:15:32.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.365+0000 7fabe1bff700 1 -- 172.21.15.156:0/2542618383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fabbc0682f0 msgr2=0x7fabbc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:32.367 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.366+0000 7fabe1bff700 1 --2- 172.21.15.156:0/2542618383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fabbc0682f0 0x7fabbc06a7a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00a010 tx=0x7fabcc008030 comp rx=0 tx=0).stop 2023-11-28T13:15:32.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.366+0000 7fabe1bff700 1 -- 172.21.15.156:0/2542618383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc0caf10 msgr2=0x7fabdc0aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:32.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.366+0000 7fabe1bff700 1 --2- 172.21.15.156:0/2542618383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc0caf10 0x7fabdc0aef40 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fabc400e900 tx=0x7fabc400ec10 comp rx=0 tx=0).stop 2023-11-28T13:15:32.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.366+0000 7fabe1bff700 1 -- 172.21.15.156:0/2542618383 shutdown_connections 2023-11-28T13:15:32.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.366+0000 7fabe1bff700 1 --2- 172.21.15.156:0/2542618383 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fabbc0682f0 0x7fabbc06a7a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.366+0000 7fabe1bff700 1 --2- 172.21.15.156:0/2542618383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fabdc158200 0x7fabdc0c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.366+0000 7fabe1bff700 1 --2- 172.21.15.156:0/2542618383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabdc0caa70 0x7fabdc155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.366+0000 7fabe1bff700 1 --2- 172.21.15.156:0/2542618383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc0caf10 0x7fabdc0aef40 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.366+0000 7fabe1bff700 1 -- 172.21.15.156:0/2542618383 >> 172.21.15.156:0/2542618383 conn(0x7fabdc0cb670 msgr2=0x7fabdc15ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:32.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.366+0000 7fabe1bff700 1 -- 172.21.15.156:0/2542618383 shutdown_connections 2023-11-28T13:15:32.369 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.366+0000 7fabe1bff700 1 -- 172.21.15.156:0/2542618383 wait complete. 2023-11-28T13:15:32.369 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T13:15:32.385 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2023-11-28T13:15:32.134137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":100436,"kb_used_data":7124,"kb_used_omap":0,"kb_used_meta":93248,"kb_avail":754874284,"statfs":{"total":773094113280,"available":772991266816,"internally_reserved":0,"allocated":7294976,"data_stored":3881970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95485952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12572,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-80,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.529920"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:15:25.993449+0000","last_change":"2023-11-28T13:07:56.890378+0000","last_active":"2023-11-28T13:15:25.993449+0000","last_peered":"2023-11-28T13:15:25.993449+0000","last_clean":"2023-11-28T13:15:25.993449+0000","last_became_active":"2023-11-28T13:07:56.890058+0000","last_became_peered":"2023-11-28T13:07:56.890058+0000","last_unstale":"2023-11-28T13:15:25.993449+0000","last_undegraded":"2023-11-28T13:15:25.993449+0000","last_fullsized":"2023-11-28T13:15:25.993449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:05.344602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":56,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:29.649906+0000","last_change":"2023-11-28T13:07:56.844509+0000","last_active":"2023-11-28T13:15:29.649906+0000","last_peered":"2023-11-28T13:15:29.649906+0000","last_clean":"2023-11-28T13:15:29.649906+0000","last_became_active":"2023-11-28T13:07:56.844081+0000","last_became_peered":"2023-11-28T13:07:56.844081+0000","last_unstale":"2023-11-28T13:15:29.649906+0000","last_undegraded":"2023-11-28T13:15:29.649906+0000","last_fullsized":"2023-11-28T13:15:29.649906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":54,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.858689+0000","last_change":"2023-11-28T13:07:56.844699+0000","last_active":"2023-11-28T13:15:28.858689+0000","last_peered":"2023-11-28T13:15:28.858689+0000","last_clean":"2023-11-28T13:15:28.858689+0000","last_became_active":"2023-11-28T13:07:56.844391+0000","last_became_peered":"2023-11-28T13:07:56.844391+0000","last_unstale":"2023-11-28T13:15:28.858689+0000","last_undegraded":"2023-11-28T13:15:28.858689+0000","last_fullsized":"2023-11-28T13:15:28.858689+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":205,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.619311+0000","last_change":"2023-11-28T13:07:57.844020+0000","last_active":"2023-11-28T13:15:28.619311+0000","last_peered":"2023-11-28T13:15:28.619311+0000","last_clean":"2023-11-28T13:15:28.619311+0000","last_became_active":"2023-11-28T13:07:57.842981+0000","last_became_peered":"2023-11-28T13:07:57.842981+0000","last_unstale":"2023-11-28T13:15:28.619311+0000","last_undegraded":"2023-11-28T13:15:28.619311+0000","last_fullsized":"2023-11-28T13:15:28.619311+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_clean_scrub_stamp":"2023-11-28T13:07:56.825711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:44.973218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":56,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.621568+0000","last_change":"2023-11-28T13:07:57.306318+0000","last_active":"2023-11-28T13:15:28.621568+0000","last_peered":"2023-11-28T13:15:28.621568+0000","last_clean":"2023-11-28T13:15:28.621568+0000","last_became_active":"2023-11-28T13:07:57.305937+0000","last_became_peered":"2023-11-28T13:07:57.305937+0000","last_unstale":"2023-11-28T13:15:28.621568+0000","last_undegraded":"2023-11-28T13:15:28.621568+0000","last_fullsized":"2023-11-28T13:15:28.621568+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":56,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.619321+0000","last_change":"2023-11-28T13:07:57.306277+0000","last_active":"2023-11-28T13:15:28.619321+0000","last_peered":"2023-11-28T13:15:28.619321+0000","last_clean":"2023-11-28T13:15:28.619321+0000","last_became_active":"2023-11-28T13:07:57.305983+0000","last_became_peered":"2023-11-28T13:07:57.305983+0000","last_unstale":"2023-11-28T13:15:28.619321+0000","last_undegraded":"2023-11-28T13:15:28.619321+0000","last_fullsized":"2023-11-28T13:15:28.619321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.618799+0000","last_change":"2023-11-28T13:07:57.306177+0000","last_active":"2023-11-28T13:15:28.618799+0000","last_peered":"2023-11-28T13:15:28.618799+0000","last_clean":"2023-11-28T13:15:28.618799+0000","last_became_active":"2023-11-28T13:07:57.305853+0000","last_became_peered":"2023-11-28T13:07:57.305853+0000","last_unstale":"2023-11-28T13:15:28.618799+0000","last_undegraded":"2023-11-28T13:15:28.618799+0000","last_fullsized":"2023-11-28T13:15:28.618799+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":55,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:15:25.993150+0000","last_change":"2023-11-28T13:07:57.306321+0000","last_active":"2023-11-28T13:15:25.993150+0000","last_peered":"2023-11-28T13:15:25.993150+0000","last_clean":"2023-11-28T13:15:25.993150+0000","last_became_active":"2023-11-28T13:07:57.305923+0000","last_became_peered":"2023-11-28T13:07:57.305923+0000","last_unstale":"2023-11-28T13:15:25.993150+0000","last_undegraded":"2023-11-28T13:15:25.993150+0000","last_fullsized":"2023-11-28T13:15:25.993150+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:41.104679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":55,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:15:25.993400+0000","last_change":"2023-11-28T13:07:56.844305+0000","last_active":"2023-11-28T13:15:25.993400+0000","last_peered":"2023-11-28T13:15:25.993400+0000","last_clean":"2023-11-28T13:15:25.993400+0000","last_became_active":"2023-11-28T13:07:56.843980+0000","last_became_peered":"2023-11-28T13:07:56.843980+0000","last_unstale":"2023-11-28T13:15:25.993400+0000","last_undegraded":"2023-11-28T13:15:25.993400+0000","last_fullsized":"2023-11-28T13:15:25.993400+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574943,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15108,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94356732,"statfs":{"total":96636764160,"available":96621293568,"internally_reserved":0,"allocated":1372160,"data_stored":933538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68700000000000006,"15min":0.66900000000000004},"min":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89900000000000002,"5min":0.91200000000000003,"15min":0.98399999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68999999999999995,"15min":0.64800000000000002},"min":{"1min":0.498,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82499999999999996,"5min":0.92300000000000004,"15min":1.0569999999999999},"last":0.65300000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73399999999999999,"15min":0.72299999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84199999999999997,"5min":0.93999999999999995,"15min":1.0289999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.73999999999999999,"15min":0.71399999999999997},"min":{"1min":0.57599999999999996,"5min":0.42399999999999999,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":0.95999999999999996,"15min":1.095},"last":0.72299999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.66500000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85499999999999998,"5min":0.92300000000000004,"15min":0.996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72599999999999998,"15min":0.69499999999999995},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.85699999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.68999999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.42999999999999999},"max":{"1min":0.89000000000000001,"5min":0.91600000000000004,"15min":0.93899999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73799999999999999,"15min":0.76200000000000001},"min":{"1min":0.60399999999999998,"5min":0.52200000000000002,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.98599999999999999,"15min":1.079},"last":0.62}]},{"osd":4,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.66400000000000003},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":1.008},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.628,"15min":0.70799999999999996},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":1.006,"15min":1.071},"last":0.69199999999999995}]},{"osd":5,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64200000000000002,"15min":0.73499999999999999},"min":{"1min":0.501,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86399999999999999,"5min":0.93600000000000005,"15min":1.1830000000000001},"last":0.745},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65300000000000002,"15min":0.75900000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96299999999999997,"5min":1.044,"15min":1.1839999999999999},"last":0.87}]},{"osd":6,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71799999999999997,"15min":0.80600000000000005},"min":{"1min":0.56100000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94099999999999995,"5min":0.98599999999999999,"15min":1.165},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70199999999999996,"15min":0.76500000000000001},"min":{"1min":0.50700000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92800000000000005,"5min":0.95499999999999996,"15min":1.141},"last":0.60799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9456,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362384,"statfs":{"total":96636764160,"available":96627081216,"internally_reserved":0,"allocated":761856,"data_stored":337184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.68700000000000006,"15min":0.67700000000000005},"min":{"1min":0.441,"5min":0.432,"15min":0.41599999999999998},"max":{"1min":0.88300000000000001,"5min":0.995,"15min":0.995},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70399999999999996,"15min":0.66400000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91300000000000003,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.81399999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70899999999999996,"15min":0.68700000000000006},"min":{"1min":0.56100000000000005,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.86899999999999999,"5min":1.113,"15min":1.113},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70699999999999996,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.48599999999999999,"15min":0.47899999999999998},"max":{"1min":0.85499999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.77100000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72299999999999998,"15min":0.68899999999999995},"min":{"1min":0.60399999999999998,"5min":0.45000000000000001,"15min":0.41799999999999998},"max":{"1min":0.90300000000000002,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72099999999999997,"15min":0.68500000000000005},"min":{"1min":0.51000000000000001,"5min":0.495,"15min":0.38500000000000001},"max":{"1min":0.89400000000000002,"5min":1.024,"15min":1.024},"last":0.745}]},{"osd":3,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.73599999999999999,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":1.121,"15min":1.121},"last":0.874},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.745,"15min":0.73799999999999999},"min":{"1min":0.63,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.89200000000000002,"5min":1.167,"15min":1.167},"last":0.79500000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.624,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92300000000000004,"5min":1.042,"15min":1.0429999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.79200000000000004},"min":{"1min":0.54200000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96499999999999997,"5min":1.105,"15min":1.3140000000000001},"last":0.97699999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63500000000000001,"15min":0.68500000000000005},"min":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84599999999999997,"5min":0.92100000000000004,"15min":1.0780000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.70699999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70499999999999996,"15min":0.72899999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93400000000000005,"5min":1.034,"15min":1.0980000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73399999999999999,"15min":0.76800000000000002},"min":{"1min":0.501,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94899999999999995,"5min":1.0129999999999999,"15min":1.107},"last":0.85799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14900,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":13568,"kb_avail":94356940,"statfs":{"total":96636764160,"available":96621506560,"internally_reserved":0,"allocated":1355776,"data_stored":927552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13893632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41799999999999998,"15min":0.43099999999999999},"min":{"1min":0.28100000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.496,"5min":0.75900000000000001,"15min":1.28},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.47899999999999998,"15min":0.45600000000000002},"min":{"1min":0.30499999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64000000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42499999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.46000000000000002},"min":{"1min":0.33200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60599999999999998,"5min":0.747,"15min":0.747},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.67500000000000004,"5min":0.77000000000000002,"15min":0.84699999999999998},"last":0.53600000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.436,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71099999999999997,"5min":0.88500000000000001,"15min":1.0009999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52000000000000002,"15min":0.53800000000000003},"min":{"1min":0.36899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.68899999999999995,"5min":0.746,"15min":1.1120000000000001},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73099999999999998,"15min":0.70799999999999996},"min":{"1min":0.60299999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.753,"15min":0.72199999999999998},"min":{"1min":0.624,"5min":0.54100000000000004,"15min":0.49099999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.86499999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.74299999999999999,"15min":0.745},"min":{"1min":0.61199999999999999,"5min":0.51700000000000002,"15min":0.433},"max":{"1min":1.0209999999999999,"5min":1.292,"15min":1.292},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.74299999999999999,"15min":0.72099999999999997},"min":{"1min":0.621,"5min":0.441,"15min":0.441},"max":{"1min":1.141,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.878}]},{"osd":6,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.74199999999999999},"min":{"1min":0.64300000000000002,"5min":0.55300000000000005,"15min":0.52700000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.47299999999999998},"max":{"1min":1.1240000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.88900000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.80000000000000004,"15min":0.81000000000000005},"min":{"1min":0.66900000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0820000000000001,"5min":1.357,"15min":1.357},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77800000000000002,"15min":0.78400000000000003},"min":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0389999999999999,"5min":1.24,"15min":1.3580000000000001},"last":0.95399999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574943,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13724,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":12992,"kb_avail":94358116,"statfs":{"total":96636764160,"available":96622710784,"internally_reserved":0,"allocated":741376,"data_stored":329593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13303808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.47799999999999998},"min":{"1min":0.32500000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65400000000000003,"5min":0.69699999999999995,"15min":1.3120000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47999999999999998,"15min":0.51500000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67000000000000004,"5min":0.70599999999999996,"15min":1.48},"last":0.45300000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.504,"15min":0.52100000000000002},"min":{"1min":0.41999999999999998,"5min":0.34200000000000003,"15min":0.33000000000000002},"max":{"1min":0.76700000000000002,"5min":0.874,"15min":0.874},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.53000000000000003,"15min":0.51400000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.749,"5min":0.749,"15min":0.80800000000000005},"last":0.504}]},{"osd":3,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.35599999999999998,"5min":0.33700000000000002,"15min":0.30099999999999999},"max":{"1min":0.63800000000000001,"5min":0.747,"15min":0.747},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56399999999999995,"15min":0.55500000000000005},"min":{"1min":0.42099999999999999,"5min":0.36699999999999999,"15min":0.35399999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.82199999999999995},"last":0.47699999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.75900000000000001,"15min":0.77500000000000002},"min":{"1min":0.54500000000000004,"5min":0.48399999999999999,"15min":0.45500000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.071},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.99099999999999999,"5min":1.147,"15min":1.147},"last":0.79000000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77300000000000002,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.48299999999999998},"max":{"1min":1.0660000000000001,"5min":1.1040000000000001,"15min":1.3260000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0309999999999999,"5min":1.127,"15min":1.127},"last":0.88100000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.77300000000000002,"15min":0.76100000000000001},"min":{"1min":0.58099999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0660000000000001,"5min":1.127,"15min":1.127},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80800000000000005,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.042,"5min":1.0489999999999999,"15min":1.097},"last":0.85999999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78500000000000003,"15min":0.80000000000000004},"min":{"1min":0.58999999999999997,"5min":0.55600000000000005,"15min":0.53700000000000003},"max":{"1min":1.1200000000000001,"5min":1.149,"15min":1.149},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79600000000000004,"15min":0.80200000000000005},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.996,"5min":1.163,"15min":1.163},"last":0.82199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13616,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358224,"statfs":{"total":96636764160,"available":96622821376,"internally_reserved":0,"allocated":761856,"data_stored":337184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.48099999999999998,"15min":0.44500000000000001},"min":{"1min":0.28499999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.68400000000000005,"5min":6.4130000000000003,"15min":6.4130000000000003},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.64200000000000002,"15min":0.52200000000000002},"min":{"1min":0.32700000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.625,"5min":19.190000000000001,"15min":19.190000000000001},"last":0.48699999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.33100000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66100000000000003,"5min":5.2009999999999996,"15min":5.2009999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.68899999999999995,"15min":0.57099999999999995},"min":{"1min":0.40300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.67900000000000005,"5min":19.241,"15min":19.241},"last":0.55400000000000005}]},{"osd":3,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55700000000000005,"15min":0.48499999999999999},"min":{"1min":0.34999999999999998,"5min":0.29899999999999999,"15min":0.29799999999999999},"max":{"1min":0.66500000000000004,"5min":7.0410000000000004,"15min":7.0410000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.65200000000000002,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70499999999999996,"5min":11.420999999999999,"15min":11.420999999999999},"last":0.57499999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.84199999999999997,"15min":0.82999999999999996},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.55000000000000004},"max":{"1min":1.006,"5min":5.1630000000000003,"15min":5.1630000000000003},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.80400000000000005,"15min":0.79600000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.496},"max":{"1min":0.95899999999999996,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":0.70899999999999996}]},{"osd":5,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.75700000000000001,"15min":0.73999999999999999},"min":{"1min":0.58499999999999996,"5min":0.505,"15min":0.46100000000000002},"max":{"1min":0.86899999999999999,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.81100000000000005,"15min":0.76100000000000001},"min":{"1min":0.52900000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.036,"5min":6.7569999999999997,"15min":6.7569999999999997},"last":0.80400000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.83299999999999996,"15min":0.77700000000000002},"min":{"1min":0.51100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97099999999999997,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.83799999999999997,"15min":0.80800000000000005},"min":{"1min":0.64200000000000002,"5min":0.55700000000000005,"15min":0.49199999999999999},"max":{"1min":0.95399999999999996,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":0.90600000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.79100000000000004,"15min":0.77000000000000002},"min":{"1min":0.57799999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.996,"5min":2.008,"15min":2.008},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.82399999999999995,"15min":0.80800000000000005},"min":{"1min":0.56000000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1379999999999999,"5min":6.6769999999999996,"15min":6.6769999999999996},"last":0.88100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13920,"kb_used_data":728,"kb_used_omap":0,"kb_used_meta":13184,"kb_avail":94357920,"statfs":{"total":96636764160,"available":96622510080,"internally_reserved":0,"allocated":745472,"data_stored":330579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13500416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.33100000000000002,"5min":0.26100000000000001,"15min":0.245},"max":{"1min":0.61699999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.48299999999999998,"15min":0.45700000000000002},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.27300000000000002},"max":{"1min":0.69799999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.46899999999999997,"15min":0.48099999999999998},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71199999999999997,"5min":0.73699999999999999,"15min":0.88100000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.41099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.48099999999999998,"5min":0.372,"15min":0.36399999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56999999999999995,"15min":0.54900000000000004},"min":{"1min":0.40300000000000002,"5min":0.39000000000000001,"15min":0.35399999999999998},"max":{"1min":0.80500000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.61099999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72199999999999998,"15min":0.70499999999999996},"min":{"1min":0.54400000000000004,"5min":0.50900000000000001,"15min":0.50600000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.73399999999999999,"15min":0.71399999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.46200000000000002},"max":{"1min":1.054,"5min":1.087,"15min":1.087},"last":0.73699999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.091},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.77500000000000002,"15min":0.76600000000000001},"min":{"1min":0.63400000000000001,"5min":0.50600000000000001,"15min":0.498},"max":{"1min":1.0369999999999999,"5min":1.0649999999999999,"15min":1.0680000000000001},"last":0.751}]},{"osd":6,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76600000000000001,"15min":0.73499999999999999},"min":{"1min":0.65500000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96799999999999997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.76100000000000001,"15min":0.71999999999999997},"min":{"1min":0.59299999999999997,"5min":0.50600000000000001,"15min":0.46100000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.57799999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79900000000000004,"15min":0.76900000000000002},"min":{"1min":0.64600000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.032,"5min":1.05,"15min":1.05},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.80000000000000004,"15min":0.77300000000000002},"min":{"1min":0.68400000000000005,"5min":0.53600000000000003,"15min":0.52800000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.1299999999999999},"last":0.69899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9920,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361920,"statfs":{"total":96636764160,"available":96626606080,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70499999999999996,"15min":0.63},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.91900000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72099999999999997,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.42799999999999999,"15min":0.38400000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":1.3089999999999999},"last":0.56899999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73499999999999999,"15min":0.69699999999999995},"min":{"1min":0.59399999999999997,"5min":0.51200000000000001,"15min":0.48299999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":1.161},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69399999999999995,"15min":0.63},"min":{"1min":0.47899999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88900000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.628}]},{"osd":2,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":0.67800000000000005},"min":{"1min":0.46400000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.755},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.69899999999999995},"min":{"1min":0.502,"5min":0.47599999999999998,"15min":0.441},"max":{"1min":0.91500000000000004,"5min":1.042,"15min":1.042},"last":0.70799999999999996}]},{"osd":3,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71699999999999997,"15min":0.68600000000000005},"min":{"1min":0.48699999999999999,"5min":0.46000000000000002,"15min":0.438},"max":{"1min":0.85499999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.75700000000000001,"15min":0.72799999999999998},"min":{"1min":0.54500000000000004,"5min":0.52900000000000003,"15min":0.505},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.60199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.66000000000000003},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95799999999999996,"5min":1.012,"15min":1.264},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64500000000000002,"15min":0.70299999999999996},"min":{"1min":0.49399999999999999,"5min":0.39100000000000001,"15min":0.38400000000000001},"max":{"1min":0.81999999999999995,"5min":0.93200000000000005,"15min":1.079},"last":0.72499999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65600000000000003,"15min":0.70799999999999996},"min":{"1min":0.46300000000000002,"5min":0.439,"15min":0.32700000000000001},"max":{"1min":0.89000000000000001,"5min":0.91200000000000003,"15min":1.022},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69799999999999995,"15min":0.73099999999999998},"min":{"1min":0.53800000000000003,"5min":0.48699999999999999,"15min":0.47399999999999998},"max":{"1min":0.90900000000000003,"5min":1.044,"15min":1.22},"last":0.74299999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69099999999999995,"15min":0.745},"min":{"1min":0.51700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98699999999999999,"5min":0.995,"15min":1.4530000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72899999999999998,"15min":0.76700000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.40899999999999997},"max":{"1min":1.0089999999999999,"5min":1.018,"15min":1.018},"last":0.79300000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9792,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362048,"statfs":{"total":96636764160,"available":96626737152,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69199999999999995,"15min":0.63300000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92500000000000004,"5min":1.0640000000000001,"15min":1.659},"last":0.875},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69699999999999995,"15min":0.63400000000000001},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.38900000000000001},"max":{"1min":0.95899999999999996,"5min":0.97999999999999998,"15min":2.3050000000000002},"last":0.85499999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66900000000000004,"15min":0.62},"min":{"1min":0.44600000000000001,"5min":0.38200000000000001,"15min":0.35499999999999998},"max":{"1min":0.94299999999999995,"5min":0.95699999999999996,"15min":1.2390000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72099999999999997,"15min":0.65300000000000002},"min":{"1min":0.54300000000000004,"5min":0.40500000000000003,"15min":0.39400000000000002},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.9630000000000001},"last":0.79100000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69699999999999995,"15min":0.65700000000000003},"min":{"1min":0.53500000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.86899999999999999,"5min":1.0469999999999999,"15min":2.3860000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71199999999999997,"15min":0.66400000000000003},"min":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.432},"max":{"1min":0.89300000000000002,"5min":0.97299999999999998,"15min":1.0800000000000001},"last":0.80900000000000005}]},{"osd":3,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71399999999999997,"15min":0.65900000000000003},"min":{"1min":0.53800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":1.0169999999999999,"15min":1.8140000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73699999999999999,"15min":0.71399999999999997},"min":{"1min":0.5,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91700000000000004,"5min":0.96699999999999997,"15min":2.3500000000000001},"last":0.95099999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63200000000000001,"15min":0.65300000000000002},"min":{"1min":0.495,"5min":0.379,"15min":0.379},"max":{"1min":0.79900000000000004,"5min":0.95999999999999996,"15min":1.702},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70799999999999996,"15min":0.72799999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89000000000000001,"5min":1.0529999999999999,"15min":2.0009999999999999},"last":0.73799999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64500000000000002,"15min":0.69599999999999995},"min":{"1min":0.437,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76700000000000002,"5min":0.89400000000000002,"15min":2.4969999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66900000000000004,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86899999999999999,"5min":1.012,"15min":2.0880000000000001},"last":0.57099999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69499999999999995,"15min":0.71499999999999997},"min":{"1min":0.53100000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93999999999999995,"5min":0.999,"15min":2.0449999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.49199999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.85099999999999998,"5min":0.98099999999999998,"15min":2.4249999999999998},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:15:32.388 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:15:32.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.584+0000 7ff0f23ff700 1 Processor -- start 2023-11-28T13:15:32.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.584+0000 7ff0f23ff700 1 -- start start 2023-11-28T13:15:32.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.585+0000 7ff0f23ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff0ec15c9b0 0x7ff0ec0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:32.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.585+0000 7ff0f23ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff0ec161340 0x7ff0ec0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:32.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.585+0000 7ff0f23ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0ec0d1950 con 0x7ff0ec0d2320 2023-11-28T13:15:32.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.585+0000 7ff0f23ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0ec0b5800 con 0x7ff0ec161340 2023-11-28T13:15:32.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.585+0000 7ff0f23ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0ec0b5520 con 0x7ff0ec15c9b0 2023-11-28T13:15:32.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.586+0000 7ff0eb7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff0ec161340 0x7ff0ec0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:32.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.586+0000 7ff0f099c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff0ec0d2320 0x7ff0ec0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39434/0 (socket says 172.21.15.156:39434) 2023-11-28T13:15:32.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.586+0000 7ff0f099c700 1 -- 172.21.15.156:0/586036158 learned_addr learned my addr 172.21.15.156:0/586036158 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:32.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.586+0000 7ff0ebfff700 1 --2- 172.21.15.156:0/586036158 >> v2:172.21.15.186:3300/0 conn(0x7ff0ec15c9b0 0x7ff0ec0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:32.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.586+0000 7ff0eb7fe700 1 -- 172.21.15.156:0/586036158 >> v2:172.21.15.186:3300/0 conn(0x7ff0ec15c9b0 msgr2=0x7ff0ec0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:32.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.586+0000 7ff0eb7fe700 1 --2- 172.21.15.156:0/586036158 >> v2:172.21.15.186:3300/0 conn(0x7ff0ec15c9b0 0x7ff0ec0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.586+0000 7ff0eb7fe700 1 -- 172.21.15.156:0/586036158 >> v1:172.21.15.156:6789/0 conn(0x7ff0ec0d2320 legacy=0x7ff0ec0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:32.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.586+0000 7ff0eb7fe700 1 -- 172.21.15.156:0/586036158 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec0d1300 con 0x7ff0ec161340 2023-11-28T13:15:32.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.586+0000 7ff0ebfff700 1 --2- 172.21.15.156:0/586036158 >> v2:172.21.15.186:3300/0 conn(0x7ff0ec15c9b0 0x7ff0ec0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:15:32.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.586+0000 7ff0eb7fe700 1 --2- 172.21.15.156:0/586036158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ec161340 0x7ff0ec0d1de0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec067460 tx=0x7ff0e0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9c6836ec8c100a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:32.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.587+0000 7ff0eaffd700 1 -- 172.21.15.156:0/586036158 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0011830 con 0x7ff0ec161340 2023-11-28T13:15:32.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.587+0000 7ff0eaffd700 1 -- 172.21.15.156:0/586036158 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e0011990 con 0x7ff0ec161340 2023-11-28T13:15:32.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.587+0000 7ff0eaffd700 1 -- 172.21.15.156:0/586036158 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0011d20 con 0x7ff0ec161340 2023-11-28T13:15:32.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.587+0000 7ff0f23ff700 1 -- 172.21.15.156:0/586036158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ec161340 msgr2=0x7ff0ec0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:32.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.587+0000 7ff0f23ff700 1 --2- 172.21.15.156:0/586036158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ec161340 0x7ff0ec0d1de0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec067460 tx=0x7ff0e0009b20 comp rx=0 tx=0).stop 2023-11-28T13:15:32.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.587+0000 7ff0f23ff700 1 -- 172.21.15.156:0/586036158 shutdown_connections 2023-11-28T13:15:32.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.587+0000 7ff0f23ff700 1 --2- 172.21.15.156:0/586036158 >> v2:172.21.15.186:3300/0 conn(0x7ff0ec15c9b0 0x7ff0ec0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.587+0000 7ff0f23ff700 1 --2- 172.21.15.156:0/586036158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ec161340 0x7ff0ec0d1de0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.587+0000 7ff0f23ff700 1 -- 172.21.15.156:0/586036158 >> 172.21.15.156:0/586036158 conn(0x7ff0ec072600 msgr2=0x7ff0ec0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:32.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.587+0000 7ff0f23ff700 1 -- 172.21.15.156:0/586036158 shutdown_connections 2023-11-28T13:15:32.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.587+0000 7ff0f23ff700 1 -- 172.21.15.156:0/586036158 wait complete. 2023-11-28T13:15:32.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.589+0000 7ff0f23ff700 1 Processor -- start 2023-11-28T13:15:32.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.589+0000 7ff0f23ff700 1 -- start start 2023-11-28T13:15:32.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.590+0000 7ff0f23ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ec0d2320 0x7ff0ec0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:32.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.590+0000 7ff0f23ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0ec15c9b0 0x7ff0ec1520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:32.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.590+0000 7ff0f23ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ec161340 0x7ff0ec15ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:32.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.590+0000 7ff0f23ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec150780 con 0x7ff0ec0d2320 2023-11-28T13:15:32.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.590+0000 7ff0f23ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec0c9860 con 0x7ff0ec15c9b0 2023-11-28T13:15:32.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.590+0000 7ff0f23ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec153440 con 0x7ff0ec161340 2023-11-28T13:15:32.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.590+0000 7ff0ebfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ec0d2320 0x7ff0ec0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:32.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.590+0000 7ff0ebfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ec0d2320 0x7ff0ec0c6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39508/0 (socket says 172.21.15.156:39508) 2023-11-28T13:15:32.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.590+0000 7ff0ebfff700 1 -- 172.21.15.156:0/2492498832 learned_addr learned my addr 172.21.15.156:0/2492498832 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:32.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.590+0000 7ff0f099c700 1 --2- 172.21.15.156:0/2492498832 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ec161340 0x7ff0ec15ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:32.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.591+0000 7ff0eb7fe700 1 --2- 172.21.15.156:0/2492498832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0ec15c9b0 0x7ff0ec1520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:32.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.591+0000 7ff0ebfff700 1 -- 172.21.15.156:0/2492498832 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ec161340 msgr2=0x7ff0ec15ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:32.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.591+0000 7ff0ebfff700 1 --2- 172.21.15.156:0/2492498832 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ec161340 0x7ff0ec15ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.591+0000 7ff0ebfff700 1 -- 172.21.15.156:0/2492498832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0ec15c9b0 msgr2=0x7ff0ec1520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:32.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.591+0000 7ff0ebfff700 1 --2- 172.21.15.156:0/2492498832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0ec15c9b0 0x7ff0ec1520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.591+0000 7ff0ebfff700 1 -- 172.21.15.156:0/2492498832 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e00094a0 con 0x7ff0ec0d2320 2023-11-28T13:15:32.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.591+0000 7ff0eb7fe700 1 --2- 172.21.15.156:0/2492498832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0ec15c9b0 0x7ff0ec1520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:15:32.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.591+0000 7ff0ebfff700 1 --2- 172.21.15.156:0/2492498832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ec0d2320 0x7ff0ec0c6060 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00e810 tx=0x7ff0dc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:32.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.591+0000 7ff0e8ff9700 1 -- 172.21.15.156:0/2492498832 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc013610 con 0x7ff0ec0d2320 2023-11-28T13:15:32.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.591+0000 7ff0e8ff9700 1 -- 172.21.15.156:0/2492498832 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0dc013770 con 0x7ff0ec0d2320 2023-11-28T13:15:32.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.591+0000 7ff0f23ff700 1 -- 172.21.15.156:0/2492498832 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ec0b5130 con 0x7ff0ec0d2320 2023-11-28T13:15:32.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.591+0000 7ff0e8ff9700 1 -- 172.21.15.156:0/2492498832 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc013b20 con 0x7ff0ec0d2320 2023-11-28T13:15:32.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.591+0000 7ff0f23ff700 1 -- 172.21.15.156:0/2492498832 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ec0d15e0 con 0x7ff0ec0d2320 2023-11-28T13:15:32.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.592+0000 7ff0f23ff700 1 -- 172.21.15.156:0/2492498832 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0ec0b52c0 con 0x7ff0ec0d2320 2023-11-28T13:15:32.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.593+0000 7ff0e8ff9700 1 -- 172.21.15.156:0/2492498832 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff0dc023070 con 0x7ff0ec0d2320 2023-11-28T13:15:32.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.594+0000 7ff0e8ff9700 1 --2- 172.21.15.156:0/2492498832 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff0d80682f0 0x7ff0d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:32.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.594+0000 7ff0e8ff9700 1 -- 172.21.15.156:0/2492498832 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff0dc085760 con 0x7ff0ec0d2320 2023-11-28T13:15:32.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.594+0000 7ff0eb7fe700 1 --2- 172.21.15.156:0/2492498832 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff0d80682f0 0x7ff0d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:32.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.595+0000 7ff0eb7fe700 1 --2- 172.21.15.156:0/2492498832 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff0d80682f0 0x7ff0d806a7a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00090b0 tx=0x7ff0e0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:32.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.598+0000 7ff0e8ff9700 1 -- 172.21.15.156:0/2492498832 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff0dc0521f0 con 0x7ff0ec0d2320 2023-11-28T13:15:32.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.791+0000 7ff0f23ff700 1 -- 172.21.15.156:0/2492498832 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff0ec1538d0 con 0x7ff0d80682f0 2023-11-28T13:15:32.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.794+0000 7ff0e8ff9700 1 -- 172.21.15.156:0/2492498832 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73022 (secure 0 0 0) 0x7ff0ec1538d0 con 0x7ff0d80682f0 2023-11-28T13:15:32.797 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:15:32.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.797+0000 7ff0f23ff700 1 -- 172.21.15.156:0/2492498832 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff0d80682f0 msgr2=0x7ff0d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:32.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.797+0000 7ff0f23ff700 1 --2- 172.21.15.156:0/2492498832 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff0d80682f0 0x7ff0d806a7a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00090b0 tx=0x7ff0e0008030 comp rx=0 tx=0).stop 2023-11-28T13:15:32.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.797+0000 7ff0f23ff700 1 -- 172.21.15.156:0/2492498832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ec0d2320 msgr2=0x7ff0ec0c6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:32.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.797+0000 7ff0f23ff700 1 --2- 172.21.15.156:0/2492498832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ec0d2320 0x7ff0ec0c6060 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00e810 tx=0x7ff0dc00ebd0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.797+0000 7ff0f23ff700 1 -- 172.21.15.156:0/2492498832 shutdown_connections 2023-11-28T13:15:32.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.797+0000 7ff0f23ff700 1 --2- 172.21.15.156:0/2492498832 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7ff0d80682f0 0x7ff0d806a7a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.797+0000 7ff0f23ff700 1 --2- 172.21.15.156:0/2492498832 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0ec161340 0x7ff0ec15ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.797+0000 7ff0f23ff700 1 --2- 172.21.15.156:0/2492498832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ec0d2320 0x7ff0ec0c6060 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.797+0000 7ff0f23ff700 1 --2- 172.21.15.156:0/2492498832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0ec15c9b0 0x7ff0ec1520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:32.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.797+0000 7ff0f23ff700 1 -- 172.21.15.156:0/2492498832 >> 172.21.15.156:0/2492498832 conn(0x7ff0ec072600 msgr2=0x7ff0ec15a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:32.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.797+0000 7ff0f23ff700 1 -- 172.21.15.156:0/2492498832 shutdown_connections 2023-11-28T13:15:32.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:32.797+0000 7ff0f23ff700 1 -- 172.21.15.156:0/2492498832 wait complete. 2023-11-28T13:15:32.800 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T13:15:32.818 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2023-11-28T13:15:32.134137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":100436,"kb_used_data":7124,"kb_used_omap":0,"kb_used_meta":93248,"kb_avail":754874284,"statfs":{"total":773094113280,"available":772991266816,"internally_reserved":0,"allocated":7294976,"data_stored":3881970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95485952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12572,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-80,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.529920"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:15:25.993449+0000","last_change":"2023-11-28T13:07:56.890378+0000","last_active":"2023-11-28T13:15:25.993449+0000","last_peered":"2023-11-28T13:15:25.993449+0000","last_clean":"2023-11-28T13:15:25.993449+0000","last_became_active":"2023-11-28T13:07:56.890058+0000","last_became_peered":"2023-11-28T13:07:56.890058+0000","last_unstale":"2023-11-28T13:15:25.993449+0000","last_undegraded":"2023-11-28T13:15:25.993449+0000","last_fullsized":"2023-11-28T13:15:25.993449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:05.344602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":56,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:29.649906+0000","last_change":"2023-11-28T13:07:56.844509+0000","last_active":"2023-11-28T13:15:29.649906+0000","last_peered":"2023-11-28T13:15:29.649906+0000","last_clean":"2023-11-28T13:15:29.649906+0000","last_became_active":"2023-11-28T13:07:56.844081+0000","last_became_peered":"2023-11-28T13:07:56.844081+0000","last_unstale":"2023-11-28T13:15:29.649906+0000","last_undegraded":"2023-11-28T13:15:29.649906+0000","last_fullsized":"2023-11-28T13:15:29.649906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":54,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.858689+0000","last_change":"2023-11-28T13:07:56.844699+0000","last_active":"2023-11-28T13:15:28.858689+0000","last_peered":"2023-11-28T13:15:28.858689+0000","last_clean":"2023-11-28T13:15:28.858689+0000","last_became_active":"2023-11-28T13:07:56.844391+0000","last_became_peered":"2023-11-28T13:07:56.844391+0000","last_unstale":"2023-11-28T13:15:28.858689+0000","last_undegraded":"2023-11-28T13:15:28.858689+0000","last_fullsized":"2023-11-28T13:15:28.858689+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":205,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.619311+0000","last_change":"2023-11-28T13:07:57.844020+0000","last_active":"2023-11-28T13:15:28.619311+0000","last_peered":"2023-11-28T13:15:28.619311+0000","last_clean":"2023-11-28T13:15:28.619311+0000","last_became_active":"2023-11-28T13:07:57.842981+0000","last_became_peered":"2023-11-28T13:07:57.842981+0000","last_unstale":"2023-11-28T13:15:28.619311+0000","last_undegraded":"2023-11-28T13:15:28.619311+0000","last_fullsized":"2023-11-28T13:15:28.619311+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_clean_scrub_stamp":"2023-11-28T13:07:56.825711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:44.973218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":56,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.621568+0000","last_change":"2023-11-28T13:07:57.306318+0000","last_active":"2023-11-28T13:15:28.621568+0000","last_peered":"2023-11-28T13:15:28.621568+0000","last_clean":"2023-11-28T13:15:28.621568+0000","last_became_active":"2023-11-28T13:07:57.305937+0000","last_became_peered":"2023-11-28T13:07:57.305937+0000","last_unstale":"2023-11-28T13:15:28.621568+0000","last_undegraded":"2023-11-28T13:15:28.621568+0000","last_fullsized":"2023-11-28T13:15:28.621568+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":56,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.619321+0000","last_change":"2023-11-28T13:07:57.306277+0000","last_active":"2023-11-28T13:15:28.619321+0000","last_peered":"2023-11-28T13:15:28.619321+0000","last_clean":"2023-11-28T13:15:28.619321+0000","last_became_active":"2023-11-28T13:07:57.305983+0000","last_became_peered":"2023-11-28T13:07:57.305983+0000","last_unstale":"2023-11-28T13:15:28.619321+0000","last_undegraded":"2023-11-28T13:15:28.619321+0000","last_fullsized":"2023-11-28T13:15:28.619321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.618799+0000","last_change":"2023-11-28T13:07:57.306177+0000","last_active":"2023-11-28T13:15:28.618799+0000","last_peered":"2023-11-28T13:15:28.618799+0000","last_clean":"2023-11-28T13:15:28.618799+0000","last_became_active":"2023-11-28T13:07:57.305853+0000","last_became_peered":"2023-11-28T13:07:57.305853+0000","last_unstale":"2023-11-28T13:15:28.618799+0000","last_undegraded":"2023-11-28T13:15:28.618799+0000","last_fullsized":"2023-11-28T13:15:28.618799+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":55,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:15:25.993150+0000","last_change":"2023-11-28T13:07:57.306321+0000","last_active":"2023-11-28T13:15:25.993150+0000","last_peered":"2023-11-28T13:15:25.993150+0000","last_clean":"2023-11-28T13:15:25.993150+0000","last_became_active":"2023-11-28T13:07:57.305923+0000","last_became_peered":"2023-11-28T13:07:57.305923+0000","last_unstale":"2023-11-28T13:15:25.993150+0000","last_undegraded":"2023-11-28T13:15:25.993150+0000","last_fullsized":"2023-11-28T13:15:25.993150+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:41.104679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":55,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:15:25.993400+0000","last_change":"2023-11-28T13:07:56.844305+0000","last_active":"2023-11-28T13:15:25.993400+0000","last_peered":"2023-11-28T13:15:25.993400+0000","last_clean":"2023-11-28T13:15:25.993400+0000","last_became_active":"2023-11-28T13:07:56.843980+0000","last_became_peered":"2023-11-28T13:07:56.843980+0000","last_unstale":"2023-11-28T13:15:25.993400+0000","last_undegraded":"2023-11-28T13:15:25.993400+0000","last_fullsized":"2023-11-28T13:15:25.993400+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574943,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15108,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94356732,"statfs":{"total":96636764160,"available":96621293568,"internally_reserved":0,"allocated":1372160,"data_stored":933538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68700000000000006,"15min":0.66900000000000004},"min":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89900000000000002,"5min":0.91200000000000003,"15min":0.98399999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68999999999999995,"15min":0.64800000000000002},"min":{"1min":0.498,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82499999999999996,"5min":0.92300000000000004,"15min":1.0569999999999999},"last":0.65300000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73399999999999999,"15min":0.72299999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84199999999999997,"5min":0.93999999999999995,"15min":1.0289999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.73999999999999999,"15min":0.71399999999999997},"min":{"1min":0.57599999999999996,"5min":0.42399999999999999,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":0.95999999999999996,"15min":1.095},"last":0.72299999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.66500000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85499999999999998,"5min":0.92300000000000004,"15min":0.996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72599999999999998,"15min":0.69499999999999995},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.85699999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.68999999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.42999999999999999},"max":{"1min":0.89000000000000001,"5min":0.91600000000000004,"15min":0.93899999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73799999999999999,"15min":0.76200000000000001},"min":{"1min":0.60399999999999998,"5min":0.52200000000000002,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.98599999999999999,"15min":1.079},"last":0.62}]},{"osd":4,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.66400000000000003},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":1.008},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.628,"15min":0.70799999999999996},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":1.006,"15min":1.071},"last":0.69199999999999995}]},{"osd":5,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64200000000000002,"15min":0.73499999999999999},"min":{"1min":0.501,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86399999999999999,"5min":0.93600000000000005,"15min":1.1830000000000001},"last":0.745},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65300000000000002,"15min":0.75900000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96299999999999997,"5min":1.044,"15min":1.1839999999999999},"last":0.87}]},{"osd":6,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71799999999999997,"15min":0.80600000000000005},"min":{"1min":0.56100000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94099999999999995,"5min":0.98599999999999999,"15min":1.165},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70199999999999996,"15min":0.76500000000000001},"min":{"1min":0.50700000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92800000000000005,"5min":0.95499999999999996,"15min":1.141},"last":0.60799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9456,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362384,"statfs":{"total":96636764160,"available":96627081216,"internally_reserved":0,"allocated":761856,"data_stored":337184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.68700000000000006,"15min":0.67700000000000005},"min":{"1min":0.441,"5min":0.432,"15min":0.41599999999999998},"max":{"1min":0.88300000000000001,"5min":0.995,"15min":0.995},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70399999999999996,"15min":0.66400000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91300000000000003,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.81399999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70899999999999996,"15min":0.68700000000000006},"min":{"1min":0.56100000000000005,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.86899999999999999,"5min":1.113,"15min":1.113},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70699999999999996,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.48599999999999999,"15min":0.47899999999999998},"max":{"1min":0.85499999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.77100000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72299999999999998,"15min":0.68899999999999995},"min":{"1min":0.60399999999999998,"5min":0.45000000000000001,"15min":0.41799999999999998},"max":{"1min":0.90300000000000002,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72099999999999997,"15min":0.68500000000000005},"min":{"1min":0.51000000000000001,"5min":0.495,"15min":0.38500000000000001},"max":{"1min":0.89400000000000002,"5min":1.024,"15min":1.024},"last":0.745}]},{"osd":3,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.73599999999999999,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":1.121,"15min":1.121},"last":0.874},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.745,"15min":0.73799999999999999},"min":{"1min":0.63,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.89200000000000002,"5min":1.167,"15min":1.167},"last":0.79500000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.624,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92300000000000004,"5min":1.042,"15min":1.0429999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.79200000000000004},"min":{"1min":0.54200000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96499999999999997,"5min":1.105,"15min":1.3140000000000001},"last":0.97699999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63500000000000001,"15min":0.68500000000000005},"min":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84599999999999997,"5min":0.92100000000000004,"15min":1.0780000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.70699999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70499999999999996,"15min":0.72899999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93400000000000005,"5min":1.034,"15min":1.0980000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73399999999999999,"15min":0.76800000000000002},"min":{"1min":0.501,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94899999999999995,"5min":1.0129999999999999,"15min":1.107},"last":0.85799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14900,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":13568,"kb_avail":94356940,"statfs":{"total":96636764160,"available":96621506560,"internally_reserved":0,"allocated":1355776,"data_stored":927552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13893632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41799999999999998,"15min":0.43099999999999999},"min":{"1min":0.28100000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.496,"5min":0.75900000000000001,"15min":1.28},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.47899999999999998,"15min":0.45600000000000002},"min":{"1min":0.30499999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64000000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42499999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.46000000000000002},"min":{"1min":0.33200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60599999999999998,"5min":0.747,"15min":0.747},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.67500000000000004,"5min":0.77000000000000002,"15min":0.84699999999999998},"last":0.53600000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.436,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71099999999999997,"5min":0.88500000000000001,"15min":1.0009999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52000000000000002,"15min":0.53800000000000003},"min":{"1min":0.36899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.68899999999999995,"5min":0.746,"15min":1.1120000000000001},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73099999999999998,"15min":0.70799999999999996},"min":{"1min":0.60299999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.753,"15min":0.72199999999999998},"min":{"1min":0.624,"5min":0.54100000000000004,"15min":0.49099999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.86499999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.74299999999999999,"15min":0.745},"min":{"1min":0.61199999999999999,"5min":0.51700000000000002,"15min":0.433},"max":{"1min":1.0209999999999999,"5min":1.292,"15min":1.292},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.74299999999999999,"15min":0.72099999999999997},"min":{"1min":0.621,"5min":0.441,"15min":0.441},"max":{"1min":1.141,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.878}]},{"osd":6,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.74199999999999999},"min":{"1min":0.64300000000000002,"5min":0.55300000000000005,"15min":0.52700000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.47299999999999998},"max":{"1min":1.1240000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.88900000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.80000000000000004,"15min":0.81000000000000005},"min":{"1min":0.66900000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0820000000000001,"5min":1.357,"15min":1.357},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77800000000000002,"15min":0.78400000000000003},"min":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0389999999999999,"5min":1.24,"15min":1.3580000000000001},"last":0.95399999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574943,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13724,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":12992,"kb_avail":94358116,"statfs":{"total":96636764160,"available":96622710784,"internally_reserved":0,"allocated":741376,"data_stored":329593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13303808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.47799999999999998},"min":{"1min":0.32500000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65400000000000003,"5min":0.69699999999999995,"15min":1.3120000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47999999999999998,"15min":0.51500000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67000000000000004,"5min":0.70599999999999996,"15min":1.48},"last":0.45300000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.504,"15min":0.52100000000000002},"min":{"1min":0.41999999999999998,"5min":0.34200000000000003,"15min":0.33000000000000002},"max":{"1min":0.76700000000000002,"5min":0.874,"15min":0.874},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.53000000000000003,"15min":0.51400000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.749,"5min":0.749,"15min":0.80800000000000005},"last":0.504}]},{"osd":3,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.35599999999999998,"5min":0.33700000000000002,"15min":0.30099999999999999},"max":{"1min":0.63800000000000001,"5min":0.747,"15min":0.747},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56399999999999995,"15min":0.55500000000000005},"min":{"1min":0.42099999999999999,"5min":0.36699999999999999,"15min":0.35399999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.82199999999999995},"last":0.47699999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.75900000000000001,"15min":0.77500000000000002},"min":{"1min":0.54500000000000004,"5min":0.48399999999999999,"15min":0.45500000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.071},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.99099999999999999,"5min":1.147,"15min":1.147},"last":0.79000000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77300000000000002,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.48299999999999998},"max":{"1min":1.0660000000000001,"5min":1.1040000000000001,"15min":1.3260000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0309999999999999,"5min":1.127,"15min":1.127},"last":0.88100000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.77300000000000002,"15min":0.76100000000000001},"min":{"1min":0.58099999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0660000000000001,"5min":1.127,"15min":1.127},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80800000000000005,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.042,"5min":1.0489999999999999,"15min":1.097},"last":0.85999999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78500000000000003,"15min":0.80000000000000004},"min":{"1min":0.58999999999999997,"5min":0.55600000000000005,"15min":0.53700000000000003},"max":{"1min":1.1200000000000001,"5min":1.149,"15min":1.149},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79600000000000004,"15min":0.80200000000000005},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.996,"5min":1.163,"15min":1.163},"last":0.82199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13616,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358224,"statfs":{"total":96636764160,"available":96622821376,"internally_reserved":0,"allocated":761856,"data_stored":337184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.48099999999999998,"15min":0.44500000000000001},"min":{"1min":0.28499999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.68400000000000005,"5min":6.4130000000000003,"15min":6.4130000000000003},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.64200000000000002,"15min":0.52200000000000002},"min":{"1min":0.32700000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.625,"5min":19.190000000000001,"15min":19.190000000000001},"last":0.48699999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.33100000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66100000000000003,"5min":5.2009999999999996,"15min":5.2009999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.68899999999999995,"15min":0.57099999999999995},"min":{"1min":0.40300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.67900000000000005,"5min":19.241,"15min":19.241},"last":0.55400000000000005}]},{"osd":3,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55700000000000005,"15min":0.48499999999999999},"min":{"1min":0.34999999999999998,"5min":0.29899999999999999,"15min":0.29799999999999999},"max":{"1min":0.66500000000000004,"5min":7.0410000000000004,"15min":7.0410000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.65200000000000002,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70499999999999996,"5min":11.420999999999999,"15min":11.420999999999999},"last":0.57499999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.84199999999999997,"15min":0.82999999999999996},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.55000000000000004},"max":{"1min":1.006,"5min":5.1630000000000003,"15min":5.1630000000000003},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.80400000000000005,"15min":0.79600000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.496},"max":{"1min":0.95899999999999996,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":0.70899999999999996}]},{"osd":5,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.75700000000000001,"15min":0.73999999999999999},"min":{"1min":0.58499999999999996,"5min":0.505,"15min":0.46100000000000002},"max":{"1min":0.86899999999999999,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.81100000000000005,"15min":0.76100000000000001},"min":{"1min":0.52900000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.036,"5min":6.7569999999999997,"15min":6.7569999999999997},"last":0.80400000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.83299999999999996,"15min":0.77700000000000002},"min":{"1min":0.51100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97099999999999997,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.83799999999999997,"15min":0.80800000000000005},"min":{"1min":0.64200000000000002,"5min":0.55700000000000005,"15min":0.49199999999999999},"max":{"1min":0.95399999999999996,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":0.90600000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.79100000000000004,"15min":0.77000000000000002},"min":{"1min":0.57799999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.996,"5min":2.008,"15min":2.008},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.82399999999999995,"15min":0.80800000000000005},"min":{"1min":0.56000000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1379999999999999,"5min":6.6769999999999996,"15min":6.6769999999999996},"last":0.88100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13920,"kb_used_data":728,"kb_used_omap":0,"kb_used_meta":13184,"kb_avail":94357920,"statfs":{"total":96636764160,"available":96622510080,"internally_reserved":0,"allocated":745472,"data_stored":330579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13500416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.33100000000000002,"5min":0.26100000000000001,"15min":0.245},"max":{"1min":0.61699999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.48299999999999998,"15min":0.45700000000000002},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.27300000000000002},"max":{"1min":0.69799999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.46899999999999997,"15min":0.48099999999999998},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71199999999999997,"5min":0.73699999999999999,"15min":0.88100000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.41099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.48099999999999998,"5min":0.372,"15min":0.36399999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56999999999999995,"15min":0.54900000000000004},"min":{"1min":0.40300000000000002,"5min":0.39000000000000001,"15min":0.35399999999999998},"max":{"1min":0.80500000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.61099999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72199999999999998,"15min":0.70499999999999996},"min":{"1min":0.54400000000000004,"5min":0.50900000000000001,"15min":0.50600000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.73399999999999999,"15min":0.71399999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.46200000000000002},"max":{"1min":1.054,"5min":1.087,"15min":1.087},"last":0.73699999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.091},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.77500000000000002,"15min":0.76600000000000001},"min":{"1min":0.63400000000000001,"5min":0.50600000000000001,"15min":0.498},"max":{"1min":1.0369999999999999,"5min":1.0649999999999999,"15min":1.0680000000000001},"last":0.751}]},{"osd":6,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76600000000000001,"15min":0.73499999999999999},"min":{"1min":0.65500000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96799999999999997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.76100000000000001,"15min":0.71999999999999997},"min":{"1min":0.59299999999999997,"5min":0.50600000000000001,"15min":0.46100000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.57799999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79900000000000004,"15min":0.76900000000000002},"min":{"1min":0.64600000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.032,"5min":1.05,"15min":1.05},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.80000000000000004,"15min":0.77300000000000002},"min":{"1min":0.68400000000000005,"5min":0.53600000000000003,"15min":0.52800000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.1299999999999999},"last":0.69899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9920,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361920,"statfs":{"total":96636764160,"available":96626606080,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70499999999999996,"15min":0.63},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.91900000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72099999999999997,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.42799999999999999,"15min":0.38400000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":1.3089999999999999},"last":0.56899999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73499999999999999,"15min":0.69699999999999995},"min":{"1min":0.59399999999999997,"5min":0.51200000000000001,"15min":0.48299999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":1.161},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69399999999999995,"15min":0.63},"min":{"1min":0.47899999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88900000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.628}]},{"osd":2,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":0.67800000000000005},"min":{"1min":0.46400000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.755},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.69899999999999995},"min":{"1min":0.502,"5min":0.47599999999999998,"15min":0.441},"max":{"1min":0.91500000000000004,"5min":1.042,"15min":1.042},"last":0.70799999999999996}]},{"osd":3,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71699999999999997,"15min":0.68600000000000005},"min":{"1min":0.48699999999999999,"5min":0.46000000000000002,"15min":0.438},"max":{"1min":0.85499999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.75700000000000001,"15min":0.72799999999999998},"min":{"1min":0.54500000000000004,"5min":0.52900000000000003,"15min":0.505},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.60199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.66000000000000003},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95799999999999996,"5min":1.012,"15min":1.264},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64500000000000002,"15min":0.70299999999999996},"min":{"1min":0.49399999999999999,"5min":0.39100000000000001,"15min":0.38400000000000001},"max":{"1min":0.81999999999999995,"5min":0.93200000000000005,"15min":1.079},"last":0.72499999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65600000000000003,"15min":0.70799999999999996},"min":{"1min":0.46300000000000002,"5min":0.439,"15min":0.32700000000000001},"max":{"1min":0.89000000000000001,"5min":0.91200000000000003,"15min":1.022},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69799999999999995,"15min":0.73099999999999998},"min":{"1min":0.53800000000000003,"5min":0.48699999999999999,"15min":0.47399999999999998},"max":{"1min":0.90900000000000003,"5min":1.044,"15min":1.22},"last":0.74299999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69099999999999995,"15min":0.745},"min":{"1min":0.51700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98699999999999999,"5min":0.995,"15min":1.4530000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72899999999999998,"15min":0.76700000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.40899999999999997},"max":{"1min":1.0089999999999999,"5min":1.018,"15min":1.018},"last":0.79300000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9792,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362048,"statfs":{"total":96636764160,"available":96626737152,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69199999999999995,"15min":0.63300000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92500000000000004,"5min":1.0640000000000001,"15min":1.659},"last":0.875},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69699999999999995,"15min":0.63400000000000001},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.38900000000000001},"max":{"1min":0.95899999999999996,"5min":0.97999999999999998,"15min":2.3050000000000002},"last":0.85499999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66900000000000004,"15min":0.62},"min":{"1min":0.44600000000000001,"5min":0.38200000000000001,"15min":0.35499999999999998},"max":{"1min":0.94299999999999995,"5min":0.95699999999999996,"15min":1.2390000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72099999999999997,"15min":0.65300000000000002},"min":{"1min":0.54300000000000004,"5min":0.40500000000000003,"15min":0.39400000000000002},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.9630000000000001},"last":0.79100000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69699999999999995,"15min":0.65700000000000003},"min":{"1min":0.53500000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.86899999999999999,"5min":1.0469999999999999,"15min":2.3860000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71199999999999997,"15min":0.66400000000000003},"min":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.432},"max":{"1min":0.89300000000000002,"5min":0.97299999999999998,"15min":1.0800000000000001},"last":0.80900000000000005}]},{"osd":3,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71399999999999997,"15min":0.65900000000000003},"min":{"1min":0.53800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":1.0169999999999999,"15min":1.8140000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73699999999999999,"15min":0.71399999999999997},"min":{"1min":0.5,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91700000000000004,"5min":0.96699999999999997,"15min":2.3500000000000001},"last":0.95099999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63200000000000001,"15min":0.65300000000000002},"min":{"1min":0.495,"5min":0.379,"15min":0.379},"max":{"1min":0.79900000000000004,"5min":0.95999999999999996,"15min":1.702},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70799999999999996,"15min":0.72799999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89000000000000001,"5min":1.0529999999999999,"15min":2.0009999999999999},"last":0.73799999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64500000000000002,"15min":0.69599999999999995},"min":{"1min":0.437,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76700000000000002,"5min":0.89400000000000002,"15min":2.4969999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66900000000000004,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86899999999999999,"5min":1.012,"15min":2.0880000000000001},"last":0.57099999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69499999999999995,"15min":0.71499999999999997},"min":{"1min":0.53100000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93999999999999995,"5min":0.999,"15min":2.0449999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.49199999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.85099999999999998,"5min":0.98099999999999998,"15min":2.4249999999999998},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:15:32.820 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T13:15:32.820 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:15:33.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.015+0000 7faf6b7ff700 1 Processor -- start 2023-11-28T13:15:33.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.015+0000 7faf6b7ff700 1 -- start start 2023-11-28T13:15:33.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.016+0000 7faf6b7ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7faf64161340 0x7faf640d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:33.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.016+0000 7faf6b7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf640d2320 0x7faf640cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:33.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.016+0000 7faf6b7ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf640d1950 con 0x7faf6415c9b0 2023-11-28T13:15:33.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.016+0000 7faf6b7ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf640b5800 con 0x7faf64161340 2023-11-28T13:15:33.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.016+0000 7faf6b7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf640b5300 con 0x7faf640d2320 2023-11-28T13:15:33.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.016+0000 7faf6959b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7faf6415c9b0 0x7faf640b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39442/0 (socket says 172.21.15.156:39442) 2023-11-28T13:15:33.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.016+0000 7faf6959b700 1 -- 172.21.15.156:0/3368468322 learned_addr learned my addr 172.21.15.156:0/3368468322 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:33.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.017+0000 7faf69d9c700 1 --2- 172.21.15.156:0/3368468322 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf640d2320 0x7faf640cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:33.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.017+0000 7faf68d9a700 1 --2- 172.21.15.156:0/3368468322 >> v2:172.21.15.186:3300/0 conn(0x7faf64161340 0x7faf640d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:33.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.017+0000 7faf5bfff700 1 -- 172.21.15.156:0/3368468322 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1519212869 0 0) 0x7faf640d1950 con 0x7faf6415c9b0 2023-11-28T13:15:33.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.017+0000 7faf5bfff700 1 -- 172.21.15.156:0/3368468322 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf500035b0 con 0x7faf6415c9b0 2023-11-28T13:15:33.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.017+0000 7faf69d9c700 1 -- 172.21.15.156:0/3368468322 >> v2:172.21.15.186:3300/0 conn(0x7faf64161340 msgr2=0x7faf640d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:33.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.017+0000 7faf69d9c700 1 --2- 172.21.15.156:0/3368468322 >> v2:172.21.15.186:3300/0 conn(0x7faf64161340 0x7faf640d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.017+0000 7faf69d9c700 1 -- 172.21.15.156:0/3368468322 >> v1:172.21.15.156:6789/0 conn(0x7faf6415c9b0 legacy=0x7faf640b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:33.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.017+0000 7faf69d9c700 1 -- 172.21.15.156:0/3368468322 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf640b5520 con 0x7faf640d2320 2023-11-28T13:15:33.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.017+0000 7faf68d9a700 1 --2- 172.21.15.156:0/3368468322 >> v2:172.21.15.186:3300/0 conn(0x7faf64161340 0x7faf640d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:15:33.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.017+0000 7faf69d9c700 1 --2- 172.21.15.156:0/3368468322 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf640d2320 0x7faf640cd7c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009d90 tx=0x7faf5c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85e20772dd281378 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:33.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.018+0000 7faf5bfff700 1 -- 172.21.15.156:0/3368468322 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c00e070 con 0x7faf640d2320 2023-11-28T13:15:33.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.018+0000 7faf5bfff700 1 -- 172.21.15.156:0/3368468322 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c00bb20 con 0x7faf640d2320 2023-11-28T13:15:33.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.018+0000 7faf5bfff700 1 -- 172.21.15.156:0/3368468322 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c012430 con 0x7faf640d2320 2023-11-28T13:15:33.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.018+0000 7faf6b7ff700 1 -- 172.21.15.156:0/3368468322 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf640d2320 msgr2=0x7faf640cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:33.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.018+0000 7faf6b7ff700 1 --2- 172.21.15.156:0/3368468322 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf640d2320 0x7faf640cd7c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009d90 tx=0x7faf5c00b330 comp rx=0 tx=0).stop 2023-11-28T13:15:33.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.018+0000 7faf6b7ff700 1 -- 172.21.15.156:0/3368468322 shutdown_connections 2023-11-28T13:15:33.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.018+0000 7faf6b7ff700 1 --2- 172.21.15.156:0/3368468322 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf640d2320 0x7faf640cd7c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.018+0000 7faf6b7ff700 1 --2- 172.21.15.156:0/3368468322 >> v2:172.21.15.186:3300/0 conn(0x7faf64161340 0x7faf640d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.018+0000 7faf6b7ff700 1 -- 172.21.15.156:0/3368468322 >> 172.21.15.156:0/3368468322 conn(0x7faf64072600 msgr2=0x7faf640cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:33.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.018+0000 7faf6b7ff700 1 -- 172.21.15.156:0/3368468322 shutdown_connections 2023-11-28T13:15:33.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.018+0000 7faf6b7ff700 1 -- 172.21.15.156:0/3368468322 wait complete. 2023-11-28T13:15:33.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.020+0000 7faf6b7ff700 1 Processor -- start 2023-11-28T13:15:33.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.020+0000 7faf6b7ff700 1 -- start start 2023-11-28T13:15:33.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.021+0000 7faf6b7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf640d2320 0x7faf64150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:33.027 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.021+0000 7faf6b7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6415c9b0 0x7faf640b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:33.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.021+0000 7faf6b7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf64161340 0x7faf640b8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:33.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.021+0000 7faf6b7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf641f2590 con 0x7faf64161340 2023-11-28T13:15:33.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.021+0000 7faf6b7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf640c8440 con 0x7faf6415c9b0 2023-11-28T13:15:33.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.021+0000 7faf6b7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf64150d40 con 0x7faf640d2320 2023-11-28T13:15:33.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.021+0000 7faf69d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf64161340 0x7faf640b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:33.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.021+0000 7faf6959b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf640d2320 0x7faf64150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:33.028 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.021+0000 7faf69d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf64161340 0x7faf640b8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39524/0 (socket says 172.21.15.156:39524) 2023-11-28T13:15:33.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.021+0000 7faf69d9c700 1 -- 172.21.15.156:0/890117161 learned_addr learned my addr 172.21.15.156:0/890117161 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:33.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.021+0000 7faf68d9a700 1 --2- 172.21.15.156:0/890117161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6415c9b0 0x7faf640b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:33.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.022+0000 7faf69d9c700 1 -- 172.21.15.156:0/890117161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf640d2320 msgr2=0x7faf64150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:33.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.022+0000 7faf69d9c700 1 --2- 172.21.15.156:0/890117161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf640d2320 0x7faf64150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.022+0000 7faf69d9c700 1 -- 172.21.15.156:0/890117161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6415c9b0 msgr2=0x7faf640b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:33.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.022+0000 7faf69d9c700 1 --2- 172.21.15.156:0/890117161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6415c9b0 0x7faf640b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.029 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.022+0000 7faf69d9c700 1 -- 172.21.15.156:0/890117161 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf540094a0 con 0x7faf64161340 2023-11-28T13:15:33.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.022+0000 7faf69d9c700 1 --2- 172.21.15.156:0/890117161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf64161340 0x7faf640b8140 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7faf640d05d0 tx=0x7faf5c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:33.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.022+0000 7faf59ffb700 1 -- 172.21.15.156:0/890117161 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c0129f0 con 0x7faf64161340 2023-11-28T13:15:33.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.022+0000 7faf59ffb700 1 -- 172.21.15.156:0/890117161 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c020070 con 0x7faf64161340 2023-11-28T13:15:33.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.022+0000 7faf6b7ff700 1 -- 172.21.15.156:0/890117161 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5c009a70 con 0x7faf64161340 2023-11-28T13:15:33.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.022+0000 7faf59ffb700 1 -- 172.21.15.156:0/890117161 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c012db0 con 0x7faf64161340 2023-11-28T13:15:33.030 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.022+0000 7faf6b7ff700 1 -- 172.21.15.156:0/890117161 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf640d15e0 con 0x7faf64161340 2023-11-28T13:15:33.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.023+0000 7faf6b7ff700 1 -- 172.21.15.156:0/890117161 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf640d1300 con 0x7faf64161340 2023-11-28T13:15:33.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.024+0000 7faf59ffb700 1 -- 172.21.15.156:0/890117161 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faf5c004000 con 0x7faf64161340 2023-11-28T13:15:33.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.026+0000 7faf59ffb700 1 --2- 172.21.15.156:0/890117161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faf480682f0 0x7faf4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:33.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.026+0000 7faf59ffb700 1 -- 172.21.15.156:0/890117161 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7faf5c022070 con 0x7faf64161340 2023-11-28T13:15:33.031 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.026+0000 7faf6959b700 1 --2- 172.21.15.156:0/890117161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faf480682f0 0x7faf4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:33.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.026+0000 7faf6959b700 1 --2- 172.21.15.156:0/890117161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faf480682f0 0x7faf4806a7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faf60007fd0 tx=0x7faf60007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:33.032 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.030+0000 7faf59ffb700 1 -- 172.21.15.156:0/890117161 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faf5c0546d0 con 0x7faf64161340 2023-11-28T13:15:33.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.217+0000 7faf6b7ff700 1 -- 172.21.15.156:0/890117161 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf6415cd80 con 0x7faf480682f0 2023-11-28T13:15:33.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.221+0000 7faf59ffb700 1 -- 172.21.15.156:0/890117161 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73022 (secure 0 0 0) 0x7faf6415cd80 con 0x7faf480682f0 2023-11-28T13:15:33.223 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:15:33.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.223+0000 7faf6b7ff700 1 -- 172.21.15.156:0/890117161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faf480682f0 msgr2=0x7faf4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:33.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.223+0000 7faf6b7ff700 1 --2- 172.21.15.156:0/890117161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faf480682f0 0x7faf4806a7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faf60007fd0 tx=0x7faf60007dc0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.223+0000 7faf6b7ff700 1 -- 172.21.15.156:0/890117161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf64161340 msgr2=0x7faf640b8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:33.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.223+0000 7faf6b7ff700 1 --2- 172.21.15.156:0/890117161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf64161340 0x7faf640b8140 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7faf640d05d0 tx=0x7faf5c004ad0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.223+0000 7faf6b7ff700 1 -- 172.21.15.156:0/890117161 shutdown_connections 2023-11-28T13:15:33.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.223+0000 7faf6b7ff700 1 --2- 172.21.15.156:0/890117161 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7faf480682f0 0x7faf4806a7a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.223+0000 7faf6b7ff700 1 --2- 172.21.15.156:0/890117161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf64161340 0x7faf640b8140 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.223+0000 7faf6b7ff700 1 --2- 172.21.15.156:0/890117161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faf640d2320 0x7faf64150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.223+0000 7faf6b7ff700 1 --2- 172.21.15.156:0/890117161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6415c9b0 0x7faf640b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.223+0000 7faf6b7ff700 1 -- 172.21.15.156:0/890117161 >> 172.21.15.156:0/890117161 conn(0x7faf64072600 msgr2=0x7faf6415a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:33.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.223+0000 7faf6b7ff700 1 -- 172.21.15.156:0/890117161 shutdown_connections 2023-11-28T13:15:33.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.223+0000 7faf6b7ff700 1 -- 172.21.15.156:0/890117161 wait complete. 2023-11-28T13:15:33.229 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T13:15:33.243 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2023-11-28T13:15:32.134137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":100436,"kb_used_data":7124,"kb_used_omap":0,"kb_used_meta":93248,"kb_avail":754874284,"statfs":{"total":773094113280,"available":772991266816,"internally_reserved":0,"allocated":7294976,"data_stored":3881970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95485952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12572,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-80,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.529920"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:15:25.993449+0000","last_change":"2023-11-28T13:07:56.890378+0000","last_active":"2023-11-28T13:15:25.993449+0000","last_peered":"2023-11-28T13:15:25.993449+0000","last_clean":"2023-11-28T13:15:25.993449+0000","last_became_active":"2023-11-28T13:07:56.890058+0000","last_became_peered":"2023-11-28T13:07:56.890058+0000","last_unstale":"2023-11-28T13:15:25.993449+0000","last_undegraded":"2023-11-28T13:15:25.993449+0000","last_fullsized":"2023-11-28T13:15:25.993449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:05.344602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":56,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:29.649906+0000","last_change":"2023-11-28T13:07:56.844509+0000","last_active":"2023-11-28T13:15:29.649906+0000","last_peered":"2023-11-28T13:15:29.649906+0000","last_clean":"2023-11-28T13:15:29.649906+0000","last_became_active":"2023-11-28T13:07:56.844081+0000","last_became_peered":"2023-11-28T13:07:56.844081+0000","last_unstale":"2023-11-28T13:15:29.649906+0000","last_undegraded":"2023-11-28T13:15:29.649906+0000","last_fullsized":"2023-11-28T13:15:29.649906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":54,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.858689+0000","last_change":"2023-11-28T13:07:56.844699+0000","last_active":"2023-11-28T13:15:28.858689+0000","last_peered":"2023-11-28T13:15:28.858689+0000","last_clean":"2023-11-28T13:15:28.858689+0000","last_became_active":"2023-11-28T13:07:56.844391+0000","last_became_peered":"2023-11-28T13:07:56.844391+0000","last_unstale":"2023-11-28T13:15:28.858689+0000","last_undegraded":"2023-11-28T13:15:28.858689+0000","last_fullsized":"2023-11-28T13:15:28.858689+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":205,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.619311+0000","last_change":"2023-11-28T13:07:57.844020+0000","last_active":"2023-11-28T13:15:28.619311+0000","last_peered":"2023-11-28T13:15:28.619311+0000","last_clean":"2023-11-28T13:15:28.619311+0000","last_became_active":"2023-11-28T13:07:57.842981+0000","last_became_peered":"2023-11-28T13:07:57.842981+0000","last_unstale":"2023-11-28T13:15:28.619311+0000","last_undegraded":"2023-11-28T13:15:28.619311+0000","last_fullsized":"2023-11-28T13:15:28.619311+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_clean_scrub_stamp":"2023-11-28T13:07:56.825711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:44.973218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":56,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.621568+0000","last_change":"2023-11-28T13:07:57.306318+0000","last_active":"2023-11-28T13:15:28.621568+0000","last_peered":"2023-11-28T13:15:28.621568+0000","last_clean":"2023-11-28T13:15:28.621568+0000","last_became_active":"2023-11-28T13:07:57.305937+0000","last_became_peered":"2023-11-28T13:07:57.305937+0000","last_unstale":"2023-11-28T13:15:28.621568+0000","last_undegraded":"2023-11-28T13:15:28.621568+0000","last_fullsized":"2023-11-28T13:15:28.621568+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":56,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.619321+0000","last_change":"2023-11-28T13:07:57.306277+0000","last_active":"2023-11-28T13:15:28.619321+0000","last_peered":"2023-11-28T13:15:28.619321+0000","last_clean":"2023-11-28T13:15:28.619321+0000","last_became_active":"2023-11-28T13:07:57.305983+0000","last_became_peered":"2023-11-28T13:07:57.305983+0000","last_unstale":"2023-11-28T13:15:28.619321+0000","last_undegraded":"2023-11-28T13:15:28.619321+0000","last_fullsized":"2023-11-28T13:15:28.619321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.618799+0000","last_change":"2023-11-28T13:07:57.306177+0000","last_active":"2023-11-28T13:15:28.618799+0000","last_peered":"2023-11-28T13:15:28.618799+0000","last_clean":"2023-11-28T13:15:28.618799+0000","last_became_active":"2023-11-28T13:07:57.305853+0000","last_became_peered":"2023-11-28T13:07:57.305853+0000","last_unstale":"2023-11-28T13:15:28.618799+0000","last_undegraded":"2023-11-28T13:15:28.618799+0000","last_fullsized":"2023-11-28T13:15:28.618799+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":55,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:15:25.993150+0000","last_change":"2023-11-28T13:07:57.306321+0000","last_active":"2023-11-28T13:15:25.993150+0000","last_peered":"2023-11-28T13:15:25.993150+0000","last_clean":"2023-11-28T13:15:25.993150+0000","last_became_active":"2023-11-28T13:07:57.305923+0000","last_became_peered":"2023-11-28T13:07:57.305923+0000","last_unstale":"2023-11-28T13:15:25.993150+0000","last_undegraded":"2023-11-28T13:15:25.993150+0000","last_fullsized":"2023-11-28T13:15:25.993150+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:41.104679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":55,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:15:25.993400+0000","last_change":"2023-11-28T13:07:56.844305+0000","last_active":"2023-11-28T13:15:25.993400+0000","last_peered":"2023-11-28T13:15:25.993400+0000","last_clean":"2023-11-28T13:15:25.993400+0000","last_became_active":"2023-11-28T13:07:56.843980+0000","last_became_peered":"2023-11-28T13:07:56.843980+0000","last_unstale":"2023-11-28T13:15:25.993400+0000","last_undegraded":"2023-11-28T13:15:25.993400+0000","last_fullsized":"2023-11-28T13:15:25.993400+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574943,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15108,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94356732,"statfs":{"total":96636764160,"available":96621293568,"internally_reserved":0,"allocated":1372160,"data_stored":933538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68700000000000006,"15min":0.66900000000000004},"min":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89900000000000002,"5min":0.91200000000000003,"15min":0.98399999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68999999999999995,"15min":0.64800000000000002},"min":{"1min":0.498,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82499999999999996,"5min":0.92300000000000004,"15min":1.0569999999999999},"last":0.65300000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73399999999999999,"15min":0.72299999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84199999999999997,"5min":0.93999999999999995,"15min":1.0289999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.73999999999999999,"15min":0.71399999999999997},"min":{"1min":0.57599999999999996,"5min":0.42399999999999999,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":0.95999999999999996,"15min":1.095},"last":0.72299999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.66500000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85499999999999998,"5min":0.92300000000000004,"15min":0.996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72599999999999998,"15min":0.69499999999999995},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.85699999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.68999999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.42999999999999999},"max":{"1min":0.89000000000000001,"5min":0.91600000000000004,"15min":0.93899999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73799999999999999,"15min":0.76200000000000001},"min":{"1min":0.60399999999999998,"5min":0.52200000000000002,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.98599999999999999,"15min":1.079},"last":0.62}]},{"osd":4,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.66400000000000003},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":1.008},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.628,"15min":0.70799999999999996},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":1.006,"15min":1.071},"last":0.69199999999999995}]},{"osd":5,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64200000000000002,"15min":0.73499999999999999},"min":{"1min":0.501,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86399999999999999,"5min":0.93600000000000005,"15min":1.1830000000000001},"last":0.745},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65300000000000002,"15min":0.75900000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96299999999999997,"5min":1.044,"15min":1.1839999999999999},"last":0.87}]},{"osd":6,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71799999999999997,"15min":0.80600000000000005},"min":{"1min":0.56100000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94099999999999995,"5min":0.98599999999999999,"15min":1.165},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70199999999999996,"15min":0.76500000000000001},"min":{"1min":0.50700000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92800000000000005,"5min":0.95499999999999996,"15min":1.141},"last":0.60799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9456,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362384,"statfs":{"total":96636764160,"available":96627081216,"internally_reserved":0,"allocated":761856,"data_stored":337184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.68700000000000006,"15min":0.67700000000000005},"min":{"1min":0.441,"5min":0.432,"15min":0.41599999999999998},"max":{"1min":0.88300000000000001,"5min":0.995,"15min":0.995},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70399999999999996,"15min":0.66400000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91300000000000003,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.81399999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70899999999999996,"15min":0.68700000000000006},"min":{"1min":0.56100000000000005,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.86899999999999999,"5min":1.113,"15min":1.113},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70699999999999996,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.48599999999999999,"15min":0.47899999999999998},"max":{"1min":0.85499999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.77100000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72299999999999998,"15min":0.68899999999999995},"min":{"1min":0.60399999999999998,"5min":0.45000000000000001,"15min":0.41799999999999998},"max":{"1min":0.90300000000000002,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72099999999999997,"15min":0.68500000000000005},"min":{"1min":0.51000000000000001,"5min":0.495,"15min":0.38500000000000001},"max":{"1min":0.89400000000000002,"5min":1.024,"15min":1.024},"last":0.745}]},{"osd":3,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.73599999999999999,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":1.121,"15min":1.121},"last":0.874},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.745,"15min":0.73799999999999999},"min":{"1min":0.63,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.89200000000000002,"5min":1.167,"15min":1.167},"last":0.79500000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.624,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92300000000000004,"5min":1.042,"15min":1.0429999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.79200000000000004},"min":{"1min":0.54200000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96499999999999997,"5min":1.105,"15min":1.3140000000000001},"last":0.97699999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63500000000000001,"15min":0.68500000000000005},"min":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84599999999999997,"5min":0.92100000000000004,"15min":1.0780000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.70699999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70499999999999996,"15min":0.72899999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93400000000000005,"5min":1.034,"15min":1.0980000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73399999999999999,"15min":0.76800000000000002},"min":{"1min":0.501,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94899999999999995,"5min":1.0129999999999999,"15min":1.107},"last":0.85799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14900,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":13568,"kb_avail":94356940,"statfs":{"total":96636764160,"available":96621506560,"internally_reserved":0,"allocated":1355776,"data_stored":927552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13893632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41799999999999998,"15min":0.43099999999999999},"min":{"1min":0.28100000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.496,"5min":0.75900000000000001,"15min":1.28},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.47899999999999998,"15min":0.45600000000000002},"min":{"1min":0.30499999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64000000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42499999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.46000000000000002},"min":{"1min":0.33200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60599999999999998,"5min":0.747,"15min":0.747},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.67500000000000004,"5min":0.77000000000000002,"15min":0.84699999999999998},"last":0.53600000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.436,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71099999999999997,"5min":0.88500000000000001,"15min":1.0009999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52000000000000002,"15min":0.53800000000000003},"min":{"1min":0.36899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.68899999999999995,"5min":0.746,"15min":1.1120000000000001},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73099999999999998,"15min":0.70799999999999996},"min":{"1min":0.60299999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.753,"15min":0.72199999999999998},"min":{"1min":0.624,"5min":0.54100000000000004,"15min":0.49099999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.86499999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.74299999999999999,"15min":0.745},"min":{"1min":0.61199999999999999,"5min":0.51700000000000002,"15min":0.433},"max":{"1min":1.0209999999999999,"5min":1.292,"15min":1.292},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.74299999999999999,"15min":0.72099999999999997},"min":{"1min":0.621,"5min":0.441,"15min":0.441},"max":{"1min":1.141,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.878}]},{"osd":6,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.74199999999999999},"min":{"1min":0.64300000000000002,"5min":0.55300000000000005,"15min":0.52700000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.47299999999999998},"max":{"1min":1.1240000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.88900000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.80000000000000004,"15min":0.81000000000000005},"min":{"1min":0.66900000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0820000000000001,"5min":1.357,"15min":1.357},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77800000000000002,"15min":0.78400000000000003},"min":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0389999999999999,"5min":1.24,"15min":1.3580000000000001},"last":0.95399999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574943,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13724,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":12992,"kb_avail":94358116,"statfs":{"total":96636764160,"available":96622710784,"internally_reserved":0,"allocated":741376,"data_stored":329593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13303808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.47799999999999998},"min":{"1min":0.32500000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65400000000000003,"5min":0.69699999999999995,"15min":1.3120000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47999999999999998,"15min":0.51500000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67000000000000004,"5min":0.70599999999999996,"15min":1.48},"last":0.45300000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.504,"15min":0.52100000000000002},"min":{"1min":0.41999999999999998,"5min":0.34200000000000003,"15min":0.33000000000000002},"max":{"1min":0.76700000000000002,"5min":0.874,"15min":0.874},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.53000000000000003,"15min":0.51400000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.749,"5min":0.749,"15min":0.80800000000000005},"last":0.504}]},{"osd":3,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.35599999999999998,"5min":0.33700000000000002,"15min":0.30099999999999999},"max":{"1min":0.63800000000000001,"5min":0.747,"15min":0.747},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56399999999999995,"15min":0.55500000000000005},"min":{"1min":0.42099999999999999,"5min":0.36699999999999999,"15min":0.35399999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.82199999999999995},"last":0.47699999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.75900000000000001,"15min":0.77500000000000002},"min":{"1min":0.54500000000000004,"5min":0.48399999999999999,"15min":0.45500000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.071},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.99099999999999999,"5min":1.147,"15min":1.147},"last":0.79000000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77300000000000002,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.48299999999999998},"max":{"1min":1.0660000000000001,"5min":1.1040000000000001,"15min":1.3260000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0309999999999999,"5min":1.127,"15min":1.127},"last":0.88100000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.77300000000000002,"15min":0.76100000000000001},"min":{"1min":0.58099999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0660000000000001,"5min":1.127,"15min":1.127},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80800000000000005,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.042,"5min":1.0489999999999999,"15min":1.097},"last":0.85999999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78500000000000003,"15min":0.80000000000000004},"min":{"1min":0.58999999999999997,"5min":0.55600000000000005,"15min":0.53700000000000003},"max":{"1min":1.1200000000000001,"5min":1.149,"15min":1.149},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79600000000000004,"15min":0.80200000000000005},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.996,"5min":1.163,"15min":1.163},"last":0.82199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13616,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358224,"statfs":{"total":96636764160,"available":96622821376,"internally_reserved":0,"allocated":761856,"data_stored":337184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.48099999999999998,"15min":0.44500000000000001},"min":{"1min":0.28499999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.68400000000000005,"5min":6.4130000000000003,"15min":6.4130000000000003},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.64200000000000002,"15min":0.52200000000000002},"min":{"1min":0.32700000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.625,"5min":19.190000000000001,"15min":19.190000000000001},"last":0.48699999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.33100000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66100000000000003,"5min":5.2009999999999996,"15min":5.2009999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.68899999999999995,"15min":0.57099999999999995},"min":{"1min":0.40300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.67900000000000005,"5min":19.241,"15min":19.241},"last":0.55400000000000005}]},{"osd":3,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55700000000000005,"15min":0.48499999999999999},"min":{"1min":0.34999999999999998,"5min":0.29899999999999999,"15min":0.29799999999999999},"max":{"1min":0.66500000000000004,"5min":7.0410000000000004,"15min":7.0410000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.65200000000000002,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70499999999999996,"5min":11.420999999999999,"15min":11.420999999999999},"last":0.57499999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.84199999999999997,"15min":0.82999999999999996},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.55000000000000004},"max":{"1min":1.006,"5min":5.1630000000000003,"15min":5.1630000000000003},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.80400000000000005,"15min":0.79600000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.496},"max":{"1min":0.95899999999999996,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":0.70899999999999996}]},{"osd":5,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.75700000000000001,"15min":0.73999999999999999},"min":{"1min":0.58499999999999996,"5min":0.505,"15min":0.46100000000000002},"max":{"1min":0.86899999999999999,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.81100000000000005,"15min":0.76100000000000001},"min":{"1min":0.52900000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.036,"5min":6.7569999999999997,"15min":6.7569999999999997},"last":0.80400000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.83299999999999996,"15min":0.77700000000000002},"min":{"1min":0.51100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97099999999999997,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.83799999999999997,"15min":0.80800000000000005},"min":{"1min":0.64200000000000002,"5min":0.55700000000000005,"15min":0.49199999999999999},"max":{"1min":0.95399999999999996,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":0.90600000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.79100000000000004,"15min":0.77000000000000002},"min":{"1min":0.57799999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.996,"5min":2.008,"15min":2.008},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.82399999999999995,"15min":0.80800000000000005},"min":{"1min":0.56000000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1379999999999999,"5min":6.6769999999999996,"15min":6.6769999999999996},"last":0.88100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13920,"kb_used_data":728,"kb_used_omap":0,"kb_used_meta":13184,"kb_avail":94357920,"statfs":{"total":96636764160,"available":96622510080,"internally_reserved":0,"allocated":745472,"data_stored":330579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13500416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.33100000000000002,"5min":0.26100000000000001,"15min":0.245},"max":{"1min":0.61699999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.48299999999999998,"15min":0.45700000000000002},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.27300000000000002},"max":{"1min":0.69799999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.46899999999999997,"15min":0.48099999999999998},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71199999999999997,"5min":0.73699999999999999,"15min":0.88100000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.41099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.48099999999999998,"5min":0.372,"15min":0.36399999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56999999999999995,"15min":0.54900000000000004},"min":{"1min":0.40300000000000002,"5min":0.39000000000000001,"15min":0.35399999999999998},"max":{"1min":0.80500000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.61099999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72199999999999998,"15min":0.70499999999999996},"min":{"1min":0.54400000000000004,"5min":0.50900000000000001,"15min":0.50600000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.73399999999999999,"15min":0.71399999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.46200000000000002},"max":{"1min":1.054,"5min":1.087,"15min":1.087},"last":0.73699999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.091},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.77500000000000002,"15min":0.76600000000000001},"min":{"1min":0.63400000000000001,"5min":0.50600000000000001,"15min":0.498},"max":{"1min":1.0369999999999999,"5min":1.0649999999999999,"15min":1.0680000000000001},"last":0.751}]},{"osd":6,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76600000000000001,"15min":0.73499999999999999},"min":{"1min":0.65500000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96799999999999997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.76100000000000001,"15min":0.71999999999999997},"min":{"1min":0.59299999999999997,"5min":0.50600000000000001,"15min":0.46100000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.57799999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79900000000000004,"15min":0.76900000000000002},"min":{"1min":0.64600000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.032,"5min":1.05,"15min":1.05},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.80000000000000004,"15min":0.77300000000000002},"min":{"1min":0.68400000000000005,"5min":0.53600000000000003,"15min":0.52800000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.1299999999999999},"last":0.69899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9920,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361920,"statfs":{"total":96636764160,"available":96626606080,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70499999999999996,"15min":0.63},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.91900000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72099999999999997,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.42799999999999999,"15min":0.38400000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":1.3089999999999999},"last":0.56899999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73499999999999999,"15min":0.69699999999999995},"min":{"1min":0.59399999999999997,"5min":0.51200000000000001,"15min":0.48299999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":1.161},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69399999999999995,"15min":0.63},"min":{"1min":0.47899999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88900000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.628}]},{"osd":2,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":0.67800000000000005},"min":{"1min":0.46400000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.755},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.69899999999999995},"min":{"1min":0.502,"5min":0.47599999999999998,"15min":0.441},"max":{"1min":0.91500000000000004,"5min":1.042,"15min":1.042},"last":0.70799999999999996}]},{"osd":3,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71699999999999997,"15min":0.68600000000000005},"min":{"1min":0.48699999999999999,"5min":0.46000000000000002,"15min":0.438},"max":{"1min":0.85499999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.75700000000000001,"15min":0.72799999999999998},"min":{"1min":0.54500000000000004,"5min":0.52900000000000003,"15min":0.505},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.60199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.66000000000000003},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95799999999999996,"5min":1.012,"15min":1.264},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64500000000000002,"15min":0.70299999999999996},"min":{"1min":0.49399999999999999,"5min":0.39100000000000001,"15min":0.38400000000000001},"max":{"1min":0.81999999999999995,"5min":0.93200000000000005,"15min":1.079},"last":0.72499999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65600000000000003,"15min":0.70799999999999996},"min":{"1min":0.46300000000000002,"5min":0.439,"15min":0.32700000000000001},"max":{"1min":0.89000000000000001,"5min":0.91200000000000003,"15min":1.022},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69799999999999995,"15min":0.73099999999999998},"min":{"1min":0.53800000000000003,"5min":0.48699999999999999,"15min":0.47399999999999998},"max":{"1min":0.90900000000000003,"5min":1.044,"15min":1.22},"last":0.74299999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69099999999999995,"15min":0.745},"min":{"1min":0.51700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98699999999999999,"5min":0.995,"15min":1.4530000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72899999999999998,"15min":0.76700000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.40899999999999997},"max":{"1min":1.0089999999999999,"5min":1.018,"15min":1.018},"last":0.79300000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9792,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362048,"statfs":{"total":96636764160,"available":96626737152,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69199999999999995,"15min":0.63300000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92500000000000004,"5min":1.0640000000000001,"15min":1.659},"last":0.875},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69699999999999995,"15min":0.63400000000000001},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.38900000000000001},"max":{"1min":0.95899999999999996,"5min":0.97999999999999998,"15min":2.3050000000000002},"last":0.85499999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66900000000000004,"15min":0.62},"min":{"1min":0.44600000000000001,"5min":0.38200000000000001,"15min":0.35499999999999998},"max":{"1min":0.94299999999999995,"5min":0.95699999999999996,"15min":1.2390000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72099999999999997,"15min":0.65300000000000002},"min":{"1min":0.54300000000000004,"5min":0.40500000000000003,"15min":0.39400000000000002},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.9630000000000001},"last":0.79100000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69699999999999995,"15min":0.65700000000000003},"min":{"1min":0.53500000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.86899999999999999,"5min":1.0469999999999999,"15min":2.3860000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71199999999999997,"15min":0.66400000000000003},"min":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.432},"max":{"1min":0.89300000000000002,"5min":0.97299999999999998,"15min":1.0800000000000001},"last":0.80900000000000005}]},{"osd":3,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71399999999999997,"15min":0.65900000000000003},"min":{"1min":0.53800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":1.0169999999999999,"15min":1.8140000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73699999999999999,"15min":0.71399999999999997},"min":{"1min":0.5,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91700000000000004,"5min":0.96699999999999997,"15min":2.3500000000000001},"last":0.95099999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63200000000000001,"15min":0.65300000000000002},"min":{"1min":0.495,"5min":0.379,"15min":0.379},"max":{"1min":0.79900000000000004,"5min":0.95999999999999996,"15min":1.702},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70799999999999996,"15min":0.72799999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89000000000000001,"5min":1.0529999999999999,"15min":2.0009999999999999},"last":0.73799999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64500000000000002,"15min":0.69599999999999995},"min":{"1min":0.437,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76700000000000002,"5min":0.89400000000000002,"15min":2.4969999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66900000000000004,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86899999999999999,"5min":1.012,"15min":2.0880000000000001},"last":0.57099999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69499999999999995,"15min":0.71499999999999997},"min":{"1min":0.53100000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93999999999999995,"5min":0.999,"15min":2.0449999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.49199999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.85099999999999998,"5min":0.98099999999999998,"15min":2.4249999999999998},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:15:33.246 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:15:33.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.442+0000 7f5c87bff700 1 Processor -- start 2023-11-28T13:15:33.444 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.442+0000 7f5c87bff700 1 -- start start 2023-11-28T13:15:33.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.443+0000 7f5c87bff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5c8015c9b0 0x7f5c800b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:33.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.443+0000 7f5c87bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c80161340 0x7f5c800d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:33.446 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.444+0000 7f5c87bff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5c800d2320 0x7f5c800cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:33.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.444+0000 7f5c87bff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c800b5800 con 0x7f5c8015c9b0 2023-11-28T13:15:33.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.444+0000 7f5c87bff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c800d1950 con 0x7f5c800d2320 2023-11-28T13:15:33.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.444+0000 7f5c87bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c800b5520 con 0x7f5c80161340 2023-11-28T13:15:33.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.444+0000 7f5c8599b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5c8015c9b0 0x7f5c800b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:33.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.444+0000 7f5c8599b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5c8015c9b0 0x7f5c800b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39534/0 (socket says 172.21.15.156:39534) 2023-11-28T13:15:33.447 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.444+0000 7f5c8599b700 1 -- 172.21.15.156:0/1791998688 learned_addr learned my addr 172.21.15.156:0/1791998688 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:33.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.444+0000 7f5c8519a700 1 --2- 172.21.15.156:0/1791998688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c80161340 0x7f5c800d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:33.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.444+0000 7f5c8619c700 1 --2- 172.21.15.156:0/1791998688 >> v2:172.21.15.186:3300/0 conn(0x7f5c800d2320 0x7f5c800cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:33.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.444+0000 7f5c8599b700 1 -- 172.21.15.156:0/1791998688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c80161340 msgr2=0x7f5c800d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:33.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.444+0000 7f5c8599b700 1 --2- 172.21.15.156:0/1791998688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c80161340 0x7f5c800d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.444+0000 7f5c8599b700 1 -- 172.21.15.156:0/1791998688 >> v2:172.21.15.186:3300/0 conn(0x7f5c800d2320 msgr2=0x7f5c800cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:33.448 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.444+0000 7f5c8599b700 1 --2- 172.21.15.156:0/1791998688 >> v2:172.21.15.186:3300/0 conn(0x7f5c800d2320 0x7f5c800cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.444+0000 7f5c8599b700 1 -- 172.21.15.156:0/1791998688 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c800d1300 con 0x7f5c8015c9b0 2023-11-28T13:15:33.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.445+0000 7f5c8599b700 1 --2- 172.21.15.156:0/1791998688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c8015c9b0 0x7f5c800b4bf0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f5c8008c4d0 tx=0x7f5c7c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1d9845f97b8f6e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:33.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.445+0000 7f5c84999700 1 -- 172.21.15.156:0/1791998688 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c00b870 con 0x7f5c8015c9b0 2023-11-28T13:15:33.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.445+0000 7f5c84999700 1 -- 172.21.15.156:0/1791998688 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c00b9d0 con 0x7f5c8015c9b0 2023-11-28T13:15:33.450 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.445+0000 7f5c84999700 1 -- 172.21.15.156:0/1791998688 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c00bd60 con 0x7f5c8015c9b0 2023-11-28T13:15:33.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.445+0000 7f5c87bff700 1 -- 172.21.15.156:0/1791998688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c8015c9b0 msgr2=0x7f5c800b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:33.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.445+0000 7f5c87bff700 1 --2- 172.21.15.156:0/1791998688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c8015c9b0 0x7f5c800b4bf0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f5c8008c4d0 tx=0x7f5c7c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.445+0000 7f5c87bff700 1 -- 172.21.15.156:0/1791998688 shutdown_connections 2023-11-28T13:15:33.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.445+0000 7f5c87bff700 1 --2- 172.21.15.156:0/1791998688 >> v2:172.21.15.186:3300/0 conn(0x7f5c800d2320 0x7f5c800cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.445+0000 7f5c87bff700 1 --2- 172.21.15.156:0/1791998688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c8015c9b0 0x7f5c800b4bf0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.445+0000 7f5c87bff700 1 --2- 172.21.15.156:0/1791998688 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c80161340 0x7f5c800d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.445+0000 7f5c87bff700 1 -- 172.21.15.156:0/1791998688 >> 172.21.15.156:0/1791998688 conn(0x7f5c80072600 msgr2=0x7f5c800cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:33.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.446+0000 7f5c87bff700 1 -- 172.21.15.156:0/1791998688 shutdown_connections 2023-11-28T13:15:33.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.446+0000 7f5c87bff700 1 -- 172.21.15.156:0/1791998688 wait complete. 2023-11-28T13:15:33.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.447+0000 7f5c87bff700 1 Processor -- start 2023-11-28T13:15:33.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.447+0000 7f5c87bff700 1 -- start start 2023-11-28T13:15:33.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.448+0000 7f5c87bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c800d2320 0x7f5c8015dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:33.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.448+0000 7f5c87bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c8015c9b0 0x7f5c8014dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:33.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.448+0000 7f5c87bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c80161340 0x7f5c80155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:33.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.448+0000 7f5c87bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c801e68a0 con 0x7f5c80161340 2023-11-28T13:15:33.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.448+0000 7f5c87bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8015e150 con 0x7f5c800d2320 2023-11-28T13:15:33.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.448+0000 7f5c87bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8015e3f0 con 0x7f5c8015c9b0 2023-11-28T13:15:33.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.449+0000 7f5c8619c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c80161340 0x7f5c80155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:33.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.449+0000 7f5c8599b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c800d2320 0x7f5c8015dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:33.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.449+0000 7f5c8599b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c800d2320 0x7f5c8015dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:40580/0 (socket says 172.21.15.156:40580) 2023-11-28T13:15:33.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.449+0000 7f5c8599b700 1 -- 172.21.15.156:0/1567833939 learned_addr learned my addr 172.21.15.156:0/1567833939 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:33.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.449+0000 7f5c8519a700 1 --2- 172.21.15.156:0/1567833939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c8015c9b0 0x7f5c8014dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:33.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.449+0000 7f5c8619c700 1 -- 172.21.15.156:0/1567833939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c8015c9b0 msgr2=0x7f5c8014dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:33.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.449+0000 7f5c8619c700 1 --2- 172.21.15.156:0/1567833939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c8015c9b0 0x7f5c8014dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.449+0000 7f5c8619c700 1 -- 172.21.15.156:0/1567833939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c800d2320 msgr2=0x7f5c8015dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:33.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.449+0000 7f5c8619c700 1 --2- 172.21.15.156:0/1567833939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c800d2320 0x7f5c8015dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.449+0000 7f5c8619c700 1 -- 172.21.15.156:0/1567833939 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c009550 con 0x7f5c80161340 2023-11-28T13:15:33.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.449+0000 7f5c8619c700 1 --2- 172.21.15.156:0/1567833939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c80161340 0x7f5c80155dd0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5c78007cf0 tx=0x7f5c7800e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:33.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.450+0000 7f5c767fc700 1 -- 172.21.15.156:0/1567833939 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78018070 con 0x7f5c80161340 2023-11-28T13:15:33.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.450+0000 7f5c767fc700 1 -- 172.21.15.156:0/1567833939 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c78003a20 con 0x7f5c80161340 2023-11-28T13:15:33.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.450+0000 7f5c767fc700 1 -- 172.21.15.156:0/1567833939 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78003db0 con 0x7f5c80161340 2023-11-28T13:15:33.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.450+0000 7f5c87bff700 1 -- 172.21.15.156:0/1567833939 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c800b5130 con 0x7f5c80161340 2023-11-28T13:15:33.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.450+0000 7f5c87bff700 1 -- 172.21.15.156:0/1567833939 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c800d15e0 con 0x7f5c80161340 2023-11-28T13:15:33.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.451+0000 7f5c87bff700 1 -- 172.21.15.156:0/1567833939 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c800b52c0 con 0x7f5c80161340 2023-11-28T13:15:33.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.452+0000 7f5c767fc700 1 -- 172.21.15.156:0/1567833939 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5c78012050 con 0x7f5c80161340 2023-11-28T13:15:33.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.453+0000 7f5c767fc700 1 --2- 172.21.15.156:0/1567833939 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5c640682f0 0x7f5c6406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:33.460 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.453+0000 7f5c767fc700 1 -- 172.21.15.156:0/1567833939 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5c780857d0 con 0x7f5c80161340 2023-11-28T13:15:33.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.453+0000 7f5c8599b700 1 --2- 172.21.15.156:0/1567833939 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5c640682f0 0x7f5c6406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:33.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.454+0000 7f5c8599b700 1 --2- 172.21.15.156:0/1567833939 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5c640682f0 0x7f5c6406a7a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009c10 tx=0x7f5c7c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:33.461 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.457+0000 7f5c767fc700 1 -- 172.21.15.156:0/1567833939 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5c78052260 con 0x7f5c80161340 2023-11-28T13:15:33.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.649+0000 7f5c87bff700 1 -- 172.21.15.156:0/1567833939 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c801e69e0 con 0x7f5c80161340 2023-11-28T13:15:33.652 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.650+0000 7f5c767fc700 1 -- 172.21.15.156:0/1567833939 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+14389 (secure 0 0 0) 0x7f5c78055880 con 0x7f5c80161340 2023-11-28T13:15:33.652 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:15:33.652 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":64,"fsid":"12d9e849-dc60-46f0-8e8e-9e291633f61b","created":"2023-11-28T13:07:42.605767+0000","modified":"2023-11-28T13:15:29.615306+0000","last_up_change":"2023-11-28T13:07:54.821228+0000","last_in_change":"2023-11-28T13:07:46.384751+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:07:55.600013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:07:56.083469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"220c17ef-9e08-4a62-b062-f68eec1bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":132471},{"type":"v1","addr":"172.21.15.156:6809","nonce":132471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":132471},{"type":"v1","addr":"172.21.15.156:6811","nonce":132471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6815","nonce":132471},{"type":"v1","addr":"172.21.15.156:6817","nonce":132471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":132471},{"type":"v1","addr":"172.21.15.156:6813","nonce":132471}]},"public_addr":"172.21.15.156:6809/132471","cluster_addr":"172.21.15.156:6811/132471","heartbeat_back_addr":"172.21.15.156:6817/132471","heartbeat_front_addr":"172.21.15.156:6813/132471","state":["exists","up"]},{"osd":1,"uuid":"69f34e6a-bf4b-4a4f-baff-8d6e41022f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":132479},{"type":"v1","addr":"172.21.15.156:6801","nonce":132479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":132479},{"type":"v1","addr":"172.21.15.156:6803","nonce":132479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":132479},{"type":"v1","addr":"172.21.15.156:6807","nonce":132479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":132479},{"type":"v1","addr":"172.21.15.156:6805","nonce":132479}]},"public_addr":"172.21.15.156:6801/132479","cluster_addr":"172.21.15.156:6803/132479","heartbeat_back_addr":"172.21.15.156:6807/132479","heartbeat_front_addr":"172.21.15.156:6805/132479","state":["exists","up"]},{"osd":2,"uuid":"89cd6662-def1-41b7-9cf4-4abb8d377ea0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":132475},{"type":"v1","addr":"172.21.15.156:6816","nonce":132475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":132475},{"type":"v1","addr":"172.21.15.156:6819","nonce":132475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":132475},{"type":"v1","addr":"172.21.15.156:6825","nonce":132475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":132475},{"type":"v1","addr":"172.21.15.156:6822","nonce":132475}]},"public_addr":"172.21.15.156:6816/132475","cluster_addr":"172.21.15.156:6819/132475","heartbeat_back_addr":"172.21.15.156:6825/132475","heartbeat_front_addr":"172.21.15.156:6822/132475","state":["exists","up"]},{"osd":3,"uuid":"9ac08dfd-f39c-4929-a7ae-eff60f1df906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6821","nonce":132480},{"type":"v1","addr":"172.21.15.156:6823","nonce":132480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":132480},{"type":"v1","addr":"172.21.15.156:6827","nonce":132480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":132480},{"type":"v1","addr":"172.21.15.156:6831","nonce":132480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":132480},{"type":"v1","addr":"172.21.15.156:6829","nonce":132480}]},"public_addr":"172.21.15.156:6823/132480","cluster_addr":"172.21.15.156:6827/132480","heartbeat_back_addr":"172.21.15.156:6831/132480","heartbeat_front_addr":"172.21.15.156:6829/132480","state":["exists","up"]},{"osd":4,"uuid":"1b0b7457-5a57-456a-b22d-4751c0004593","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":132937},{"type":"v1","addr":"172.21.15.186:6817","nonce":132937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":132937},{"type":"v1","addr":"172.21.15.186:6819","nonce":132937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":132937},{"type":"v1","addr":"172.21.15.186:6823","nonce":132937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":132937},{"type":"v1","addr":"172.21.15.186:6821","nonce":132937}]},"public_addr":"172.21.15.186:6817/132937","cluster_addr":"172.21.15.186:6819/132937","heartbeat_back_addr":"172.21.15.186:6823/132937","heartbeat_front_addr":"172.21.15.186:6821/132937","state":["exists","up"]},{"osd":5,"uuid":"13b142fc-4857-4039-bee3-1d616041b4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":132935},{"type":"v1","addr":"172.21.15.186:6801","nonce":132935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":132935},{"type":"v1","addr":"172.21.15.186:6803","nonce":132935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":132935},{"type":"v1","addr":"172.21.15.186:6807","nonce":132935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":132935},{"type":"v1","addr":"172.21.15.186:6805","nonce":132935}]},"public_addr":"172.21.15.186:6801/132935","cluster_addr":"172.21.15.186:6803/132935","heartbeat_back_addr":"172.21.15.186:6807/132935","heartbeat_front_addr":"172.21.15.186:6805/132935","state":["exists","up"]},{"osd":6,"uuid":"568bcbd6-4435-4c87-aada-29a5b9499432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":132946},{"type":"v1","addr":"172.21.15.186:6825","nonce":132946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":132946},{"type":"v1","addr":"172.21.15.186:6827","nonce":132946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":132946},{"type":"v1","addr":"172.21.15.186:6831","nonce":132946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":132946},{"type":"v1","addr":"172.21.15.186:6829","nonce":132946}]},"public_addr":"172.21.15.186:6825/132946","cluster_addr":"172.21.15.186:6827/132946","heartbeat_back_addr":"172.21.15.186:6831/132946","heartbeat_front_addr":"172.21.15.186:6829/132946","state":["exists","up"]},{"osd":7,"uuid":"6b2a114b-2a7d-4458-a8a8-8cbc1cf9f7b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":132936},{"type":"v1","addr":"172.21.15.186:6809","nonce":132936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":132936},{"type":"v1","addr":"172.21.15.186:6811","nonce":132936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":132936},{"type":"v1","addr":"172.21.15.186:6815","nonce":132936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":132936},{"type":"v1","addr":"172.21.15.186:6813","nonce":132936}]},"public_addr":"172.21.15.186:6809/132936","cluster_addr":"172.21.15.186:6811/132936","heartbeat_back_addr":"172.21.15.186:6815/132936","heartbeat_front_addr":"172.21.15.186:6813/132936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.709231+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.499998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.723509+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.943667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.011201+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.978664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:52.034928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:07:51.917236+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.156:6834/3314379816":"2023-11-29T13:15:25.601380+0000","172.21.15.156:6832/3314379816":"2023-11-29T13:15:25.601380+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:15:33.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.653+0000 7f5c87bff700 1 -- 172.21.15.156:0/1567833939 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5c640682f0 msgr2=0x7f5c6406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:33.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.653+0000 7f5c87bff700 1 --2- 172.21.15.156:0/1567833939 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5c640682f0 0x7f5c6406a7a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009c10 tx=0x7f5c7c008030 comp rx=0 tx=0).stop 2023-11-28T13:15:33.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.653+0000 7f5c87bff700 1 -- 172.21.15.156:0/1567833939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c80161340 msgr2=0x7f5c80155dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:33.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.653+0000 7f5c87bff700 1 --2- 172.21.15.156:0/1567833939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c80161340 0x7f5c80155dd0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5c78007cf0 tx=0x7f5c7800e0e0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.653+0000 7f5c87bff700 1 -- 172.21.15.156:0/1567833939 shutdown_connections 2023-11-28T13:15:33.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.653+0000 7f5c87bff700 1 --2- 172.21.15.156:0/1567833939 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5c640682f0 0x7f5c6406a7a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.653+0000 7f5c87bff700 1 --2- 172.21.15.156:0/1567833939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c80161340 0x7f5c80155dd0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.653+0000 7f5c87bff700 1 --2- 172.21.15.156:0/1567833939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c800d2320 0x7f5c8015dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.653+0000 7f5c87bff700 1 --2- 172.21.15.156:0/1567833939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c8015c9b0 0x7f5c8014dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:33.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.653+0000 7f5c87bff700 1 -- 172.21.15.156:0/1567833939 >> 172.21.15.156:0/1567833939 conn(0x7f5c80072600 msgr2=0x7f5c8015ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:33.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.653+0000 7f5c87bff700 1 -- 172.21.15.156:0/1567833939 shutdown_connections 2023-11-28T13:15:33.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:33.653+0000 7f5c87bff700 1 -- 172.21.15.156:0/1567833939 wait complete. 2023-11-28T13:15:34.675 INFO:tasks.ceph:Scrubbing osd.0 2023-11-28T13:15:34.675 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:15:34.874 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:15:34.879 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:15:34.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.886+0000 7f809d9ff700 1 Processor -- start 2023-11-28T13:15:34.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.886+0000 7f809d9ff700 1 -- start start 2023-11-28T13:15:34.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.887+0000 7f809d9ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8098161200 0x7f809815dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:34.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.887+0000 7f809d9ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8098160d70 con 0x7f809815a760 2023-11-28T13:15:34.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.887+0000 7f809d9ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80980b5390 con 0x7f80980cded0 2023-11-28T13:15:34.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.887+0000 7f809d9ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f809815ab30 con 0x7f8098161200 2023-11-28T13:15:34.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.887+0000 7f80977fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8098161200 0x7f809815dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:34.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.888+0000 7f8096ffd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f809815a760 0x7f80980cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39458/0 (socket says 172.21.15.156:39458) 2023-11-28T13:15:34.890 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.888+0000 7f8096ffd700 1 -- 172.21.15.156:0/1356686172 learned_addr learned my addr 172.21.15.156:0/1356686172 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:34.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.888+0000 7f80977fe700 1 -- 172.21.15.156:0/1356686172 >> v1:172.21.15.186:6789/0 conn(0x7f80980cded0 legacy=0x7f80980d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:34.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.888+0000 7f80977fe700 1 -- 172.21.15.156:0/1356686172 >> v1:172.21.15.156:6789/0 conn(0x7f809815a760 legacy=0x7f80980cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:34.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.888+0000 7f80977fe700 1 -- 172.21.15.156:0/1356686172 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80980b5670 con 0x7f8098161200 2023-11-28T13:15:34.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.888+0000 7f8095ffb700 1 -- 172.21.15.156:0/1356686172 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71638879 0 0) 0x7f8098160d70 con 0x7f809815a760 2023-11-28T13:15:34.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.888+0000 7f80977fe700 1 --2- 172.21.15.156:0/1356686172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8098161200 0x7f809815dc30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8090009d90 tx=0x7f809000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda9c6ebb3e21a8d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:34.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.889+0000 7f8095ffb700 1 -- 172.21.15.156:0/1356686172 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809000e070 con 0x7f8098161200 2023-11-28T13:15:34.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.889+0000 7f8095ffb700 1 -- 172.21.15.156:0/1356686172 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809000bbd0 con 0x7f8098161200 2023-11-28T13:15:34.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.889+0000 7f8095ffb700 1 -- 172.21.15.156:0/1356686172 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8090012430 con 0x7f8098161200 2023-11-28T13:15:34.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.889+0000 7f809d9ff700 1 -- 172.21.15.156:0/1356686172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8098161200 msgr2=0x7f809815dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:34.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.889+0000 7f809d9ff700 1 --2- 172.21.15.156:0/1356686172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8098161200 0x7f809815dc30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8090009d90 tx=0x7f809000b330 comp rx=0 tx=0).stop 2023-11-28T13:15:34.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.889+0000 7f809d9ff700 1 -- 172.21.15.156:0/1356686172 shutdown_connections 2023-11-28T13:15:34.892 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.889+0000 7f809d9ff700 1 --2- 172.21.15.156:0/1356686172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8098161200 0x7f809815dc30 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:34.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.889+0000 7f809d9ff700 1 -- 172.21.15.156:0/1356686172 >> 172.21.15.156:0/1356686172 conn(0x7f8098072620 msgr2=0x7f80980cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:34.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.889+0000 7f809d9ff700 1 -- 172.21.15.156:0/1356686172 shutdown_connections 2023-11-28T13:15:34.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.889+0000 7f809d9ff700 1 -- 172.21.15.156:0/1356686172 wait complete. 2023-11-28T13:15:34.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.892+0000 7f809d9ff700 1 Processor -- start 2023-11-28T13:15:34.893 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.892+0000 7f809d9ff700 1 -- start start 2023-11-28T13:15:34.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.893+0000 7f809d9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80980cded0 0x7f8098155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:34.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.893+0000 7f809d9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f809815a760 0x7f8098154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:34.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.893+0000 7f809d9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8098161200 0x7f80980cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:34.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.893+0000 7f809d9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80980cb0a0 con 0x7f8098161200 2023-11-28T13:15:34.895 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.893+0000 7f809d9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80980ca7b0 con 0x7f809815a760 2023-11-28T13:15:34.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.893+0000 7f809d9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80981552d0 con 0x7f80980cded0 2023-11-28T13:15:34.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.893+0000 7f80977fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8098161200 0x7f80980cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:34.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.893+0000 7f80977fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8098161200 0x7f80980cc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39548/0 (socket says 172.21.15.156:39548) 2023-11-28T13:15:34.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.893+0000 7f80977fe700 1 -- 172.21.15.156:0/2815691166 learned_addr learned my addr 172.21.15.156:0/2815691166 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:34.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.893+0000 7f8096ffd700 1 --2- 172.21.15.156:0/2815691166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80980cded0 0x7f8098155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:34.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.893+0000 7f80967fc700 1 --2- 172.21.15.156:0/2815691166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f809815a760 0x7f8098154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:34.896 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.894+0000 7f80977fe700 1 -- 172.21.15.156:0/2815691166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80980cded0 msgr2=0x7f8098155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:34.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.894+0000 7f80977fe700 1 --2- 172.21.15.156:0/2815691166 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80980cded0 0x7f8098155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:34.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.894+0000 7f80977fe700 1 -- 172.21.15.156:0/2815691166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f809815a760 msgr2=0x7f8098154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:34.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.894+0000 7f80977fe700 1 --2- 172.21.15.156:0/2815691166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f809815a760 0x7f8098154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:34.897 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.894+0000 7f80977fe700 1 -- 172.21.15.156:0/2815691166 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8090009a70 con 0x7f8098161200 2023-11-28T13:15:34.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.894+0000 7f80977fe700 1 --2- 172.21.15.156:0/2815691166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8098161200 0x7f80980cc9f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8090014040 tx=0x7f8090004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:34.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.894+0000 7f807ffff700 1 -- 172.21.15.156:0/2815691166 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809001b810 con 0x7f8098161200 2023-11-28T13:15:34.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.894+0000 7f807ffff700 1 -- 172.21.15.156:0/2815691166 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8090020070 con 0x7f8098161200 2023-11-28T13:15:34.898 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.894+0000 7f809d9ff700 1 -- 172.21.15.156:0/2815691166 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8098160a00 con 0x7f8098161200 2023-11-28T13:15:34.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.894+0000 7f807ffff700 1 -- 172.21.15.156:0/2815691166 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809001bbb0 con 0x7f8098161200 2023-11-28T13:15:34.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.895+0000 7f809d9ff700 1 -- 172.21.15.156:0/2815691166 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8098160720 con 0x7f8098161200 2023-11-28T13:15:34.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.896+0000 7f809d9ff700 1 -- 172.21.15.156:0/2815691166 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f80980b4bf0 con 0x7f8098161200 2023-11-28T13:15:34.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.896+0000 7f807ffff700 1 -- 172.21.15.156:0/2815691166 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f809002e070 con 0x7f8098161200 2023-11-28T13:15:34.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.898+0000 7f807ffff700 1 --2- 172.21.15.156:0/2815691166 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f80840682f0 0x7f808406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:34.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.898+0000 7f807ffff700 1 -- 172.21.15.156:0/2815691166 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8090022070 con 0x7f8098161200 2023-11-28T13:15:34.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.898+0000 7f807ffff700 1 --2- 172.21.15.156:0/2815691166 >> [v2:172.21.15.156:6808/132471,v1:172.21.15.156:6809/132471] conn(0x7f808406e760 0x7f8084070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:34.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.898+0000 7f807ffff700 1 -- 172.21.15.156:0/2815691166 --> [v2:172.21.15.156:6808/132471,v1:172.21.15.156:6809/132471] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8084071240 con 0x7f808406e760 2023-11-28T13:15:34.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.898+0000 7f807ffff700 1 -- 172.21.15.156:0/2815691166 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f809008c2a0 con 0x7f8098161200 2023-11-28T13:15:34.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.898+0000 7f8096ffd700 1 --2- 172.21.15.156:0/2815691166 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f80840682f0 0x7f808406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:34.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.899+0000 7f80967fc700 1 --2- 172.21.15.156:0/2815691166 >> [v2:172.21.15.156:6808/132471,v1:172.21.15.156:6809/132471] conn(0x7f808406e760 0x7f8084070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:34.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.899+0000 7f80967fc700 1 --2- 172.21.15.156:0/2815691166 >> [v2:172.21.15.156:6808/132471,v1:172.21.15.156:6809/132471] conn(0x7f808406e760 0x7f8084070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:34.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.899+0000 7f8096ffd700 1 --2- 172.21.15.156:0/2815691166 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f80840682f0 0x7f808406a7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8088007b10 tx=0x7f8088007a60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:34.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.900+0000 7f807ffff700 1 -- 172.21.15.156:0/2815691166 <== osd.0 v2:172.21.15.156:6808/132471 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8084071240 con 0x7f808406e760 2023-11-28T13:15:34.916 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.915+0000 7f809d9ff700 1 -- 172.21.15.156:0/2815691166 --> [v2:172.21.15.156:6808/132471,v1:172.21.15.156:6809/132471] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f809804e620 con 0x7f808406e760 2023-11-28T13:15:34.917 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.915+0000 7f807ffff700 1 -- 172.21.15.156:0/2815691166 <== osd.0 v2:172.21.15.156:6808/132471 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f809804e620 con 0x7f808406e760 2023-11-28T13:15:34.917 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:15:34.917 INFO:teuthology.orchestra.run.smithi156.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:15:34.918 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:15:34.918 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.916+0000 7f809d9ff700 1 -- 172.21.15.156:0/2815691166 >> [v2:172.21.15.156:6808/132471,v1:172.21.15.156:6809/132471] conn(0x7f808406e760 msgr2=0x7f8084070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:34.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.916+0000 7f809d9ff700 1 --2- 172.21.15.156:0/2815691166 >> [v2:172.21.15.156:6808/132471,v1:172.21.15.156:6809/132471] conn(0x7f808406e760 0x7f8084070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:34.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.916+0000 7f809d9ff700 1 -- 172.21.15.156:0/2815691166 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f80840682f0 msgr2=0x7f808406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:34.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.916+0000 7f809d9ff700 1 --2- 172.21.15.156:0/2815691166 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f80840682f0 0x7f808406a7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8088007b10 tx=0x7f8088007a60 comp rx=0 tx=0).stop 2023-11-28T13:15:34.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.916+0000 7f809d9ff700 1 -- 172.21.15.156:0/2815691166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8098161200 msgr2=0x7f80980cc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:34.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.916+0000 7f809d9ff700 1 --2- 172.21.15.156:0/2815691166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8098161200 0x7f80980cc9f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8090014040 tx=0x7f8090004ad0 comp rx=0 tx=0).stop 2023-11-28T13:15:34.919 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.916+0000 7f80977fe700 1 -- 172.21.15.156:0/2815691166 reap_dead start 2023-11-28T13:15:34.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.916+0000 7f809d9ff700 1 -- 172.21.15.156:0/2815691166 shutdown_connections 2023-11-28T13:15:34.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.916+0000 7f809d9ff700 1 -- 172.21.15.156:0/2815691166 >> 172.21.15.156:0/2815691166 conn(0x7f8098072620 msgr2=0x7f809815e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:34.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.916+0000 7f809d9ff700 1 -- 172.21.15.156:0/2815691166 shutdown_connections 2023-11-28T13:15:34.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:34.916+0000 7f809d9ff700 1 -- 172.21.15.156:0/2815691166 wait complete. 2023-11-28T13:15:34.937 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-28T13:15:35.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.173+0000 7f548b5ff700 1 Processor -- start 2023-11-28T13:15:35.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.174+0000 7f548b5ff700 1 -- start start 2023-11-28T13:15:35.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.175+0000 7f548b5ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f548415c9b0 0x7f54840b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:35.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.175+0000 7f548b5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54840cd7c0 0x7f54840cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:35.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.175+0000 7f548b5ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54840d1950 con 0x7f5484161340 2023-11-28T13:15:35.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.175+0000 7f548b5ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54840b5800 con 0x7f548415c9b0 2023-11-28T13:15:35.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.175+0000 7f548b5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54840ce140 con 0x7f54840cd7c0 2023-11-28T13:15:35.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.175+0000 7f548939b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f548415c9b0 0x7f54840b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:35.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.175+0000 7f548939b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f548415c9b0 0x7f54840b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39562/0 (socket says 172.21.15.156:39562) 2023-11-28T13:15:35.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.175+0000 7f548939b700 1 -- 172.21.15.156:0/299592456 learned_addr learned my addr 172.21.15.156:0/299592456 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:35.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.175+0000 7f5489b9c700 1 --2- 172.21.15.156:0/299592456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54840cd7c0 0x7f54840cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:35.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.176+0000 7f548939b700 1 -- 172.21.15.156:0/299592456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54840cd7c0 msgr2=0x7f54840cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:35.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.176+0000 7f547bfff700 1 -- 172.21.15.156:0/299592456 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 848039102 0 0) 0x7f54840d1950 con 0x7f5484161340 2023-11-28T13:15:35.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.176+0000 7f548939b700 1 --2- 172.21.15.156:0/299592456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54840cd7c0 0x7f54840cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:35.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.176+0000 7f548939b700 1 -- 172.21.15.156:0/299592456 >> v1:172.21.15.186:6789/0 conn(0x7f5484161340 legacy=0x7f54840d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.176+0000 7f548939b700 1 -- 172.21.15.156:0/299592456 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54840d24f0 con 0x7f548415c9b0 2023-11-28T13:15:35.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.176+0000 7f548939b700 1 --2- 172.21.15.156:0/299592456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f548415c9b0 0x7f54840b4bf0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f54800098f0 tx=0x7f5480009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca214e8cf8d63b95 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:35.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.176+0000 7f547bfff700 1 -- 172.21.15.156:0/299592456 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480011830 con 0x7f548415c9b0 2023-11-28T13:15:35.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.176+0000 7f547bfff700 1 -- 172.21.15.156:0/299592456 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5480011990 con 0x7f548415c9b0 2023-11-28T13:15:35.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.176+0000 7f547bfff700 1 -- 172.21.15.156:0/299592456 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480011d20 con 0x7f548415c9b0 2023-11-28T13:15:35.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.177+0000 7f548b5ff700 1 -- 172.21.15.156:0/299592456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f548415c9b0 msgr2=0x7f54840b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.177+0000 7f548b5ff700 1 --2- 172.21.15.156:0/299592456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f548415c9b0 0x7f54840b4bf0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f54800098f0 tx=0x7f5480009c00 comp rx=0 tx=0).stop 2023-11-28T13:15:35.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.177+0000 7f548b5ff700 1 -- 172.21.15.156:0/299592456 shutdown_connections 2023-11-28T13:15:35.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.177+0000 7f548b5ff700 1 --2- 172.21.15.156:0/299592456 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54840cd7c0 0x7f54840cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:35.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.177+0000 7f548b5ff700 1 --2- 172.21.15.156:0/299592456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f548415c9b0 0x7f54840b4bf0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:35.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.177+0000 7f548b5ff700 1 -- 172.21.15.156:0/299592456 >> 172.21.15.156:0/299592456 conn(0x7f5484072600 msgr2=0x7f54840cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:35.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.177+0000 7f548b5ff700 1 -- 172.21.15.156:0/299592456 shutdown_connections 2023-11-28T13:15:35.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.177+0000 7f548b5ff700 1 -- 172.21.15.156:0/299592456 wait complete. 2023-11-28T13:15:35.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.179+0000 7f548b5ff700 1 Processor -- start 2023-11-28T13:15:35.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.180+0000 7f548b5ff700 1 -- start start 2023-11-28T13:15:35.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.181+0000 7f548b5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54840cd7c0 0x7f54841e7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:35.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.181+0000 7f548b5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548415c9b0 0x7f5484156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:35.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.181+0000 7f548b5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5484161340 0x7f54840c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:35.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.181+0000 7f548b5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54840cb530 con 0x7f54840cd7c0 2023-11-28T13:15:35.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.181+0000 7f548b5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54840cb810 con 0x7f548415c9b0 2023-11-28T13:15:35.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.181+0000 7f548b5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54840cbaf0 con 0x7f5484161340 2023-11-28T13:15:35.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.181+0000 7f548939b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54840cd7c0 0x7f54841e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:35.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.181+0000 7f548939b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54840cd7c0 0x7f54841e7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39564/0 (socket says 172.21.15.156:39564) 2023-11-28T13:15:35.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.181+0000 7f548939b700 1 -- 172.21.15.156:0/3098031676 learned_addr learned my addr 172.21.15.156:0/3098031676 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:35.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.181+0000 7f5489b9c700 1 --2- 172.21.15.156:0/3098031676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5484161340 0x7f54840c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:35.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.181+0000 7f5488b9a700 1 --2- 172.21.15.156:0/3098031676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548415c9b0 0x7f5484156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:35.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.181+0000 7f548939b700 1 -- 172.21.15.156:0/3098031676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5484161340 msgr2=0x7f54840c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.181+0000 7f548939b700 1 --2- 172.21.15.156:0/3098031676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5484161340 0x7f54840c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:35.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.181+0000 7f548939b700 1 -- 172.21.15.156:0/3098031676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548415c9b0 msgr2=0x7f5484156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.181+0000 7f548939b700 1 --2- 172.21.15.156:0/3098031676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548415c9b0 0x7f5484156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:35.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.181+0000 7f548939b700 1 -- 172.21.15.156:0/3098031676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5480009550 con 0x7f54840cd7c0 2023-11-28T13:15:35.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.181+0000 7f5488b9a700 1 --2- 172.21.15.156:0/3098031676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548415c9b0 0x7f5484156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:15:35.187 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.182+0000 7f548939b700 1 --2- 172.21.15.156:0/3098031676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54840cd7c0 0x7f54841e7480 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f5480009c40 tx=0x7f5480004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:35.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.182+0000 7f5479ffb700 1 -- 172.21.15.156:0/3098031676 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480004590 con 0x7f54840cd7c0 2023-11-28T13:15:35.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.182+0000 7f5479ffb700 1 -- 172.21.15.156:0/3098031676 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548001e070 con 0x7f54840cd7c0 2023-11-28T13:15:35.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.182+0000 7f548b5ff700 1 -- 172.21.15.156:0/3098031676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54840d1300 con 0x7f54840cd7c0 2023-11-28T13:15:35.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.182+0000 7f5479ffb700 1 -- 172.21.15.156:0/3098031676 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54800038c0 con 0x7f54840cd7c0 2023-11-28T13:15:35.188 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.182+0000 7f548b5ff700 1 -- 172.21.15.156:0/3098031676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54840b5520 con 0x7f54840cd7c0 2023-11-28T13:15:35.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.183+0000 7f548b5ff700 1 -- 172.21.15.156:0/3098031676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54840d15e0 con 0x7f54840cd7c0 2023-11-28T13:15:35.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.184+0000 7f5479ffb700 1 -- 172.21.15.156:0/3098031676 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f548000f050 con 0x7f54840cd7c0 2023-11-28T13:15:35.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.185+0000 7f5479ffb700 1 --2- 172.21.15.156:0/3098031676 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f54680682f0 0x7f546806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:35.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.185+0000 7f5479ffb700 1 -- 172.21.15.156:0/3098031676 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f54800869b0 con 0x7f54840cd7c0 2023-11-28T13:15:35.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.186+0000 7f5488b9a700 1 --2- 172.21.15.156:0/3098031676 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f54680682f0 0x7f546806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:35.190 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.186+0000 7f5488b9a700 1 --2- 172.21.15.156:0/3098031676 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f54680682f0 0x7f546806a7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f548406ae80 tx=0x7f5474009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:35.191 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.189+0000 7f5479ffb700 1 -- 172.21.15.156:0/3098031676 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5480053440 con 0x7f54840cd7c0 2023-11-28T13:15:35.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.379+0000 7f548b5ff700 1 -- 172.21.15.156:0/3098031676 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f54840745c0 con 0x7f54680682f0 2023-11-28T13:15:35.381 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.380+0000 7f5479ffb700 1 -- 172.21.15.156:0/3098031676 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f54840745c0 con 0x7f54680682f0 2023-11-28T13:15:35.384 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.382+0000 7f548b5ff700 1 -- 172.21.15.156:0/3098031676 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f54680682f0 msgr2=0x7f546806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.382+0000 7f548b5ff700 1 --2- 172.21.15.156:0/3098031676 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f54680682f0 0x7f546806a7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f548406ae80 tx=0x7f5474009040 comp rx=0 tx=0).stop 2023-11-28T13:15:35.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.382+0000 7f548b5ff700 1 -- 172.21.15.156:0/3098031676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54840cd7c0 msgr2=0x7f54841e7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.382+0000 7f548b5ff700 1 --2- 172.21.15.156:0/3098031676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54840cd7c0 0x7f54841e7480 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f5480009c40 tx=0x7f5480004750 comp rx=0 tx=0).stop 2023-11-28T13:15:35.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.383+0000 7f548b5ff700 1 -- 172.21.15.156:0/3098031676 shutdown_connections 2023-11-28T13:15:35.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.383+0000 7f548b5ff700 1 --2- 172.21.15.156:0/3098031676 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f54680682f0 0x7f546806a7a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:35.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.383+0000 7f548b5ff700 1 --2- 172.21.15.156:0/3098031676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5484161340 0x7f54840c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:35.385 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.383+0000 7f548b5ff700 1 --2- 172.21.15.156:0/3098031676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54840cd7c0 0x7f54841e7480 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:35.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.383+0000 7f548b5ff700 1 --2- 172.21.15.156:0/3098031676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548415c9b0 0x7f5484156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:35.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.383+0000 7f548b5ff700 1 -- 172.21.15.156:0/3098031676 >> 172.21.15.156:0/3098031676 conn(0x7f5484072600 msgr2=0x7f54840b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:35.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.383+0000 7f548b5ff700 1 -- 172.21.15.156:0/3098031676 shutdown_connections 2023-11-28T13:15:35.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.383+0000 7f548b5ff700 1 -- 172.21.15.156:0/3098031676 wait complete. 2023-11-28T13:15:35.386 INFO:teuthology.orchestra.run.smithi156.stderr:instructed osd(s) 0 to deep-scrub 2023-11-28T13:15:35.403 INFO:tasks.ceph:Scrubbing osd.1 2023-11-28T13:15:35.403 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:15:35.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.602+0000 7f3bf59ff700 1 Processor -- start 2023-11-28T13:15:35.604 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.602+0000 7f3bf59ff700 1 -- start start 2023-11-28T13:15:35.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.603+0000 7f3bf59ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3bf015a760 0x7f3bf0161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:35.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.603+0000 7f3bf59ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf015e0a0 con 0x7f3bf015dc30 2023-11-28T13:15:35.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.603+0000 7f3bf59ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf0160d70 con 0x7f3bf00cded0 2023-11-28T13:15:35.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.603+0000 7f3bf59ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf015ab30 con 0x7f3bf015a760 2023-11-28T13:15:35.607 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.603+0000 7f3bee7fc700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f3bf015dc30 0x7f3bf00cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:39462/0 (socket says 172.21.15.156:39462) 2023-11-28T13:15:35.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.603+0000 7f3bee7fc700 1 -- 172.21.15.156:0/116873423 learned_addr learned my addr 172.21.15.156:0/116873423 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:35.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.604+0000 7f3beeffd700 1 --2- 172.21.15.156:0/116873423 >> v2:172.21.15.186:3300/0 conn(0x7f3bf015a760 0x7f3bf0161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:35.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.604+0000 7f3bedffb700 1 -- 172.21.15.156:0/116873423 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3837372764 0 0) 0x7f3bf015e0a0 con 0x7f3bf015dc30 2023-11-28T13:15:35.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.604+0000 7f3bedffb700 1 -- 172.21.15.156:0/116873423 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bd80035b0 con 0x7f3bf015dc30 2023-11-28T13:15:35.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.604+0000 7f3bedffb700 1 -- 172.21.15.156:0/116873423 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2901165953 0 0) 0x7f3bf0160d70 con 0x7f3bf00cded0 2023-11-28T13:15:35.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.604+0000 7f3bedffb700 1 -- 172.21.15.156:0/116873423 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bf015e0a0 con 0x7f3bf00cded0 2023-11-28T13:15:35.608 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.604+0000 7f3beeffd700 1 -- 172.21.15.156:0/116873423 >> v1:172.21.15.186:6789/0 conn(0x7f3bf00cded0 legacy=0x7f3bf00d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.604+0000 7f3bedffb700 1 -- 172.21.15.156:0/116873423 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 957870136 0 0) 0x7f3bd80035b0 con 0x7f3bf015dc30 2023-11-28T13:15:35.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.604+0000 7f3beeffd700 1 -- 172.21.15.156:0/116873423 >> v1:172.21.15.156:6789/0 conn(0x7f3bf015dc30 legacy=0x7f3bf00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.604+0000 7f3beeffd700 1 -- 172.21.15.156:0/116873423 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf00b5390 con 0x7f3bf015a760 2023-11-28T13:15:35.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.604+0000 7f3beeffd700 1 --2- 172.21.15.156:0/116873423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bf015a760 0x7f3bf0161200 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009840 tx=0x7f3be0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d70a153851cac13 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:35.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.605+0000 7f3bedffb700 1 -- 172.21.15.156:0/116873423 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be4001670 con 0x7f3bf015a760 2023-11-28T13:15:35.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.605+0000 7f3bedffb700 1 -- 172.21.15.156:0/116873423 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be0004030 con 0x7f3bf015a760 2023-11-28T13:15:35.609 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.605+0000 7f3bedffb700 1 -- 172.21.15.156:0/116873423 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be00043c0 con 0x7f3bf015a760 2023-11-28T13:15:35.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.605+0000 7f3bf59ff700 1 -- 172.21.15.156:0/116873423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bf015a760 msgr2=0x7f3bf0161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.605+0000 7f3bf59ff700 1 --2- 172.21.15.156:0/116873423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bf015a760 0x7f3bf0161200 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009840 tx=0x7f3be0009b50 comp rx=0 tx=0).stop 2023-11-28T13:15:35.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.605+0000 7f3bf59ff700 1 -- 172.21.15.156:0/116873423 shutdown_connections 2023-11-28T13:15:35.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.605+0000 7f3bf59ff700 1 --2- 172.21.15.156:0/116873423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bf015a760 0x7f3bf0161200 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:35.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.605+0000 7f3bf59ff700 1 -- 172.21.15.156:0/116873423 >> 172.21.15.156:0/116873423 conn(0x7f3bf0072620 msgr2=0x7f3bf00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:35.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.605+0000 7f3bf59ff700 1 -- 172.21.15.156:0/116873423 shutdown_connections 2023-11-28T13:15:35.610 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.605+0000 7f3bf59ff700 1 -- 172.21.15.156:0/116873423 wait complete. 2023-11-28T13:15:35.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.607+0000 7f3bf59ff700 1 Processor -- start 2023-11-28T13:15:35.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.607+0000 7f3bf59ff700 1 -- start start 2023-11-28T13:15:35.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.608+0000 7f3bf59ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bf00cded0 0x7f3bf00cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:35.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.608+0000 7f3bf59ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bf015a760 0x7f3bf01586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:35.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.608+0000 7f3bf59ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bf015dc30 0x7f3bf01f2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:35.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.608+0000 7f3bf59ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf00b7610 con 0x7f3bf015dc30 2023-11-28T13:15:35.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.608+0000 7f3bf59ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf00b8f50 con 0x7f3bf015a760 2023-11-28T13:15:35.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.608+0000 7f3bf59ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf00b7050 con 0x7f3bf00cded0 2023-11-28T13:15:35.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.608+0000 7f3bef7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bf015dc30 0x7f3bf01f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:35.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.609+0000 7f3beeffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bf00cded0 0x7f3bf00cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:35.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.609+0000 7f3bee7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bf015a760 0x7f3bf01586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:35.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.609+0000 7f3beeffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bf00cded0 0x7f3bf00cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:39284/0 (socket says 172.21.15.156:39284) 2023-11-28T13:15:35.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.609+0000 7f3beeffd700 1 -- 172.21.15.156:0/4001786168 learned_addr learned my addr 172.21.15.156:0/4001786168 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:35.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.609+0000 7f3bef7fe700 1 -- 172.21.15.156:0/4001786168 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bf00cded0 msgr2=0x7f3bf00cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.609+0000 7f3bef7fe700 1 --2- 172.21.15.156:0/4001786168 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bf00cded0 0x7f3bf00cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:35.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.609+0000 7f3bef7fe700 1 -- 172.21.15.156:0/4001786168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bf015a760 msgr2=0x7f3bf01586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.609+0000 7f3bef7fe700 1 --2- 172.21.15.156:0/4001786168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bf015a760 0x7f3bf01586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:35.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.609+0000 7f3bef7fe700 1 -- 172.21.15.156:0/4001786168 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be00094a0 con 0x7f3bf015dc30 2023-11-28T13:15:35.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.609+0000 7f3beeffd700 1 --2- 172.21.15.156:0/4001786168 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3bf00cded0 0x7f3bf00cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:15:35.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.609+0000 7f3bef7fe700 1 --2- 172.21.15.156:0/4001786168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bf015dc30 0x7f3bf01f2780 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3bf006c170 tx=0x7f3be800ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:35.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.609+0000 7f3bd7fff700 1 -- 172.21.15.156:0/4001786168 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be8003c00 con 0x7f3bf015dc30 2023-11-28T13:15:35.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.609+0000 7f3bd7fff700 1 -- 172.21.15.156:0/4001786168 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be8003d60 con 0x7f3bf015dc30 2023-11-28T13:15:35.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.609+0000 7f3bd7fff700 1 -- 172.21.15.156:0/4001786168 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be800a9c0 con 0x7f3bf015dc30 2023-11-28T13:15:35.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.609+0000 7f3bf59ff700 1 -- 172.21.15.156:0/4001786168 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf0160720 con 0x7f3bf015dc30 2023-11-28T13:15:35.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.610+0000 7f3bf59ff700 1 -- 172.21.15.156:0/4001786168 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf00b5670 con 0x7f3bf015dc30 2023-11-28T13:15:35.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.610+0000 7f3bf59ff700 1 -- 172.21.15.156:0/4001786168 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3bf0160a00 con 0x7f3bf015dc30 2023-11-28T13:15:35.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.612+0000 7f3bd7fff700 1 -- 172.21.15.156:0/4001786168 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3be8010050 con 0x7f3bf015dc30 2023-11-28T13:15:35.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.612+0000 7f3bd7fff700 1 --2- 172.21.15.156:0/4001786168 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3bdc0682f0 0x7f3bdc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:35.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.612+0000 7f3bd7fff700 1 -- 172.21.15.156:0/4001786168 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3be8086280 con 0x7f3bf015dc30 2023-11-28T13:15:35.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.613+0000 7f3bd7fff700 1 --2- 172.21.15.156:0/4001786168 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x7f3bdc06e760 0x7f3bdc070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:35.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.613+0000 7f3bd7fff700 1 -- 172.21.15.156:0/4001786168 --> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3bdc071240 con 0x7f3bdc06e760 2023-11-28T13:15:35.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.613+0000 7f3bd7fff700 1 -- 172.21.15.156:0/4001786168 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3be80865f0 con 0x7f3bf015dc30 2023-11-28T13:15:35.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.613+0000 7f3beeffd700 1 --2- 172.21.15.156:0/4001786168 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3bdc0682f0 0x7f3bdc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:35.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.613+0000 7f3bee7fc700 1 --2- 172.21.15.156:0/4001786168 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x7f3bdc06e760 0x7f3bdc070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:35.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.613+0000 7f3beeffd700 1 --2- 172.21.15.156:0/4001786168 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3bdc0682f0 0x7f3bdc06a7a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3be0000c00 tx=0x7f3be0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:35.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.613+0000 7f3bee7fc700 1 --2- 172.21.15.156:0/4001786168 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x7f3bdc06e760 0x7f3bdc070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:35.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.614+0000 7f3bd7fff700 1 -- 172.21.15.156:0/4001786168 <== osd.1 v2:172.21.15.156:6800/132479 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3bdc071240 con 0x7f3bdc06e760 2023-11-28T13:15:35.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.629+0000 7f3bf59ff700 1 -- 172.21.15.156:0/4001786168 --> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3bf004e620 con 0x7f3bdc06e760 2023-11-28T13:15:35.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.629+0000 7f3bd7fff700 1 -- 172.21.15.156:0/4001786168 <== osd.1 v2:172.21.15.156:6800/132479 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3bf004e620 con 0x7f3bdc06e760 2023-11-28T13:15:35.631 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:15:35.631 INFO:teuthology.orchestra.run.smithi156.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:15:35.632 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:15:35.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.630+0000 7f3bf59ff700 1 -- 172.21.15.156:0/4001786168 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x7f3bdc06e760 msgr2=0x7f3bdc070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.630+0000 7f3bf59ff700 1 --2- 172.21.15.156:0/4001786168 >> [v2:172.21.15.156:6800/132479,v1:172.21.15.156:6801/132479] conn(0x7f3bdc06e760 0x7f3bdc070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:35.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.630+0000 7f3bf59ff700 1 -- 172.21.15.156:0/4001786168 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3bdc0682f0 msgr2=0x7f3bdc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.630+0000 7f3bf59ff700 1 --2- 172.21.15.156:0/4001786168 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f3bdc0682f0 0x7f3bdc06a7a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3be0000c00 tx=0x7f3be0008030 comp rx=0 tx=0).stop 2023-11-28T13:15:35.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.630+0000 7f3bf59ff700 1 -- 172.21.15.156:0/4001786168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bf015dc30 msgr2=0x7f3bf01f2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.630+0000 7f3bf59ff700 1 --2- 172.21.15.156:0/4001786168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bf015dc30 0x7f3bf01f2780 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3bf006c170 tx=0x7f3be800ce70 comp rx=0 tx=0).stop 2023-11-28T13:15:35.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.630+0000 7f3bef7fe700 1 -- 172.21.15.156:0/4001786168 reap_dead start 2023-11-28T13:15:35.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.630+0000 7f3bf59ff700 1 -- 172.21.15.156:0/4001786168 shutdown_connections 2023-11-28T13:15:35.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.630+0000 7f3bf59ff700 1 -- 172.21.15.156:0/4001786168 >> 172.21.15.156:0/4001786168 conn(0x7f3bf0072620 msgr2=0x7f3bf00b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:35.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.630+0000 7f3bf59ff700 1 -- 172.21.15.156:0/4001786168 shutdown_connections 2023-11-28T13:15:35.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.630+0000 7f3bf59ff700 1 -- 172.21.15.156:0/4001786168 wait complete. 2023-11-28T13:15:35.649 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-28T13:15:35.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.845+0000 7f647f5ff700 1 Processor -- start 2023-11-28T13:15:35.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.845+0000 7f647f5ff700 1 -- start start 2023-11-28T13:15:35.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.847+0000 7f647f5ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6478161340 0x7f64780d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:35.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.847+0000 7f647f5ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64780d1950 con 0x7f647815c9b0 2023-11-28T13:15:35.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.847+0000 7f647f5ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64780b5520 con 0x7f64780d2320 2023-11-28T13:15:35.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.847+0000 7f647f5ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64780b5800 con 0x7f6478161340 2023-11-28T13:15:35.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.847+0000 7f647d39b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f647815c9b0 0x7f64780b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:40832/0 (socket says 172.21.15.156:40832) 2023-11-28T13:15:35.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.847+0000 7f647d39b700 1 -- 172.21.15.156:0/2950091349 learned_addr learned my addr 172.21.15.156:0/2950091349 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:35.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.847+0000 7f647cb9a700 1 --2- 172.21.15.156:0/2950091349 >> v2:172.21.15.186:3300/0 conn(0x7f6478161340 0x7f64780d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:35.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.847+0000 7f646ffff700 1 -- 172.21.15.156:0/2950091349 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1778535730 0 0) 0x7f64780d1950 con 0x7f647815c9b0 2023-11-28T13:15:35.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.847+0000 7f646ffff700 1 -- 172.21.15.156:0/2950091349 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64640035b0 con 0x7f647815c9b0 2023-11-28T13:15:35.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.848+0000 7f647cb9a700 1 -- 172.21.15.156:0/2950091349 >> v1:172.21.15.186:6789/0 conn(0x7f64780d2320 legacy=0x7f64780cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.848+0000 7f646ffff700 1 -- 172.21.15.156:0/2950091349 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1033328329 0 0) 0x7f64780b5520 con 0x7f64780d2320 2023-11-28T13:15:35.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.848+0000 7f647cb9a700 1 -- 172.21.15.156:0/2950091349 >> v1:172.21.15.156:6789/0 conn(0x7f647815c9b0 legacy=0x7f64780b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.848+0000 7f647cb9a700 1 -- 172.21.15.156:0/2950091349 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64780d1300 con 0x7f6478161340 2023-11-28T13:15:35.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.848+0000 7f647cb9a700 1 --2- 172.21.15.156:0/2950091349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6478161340 0x7f64780d1de0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6468009840 tx=0x7f6468009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f20a7584512c9036 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:35.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.848+0000 7f646ffff700 1 -- 172.21.15.156:0/2950091349 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6468011830 con 0x7f6478161340 2023-11-28T13:15:35.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.848+0000 7f646ffff700 1 -- 172.21.15.156:0/2950091349 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6468011990 con 0x7f6478161340 2023-11-28T13:15:35.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.848+0000 7f646ffff700 1 -- 172.21.15.156:0/2950091349 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6468011d20 con 0x7f6478161340 2023-11-28T13:15:35.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.849+0000 7f647f5ff700 1 -- 172.21.15.156:0/2950091349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6478161340 msgr2=0x7f64780d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.853 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.849+0000 7f647f5ff700 1 --2- 172.21.15.156:0/2950091349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6478161340 0x7f64780d1de0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6468009840 tx=0x7f6468009b50 comp rx=0 tx=0).stop 2023-11-28T13:15:35.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.849+0000 7f647f5ff700 1 -- 172.21.15.156:0/2950091349 shutdown_connections 2023-11-28T13:15:35.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.849+0000 7f647f5ff700 1 --2- 172.21.15.156:0/2950091349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6478161340 0x7f64780d1de0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:35.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.849+0000 7f647f5ff700 1 -- 172.21.15.156:0/2950091349 >> 172.21.15.156:0/2950091349 conn(0x7f6478072600 msgr2=0x7f64780cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:35.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.849+0000 7f647f5ff700 1 -- 172.21.15.156:0/2950091349 shutdown_connections 2023-11-28T13:15:35.854 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.849+0000 7f647f5ff700 1 -- 172.21.15.156:0/2950091349 wait complete. 2023-11-28T13:15:35.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.851+0000 7f647f5ff700 1 Processor -- start 2023-11-28T13:15:35.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.851+0000 7f647f5ff700 1 -- start start 2023-11-28T13:15:35.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647f5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64780d2320 0x7f6478150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:35.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647f5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f647815c9b0 0x7f6478155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:35.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647f5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6478161340 0x7f6478156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:35.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647f5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64781575f0 con 0x7f64780d2320 2023-11-28T13:15:35.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647f5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6478156770 con 0x7f647815c9b0 2023-11-28T13:15:35.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647f5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64781569e0 con 0x7f6478161340 2023-11-28T13:15:35.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647d39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64780d2320 0x7f6478150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:35.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647d39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64780d2320 0x7f6478150550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57462/0 (socket says 172.21.15.156:57462) 2023-11-28T13:15:35.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647d39b700 1 -- 172.21.15.156:0/2001122729 learned_addr learned my addr 172.21.15.156:0/2001122729 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:35.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647cb9a700 1 --2- 172.21.15.156:0/2001122729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f647815c9b0 0x7f6478155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:35.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647db9c700 1 --2- 172.21.15.156:0/2001122729 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6478161340 0x7f6478156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:35.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647d39b700 1 -- 172.21.15.156:0/2001122729 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6478161340 msgr2=0x7f6478156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647d39b700 1 --2- 172.21.15.156:0/2001122729 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6478161340 0x7f6478156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:35.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647d39b700 1 -- 172.21.15.156:0/2001122729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f647815c9b0 msgr2=0x7f6478155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:35.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647d39b700 1 --2- 172.21.15.156:0/2001122729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f647815c9b0 0x7f6478155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:35.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647d39b700 1 -- 172.21.15.156:0/2001122729 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64680094a0 con 0x7f64780d2320 2023-11-28T13:15:35.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647db9c700 1 --2- 172.21.15.156:0/2001122729 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6478161340 0x7f6478156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:15:35.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.852+0000 7f647d39b700 1 --2- 172.21.15.156:0/2001122729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64780d2320 0x7f6478150550 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f647400c500 tx=0x7f647400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:35.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.853+0000 7f646dffb700 1 -- 172.21.15.156:0/2001122729 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6474005d70 con 0x7f64780d2320 2023-11-28T13:15:35.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.853+0000 7f646dffb700 1 -- 172.21.15.156:0/2001122729 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64740070d0 con 0x7f64780d2320 2023-11-28T13:15:35.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.853+0000 7f647f5ff700 1 -- 172.21.15.156:0/2001122729 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647815cd80 con 0x7f64780d2320 2023-11-28T13:15:35.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.853+0000 7f646dffb700 1 -- 172.21.15.156:0/2001122729 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6474007440 con 0x7f64780d2320 2023-11-28T13:15:35.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.853+0000 7f647f5ff700 1 -- 172.21.15.156:0/2001122729 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64780d15e0 con 0x7f64780d2320 2023-11-28T13:15:35.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.854+0000 7f647f5ff700 1 -- 172.21.15.156:0/2001122729 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f647815cf10 con 0x7f64780d2320 2023-11-28T13:15:35.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.855+0000 7f646dffb700 1 -- 172.21.15.156:0/2001122729 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6474019030 con 0x7f64780d2320 2023-11-28T13:15:35.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.856+0000 7f646dffb700 1 --2- 172.21.15.156:0/2001122729 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f645c0682f0 0x7f645c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:35.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.856+0000 7f646dffb700 1 -- 172.21.15.156:0/2001122729 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6474084cf0 con 0x7f64780d2320 2023-11-28T13:15:35.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.856+0000 7f647cb9a700 1 --2- 172.21.15.156:0/2001122729 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f645c0682f0 0x7f645c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:35.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.857+0000 7f647cb9a700 1 --2- 172.21.15.156:0/2001122729 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f645c0682f0 0x7f645c06a7a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6468009fd0 tx=0x7f646801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:35.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:35.860+0000 7f646dffb700 1 -- 172.21.15.156:0/2001122729 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f647404b5e0 con 0x7f64780d2320 2023-11-28T13:15:36.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.051+0000 7f647f5ff700 1 -- 172.21.15.156:0/2001122729 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6478154cd0 con 0x7f645c0682f0 2023-11-28T13:15:36.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.051+0000 7f646dffb700 1 -- 172.21.15.156:0/2001122729 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6478154cd0 con 0x7f645c0682f0 2023-11-28T13:15:36.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.054+0000 7f647f5ff700 1 -- 172.21.15.156:0/2001122729 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f645c0682f0 msgr2=0x7f645c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.054+0000 7f647f5ff700 1 --2- 172.21.15.156:0/2001122729 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f645c0682f0 0x7f645c06a7a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6468009fd0 tx=0x7f646801c040 comp rx=0 tx=0).stop 2023-11-28T13:15:36.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.054+0000 7f647f5ff700 1 -- 172.21.15.156:0/2001122729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64780d2320 msgr2=0x7f6478150550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.054+0000 7f647f5ff700 1 --2- 172.21.15.156:0/2001122729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64780d2320 0x7f6478150550 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f647400c500 tx=0x7f647400c810 comp rx=0 tx=0).stop 2023-11-28T13:15:36.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.054+0000 7f647f5ff700 1 -- 172.21.15.156:0/2001122729 shutdown_connections 2023-11-28T13:15:36.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.054+0000 7f647f5ff700 1 --2- 172.21.15.156:0/2001122729 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f645c0682f0 0x7f645c06a7a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.054+0000 7f647f5ff700 1 --2- 172.21.15.156:0/2001122729 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6478161340 0x7f6478156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.054+0000 7f647f5ff700 1 --2- 172.21.15.156:0/2001122729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64780d2320 0x7f6478150550 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.054+0000 7f647f5ff700 1 --2- 172.21.15.156:0/2001122729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f647815c9b0 0x7f6478155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.054+0000 7f647f5ff700 1 -- 172.21.15.156:0/2001122729 >> 172.21.15.156:0/2001122729 conn(0x7f6478072600 msgr2=0x7f647815fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:36.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.054+0000 7f647f5ff700 1 -- 172.21.15.156:0/2001122729 shutdown_connections 2023-11-28T13:15:36.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.054+0000 7f647f5ff700 1 -- 172.21.15.156:0/2001122729 wait complete. 2023-11-28T13:15:36.059 INFO:teuthology.orchestra.run.smithi156.stderr:instructed osd(s) 1 to deep-scrub 2023-11-28T13:15:36.074 INFO:tasks.ceph:Scrubbing osd.2 2023-11-28T13:15:36.074 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:15:36.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.273+0000 7f5bffbff700 1 Processor -- start 2023-11-28T13:15:36.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.273+0000 7f5bffbff700 1 -- start start 2023-11-28T13:15:36.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.274+0000 7f5bffbff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5bf815a760 0x7f5bf8161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.274+0000 7f5bffbff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5bf815dc30 0x7f5bf80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.274+0000 7f5bffbff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf815e0a0 con 0x7f5bf80cd7c0 2023-11-28T13:15:36.276 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.274+0000 7f5bffbff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf815ab30 con 0x7f5bf815dc30 2023-11-28T13:15:36.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.274+0000 7f5bffbff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf8160d70 con 0x7f5bf815a760 2023-11-28T13:15:36.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.275+0000 7f5bfd19a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5bf815dc30 0x7f5bf80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.275+0000 7f5bfe19c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f5bf80cd7c0 0x7f5bf80d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:40840/0 (socket says 172.21.15.156:40840) 2023-11-28T13:15:36.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.275+0000 7f5bfe19c700 1 -- 172.21.15.156:0/2680873301 learned_addr learned my addr 172.21.15.156:0/2680873301 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:36.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.275+0000 7f5bfd99b700 1 --2- 172.21.15.156:0/2680873301 >> v2:172.21.15.186:3300/0 conn(0x7f5bf815a760 0x7f5bf8161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.275+0000 7f5bfd19a700 1 -- 172.21.15.156:0/2680873301 >> v2:172.21.15.186:3300/0 conn(0x7f5bf815a760 msgr2=0x7f5bf8161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:36.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.275+0000 7f5bfd19a700 1 --2- 172.21.15.156:0/2680873301 >> v2:172.21.15.186:3300/0 conn(0x7f5bf815a760 0x7f5bf8161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.275+0000 7f5bfd19a700 1 -- 172.21.15.156:0/2680873301 >> v1:172.21.15.156:6789/0 conn(0x7f5bf80cd7c0 legacy=0x7f5bf80d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.275+0000 7f5bfd19a700 1 -- 172.21.15.156:0/2680873301 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf80b5130 con 0x7f5bf815dc30 2023-11-28T13:15:36.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.275+0000 7f5bfc999700 1 -- 172.21.15.156:0/2680873301 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823633364 0 0) 0x7f5bf815e0a0 con 0x7f5bf80cd7c0 2023-11-28T13:15:36.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.275+0000 7f5bfd19a700 1 --2- 172.21.15.156:0/2680873301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf815dc30 0x7f5bf80b4bf0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5bf808bff0 tx=0x7f5be8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8a72f5aeda2f8c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:36.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.276+0000 7f5bfc999700 1 -- 172.21.15.156:0/2680873301 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be8011830 con 0x7f5bf815dc30 2023-11-28T13:15:36.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.276+0000 7f5bfc999700 1 -- 172.21.15.156:0/2680873301 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be8011990 con 0x7f5bf815dc30 2023-11-28T13:15:36.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.276+0000 7f5bfc999700 1 -- 172.21.15.156:0/2680873301 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be8011d20 con 0x7f5bf815dc30 2023-11-28T13:15:36.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.276+0000 7f5bffbff700 1 -- 172.21.15.156:0/2680873301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf815dc30 msgr2=0x7f5bf80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.276+0000 7f5bffbff700 1 --2- 172.21.15.156:0/2680873301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf815dc30 0x7f5bf80b4bf0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5bf808bff0 tx=0x7f5be8009b20 comp rx=0 tx=0).stop 2023-11-28T13:15:36.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.276+0000 7f5bffbff700 1 -- 172.21.15.156:0/2680873301 shutdown_connections 2023-11-28T13:15:36.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.276+0000 7f5bffbff700 1 --2- 172.21.15.156:0/2680873301 >> v2:172.21.15.186:3300/0 conn(0x7f5bf815a760 0x7f5bf8161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.276+0000 7f5bffbff700 1 --2- 172.21.15.156:0/2680873301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf815dc30 0x7f5bf80b4bf0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.276+0000 7f5bffbff700 1 -- 172.21.15.156:0/2680873301 >> 172.21.15.156:0/2680873301 conn(0x7f5bf8072620 msgr2=0x7f5bf80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:36.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.276+0000 7f5bffbff700 1 -- 172.21.15.156:0/2680873301 shutdown_connections 2023-11-28T13:15:36.281 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.276+0000 7f5bffbff700 1 -- 172.21.15.156:0/2680873301 wait complete. 2023-11-28T13:15:36.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.278+0000 7f5bffbff700 1 Processor -- start 2023-11-28T13:15:36.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.278+0000 7f5bffbff700 1 -- start start 2023-11-28T13:15:36.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.279+0000 7f5bffbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bf80cd7c0 0x7f5bf80c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.279+0000 7f5bffbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf815a760 0x7f5bf81f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.279+0000 7f5bffbff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5bf815dc30 0x7f5bf80b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.279+0000 7f5bffbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf80c9050 con 0x7f5bf815a760 2023-11-28T13:15:36.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.279+0000 7f5bffbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf80c71c0 con 0x7f5bf80cd7c0 2023-11-28T13:15:36.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.279+0000 7f5bffbff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf80c7f70 con 0x7f5bf815dc30 2023-11-28T13:15:36.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.280+0000 7f5bfd19a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf815a760 0x7f5bf81f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.280+0000 7f5bfd19a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf815a760 0x7f5bf81f1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57490/0 (socket says 172.21.15.156:57490) 2023-11-28T13:15:36.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.280+0000 7f5bfd19a700 1 -- 172.21.15.156:0/1836005434 learned_addr learned my addr 172.21.15.156:0/1836005434 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:36.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.280+0000 7f5bfd99b700 1 --2- 172.21.15.156:0/1836005434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bf80cd7c0 0x7f5bf80c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.280+0000 7f5bfe19c700 1 --2- 172.21.15.156:0/1836005434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5bf815dc30 0x7f5bf80b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.280+0000 7f5bfd19a700 1 -- 172.21.15.156:0/1836005434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5bf815dc30 msgr2=0x7f5bf80b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.280+0000 7f5bfd19a700 1 --2- 172.21.15.156:0/1836005434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5bf815dc30 0x7f5bf80b3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.280+0000 7f5bfd19a700 1 -- 172.21.15.156:0/1836005434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bf80cd7c0 msgr2=0x7f5bf80c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.280+0000 7f5bfd19a700 1 --2- 172.21.15.156:0/1836005434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bf80cd7c0 0x7f5bf80c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.280+0000 7f5bfd19a700 1 -- 172.21.15.156:0/1836005434 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be80094a0 con 0x7f5bf815a760 2023-11-28T13:15:36.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.280+0000 7f5bfd19a700 1 --2- 172.21.15.156:0/1836005434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf815a760 0x7f5bf81f1cf0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5be80090b0 tx=0x7f5be8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:36.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.280+0000 7f5bee7fc700 1 -- 172.21.15.156:0/1836005434 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be80036a0 con 0x7f5bf815a760 2023-11-28T13:15:36.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.281+0000 7f5bee7fc700 1 -- 172.21.15.156:0/1836005434 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be801e070 con 0x7f5bf815a760 2023-11-28T13:15:36.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.281+0000 7f5bee7fc700 1 -- 172.21.15.156:0/1836005434 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be8003a40 con 0x7f5bf815a760 2023-11-28T13:15:36.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.281+0000 7f5bffbff700 1 -- 172.21.15.156:0/1836005434 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf80b5670 con 0x7f5bf815a760 2023-11-28T13:15:36.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.281+0000 7f5bffbff700 1 -- 172.21.15.156:0/1836005434 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf80b5390 con 0x7f5bf815a760 2023-11-28T13:15:36.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.282+0000 7f5bffbff700 1 -- 172.21.15.156:0/1836005434 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5bf8160720 con 0x7f5bf815a760 2023-11-28T13:15:36.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.283+0000 7f5bee7fc700 1 -- 172.21.15.156:0/1836005434 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5be800f050 con 0x7f5bf815a760 2023-11-28T13:15:36.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.284+0000 7f5bee7fc700 1 --2- 172.21.15.156:0/1836005434 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5bdc0682f0 0x7f5bdc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.284+0000 7f5bee7fc700 1 -- 172.21.15.156:0/1836005434 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5be8086d90 con 0x7f5bf815a760 2023-11-28T13:15:36.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.284+0000 7f5bee7fc700 1 --2- 172.21.15.156:0/1836005434 >> [v2:172.21.15.156:6814/132475,v1:172.21.15.156:6816/132475] conn(0x7f5bdc06e760 0x7f5bdc070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.284+0000 7f5bee7fc700 1 -- 172.21.15.156:0/1836005434 --> [v2:172.21.15.156:6814/132475,v1:172.21.15.156:6816/132475] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5bdc071240 con 0x7f5bdc06e760 2023-11-28T13:15:36.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.284+0000 7f5bee7fc700 1 -- 172.21.15.156:0/1836005434 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5be80871d0 con 0x7f5bf815a760 2023-11-28T13:15:36.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.284+0000 7f5bfd99b700 1 --2- 172.21.15.156:0/1836005434 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5bdc0682f0 0x7f5bdc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.284+0000 7f5bfe19c700 1 --2- 172.21.15.156:0/1836005434 >> [v2:172.21.15.156:6814/132475,v1:172.21.15.156:6816/132475] conn(0x7f5bdc06e760 0x7f5bdc070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.285+0000 7f5bfe19c700 1 --2- 172.21.15.156:0/1836005434 >> [v2:172.21.15.156:6814/132475,v1:172.21.15.156:6816/132475] conn(0x7f5bdc06e760 0x7f5bdc070b70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:36.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.285+0000 7f5bfd99b700 1 --2- 172.21.15.156:0/1836005434 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5bdc0682f0 0x7f5bdc06a7a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009be0 tx=0x7f5bf4009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:36.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.285+0000 7f5bee7fc700 1 -- 172.21.15.156:0/1836005434 <== osd.2 v2:172.21.15.156:6814/132475 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5bdc071240 con 0x7f5bdc06e760 2023-11-28T13:15:36.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.300+0000 7f5bffbff700 1 -- 172.21.15.156:0/1836005434 --> [v2:172.21.15.156:6814/132475,v1:172.21.15.156:6816/132475] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5bf804e620 con 0x7f5bdc06e760 2023-11-28T13:15:36.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.301+0000 7f5bee7fc700 1 -- 172.21.15.156:0/1836005434 <== osd.2 v2:172.21.15.156:6814/132475 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5bf804e620 con 0x7f5bdc06e760 2023-11-28T13:15:36.304 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:15:36.305 INFO:teuthology.orchestra.run.smithi156.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:15:36.305 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:15:36.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.301+0000 7f5bffbff700 1 -- 172.21.15.156:0/1836005434 >> [v2:172.21.15.156:6814/132475,v1:172.21.15.156:6816/132475] conn(0x7f5bdc06e760 msgr2=0x7f5bdc070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.301+0000 7f5bffbff700 1 --2- 172.21.15.156:0/1836005434 >> [v2:172.21.15.156:6814/132475,v1:172.21.15.156:6816/132475] conn(0x7f5bdc06e760 0x7f5bdc070b70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.301+0000 7f5bffbff700 1 -- 172.21.15.156:0/1836005434 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5bdc0682f0 msgr2=0x7f5bdc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.301+0000 7f5bffbff700 1 --2- 172.21.15.156:0/1836005434 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f5bdc0682f0 0x7f5bdc06a7a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009be0 tx=0x7f5bf4009210 comp rx=0 tx=0).stop 2023-11-28T13:15:36.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.301+0000 7f5bffbff700 1 -- 172.21.15.156:0/1836005434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf815a760 msgr2=0x7f5bf81f1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.301+0000 7f5bffbff700 1 --2- 172.21.15.156:0/1836005434 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf815a760 0x7f5bf81f1cf0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5be80090b0 tx=0x7f5be8004750 comp rx=0 tx=0).stop 2023-11-28T13:15:36.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.301+0000 7f5bfe19c700 1 -- 172.21.15.156:0/1836005434 reap_dead start 2023-11-28T13:15:36.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.301+0000 7f5bffbff700 1 -- 172.21.15.156:0/1836005434 shutdown_connections 2023-11-28T13:15:36.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.301+0000 7f5bffbff700 1 -- 172.21.15.156:0/1836005434 >> 172.21.15.156:0/1836005434 conn(0x7f5bf8072620 msgr2=0x7f5bf815f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:36.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.301+0000 7f5bffbff700 1 -- 172.21.15.156:0/1836005434 shutdown_connections 2023-11-28T13:15:36.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.302+0000 7f5bffbff700 1 -- 172.21.15.156:0/1836005434 wait complete. 2023-11-28T13:15:36.320 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-28T13:15:36.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.518+0000 7f9dd99ff700 1 Processor -- start 2023-11-28T13:15:36.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.518+0000 7f9dd99ff700 1 -- start start 2023-11-28T13:15:36.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.519+0000 7f9dd99ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9dd4161340 0x7f9dd40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.520+0000 7f9dd99ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd40d1950 con 0x7f9dd40d2320 2023-11-28T13:15:36.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.520+0000 7f9dd99ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd40b5520 con 0x7f9dd415c9b0 2023-11-28T13:15:36.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.520+0000 7f9dd99ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dd40b5800 con 0x7f9dd4161340 2023-11-28T13:15:36.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.520+0000 7f9dcbfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9dd4161340 0x7f9dd40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.520+0000 7f9dd37fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f9dd40d2320 0x7f9dd40cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:40854/0 (socket says 172.21.15.156:40854) 2023-11-28T13:15:36.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.520+0000 7f9dd37fe700 1 -- 172.21.15.156:0/963343568 learned_addr learned my addr 172.21.15.156:0/963343568 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:36.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.520+0000 7f9dcbfff700 1 -- 172.21.15.156:0/963343568 >> v1:172.21.15.186:6789/0 conn(0x7f9dd415c9b0 legacy=0x7f9dd40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.520+0000 7f9dcbfff700 1 -- 172.21.15.156:0/963343568 >> v1:172.21.15.156:6789/0 conn(0x7f9dd40d2320 legacy=0x7f9dd40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.524 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.520+0000 7f9dcbfff700 1 -- 172.21.15.156:0/963343568 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd40d1300 con 0x7f9dd4161340 2023-11-28T13:15:36.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.520+0000 7f9dcbfff700 1 --2- 172.21.15.156:0/963343568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dd4161340 0x7f9dd40d1de0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009840 tx=0x7f9dbc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f3c299a10e88d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:36.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.521+0000 7f9dd27fc700 1 -- 172.21.15.156:0/963343568 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc011830 con 0x7f9dd4161340 2023-11-28T13:15:36.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.521+0000 7f9dd27fc700 1 -- 172.21.15.156:0/963343568 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc011990 con 0x7f9dd4161340 2023-11-28T13:15:36.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.521+0000 7f9dd27fc700 1 -- 172.21.15.156:0/963343568 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc011d20 con 0x7f9dd4161340 2023-11-28T13:15:36.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.521+0000 7f9dd99ff700 1 -- 172.21.15.156:0/963343568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dd4161340 msgr2=0x7f9dd40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.521+0000 7f9dd99ff700 1 --2- 172.21.15.156:0/963343568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dd4161340 0x7f9dd40d1de0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009840 tx=0x7f9dbc009b50 comp rx=0 tx=0).stop 2023-11-28T13:15:36.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.521+0000 7f9dd99ff700 1 -- 172.21.15.156:0/963343568 shutdown_connections 2023-11-28T13:15:36.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.521+0000 7f9dd99ff700 1 --2- 172.21.15.156:0/963343568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dd4161340 0x7f9dd40d1de0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.521+0000 7f9dd99ff700 1 -- 172.21.15.156:0/963343568 >> 172.21.15.156:0/963343568 conn(0x7f9dd4072600 msgr2=0x7f9dd40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:36.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.521+0000 7f9dd99ff700 1 -- 172.21.15.156:0/963343568 shutdown_connections 2023-11-28T13:15:36.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.521+0000 7f9dd99ff700 1 -- 172.21.15.156:0/963343568 wait complete. 2023-11-28T13:15:36.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.523+0000 7f9dd99ff700 1 Processor -- start 2023-11-28T13:15:36.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.523+0000 7f9dd99ff700 1 -- start start 2023-11-28T13:15:36.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.524+0000 7f9dd99ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9dd40d2320 0x7f9dd4156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.529 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.524+0000 7f9dd99ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9dd415c9b0 0x7f9dd41e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.524+0000 7f9dd99ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dd4161340 0x7f9dd4151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.524+0000 7f9dd99ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd4152310 con 0x7f9dd4161340 2023-11-28T13:15:36.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.524+0000 7f9dd99ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd41e78c0 con 0x7f9dd40d2320 2023-11-28T13:15:36.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.524+0000 7f9dd99ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd41e7ba0 con 0x7f9dd415c9b0 2023-11-28T13:15:36.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.524+0000 7f9dd37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dd4161340 0x7f9dd4151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.524+0000 7f9dd2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9dd40d2320 0x7f9dd4156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.524+0000 7f9dd2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9dd40d2320 0x7f9dd4156860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.156:59792/0 (socket says 172.21.15.156:59792) 2023-11-28T13:15:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.524+0000 7f9dcbfff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9dd415c9b0 0x7f9dd41e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.524+0000 7f9dd37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dd4161340 0x7f9dd4151260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57516/0 (socket says 172.21.15.156:57516) 2023-11-28T13:15:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.524+0000 7f9dd2ffd700 1 -- 172.21.15.156:0/1131937952 learned_addr learned my addr 172.21.15.156:0/1131937952 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.525+0000 7f9dd2ffd700 1 -- 172.21.15.156:0/1131937952 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9dd415c9b0 msgr2=0x7f9dd41e6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.525+0000 7f9dd2ffd700 1 --2- 172.21.15.156:0/1131937952 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9dd415c9b0 0x7f9dd41e6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.525+0000 7f9dd2ffd700 1 -- 172.21.15.156:0/1131937952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dd4161340 msgr2=0x7f9dd4151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.525+0000 7f9dd2ffd700 1 --2- 172.21.15.156:0/1131937952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dd4161340 0x7f9dd4151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.525+0000 7f9dd2ffd700 1 -- 172.21.15.156:0/1131937952 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc0094a0 con 0x7f9dd40d2320 2023-11-28T13:15:36.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.525+0000 7f9dd37fe700 1 --2- 172.21.15.156:0/1131937952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dd4161340 0x7f9dd4151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:15:36.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.525+0000 7f9dd2ffd700 1 --2- 172.21.15.156:0/1131937952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9dd40d2320 0x7f9dd4156860 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400c4e0 tx=0x7f9dc400c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:36.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.525+0000 7f9dd0ff9700 1 -- 172.21.15.156:0/1131937952 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40052d0 con 0x7f9dd40d2320 2023-11-28T13:15:36.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.525+0000 7f9dd0ff9700 1 -- 172.21.15.156:0/1131937952 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc4005430 con 0x7f9dd40d2320 2023-11-28T13:15:36.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.525+0000 7f9dd0ff9700 1 -- 172.21.15.156:0/1131937952 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40038c0 con 0x7f9dd40d2320 2023-11-28T13:15:36.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.525+0000 7f9dd99ff700 1 -- 172.21.15.156:0/1131937952 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd415cd80 con 0x7f9dd40d2320 2023-11-28T13:15:36.532 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.526+0000 7f9dd99ff700 1 -- 172.21.15.156:0/1131937952 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd40d15e0 con 0x7f9dd40d2320 2023-11-28T13:15:36.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.527+0000 7f9dd99ff700 1 -- 172.21.15.156:0/1131937952 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd415cf10 con 0x7f9dd40d2320 2023-11-28T13:15:36.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.527+0000 7f9dd0ff9700 1 -- 172.21.15.156:0/1131937952 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9dc4019030 con 0x7f9dd40d2320 2023-11-28T13:15:36.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.528+0000 7f9dd0ff9700 1 --2- 172.21.15.156:0/1131937952 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9db40682f0 0x7f9db406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.528+0000 7f9dd0ff9700 1 -- 172.21.15.156:0/1131937952 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9dc4084c80 con 0x7f9dd40d2320 2023-11-28T13:15:36.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.529+0000 7f9dcbfff700 1 --2- 172.21.15.156:0/1131937952 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9db40682f0 0x7f9db406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.529+0000 7f9dcbfff700 1 --2- 172.21.15.156:0/1131937952 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9db40682f0 0x7f9db406a7a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0090b0 tx=0x7f9dbc01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:36.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.538+0000 7f9dd0ff9700 1 -- 172.21.15.156:0/1131937952 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9dc408b080 con 0x7f9dd40d2320 2023-11-28T13:15:36.734 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.733+0000 7f9dd99ff700 1 -- 172.21.15.156:0/1131937952 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9dd40745c0 con 0x7f9db40682f0 2023-11-28T13:15:36.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.733+0000 7f9dd0ff9700 1 -- 172.21.15.156:0/1131937952 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9dd40745c0 con 0x7f9db40682f0 2023-11-28T13:15:36.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.736+0000 7f9dd99ff700 1 -- 172.21.15.156:0/1131937952 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9db40682f0 msgr2=0x7f9db406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.736+0000 7f9dd99ff700 1 --2- 172.21.15.156:0/1131937952 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9db40682f0 0x7f9db406a7a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0090b0 tx=0x7f9dbc01c040 comp rx=0 tx=0).stop 2023-11-28T13:15:36.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.736+0000 7f9dd99ff700 1 -- 172.21.15.156:0/1131937952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9dd40d2320 msgr2=0x7f9dd4156860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.736+0000 7f9dd99ff700 1 --2- 172.21.15.156:0/1131937952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9dd40d2320 0x7f9dd4156860 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400c4e0 tx=0x7f9dc400c7f0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.736+0000 7f9dd99ff700 1 -- 172.21.15.156:0/1131937952 shutdown_connections 2023-11-28T13:15:36.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.736+0000 7f9dd99ff700 1 --2- 172.21.15.156:0/1131937952 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f9db40682f0 0x7f9db406a7a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.736+0000 7f9dd99ff700 1 --2- 172.21.15.156:0/1131937952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dd4161340 0x7f9dd4151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.736+0000 7f9dd99ff700 1 --2- 172.21.15.156:0/1131937952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9dd40d2320 0x7f9dd4156860 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.736+0000 7f9dd99ff700 1 --2- 172.21.15.156:0/1131937952 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9dd415c9b0 0x7f9dd41e6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.736+0000 7f9dd99ff700 1 -- 172.21.15.156:0/1131937952 >> 172.21.15.156:0/1131937952 conn(0x7f9dd4072600 msgr2=0x7f9dd415ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:36.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.736+0000 7f9dd99ff700 1 -- 172.21.15.156:0/1131937952 shutdown_connections 2023-11-28T13:15:36.739 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.736+0000 7f9dd99ff700 1 -- 172.21.15.156:0/1131937952 wait complete. 2023-11-28T13:15:36.740 INFO:teuthology.orchestra.run.smithi156.stderr:instructed osd(s) 2 to deep-scrub 2023-11-28T13:15:36.756 INFO:tasks.ceph:Scrubbing osd.3 2023-11-28T13:15:36.756 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:15:36.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.955+0000 7fbd3c3ff700 1 Processor -- start 2023-11-28T13:15:36.957 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.955+0000 7fbd3c3ff700 1 -- start start 2023-11-28T13:15:36.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.957+0000 7fbd3c3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd340cded0 0x7fbd34161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.957+0000 7fbd3c3ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbd3415dc30 0x7fbd340b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.957+0000 7fbd3c3ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3415e0a0 con 0x7fbd3415a760 2023-11-28T13:15:36.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.957+0000 7fbd3c3ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd3415ab30 con 0x7fbd3415dc30 2023-11-28T13:15:36.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.957+0000 7fbd3c3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd34160d70 con 0x7fbd340cded0 2023-11-28T13:15:36.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.957+0000 7fbd3a19b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fbd3415a760 0x7fbd340cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:40860/0 (socket says 172.21.15.156:40860) 2023-11-28T13:15:36.960 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.957+0000 7fbd3a19b700 1 -- 172.21.15.156:0/1705865136 learned_addr learned my addr 172.21.15.156:0/1705865136 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:36.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.957+0000 7fbd3999a700 1 --2- 172.21.15.156:0/1705865136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd340cded0 0x7fbd34161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.957+0000 7fbd3a99c700 1 --2- 172.21.15.156:0/1705865136 >> v2:172.21.15.186:3300/0 conn(0x7fbd3415dc30 0x7fbd340b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.957+0000 7fbd39199700 1 -- 172.21.15.156:0/1705865136 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066664697 0 0) 0x7fbd3415e0a0 con 0x7fbd3415a760 2023-11-28T13:15:36.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.957+0000 7fbd39199700 1 -- 172.21.15.156:0/1705865136 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd200035b0 con 0x7fbd3415a760 2023-11-28T13:15:36.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.958+0000 7fbd3999a700 1 -- 172.21.15.156:0/1705865136 >> v2:172.21.15.186:3300/0 conn(0x7fbd3415dc30 msgr2=0x7fbd340b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:36.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.958+0000 7fbd3999a700 1 --2- 172.21.15.156:0/1705865136 >> v2:172.21.15.186:3300/0 conn(0x7fbd3415dc30 0x7fbd340b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.961 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.958+0000 7fbd3999a700 1 -- 172.21.15.156:0/1705865136 >> v1:172.21.15.156:6789/0 conn(0x7fbd3415a760 legacy=0x7fbd340cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.958+0000 7fbd3999a700 1 -- 172.21.15.156:0/1705865136 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd340b5130 con 0x7fbd340cded0 2023-11-28T13:15:36.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.958+0000 7fbd3999a700 1 --2- 172.21.15.156:0/1705865136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd340cded0 0x7fbd34161200 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009840 tx=0x7fbd24009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6888805b11255ec6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:36.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.958+0000 7fbd39199700 1 -- 172.21.15.156:0/1705865136 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd24011830 con 0x7fbd340cded0 2023-11-28T13:15:36.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.958+0000 7fbd39199700 1 -- 172.21.15.156:0/1705865136 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd24011990 con 0x7fbd340cded0 2023-11-28T13:15:36.962 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.958+0000 7fbd39199700 1 -- 172.21.15.156:0/1705865136 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd24011d20 con 0x7fbd340cded0 2023-11-28T13:15:36.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.959+0000 7fbd3c3ff700 1 -- 172.21.15.156:0/1705865136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd340cded0 msgr2=0x7fbd34161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.959+0000 7fbd3c3ff700 1 --2- 172.21.15.156:0/1705865136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd340cded0 0x7fbd34161200 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009840 tx=0x7fbd24009b50 comp rx=0 tx=0).stop 2023-11-28T13:15:36.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.959+0000 7fbd3c3ff700 1 -- 172.21.15.156:0/1705865136 shutdown_connections 2023-11-28T13:15:36.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.959+0000 7fbd3c3ff700 1 --2- 172.21.15.156:0/1705865136 >> v2:172.21.15.186:3300/0 conn(0x7fbd3415dc30 0x7fbd340b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.959+0000 7fbd3c3ff700 1 --2- 172.21.15.156:0/1705865136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd340cded0 0x7fbd34161200 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.959+0000 7fbd3c3ff700 1 -- 172.21.15.156:0/1705865136 >> 172.21.15.156:0/1705865136 conn(0x7fbd34072620 msgr2=0x7fbd340cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:36.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.959+0000 7fbd3c3ff700 1 -- 172.21.15.156:0/1705865136 shutdown_connections 2023-11-28T13:15:36.963 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.959+0000 7fbd3c3ff700 1 -- 172.21.15.156:0/1705865136 wait complete. 2023-11-28T13:15:36.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.961+0000 7fbd3c3ff700 1 Processor -- start 2023-11-28T13:15:36.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.961+0000 7fbd3c3ff700 1 -- start start 2023-11-28T13:15:36.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.962+0000 7fbd3c3ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd340cded0 0x7fbd340b8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.962+0000 7fbd3c3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd3415a760 0x7fbd340b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.966 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.962+0000 7fbd3c3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd3415dc30 0x7fbd340b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.962+0000 7fbd3c3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd340b6790 con 0x7fbd3415dc30 2023-11-28T13:15:36.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.962+0000 7fbd3c3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd340b8e40 con 0x7fbd3415a760 2023-11-28T13:15:36.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.962+0000 7fbd3c3ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd340cb0b0 con 0x7fbd340cded0 2023-11-28T13:15:36.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.962+0000 7fbd3a99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd3415dc30 0x7fbd340b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.962+0000 7fbd3a19b700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd340cded0 0x7fbd340b8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.962+0000 7fbd3a99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd3415dc30 0x7fbd340b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57526/0 (socket says 172.21.15.156:57526) 2023-11-28T13:15:36.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.962+0000 7fbd3a99c700 1 -- 172.21.15.156:0/1950143390 learned_addr learned my addr 172.21.15.156:0/1950143390 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:36.967 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.962+0000 7fbd3999a700 1 --2- 172.21.15.156:0/1950143390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd3415a760 0x7fbd340b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.962+0000 7fbd3a99c700 1 -- 172.21.15.156:0/1950143390 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd340cded0 msgr2=0x7fbd340b8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.962+0000 7fbd3a99c700 1 --2- 172.21.15.156:0/1950143390 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd340cded0 0x7fbd340b8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.962+0000 7fbd3a99c700 1 -- 172.21.15.156:0/1950143390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd3415a760 msgr2=0x7fbd340b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.962+0000 7fbd3a99c700 1 --2- 172.21.15.156:0/1950143390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd3415a760 0x7fbd340b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.962+0000 7fbd3a99c700 1 -- 172.21.15.156:0/1950143390 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd240094a0 con 0x7fbd3415dc30 2023-11-28T13:15:36.968 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.963+0000 7fbd3a99c700 1 --2- 172.21.15.156:0/1950143390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd3415dc30 0x7fbd340b3ef0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00efc0 tx=0x7fbd2c00c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:36.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.963+0000 7fbd2affd700 1 -- 172.21.15.156:0/1950143390 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c00cad0 con 0x7fbd3415dc30 2023-11-28T13:15:36.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.963+0000 7fbd2affd700 1 -- 172.21.15.156:0/1950143390 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2c00f040 con 0x7fbd3415dc30 2023-11-28T13:15:36.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.963+0000 7fbd2affd700 1 -- 172.21.15.156:0/1950143390 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c00ce70 con 0x7fbd3415dc30 2023-11-28T13:15:36.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.963+0000 7fbd3c3ff700 1 -- 172.21.15.156:0/1950143390 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd340b5670 con 0x7fbd3415dc30 2023-11-28T13:15:36.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.963+0000 7fbd3c3ff700 1 -- 172.21.15.156:0/1950143390 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd340b5390 con 0x7fbd3415dc30 2023-11-28T13:15:36.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.964+0000 7fbd3c3ff700 1 -- 172.21.15.156:0/1950143390 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbd34160720 con 0x7fbd3415dc30 2023-11-28T13:15:36.970 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.965+0000 7fbd2affd700 1 -- 172.21.15.156:0/1950143390 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbd2c014480 con 0x7fbd3415dc30 2023-11-28T13:15:36.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.966+0000 7fbd2affd700 1 --2- 172.21.15.156:0/1950143390 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbd180682f0 0x7fbd1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.966+0000 7fbd2affd700 1 -- 172.21.15.156:0/1950143390 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbd2c0862f0 con 0x7fbd3415dc30 2023-11-28T13:15:36.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.966+0000 7fbd2affd700 1 --2- 172.21.15.156:0/1950143390 >> [v2:172.21.15.156:6821/132480,v1:172.21.15.156:6823/132480] conn(0x7fbd1806e760 0x7fbd18070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:36.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.966+0000 7fbd2affd700 1 -- 172.21.15.156:0/1950143390 --> [v2:172.21.15.156:6821/132480,v1:172.21.15.156:6823/132480] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbd18071240 con 0x7fbd1806e760 2023-11-28T13:15:36.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.966+0000 7fbd2affd700 1 -- 172.21.15.156:0/1950143390 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd2c0888d0 con 0x7fbd3415dc30 2023-11-28T13:15:36.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.966+0000 7fbd3a19b700 1 --2- 172.21.15.156:0/1950143390 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbd180682f0 0x7fbd1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.971 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.966+0000 7fbd3999a700 1 --2- 172.21.15.156:0/1950143390 >> [v2:172.21.15.156:6821/132480,v1:172.21.15.156:6823/132480] conn(0x7fbd1806e760 0x7fbd18070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:36.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.967+0000 7fbd3a19b700 1 --2- 172.21.15.156:0/1950143390 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbd180682f0 0x7fbd1806a7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbd30007fd0 tx=0x7fbd30007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:36.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.967+0000 7fbd3999a700 1 --2- 172.21.15.156:0/1950143390 >> [v2:172.21.15.156:6821/132480,v1:172.21.15.156:6823/132480] conn(0x7fbd1806e760 0x7fbd18070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:36.972 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.968+0000 7fbd2affd700 1 -- 172.21.15.156:0/1950143390 <== osd.3 v2:172.21.15.156:6821/132480 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbd18071240 con 0x7fbd1806e760 2023-11-28T13:15:36.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.983+0000 7fbd3c3ff700 1 -- 172.21.15.156:0/1950143390 --> [v2:172.21.15.156:6821/132480,v1:172.21.15.156:6823/132480] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbd340b68d0 con 0x7fbd1806e760 2023-11-28T13:15:36.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.983+0000 7fbd2affd700 1 -- 172.21.15.156:0/1950143390 <== osd.3 v2:172.21.15.156:6821/132480 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fbd340b68d0 con 0x7fbd1806e760 2023-11-28T13:15:36.985 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:15:36.985 INFO:teuthology.orchestra.run.smithi156.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:15:36.985 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:15:36.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.983+0000 7fbd3c3ff700 1 -- 172.21.15.156:0/1950143390 >> [v2:172.21.15.156:6821/132480,v1:172.21.15.156:6823/132480] conn(0x7fbd1806e760 msgr2=0x7fbd18070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.983+0000 7fbd3c3ff700 1 --2- 172.21.15.156:0/1950143390 >> [v2:172.21.15.156:6821/132480,v1:172.21.15.156:6823/132480] conn(0x7fbd1806e760 0x7fbd18070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.983+0000 7fbd3c3ff700 1 -- 172.21.15.156:0/1950143390 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbd180682f0 msgr2=0x7fbd1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.983+0000 7fbd3c3ff700 1 --2- 172.21.15.156:0/1950143390 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbd180682f0 0x7fbd1806a7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbd30007fd0 tx=0x7fbd30007dc0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.983+0000 7fbd3c3ff700 1 -- 172.21.15.156:0/1950143390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd3415dc30 msgr2=0x7fbd340b3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:36.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.983+0000 7fbd3c3ff700 1 --2- 172.21.15.156:0/1950143390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd3415dc30 0x7fbd340b3ef0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00efc0 tx=0x7fbd2c00c0c0 comp rx=0 tx=0).stop 2023-11-28T13:15:36.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.983+0000 7fbd3a99c700 1 -- 172.21.15.156:0/1950143390 reap_dead start 2023-11-28T13:15:36.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.984+0000 7fbd3c3ff700 1 -- 172.21.15.156:0/1950143390 shutdown_connections 2023-11-28T13:15:36.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.984+0000 7fbd3c3ff700 1 -- 172.21.15.156:0/1950143390 >> 172.21.15.156:0/1950143390 conn(0x7fbd34072620 msgr2=0x7fbd340d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:36.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.984+0000 7fbd3c3ff700 1 -- 172.21.15.156:0/1950143390 shutdown_connections 2023-11-28T13:15:36.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:36.984+0000 7fbd3c3ff700 1 -- 172.21.15.156:0/1950143390 wait complete. 2023-11-28T13:15:37.002 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-28T13:15:37.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.198+0000 7fe3009ff700 1 Processor -- start 2023-11-28T13:15:37.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.199+0000 7fe3009ff700 1 -- start start 2023-11-28T13:15:37.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.200+0000 7fe3009ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe2fc15c9b0 0x7fe2fc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.200+0000 7fe3009ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2fc161340 0x7fe2fc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.200+0000 7fe3009ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2fc0d1950 con 0x7fe2fc0d2320 2023-11-28T13:15:37.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.200+0000 7fe3009ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2fc0b5800 con 0x7fe2fc15c9b0 2023-11-28T13:15:37.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.200+0000 7fe3009ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc0b5520 con 0x7fe2fc161340 2023-11-28T13:15:37.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.200+0000 7fe2fa59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe2fc15c9b0 0x7fe2fc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.200+0000 7fe2fa59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe2fc15c9b0 0x7fe2fc0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57542/0 (socket says 172.21.15.156:57542) 2023-11-28T13:15:37.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.200+0000 7fe2fa59c700 1 -- 172.21.15.156:0/2661640980 learned_addr learned my addr 172.21.15.156:0/2661640980 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:37.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.200+0000 7fe2f9d9b700 1 --2- 172.21.15.156:0/2661640980 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2fc161340 0x7fe2fc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.201+0000 7fe2fa59c700 1 -- 172.21.15.156:0/2661640980 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2fc161340 msgr2=0x7fe2fc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:37.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.201+0000 7fe2fa59c700 1 --2- 172.21.15.156:0/2661640980 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2fc161340 0x7fe2fc0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.201+0000 7fe2fa59c700 1 -- 172.21.15.156:0/2661640980 >> v1:172.21.15.186:6789/0 conn(0x7fe2fc0d2320 legacy=0x7fe2fc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:37.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.201+0000 7fe2fa59c700 1 -- 172.21.15.156:0/2661640980 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2fc0d1300 con 0x7fe2fc15c9b0 2023-11-28T13:15:37.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.201+0000 7fe2fa59c700 1 --2- 172.21.15.156:0/2661640980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2fc15c9b0 0x7fe2fc0b4bf0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0098f0 tx=0x7fe2ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0004780e4af248c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:37.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.201+0000 7fe2f959a700 1 -- 172.21.15.156:0/2661640980 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec011830 con 0x7fe2fc15c9b0 2023-11-28T13:15:37.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.201+0000 7fe2f959a700 1 -- 172.21.15.156:0/2661640980 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec011990 con 0x7fe2fc15c9b0 2023-11-28T13:15:37.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.201+0000 7fe2f959a700 1 -- 172.21.15.156:0/2661640980 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec011d20 con 0x7fe2fc15c9b0 2023-11-28T13:15:37.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.202+0000 7fe3009ff700 1 -- 172.21.15.156:0/2661640980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2fc15c9b0 msgr2=0x7fe2fc0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:37.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.202+0000 7fe3009ff700 1 --2- 172.21.15.156:0/2661640980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2fc15c9b0 0x7fe2fc0b4bf0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0098f0 tx=0x7fe2ec009c00 comp rx=0 tx=0).stop 2023-11-28T13:15:37.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.202+0000 7fe3009ff700 1 -- 172.21.15.156:0/2661640980 shutdown_connections 2023-11-28T13:15:37.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.202+0000 7fe3009ff700 1 --2- 172.21.15.156:0/2661640980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2fc15c9b0 0x7fe2fc0b4bf0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.202+0000 7fe3009ff700 1 --2- 172.21.15.156:0/2661640980 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2fc161340 0x7fe2fc0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.202+0000 7fe3009ff700 1 -- 172.21.15.156:0/2661640980 >> 172.21.15.156:0/2661640980 conn(0x7fe2fc072600 msgr2=0x7fe2fc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:37.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.202+0000 7fe3009ff700 1 -- 172.21.15.156:0/2661640980 shutdown_connections 2023-11-28T13:15:37.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.202+0000 7fe3009ff700 1 -- 172.21.15.156:0/2661640980 wait complete. 2023-11-28T13:15:37.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.203+0000 7fe3009ff700 1 Processor -- start 2023-11-28T13:15:37.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.204+0000 7fe3009ff700 1 -- start start 2023-11-28T13:15:37.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.204+0000 7fe3009ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2fc0d2320 0x7fe2fc14da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.205+0000 7fe3009ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2fc15c9b0 0x7fe2fc15f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.205+0000 7fe3009ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2fc161340 0x7fe2fc150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.205+0000 7fe3009ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc14dfd0 con 0x7fe2fc0d2320 2023-11-28T13:15:37.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.205+0000 7fe3009ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc1f5fa0 con 0x7fe2fc161340 2023-11-28T13:15:37.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.205+0000 7fe3009ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc1f6280 con 0x7fe2fc15c9b0 2023-11-28T13:15:37.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.205+0000 7fe2fa59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2fc0d2320 0x7fe2fc14da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.205+0000 7fe2fa59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2fc0d2320 0x7fe2fc14da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57550/0 (socket says 172.21.15.156:57550) 2023-11-28T13:15:37.209 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.205+0000 7fe2fa59c700 1 -- 172.21.15.156:0/2790262097 learned_addr learned my addr 172.21.15.156:0/2790262097 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:37.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.205+0000 7fe2f9d9b700 1 --2- 172.21.15.156:0/2790262097 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2fc15c9b0 0x7fe2fc15f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.205+0000 7fe2fad9d700 1 --2- 172.21.15.156:0/2790262097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2fc161340 0x7fe2fc150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.205+0000 7fe2fa59c700 1 -- 172.21.15.156:0/2790262097 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2fc15c9b0 msgr2=0x7fe2fc15f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:37.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.205+0000 7fe2fa59c700 1 --2- 172.21.15.156:0/2790262097 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2fc15c9b0 0x7fe2fc15f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.205+0000 7fe2fa59c700 1 -- 172.21.15.156:0/2790262097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2fc161340 msgr2=0x7fe2fc150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:37.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.205+0000 7fe2fa59c700 1 --2- 172.21.15.156:0/2790262097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2fc161340 0x7fe2fc150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.205+0000 7fe2fa59c700 1 -- 172.21.15.156:0/2790262097 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec009550 con 0x7fe2fc0d2320 2023-11-28T13:15:37.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.205+0000 7fe2fad9d700 1 --2- 172.21.15.156:0/2790262097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2fc161340 0x7fe2fc150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:15:37.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.205+0000 7fe2fa59c700 1 --2- 172.21.15.156:0/2790262097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2fc0d2320 0x7fe2fc14da90 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec000c00 tx=0x7fe2ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:37.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.206+0000 7fe2eaffd700 1 -- 172.21.15.156:0/2790262097 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec004480 con 0x7fe2fc0d2320 2023-11-28T13:15:37.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.206+0000 7fe2eaffd700 1 -- 172.21.15.156:0/2790262097 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec01f070 con 0x7fe2fc0d2320 2023-11-28T13:15:37.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.206+0000 7fe2eaffd700 1 -- 172.21.15.156:0/2790262097 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec0038c0 con 0x7fe2fc0d2320 2023-11-28T13:15:37.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.206+0000 7fe3009ff700 1 -- 172.21.15.156:0/2790262097 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2fc0b5130 con 0x7fe2fc0d2320 2023-11-28T13:15:37.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.206+0000 7fe3009ff700 1 -- 172.21.15.156:0/2790262097 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2fc0d15e0 con 0x7fe2fc0d2320 2023-11-28T13:15:37.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.207+0000 7fe3009ff700 1 -- 172.21.15.156:0/2790262097 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2fc0b52c0 con 0x7fe2fc0d2320 2023-11-28T13:15:37.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.208+0000 7fe2eaffd700 1 -- 172.21.15.156:0/2790262097 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe2ec02a020 con 0x7fe2fc0d2320 2023-11-28T13:15:37.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.209+0000 7fe2eaffd700 1 --2- 172.21.15.156:0/2790262097 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe2dc0682f0 0x7fe2dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.209+0000 7fe2eaffd700 1 -- 172.21.15.156:0/2790262097 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe2ec086f90 con 0x7fe2fc0d2320 2023-11-28T13:15:37.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.209+0000 7fe2f9d9b700 1 --2- 172.21.15.156:0/2790262097 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe2dc0682f0 0x7fe2dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.209+0000 7fe2f9d9b700 1 --2- 172.21.15.156:0/2790262097 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe2dc0682f0 0x7fe2dc06a7a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc14ece0 tx=0x7fe2f0009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:37.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.212+0000 7fe2eaffd700 1 -- 172.21.15.156:0/2790262097 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe2ec0b85e0 con 0x7fe2fc0d2320 2023-11-28T13:15:37.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.403+0000 7fe3009ff700 1 -- 172.21.15.156:0/2790262097 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2fc1e76b0 con 0x7fe2dc0682f0 2023-11-28T13:15:37.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.403+0000 7fe2eaffd700 1 -- 172.21.15.156:0/2790262097 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe2fc1e76b0 con 0x7fe2dc0682f0 2023-11-28T13:15:37.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.406+0000 7fe3009ff700 1 -- 172.21.15.156:0/2790262097 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe2dc0682f0 msgr2=0x7fe2dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:37.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.406+0000 7fe3009ff700 1 --2- 172.21.15.156:0/2790262097 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe2dc0682f0 0x7fe2dc06a7a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc14ece0 tx=0x7fe2f0009210 comp rx=0 tx=0).stop 2023-11-28T13:15:37.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.406+0000 7fe3009ff700 1 -- 172.21.15.156:0/2790262097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2fc0d2320 msgr2=0x7fe2fc14da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:37.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.406+0000 7fe3009ff700 1 --2- 172.21.15.156:0/2790262097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2fc0d2320 0x7fe2fc14da90 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec000c00 tx=0x7fe2ec004750 comp rx=0 tx=0).stop 2023-11-28T13:15:37.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.406+0000 7fe3009ff700 1 -- 172.21.15.156:0/2790262097 shutdown_connections 2023-11-28T13:15:37.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.406+0000 7fe3009ff700 1 --2- 172.21.15.156:0/2790262097 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe2dc0682f0 0x7fe2dc06a7a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.406+0000 7fe3009ff700 1 --2- 172.21.15.156:0/2790262097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2fc161340 0x7fe2fc150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.406+0000 7fe3009ff700 1 --2- 172.21.15.156:0/2790262097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2fc0d2320 0x7fe2fc14da90 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.406+0000 7fe3009ff700 1 --2- 172.21.15.156:0/2790262097 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2fc15c9b0 0x7fe2fc15f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.406+0000 7fe3009ff700 1 -- 172.21.15.156:0/2790262097 >> 172.21.15.156:0/2790262097 conn(0x7fe2fc072600 msgr2=0x7fe2fc0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:37.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.406+0000 7fe3009ff700 1 -- 172.21.15.156:0/2790262097 shutdown_connections 2023-11-28T13:15:37.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.406+0000 7fe3009ff700 1 -- 172.21.15.156:0/2790262097 wait complete. 2023-11-28T13:15:37.410 INFO:teuthology.orchestra.run.smithi156.stderr:instructed osd(s) 3 to deep-scrub 2023-11-28T13:15:37.426 INFO:tasks.ceph:Scrubbing osd.4 2023-11-28T13:15:37.427 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:15:37.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.627+0000 7f63af9ff700 1 Processor -- start 2023-11-28T13:15:37.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.627+0000 7f63af9ff700 1 -- start start 2023-11-28T13:15:37.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.628+0000 7f63af9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a815a760 0x7f63a8161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.628+0000 7f63af9ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f63a80cded0 0x7f63a80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.628+0000 7f63af9ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a815e0a0 con 0x7f63a815dc30 2023-11-28T13:15:37.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.628+0000 7f63af9ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a815ab30 con 0x7f63a80cded0 2023-11-28T13:15:37.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.628+0000 7f63af9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a8160d70 con 0x7f63a815a760 2023-11-28T13:15:37.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.629+0000 7f63acf9a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f63a815dc30 0x7f63a80cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:40874/0 (socket says 172.21.15.156:40874) 2023-11-28T13:15:37.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.629+0000 7f63acf9a700 1 -- 172.21.15.156:0/2650953361 learned_addr learned my addr 172.21.15.156:0/2650953361 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:37.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.629+0000 7f63adf9c700 1 --2- 172.21.15.156:0/2650953361 >> v2:172.21.15.156:3300/0 conn(0x7f63a80cded0 0x7f63a80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.629+0000 7f63ad79b700 1 --2- 172.21.15.156:0/2650953361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a815a760 0x7f63a8161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.629+0000 7f63adf9c700 1 -- 172.21.15.156:0/2650953361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a815a760 msgr2=0x7f63a8161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:37.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.629+0000 7f63adf9c700 1 --2- 172.21.15.156:0/2650953361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a815a760 0x7f63a8161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.629+0000 7f63adf9c700 1 -- 172.21.15.156:0/2650953361 >> v1:172.21.15.156:6789/0 conn(0x7f63a815dc30 legacy=0x7f63a80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:37.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.629+0000 7f63adf9c700 1 -- 172.21.15.156:0/2650953361 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a80b5130 con 0x7f63a80cded0 2023-11-28T13:15:37.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.629+0000 7f63adf9c700 1 --2- 172.21.15.156:0/2650953361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63a80cded0 0x7f63a80b4bf0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009d90 tx=0x7f63a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171f030b8d861d31 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:37.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.629+0000 7f639ffff700 1 -- 172.21.15.156:0/2650953361 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a000e070 con 0x7f63a80cded0 2023-11-28T13:15:37.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.630+0000 7f639ffff700 1 -- 172.21.15.156:0/2650953361 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a000bb20 con 0x7f63a80cded0 2023-11-28T13:15:37.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.630+0000 7f639ffff700 1 -- 172.21.15.156:0/2650953361 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0012430 con 0x7f63a80cded0 2023-11-28T13:15:37.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.630+0000 7f63af9ff700 1 -- 172.21.15.156:0/2650953361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63a80cded0 msgr2=0x7f63a80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:37.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.630+0000 7f63af9ff700 1 --2- 172.21.15.156:0/2650953361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63a80cded0 0x7f63a80b4bf0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009d90 tx=0x7f63a000b330 comp rx=0 tx=0).stop 2023-11-28T13:15:37.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.630+0000 7f63af9ff700 1 -- 172.21.15.156:0/2650953361 shutdown_connections 2023-11-28T13:15:37.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.630+0000 7f63af9ff700 1 --2- 172.21.15.156:0/2650953361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63a80cded0 0x7f63a80b4bf0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.630+0000 7f63af9ff700 1 --2- 172.21.15.156:0/2650953361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a815a760 0x7f63a8161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.630+0000 7f63af9ff700 1 -- 172.21.15.156:0/2650953361 >> 172.21.15.156:0/2650953361 conn(0x7f63a8072620 msgr2=0x7f63a80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:37.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.630+0000 7f63af9ff700 1 -- 172.21.15.156:0/2650953361 shutdown_connections 2023-11-28T13:15:37.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.630+0000 7f63af9ff700 1 -- 172.21.15.156:0/2650953361 wait complete. 2023-11-28T13:15:37.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.632+0000 7f63af9ff700 1 Processor -- start 2023-11-28T13:15:37.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.632+0000 7f63af9ff700 1 -- start start 2023-11-28T13:15:37.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.633+0000 7f63af9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63a80cded0 0x7f63a80b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.633+0000 7f63af9ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a815a760 0x7f63a81533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.633+0000 7f63af9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63a815dc30 0x7f63a815b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.633+0000 7f63af9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a815c700 con 0x7f63a815dc30 2023-11-28T13:15:37.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.633+0000 7f63af9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a80c6e40 con 0x7f63a80cded0 2023-11-28T13:15:37.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.633+0000 7f63af9ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a815bbc0 con 0x7f63a815a760 2023-11-28T13:15:37.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.633+0000 7f63adf9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63a815dc30 0x7f63a815b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.633+0000 7f63ad79b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63a80cded0 0x7f63a80b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.633+0000 7f63adf9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63a815dc30 0x7f63a815b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57566/0 (socket says 172.21.15.156:57566) 2023-11-28T13:15:37.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.633+0000 7f63adf9c700 1 -- 172.21.15.156:0/323738936 learned_addr learned my addr 172.21.15.156:0/323738936 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:37.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.633+0000 7f63acf9a700 1 --2- 172.21.15.156:0/323738936 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a815a760 0x7f63a81533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.634+0000 7f63adf9c700 1 -- 172.21.15.156:0/323738936 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a815a760 msgr2=0x7f63a81533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:37.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.634+0000 7f63adf9c700 1 --2- 172.21.15.156:0/323738936 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63a815a760 0x7f63a81533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.634+0000 7f63adf9c700 1 -- 172.21.15.156:0/323738936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63a80cded0 msgr2=0x7f63a80b8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:37.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.634+0000 7f63adf9c700 1 --2- 172.21.15.156:0/323738936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63a80cded0 0x7f63a80b8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.634+0000 7f63adf9c700 1 -- 172.21.15.156:0/323738936 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a0009a70 con 0x7f63a815dc30 2023-11-28T13:15:37.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.634+0000 7f63ad79b700 1 --2- 172.21.15.156:0/323738936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63a80cded0 0x7f63a80b8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:15:37.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.634+0000 7f63adf9c700 1 --2- 172.21.15.156:0/323738936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63a815dc30 0x7f63a815b680 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f63a0014040 tx=0x7f63a0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:37.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.634+0000 7f639dffb700 1 -- 172.21.15.156:0/323738936 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0012430 con 0x7f63a815dc30 2023-11-28T13:15:37.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.634+0000 7f639dffb700 1 -- 172.21.15.156:0/323738936 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a0020070 con 0x7f63a815dc30 2023-11-28T13:15:37.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.634+0000 7f639dffb700 1 -- 172.21.15.156:0/323738936 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0003ea0 con 0x7f63a815dc30 2023-11-28T13:15:37.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.634+0000 7f63af9ff700 1 -- 172.21.15.156:0/323738936 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a80b5670 con 0x7f63a815dc30 2023-11-28T13:15:37.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.634+0000 7f63af9ff700 1 -- 172.21.15.156:0/323738936 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a80b5390 con 0x7f63a815dc30 2023-11-28T13:15:37.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.635+0000 7f63af9ff700 1 -- 172.21.15.156:0/323738936 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f63a8160720 con 0x7f63a815dc30 2023-11-28T13:15:37.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.636+0000 7f639dffb700 1 -- 172.21.15.156:0/323738936 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f63a002e070 con 0x7f63a815dc30 2023-11-28T13:15:37.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.637+0000 7f639dffb700 1 --2- 172.21.15.156:0/323738936 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f638c0682f0 0x7f638c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.637+0000 7f639dffb700 1 -- 172.21.15.156:0/323738936 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f63a0022070 con 0x7f63a815dc30 2023-11-28T13:15:37.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.637+0000 7f639dffb700 1 --2- 172.21.15.156:0/323738936 >> [v2:172.21.15.186:6816/132937,v1:172.21.15.186:6817/132937] conn(0x7f638c06e760 0x7f638c070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.638+0000 7f639dffb700 1 -- 172.21.15.156:0/323738936 --> [v2:172.21.15.186:6816/132937,v1:172.21.15.186:6817/132937] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f638c071240 con 0x7f638c06e760 2023-11-28T13:15:37.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.638+0000 7f639dffb700 1 -- 172.21.15.156:0/323738936 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63a0089ad0 con 0x7f63a815dc30 2023-11-28T13:15:37.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.638+0000 7f63ad79b700 1 --2- 172.21.15.156:0/323738936 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f638c0682f0 0x7f638c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.638+0000 7f63acf9a700 1 --2- 172.21.15.156:0/323738936 >> [v2:172.21.15.186:6816/132937,v1:172.21.15.186:6817/132937] conn(0x7f638c06e760 0x7f638c070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.638+0000 7f63ad79b700 1 --2- 172.21.15.156:0/323738936 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f638c0682f0 0x7f638c06a7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009fd0 tx=0x7f63a4009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:37.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.639+0000 7f63acf9a700 1 --2- 172.21.15.156:0/323738936 >> [v2:172.21.15.186:6816/132937,v1:172.21.15.186:6817/132937] conn(0x7f638c06e760 0x7f638c070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:37.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.639+0000 7f639dffb700 1 -- 172.21.15.156:0/323738936 <== osd.4 v2:172.21.15.186:6816/132937 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f638c071240 con 0x7f638c06e760 2023-11-28T13:15:37.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.655+0000 7f63af9ff700 1 -- 172.21.15.156:0/323738936 --> [v2:172.21.15.186:6816/132937,v1:172.21.15.186:6817/132937] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f63a80c6fb0 con 0x7f638c06e760 2023-11-28T13:15:37.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.655+0000 7f639dffb700 1 -- 172.21.15.156:0/323738936 <== osd.4 v2:172.21.15.186:6816/132937 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f63a80c6fb0 con 0x7f638c06e760 2023-11-28T13:15:37.657 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:15:37.657 INFO:teuthology.orchestra.run.smithi156.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:15:37.657 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:15:37.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.656+0000 7f63af9ff700 1 -- 172.21.15.156:0/323738936 >> [v2:172.21.15.186:6816/132937,v1:172.21.15.186:6817/132937] conn(0x7f638c06e760 msgr2=0x7f638c070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:37.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.656+0000 7f63af9ff700 1 --2- 172.21.15.156:0/323738936 >> [v2:172.21.15.186:6816/132937,v1:172.21.15.186:6817/132937] conn(0x7f638c06e760 0x7f638c070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.656+0000 7f63af9ff700 1 -- 172.21.15.156:0/323738936 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f638c0682f0 msgr2=0x7f638c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:37.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.656+0000 7f63af9ff700 1 --2- 172.21.15.156:0/323738936 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f638c0682f0 0x7f638c06a7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009fd0 tx=0x7f63a4009210 comp rx=0 tx=0).stop 2023-11-28T13:15:37.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.656+0000 7f63af9ff700 1 -- 172.21.15.156:0/323738936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63a815dc30 msgr2=0x7f63a815b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:37.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.656+0000 7f63af9ff700 1 --2- 172.21.15.156:0/323738936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63a815dc30 0x7f63a815b680 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f63a0014040 tx=0x7f63a0004ad0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.656+0000 7f63adf9c700 1 -- 172.21.15.156:0/323738936 reap_dead start 2023-11-28T13:15:37.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.656+0000 7f63af9ff700 1 -- 172.21.15.156:0/323738936 shutdown_connections 2023-11-28T13:15:37.659 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.656+0000 7f63af9ff700 1 -- 172.21.15.156:0/323738936 >> 172.21.15.156:0/323738936 conn(0x7f63a8072620 msgr2=0x7f63a815f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:37.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.656+0000 7f63af9ff700 1 -- 172.21.15.156:0/323738936 shutdown_connections 2023-11-28T13:15:37.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.656+0000 7f63af9ff700 1 -- 172.21.15.156:0/323738936 wait complete. 2023-11-28T13:15:37.675 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-28T13:15:37.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.872+0000 7fbebe5ff700 1 Processor -- start 2023-11-28T13:15:37.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.873+0000 7fbebe5ff700 1 -- start start 2023-11-28T13:15:37.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.874+0000 7fbebe5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbeb815c9b0 0x7fbeb80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.874+0000 7fbebe5ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbeb8161340 0x7fbeb80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.874+0000 7fbebe5ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb80d1950 con 0x7fbeb80d2320 2023-11-28T13:15:37.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.874+0000 7fbebe5ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb80b5800 con 0x7fbeb8161340 2023-11-28T13:15:37.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.874+0000 7fbebe5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb80b5520 con 0x7fbeb815c9b0 2023-11-28T13:15:37.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.874+0000 7fbeb77fe700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbeb8161340 0x7fbeb80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.877 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.874+0000 7fbeb7fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbeb815c9b0 0x7fbeb80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.874+0000 7fbebcb9c700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fbeb80d2320 0x7fbeb80cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:56544/0 (socket says 172.21.15.156:56544) 2023-11-28T13:15:37.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.874+0000 7fbebcb9c700 1 -- 172.21.15.156:0/3092129608 learned_addr learned my addr 172.21.15.156:0/3092129608 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:37.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.875+0000 7fbeb77fe700 1 -- 172.21.15.156:0/3092129608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbeb815c9b0 msgr2=0x7fbeb80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:37.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.875+0000 7fbeb77fe700 1 --2- 172.21.15.156:0/3092129608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbeb815c9b0 0x7fbeb80b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.875+0000 7fbeb77fe700 1 -- 172.21.15.156:0/3092129608 >> v1:172.21.15.186:6789/0 conn(0x7fbeb80d2320 legacy=0x7fbeb80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:37.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.875+0000 7fbeb77fe700 1 -- 172.21.15.156:0/3092129608 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb80d1300 con 0x7fbeb8161340 2023-11-28T13:15:37.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.875+0000 7fbeb6ffd700 1 -- 172.21.15.156:0/3092129608 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132068639 0 0) 0x7fbeb80d1950 con 0x7fbeb80d2320 2023-11-28T13:15:37.878 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.875+0000 7fbeb77fe700 1 --2- 172.21.15.156:0/3092129608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeb8161340 0x7fbeb80d1de0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009840 tx=0x7fbeac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bed86d37ea98eb6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:37.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.875+0000 7fbeb6ffd700 1 -- 172.21.15.156:0/3092129608 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac011830 con 0x7fbeb8161340 2023-11-28T13:15:37.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.875+0000 7fbeb6ffd700 1 -- 172.21.15.156:0/3092129608 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeac011990 con 0x7fbeb8161340 2023-11-28T13:15:37.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.875+0000 7fbeb6ffd700 1 -- 172.21.15.156:0/3092129608 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac011d20 con 0x7fbeb8161340 2023-11-28T13:15:37.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.876+0000 7fbebe5ff700 1 -- 172.21.15.156:0/3092129608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeb8161340 msgr2=0x7fbeb80d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:37.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.876+0000 7fbebe5ff700 1 --2- 172.21.15.156:0/3092129608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeb8161340 0x7fbeb80d1de0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009840 tx=0x7fbeac009b50 comp rx=0 tx=0).stop 2023-11-28T13:15:37.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.876+0000 7fbebe5ff700 1 -- 172.21.15.156:0/3092129608 shutdown_connections 2023-11-28T13:15:37.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.876+0000 7fbebe5ff700 1 --2- 172.21.15.156:0/3092129608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbeb815c9b0 0x7fbeb80b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.879 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.876+0000 7fbebe5ff700 1 --2- 172.21.15.156:0/3092129608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeb8161340 0x7fbeb80d1de0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.876+0000 7fbebe5ff700 1 -- 172.21.15.156:0/3092129608 >> 172.21.15.156:0/3092129608 conn(0x7fbeb8072600 msgr2=0x7fbeb80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:37.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.876+0000 7fbebe5ff700 1 -- 172.21.15.156:0/3092129608 shutdown_connections 2023-11-28T13:15:37.880 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.876+0000 7fbebe5ff700 1 -- 172.21.15.156:0/3092129608 wait complete. 2023-11-28T13:15:37.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.877+0000 7fbebe5ff700 1 Processor -- start 2023-11-28T13:15:37.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.878+0000 7fbebe5ff700 1 -- start start 2023-11-28T13:15:37.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.878+0000 7fbebe5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbeb80d2320 0x7fbeb815f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.879+0000 7fbebe5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeb815c9b0 0x7fbeb8156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.879+0000 7fbebe5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbeb8161340 0x7fbeb8156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.879+0000 7fbebe5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb81570a0 con 0x7fbeb8161340 2023-11-28T13:15:37.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.879+0000 7fbebe5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb81e6460 con 0x7fbeb815c9b0 2023-11-28T13:15:37.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.879+0000 7fbebe5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb81e6740 con 0x7fbeb80d2320 2023-11-28T13:15:37.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.879+0000 7fbebcb9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbeb8161340 0x7fbeb8156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.879+0000 7fbebcb9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbeb8161340 0x7fbeb8156b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57570/0 (socket says 172.21.15.156:57570) 2023-11-28T13:15:37.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.879+0000 7fbebcb9c700 1 -- 172.21.15.156:0/355598756 learned_addr learned my addr 172.21.15.156:0/355598756 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:37.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.879+0000 7fbeb7fff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbeb80d2320 0x7fbeb815f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.883 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.879+0000 7fbeb77fe700 1 --2- 172.21.15.156:0/355598756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeb815c9b0 0x7fbeb8156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.879+0000 7fbeb7fff700 1 -- 172.21.15.156:0/355598756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeb815c9b0 msgr2=0x7fbeb8156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:37.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.879+0000 7fbeb7fff700 1 --2- 172.21.15.156:0/355598756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeb815c9b0 0x7fbeb8156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.879+0000 7fbeb7fff700 1 -- 172.21.15.156:0/355598756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbeb8161340 msgr2=0x7fbeb8156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:37.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.879+0000 7fbeb7fff700 1 --2- 172.21.15.156:0/355598756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbeb8161340 0x7fbeb8156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:37.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.879+0000 7fbeb7fff700 1 -- 172.21.15.156:0/355598756 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac0094a0 con 0x7fbeb80d2320 2023-11-28T13:15:37.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.879+0000 7fbebcb9c700 1 --2- 172.21.15.156:0/355598756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbeb8161340 0x7fbeb8156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:15:37.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.880+0000 7fbeb7fff700 1 --2- 172.21.15.156:0/355598756 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbeb80d2320 0x7fbeb815f1f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbea800e900 tx=0x7fbea800ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:37.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.880+0000 7fbeb4ff9700 1 -- 172.21.15.156:0/355598756 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea8013680 con 0x7fbeb80d2320 2023-11-28T13:15:37.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.880+0000 7fbeb4ff9700 1 -- 172.21.15.156:0/355598756 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea80137e0 con 0x7fbeb80d2320 2023-11-28T13:15:37.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.880+0000 7fbebe5ff700 1 -- 172.21.15.156:0/355598756 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb80b5130 con 0x7fbeb80d2320 2023-11-28T13:15:37.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.880+0000 7fbeb4ff9700 1 -- 172.21.15.156:0/355598756 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea8013b50 con 0x7fbeb80d2320 2023-11-28T13:15:37.885 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.880+0000 7fbebe5ff700 1 -- 172.21.15.156:0/355598756 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb80d15e0 con 0x7fbeb80d2320 2023-11-28T13:15:37.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.881+0000 7fbebe5ff700 1 -- 172.21.15.156:0/355598756 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb80b52c0 con 0x7fbeb80d2320 2023-11-28T13:15:37.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.882+0000 7fbeb4ff9700 1 -- 172.21.15.156:0/355598756 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbea8019030 con 0x7fbeb80d2320 2023-11-28T13:15:37.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.883+0000 7fbeb4ff9700 1 --2- 172.21.15.156:0/355598756 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbea40682f0 0x7fbea406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:37.886 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.883+0000 7fbeb4ff9700 1 -- 172.21.15.156:0/355598756 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbea8085bb0 con 0x7fbeb80d2320 2023-11-28T13:15:37.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.883+0000 7fbeb77fe700 1 --2- 172.21.15.156:0/355598756 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbea40682f0 0x7fbea406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:37.887 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.884+0000 7fbeb77fe700 1 --2- 172.21.15.156:0/355598756 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbea40682f0 0x7fbea406a7a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0090b0 tx=0x7fbeac008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:37.889 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:37.887+0000 7fbeb4ff9700 1 -- 172.21.15.156:0/355598756 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbea8052640 con 0x7fbeb80d2320 2023-11-28T13:15:38.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.079+0000 7fbebe5ff700 1 -- 172.21.15.156:0/355598756 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbeb80745c0 con 0x7fbea40682f0 2023-11-28T13:15:38.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.080+0000 7fbeb4ff9700 1 -- 172.21.15.156:0/355598756 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbeb80745c0 con 0x7fbea40682f0 2023-11-28T13:15:38.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.082+0000 7fbebe5ff700 1 -- 172.21.15.156:0/355598756 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbea40682f0 msgr2=0x7fbea406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.082+0000 7fbebe5ff700 1 --2- 172.21.15.156:0/355598756 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbea40682f0 0x7fbea406a7a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0090b0 tx=0x7fbeac008030 comp rx=0 tx=0).stop 2023-11-28T13:15:38.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.082+0000 7fbebe5ff700 1 -- 172.21.15.156:0/355598756 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbeb80d2320 msgr2=0x7fbeb815f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.082+0000 7fbebe5ff700 1 --2- 172.21.15.156:0/355598756 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbeb80d2320 0x7fbeb815f1f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbea800e900 tx=0x7fbea800ec10 comp rx=0 tx=0).stop 2023-11-28T13:15:38.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.082+0000 7fbebe5ff700 1 -- 172.21.15.156:0/355598756 shutdown_connections 2023-11-28T13:15:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.082+0000 7fbebe5ff700 1 --2- 172.21.15.156:0/355598756 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fbea40682f0 0x7fbea406a7a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.082+0000 7fbebe5ff700 1 --2- 172.21.15.156:0/355598756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbeb8161340 0x7fbeb8156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.082+0000 7fbebe5ff700 1 --2- 172.21.15.156:0/355598756 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbeb80d2320 0x7fbeb815f1f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.082+0000 7fbebe5ff700 1 --2- 172.21.15.156:0/355598756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeb815c9b0 0x7fbeb8156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.082+0000 7fbebe5ff700 1 -- 172.21.15.156:0/355598756 >> 172.21.15.156:0/355598756 conn(0x7fbeb8072600 msgr2=0x7fbeb80b8870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:38.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.082+0000 7fbebe5ff700 1 -- 172.21.15.156:0/355598756 shutdown_connections 2023-11-28T13:15:38.086 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.082+0000 7fbebe5ff700 1 -- 172.21.15.156:0/355598756 wait complete. 2023-11-28T13:15:38.086 INFO:teuthology.orchestra.run.smithi156.stderr:instructed osd(s) 4 to deep-scrub 2023-11-28T13:15:38.107 INFO:tasks.ceph:Scrubbing osd.5 2023-11-28T13:15:38.107 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:15:38.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.307+0000 7f99f9dff700 1 Processor -- start 2023-11-28T13:15:38.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.307+0000 7f99f9dff700 1 -- start start 2023-11-28T13:15:38.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.308+0000 7f99f9dff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f99f4161200 0x7f99f415dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.308+0000 7f99f9dff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f4160d70 con 0x7f99f40cded0 2023-11-28T13:15:38.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.308+0000 7f99f9dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f40b5390 con 0x7f99f415a760 2023-11-28T13:15:38.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.308+0000 7f99f9dff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f415ab30 con 0x7f99f4161200 2023-11-28T13:15:38.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.308+0000 7f99ebfff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f99f40cded0 0x7f99f40d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:40886/0 (socket says 172.21.15.156:40886) 2023-11-28T13:15:38.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.308+0000 7f99ebfff700 1 -- 172.21.15.156:0/2931937614 learned_addr learned my addr 172.21.15.156:0/2931937614 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:38.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.308+0000 7f99f3fff700 1 --2- 172.21.15.156:0/2931937614 >> v2:172.21.15.156:3300/0 conn(0x7f99f4161200 0x7f99f415dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.309+0000 7f99f3fff700 1 -- 172.21.15.156:0/2931937614 >> v1:172.21.15.186:6789/0 conn(0x7f99f415a760 legacy=0x7f99f40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.309+0000 7f99f3fff700 1 -- 172.21.15.156:0/2931937614 >> v1:172.21.15.156:6789/0 conn(0x7f99f40cded0 legacy=0x7f99f40d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.309+0000 7f99f2ffd700 1 -- 172.21.15.156:0/2931937614 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943198117 0 0) 0x7f99f4160d70 con 0x7f99f40cded0 2023-11-28T13:15:38.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.309+0000 7f99f3fff700 1 -- 172.21.15.156:0/2931937614 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f40b5670 con 0x7f99f4161200 2023-11-28T13:15:38.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.309+0000 7f99f3fff700 1 --2- 172.21.15.156:0/2931937614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f4161200 0x7f99f415dc30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009d90 tx=0x7f99ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d736d7f7c661c2ef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:38.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.309+0000 7f99f2ffd700 1 -- 172.21.15.156:0/2931937614 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec00e070 con 0x7f99f4161200 2023-11-28T13:15:38.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.309+0000 7f99f2ffd700 1 -- 172.21.15.156:0/2931937614 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec00bbd0 con 0x7f99f4161200 2023-11-28T13:15:38.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.309+0000 7f99f2ffd700 1 -- 172.21.15.156:0/2931937614 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec012430 con 0x7f99f4161200 2023-11-28T13:15:38.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.310+0000 7f99f9dff700 1 -- 172.21.15.156:0/2931937614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f4161200 msgr2=0x7f99f415dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.310+0000 7f99f9dff700 1 --2- 172.21.15.156:0/2931937614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f4161200 0x7f99f415dc30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009d90 tx=0x7f99ec00b330 comp rx=0 tx=0).stop 2023-11-28T13:15:38.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.310+0000 7f99f9dff700 1 -- 172.21.15.156:0/2931937614 shutdown_connections 2023-11-28T13:15:38.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.310+0000 7f99f9dff700 1 --2- 172.21.15.156:0/2931937614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f4161200 0x7f99f415dc30 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.310+0000 7f99f9dff700 1 -- 172.21.15.156:0/2931937614 >> 172.21.15.156:0/2931937614 conn(0x7f99f4072620 msgr2=0x7f99f40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:38.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.310+0000 7f99f9dff700 1 -- 172.21.15.156:0/2931937614 shutdown_connections 2023-11-28T13:15:38.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.310+0000 7f99f9dff700 1 -- 172.21.15.156:0/2931937614 wait complete. 2023-11-28T13:15:38.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.312+0000 7f99f9dff700 1 Processor -- start 2023-11-28T13:15:38.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.312+0000 7f99f9dff700 1 -- start start 2023-11-28T13:15:38.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.313+0000 7f99f9dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99f40cded0 0x7f99f40b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.313+0000 7f99f9dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99f415a760 0x7f99f40b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.313+0000 7f99f9dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f4161200 0x7f99f40c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.313+0000 7f99f9dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f40c88e0 con 0x7f99f4161200 2023-11-28T13:15:38.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.313+0000 7f99f9dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f40b75b0 con 0x7f99f40cded0 2023-11-28T13:15:38.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.313+0000 7f99f9dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f40c7d60 con 0x7f99f415a760 2023-11-28T13:15:38.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.313+0000 7f99f3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f4161200 0x7f99f40c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.313+0000 7f99f3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f4161200 0x7f99f40c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57582/0 (socket says 172.21.15.156:57582) 2023-11-28T13:15:38.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.313+0000 7f99f3fff700 1 -- 172.21.15.156:0/318100595 learned_addr learned my addr 172.21.15.156:0/318100595 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:38.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.313+0000 7f99ebfff700 1 --2- 172.21.15.156:0/318100595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99f415a760 0x7f99f40b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.313+0000 7f99f37fe700 1 --2- 172.21.15.156:0/318100595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99f40cded0 0x7f99f40b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.313+0000 7f99f3fff700 1 -- 172.21.15.156:0/318100595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99f415a760 msgr2=0x7f99f40b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.313+0000 7f99f3fff700 1 --2- 172.21.15.156:0/318100595 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f99f415a760 0x7f99f40b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.313+0000 7f99f3fff700 1 -- 172.21.15.156:0/318100595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99f40cded0 msgr2=0x7f99f40b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.313+0000 7f99f3fff700 1 --2- 172.21.15.156:0/318100595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99f40cded0 0x7f99f40b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.313+0000 7f99f3fff700 1 -- 172.21.15.156:0/318100595 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec009a70 con 0x7f99f4161200 2023-11-28T13:15:38.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.314+0000 7f99f3fff700 1 --2- 172.21.15.156:0/318100595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f4161200 0x7f99f40c9040 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f99ec014040 tx=0x7f99ec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:38.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.314+0000 7f99f0ff9700 1 -- 172.21.15.156:0/318100595 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec01b810 con 0x7f99f4161200 2023-11-28T13:15:38.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.314+0000 7f99f0ff9700 1 -- 172.21.15.156:0/318100595 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec020070 con 0x7f99f4161200 2023-11-28T13:15:38.320 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.314+0000 7f99f0ff9700 1 -- 172.21.15.156:0/318100595 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec01bbb0 con 0x7f99f4161200 2023-11-28T13:15:38.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.314+0000 7f99f9dff700 1 -- 172.21.15.156:0/318100595 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f4160a00 con 0x7f99f4161200 2023-11-28T13:15:38.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.314+0000 7f99f9dff700 1 -- 172.21.15.156:0/318100595 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f4160720 con 0x7f99f4161200 2023-11-28T13:15:38.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.315+0000 7f99f9dff700 1 -- 172.21.15.156:0/318100595 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f99f40b4bf0 con 0x7f99f4161200 2023-11-28T13:15:38.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.316+0000 7f99f0ff9700 1 -- 172.21.15.156:0/318100595 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f99ec02e070 con 0x7f99f4161200 2023-11-28T13:15:38.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.317+0000 7f99f0ff9700 1 --2- 172.21.15.156:0/318100595 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f99d40682f0 0x7f99d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.317+0000 7f99f0ff9700 1 -- 172.21.15.156:0/318100595 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f99ec022070 con 0x7f99f4161200 2023-11-28T13:15:38.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.317+0000 7f99f0ff9700 1 --2- 172.21.15.156:0/318100595 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x7f99d406e760 0x7f99d4070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.317+0000 7f99f0ff9700 1 -- 172.21.15.156:0/318100595 --> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99d4071240 con 0x7f99d406e760 2023-11-28T13:15:38.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.317+0000 7f99f0ff9700 1 -- 172.21.15.156:0/318100595 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99ec08c2a0 con 0x7f99f4161200 2023-11-28T13:15:38.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.317+0000 7f99f37fe700 1 --2- 172.21.15.156:0/318100595 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f99d40682f0 0x7f99d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.318+0000 7f99ebfff700 1 --2- 172.21.15.156:0/318100595 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x7f99d406e760 0x7f99d4070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.318+0000 7f99f37fe700 1 --2- 172.21.15.156:0/318100595 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f99d40682f0 0x7f99d406a7a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f99e4007fd0 tx=0x7f99e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:38.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.318+0000 7f99ebfff700 1 --2- 172.21.15.156:0/318100595 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x7f99d406e760 0x7f99d4070b70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:38.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.319+0000 7f99f0ff9700 1 -- 172.21.15.156:0/318100595 <== osd.5 v2:172.21.15.186:6800/132935 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f99d4071240 con 0x7f99d406e760 2023-11-28T13:15:38.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.334+0000 7f99f9dff700 1 -- 172.21.15.156:0/318100595 --> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f99f40b77b0 con 0x7f99d406e760 2023-11-28T13:15:38.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.334+0000 7f99f0ff9700 1 -- 172.21.15.156:0/318100595 <== osd.5 v2:172.21.15.186:6800/132935 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f99f40b77b0 con 0x7f99d406e760 2023-11-28T13:15:38.337 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:15:38.337 INFO:teuthology.orchestra.run.smithi156.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:15:38.338 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:15:38.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.335+0000 7f99f9dff700 1 -- 172.21.15.156:0/318100595 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x7f99d406e760 msgr2=0x7f99d4070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.335+0000 7f99f9dff700 1 --2- 172.21.15.156:0/318100595 >> [v2:172.21.15.186:6800/132935,v1:172.21.15.186:6801/132935] conn(0x7f99d406e760 0x7f99d4070b70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.335+0000 7f99f9dff700 1 -- 172.21.15.156:0/318100595 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f99d40682f0 msgr2=0x7f99d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.335+0000 7f99f9dff700 1 --2- 172.21.15.156:0/318100595 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f99d40682f0 0x7f99d406a7a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f99e4007fd0 tx=0x7f99e4009040 comp rx=0 tx=0).stop 2023-11-28T13:15:38.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.335+0000 7f99f9dff700 1 -- 172.21.15.156:0/318100595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f4161200 msgr2=0x7f99f40c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.335+0000 7f99f9dff700 1 --2- 172.21.15.156:0/318100595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f4161200 0x7f99f40c9040 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f99ec014040 tx=0x7f99ec004ad0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.335+0000 7f99f3fff700 1 -- 172.21.15.156:0/318100595 reap_dead start 2023-11-28T13:15:38.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.335+0000 7f99f9dff700 1 -- 172.21.15.156:0/318100595 shutdown_connections 2023-11-28T13:15:38.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.335+0000 7f99f9dff700 1 -- 172.21.15.156:0/318100595 >> 172.21.15.156:0/318100595 conn(0x7f99f4072620 msgr2=0x7f99f40d28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:38.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.335+0000 7f99f9dff700 1 -- 172.21.15.156:0/318100595 shutdown_connections 2023-11-28T13:15:38.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.335+0000 7f99f9dff700 1 -- 172.21.15.156:0/318100595 wait complete. 2023-11-28T13:15:38.354 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-28T13:15:38.552 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.551+0000 7f260b7ff700 1 Processor -- start 2023-11-28T13:15:38.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.551+0000 7f260b7ff700 1 -- start start 2023-11-28T13:15:38.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.552+0000 7f260b7ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f260415c9b0 0x7f26040b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.552+0000 7f260b7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f26040cd7c0 0x7f26040cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.552+0000 7f260b7ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26040d1950 con 0x7f2604161340 2023-11-28T13:15:38.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.552+0000 7f260b7ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26040b5800 con 0x7f260415c9b0 2023-11-28T13:15:38.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.552+0000 7f260b7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26040ce140 con 0x7f26040cd7c0 2023-11-28T13:15:38.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.552+0000 7f2608d9a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f2604161340 0x7f26040d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:40900/0 (socket says 172.21.15.156:40900) 2023-11-28T13:15:38.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.552+0000 7f2608d9a700 1 -- 172.21.15.156:0/649869515 learned_addr learned my addr 172.21.15.156:0/649869515 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:38.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.553+0000 7f2609d9c700 1 --2- 172.21.15.156:0/649869515 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f26040cd7c0 0x7f26040cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.553+0000 7f260959b700 1 --2- 172.21.15.156:0/649869515 >> v2:172.21.15.186:3300/0 conn(0x7f260415c9b0 0x7f26040b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.553+0000 7f25fbfff700 1 -- 172.21.15.156:0/649869515 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498639536 0 0) 0x7f26040d1950 con 0x7f2604161340 2023-11-28T13:15:38.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.553+0000 7f25fbfff700 1 -- 172.21.15.156:0/649869515 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25f00035b0 con 0x7f2604161340 2023-11-28T13:15:38.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.553+0000 7f260959b700 1 -- 172.21.15.156:0/649869515 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f26040cd7c0 msgr2=0x7f26040cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:38.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.553+0000 7f260959b700 1 --2- 172.21.15.156:0/649869515 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f26040cd7c0 0x7f26040cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.553+0000 7f260959b700 1 -- 172.21.15.156:0/649869515 >> v1:172.21.15.156:6789/0 conn(0x7f2604161340 legacy=0x7f26040d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.553+0000 7f260959b700 1 -- 172.21.15.156:0/649869515 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26040d24f0 con 0x7f260415c9b0 2023-11-28T13:15:38.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.553+0000 7f260959b700 1 --2- 172.21.15.156:0/649869515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f260415c9b0 0x7f26040b4bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2600009840 tx=0x7f2600009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80de61010beddd05 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:38.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.554+0000 7f25fbfff700 1 -- 172.21.15.156:0/649869515 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600011830 con 0x7f260415c9b0 2023-11-28T13:15:38.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.554+0000 7f25fbfff700 1 -- 172.21.15.156:0/649869515 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2600011990 con 0x7f260415c9b0 2023-11-28T13:15:38.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.554+0000 7f25fbfff700 1 -- 172.21.15.156:0/649869515 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600011d00 con 0x7f260415c9b0 2023-11-28T13:15:38.557 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.554+0000 7f260b7ff700 1 -- 172.21.15.156:0/649869515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f260415c9b0 msgr2=0x7f26040b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.558 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.554+0000 7f260b7ff700 1 --2- 172.21.15.156:0/649869515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f260415c9b0 0x7f26040b4bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2600009840 tx=0x7f2600009b50 comp rx=0 tx=0).stop 2023-11-28T13:15:38.558 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.554+0000 7f260b7ff700 1 -- 172.21.15.156:0/649869515 shutdown_connections 2023-11-28T13:15:38.558 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.554+0000 7f260b7ff700 1 --2- 172.21.15.156:0/649869515 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f26040cd7c0 0x7f26040cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.558 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.554+0000 7f260b7ff700 1 --2- 172.21.15.156:0/649869515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f260415c9b0 0x7f26040b4bf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.554+0000 7f260b7ff700 1 -- 172.21.15.156:0/649869515 >> 172.21.15.156:0/649869515 conn(0x7f2604072600 msgr2=0x7f26040cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:38.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.554+0000 7f260b7ff700 1 -- 172.21.15.156:0/649869515 shutdown_connections 2023-11-28T13:15:38.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.555+0000 7f260b7ff700 1 -- 172.21.15.156:0/649869515 wait complete. 2023-11-28T13:15:38.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.556+0000 7f260b7ff700 1 Processor -- start 2023-11-28T13:15:38.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.556+0000 7f260b7ff700 1 -- start start 2023-11-28T13:15:38.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.557+0000 7f260b7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26040cd7c0 0x7f2604156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.557+0000 7f260b7ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f260415c9b0 0x7f260415e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.557+0000 7f260b7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2604161340 0x7f260415ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.557+0000 7f260b7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26041f6b10 con 0x7f2604161340 2023-11-28T13:15:38.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.557+0000 7f260b7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26040cd520 con 0x7f26040cd7c0 2023-11-28T13:15:38.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.557+0000 7f260b7ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26041f5d50 con 0x7f260415c9b0 2023-11-28T13:15:38.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.558+0000 7f2609d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2604161340 0x7f260415ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.558+0000 7f2609d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2604161340 0x7f260415ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57594/0 (socket says 172.21.15.156:57594) 2023-11-28T13:15:38.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.558+0000 7f2609d9c700 1 -- 172.21.15.156:0/1400522340 learned_addr learned my addr 172.21.15.156:0/1400522340 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:38.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.558+0000 7f2608d9a700 1 --2- 172.21.15.156:0/1400522340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f260415c9b0 0x7f260415e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.558+0000 7f260959b700 1 --2- 172.21.15.156:0/1400522340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26040cd7c0 0x7f2604156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.558+0000 7f2609d9c700 1 -- 172.21.15.156:0/1400522340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f260415c9b0 msgr2=0x7f260415e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.558+0000 7f2609d9c700 1 --2- 172.21.15.156:0/1400522340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f260415c9b0 0x7f260415e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.558+0000 7f2609d9c700 1 -- 172.21.15.156:0/1400522340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26040cd7c0 msgr2=0x7f2604156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.558+0000 7f2609d9c700 1 --2- 172.21.15.156:0/1400522340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26040cd7c0 0x7f2604156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.558+0000 7f2609d9c700 1 -- 172.21.15.156:0/1400522340 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26000094a0 con 0x7f2604161340 2023-11-28T13:15:38.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.558+0000 7f2608d9a700 1 --2- 172.21.15.156:0/1400522340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f260415c9b0 0x7f260415e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:15:38.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.558+0000 7f260959b700 1 --2- 172.21.15.156:0/1400522340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26040cd7c0 0x7f2604156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:15:38.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.558+0000 7f2609d9c700 1 --2- 172.21.15.156:0/1400522340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2604161340 0x7f260415ea30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00efc0 tx=0x7f25fc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:38.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.558+0000 7f25f9ffb700 1 -- 172.21.15.156:0/1400522340 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc009ba0 con 0x7f2604161340 2023-11-28T13:15:38.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.558+0000 7f25f9ffb700 1 -- 172.21.15.156:0/1400522340 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25fc00f040 con 0x7f2604161340 2023-11-28T13:15:38.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.559+0000 7f260b7ff700 1 -- 172.21.15.156:0/1400522340 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26040d1300 con 0x7f2604161340 2023-11-28T13:15:38.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.559+0000 7f25f9ffb700 1 -- 172.21.15.156:0/1400522340 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc014480 con 0x7f2604161340 2023-11-28T13:15:38.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.559+0000 7f260b7ff700 1 -- 172.21.15.156:0/1400522340 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26040b5520 con 0x7f2604161340 2023-11-28T13:15:38.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.560+0000 7f260b7ff700 1 -- 172.21.15.156:0/1400522340 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26040d15e0 con 0x7f2604161340 2023-11-28T13:15:38.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.560+0000 7f25f9ffb700 1 -- 172.21.15.156:0/1400522340 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f25fc0145e0 con 0x7f2604161340 2023-11-28T13:15:38.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.561+0000 7f25f9ffb700 1 --2- 172.21.15.156:0/1400522340 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25e80682f0 0x7f25e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.561+0000 7f25f9ffb700 1 -- 172.21.15.156:0/1400522340 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f25fc086460 con 0x7f2604161340 2023-11-28T13:15:38.566 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.562+0000 7f260959b700 1 --2- 172.21.15.156:0/1400522340 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25e80682f0 0x7f25e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.567 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.562+0000 7f260959b700 1 --2- 172.21.15.156:0/1400522340 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25e80682f0 0x7f25e806a7a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2600009fd0 tx=0x7f2600008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:38.568 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.565+0000 7f25f9ffb700 1 -- 172.21.15.156:0/1400522340 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f25fc052ef0 con 0x7f2604161340 2023-11-28T13:15:38.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.757+0000 7f260b7ff700 1 -- 172.21.15.156:0/1400522340 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f26040745c0 con 0x7f25e80682f0 2023-11-28T13:15:38.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.758+0000 7f25f9ffb700 1 -- 172.21.15.156:0/1400522340 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f26040745c0 con 0x7f25e80682f0 2023-11-28T13:15:38.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.760+0000 7f260b7ff700 1 -- 172.21.15.156:0/1400522340 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25e80682f0 msgr2=0x7f25e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.760+0000 7f260b7ff700 1 --2- 172.21.15.156:0/1400522340 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25e80682f0 0x7f25e806a7a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2600009fd0 tx=0x7f2600008030 comp rx=0 tx=0).stop 2023-11-28T13:15:38.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.760+0000 7f260b7ff700 1 -- 172.21.15.156:0/1400522340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2604161340 msgr2=0x7f260415ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.760+0000 7f260b7ff700 1 --2- 172.21.15.156:0/1400522340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2604161340 0x7f260415ea30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00efc0 tx=0x7f25fc00c3a0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.760+0000 7f260b7ff700 1 -- 172.21.15.156:0/1400522340 shutdown_connections 2023-11-28T13:15:38.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.760+0000 7f260b7ff700 1 --2- 172.21.15.156:0/1400522340 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f25e80682f0 0x7f25e806a7a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.760+0000 7f260b7ff700 1 --2- 172.21.15.156:0/1400522340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2604161340 0x7f260415ea30 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.760+0000 7f260b7ff700 1 --2- 172.21.15.156:0/1400522340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26040cd7c0 0x7f2604156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.760+0000 7f260b7ff700 1 --2- 172.21.15.156:0/1400522340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f260415c9b0 0x7f260415e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.760+0000 7f260b7ff700 1 -- 172.21.15.156:0/1400522340 >> 172.21.15.156:0/1400522340 conn(0x7f2604072600 msgr2=0x7f26040b7350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:38.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.760+0000 7f260b7ff700 1 -- 172.21.15.156:0/1400522340 shutdown_connections 2023-11-28T13:15:38.763 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.760+0000 7f260b7ff700 1 -- 172.21.15.156:0/1400522340 wait complete. 2023-11-28T13:15:38.763 INFO:teuthology.orchestra.run.smithi156.stderr:instructed osd(s) 5 to deep-scrub 2023-11-28T13:15:38.780 INFO:tasks.ceph:Scrubbing osd.6 2023-11-28T13:15:38.780 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:15:38.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.979+0000 7fcff15ff700 1 Processor -- start 2023-11-28T13:15:38.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.979+0000 7fcff15ff700 1 -- start start 2023-11-28T13:15:38.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.980+0000 7fcff15ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfec0cded0 0x7fcfec161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.982 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.980+0000 7fcff15ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcfec15dc30 0x7fcfec0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.981+0000 7fcff15ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfec15e0a0 con 0x7fcfec15a760 2023-11-28T13:15:38.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.981+0000 7fcff15ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfec15ab30 con 0x7fcfec15dc30 2023-11-28T13:15:38.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.981+0000 7fcff15ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec160d70 con 0x7fcfec0cded0 2023-11-28T13:15:38.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.981+0000 7fcfeaffd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fcfec15a760 0x7fcfec0cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:40910/0 (socket says 172.21.15.156:40910) 2023-11-28T13:15:38.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.981+0000 7fcfeaffd700 1 -- 172.21.15.156:0/1685140190 learned_addr learned my addr 172.21.15.156:0/1685140190 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:38.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.981+0000 7fcfeb7fe700 1 --2- 172.21.15.156:0/1685140190 >> v2:172.21.15.156:3300/0 conn(0x7fcfec15dc30 0x7fcfec0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.981+0000 7fcfe3fff700 1 --2- 172.21.15.156:0/1685140190 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfec0cded0 0x7fcfec161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.981+0000 7fcfea7fc700 1 -- 172.21.15.156:0/1685140190 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701265142 0 0) 0x7fcfec15e0a0 con 0x7fcfec15a760 2023-11-28T13:15:38.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.981+0000 7fcfea7fc700 1 -- 172.21.15.156:0/1685140190 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfd80035b0 con 0x7fcfec15a760 2023-11-28T13:15:38.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.981+0000 7fcfeb7fe700 1 -- 172.21.15.156:0/1685140190 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfec0cded0 msgr2=0x7fcfec161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:38.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.981+0000 7fcfeb7fe700 1 --2- 172.21.15.156:0/1685140190 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfec0cded0 0x7fcfec161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.981+0000 7fcfeb7fe700 1 -- 172.21.15.156:0/1685140190 >> v1:172.21.15.156:6789/0 conn(0x7fcfec15a760 legacy=0x7fcfec0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.981+0000 7fcfeb7fe700 1 -- 172.21.15.156:0/1685140190 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec0b5130 con 0x7fcfec15dc30 2023-11-28T13:15:38.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.982+0000 7fcfeb7fe700 1 --2- 172.21.15.156:0/1685140190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfec15dc30 0x7fcfec0b4bf0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009d90 tx=0x7fcfe400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7c722394627ed7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:38.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.982+0000 7fcfea7fc700 1 -- 172.21.15.156:0/1685140190 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe400e070 con 0x7fcfec15dc30 2023-11-28T13:15:38.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.982+0000 7fcfea7fc700 1 -- 172.21.15.156:0/1685140190 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe400bb20 con 0x7fcfec15dc30 2023-11-28T13:15:38.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.982+0000 7fcfea7fc700 1 -- 172.21.15.156:0/1685140190 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4012430 con 0x7fcfec15dc30 2023-11-28T13:15:38.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.982+0000 7fcff15ff700 1 -- 172.21.15.156:0/1685140190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfec15dc30 msgr2=0x7fcfec0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.982+0000 7fcff15ff700 1 --2- 172.21.15.156:0/1685140190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfec15dc30 0x7fcfec0b4bf0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009d90 tx=0x7fcfe400b330 comp rx=0 tx=0).stop 2023-11-28T13:15:38.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.982+0000 7fcff15ff700 1 -- 172.21.15.156:0/1685140190 shutdown_connections 2023-11-28T13:15:38.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.982+0000 7fcff15ff700 1 --2- 172.21.15.156:0/1685140190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfec15dc30 0x7fcfec0b4bf0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.983+0000 7fcff15ff700 1 --2- 172.21.15.156:0/1685140190 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfec0cded0 0x7fcfec161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.983+0000 7fcff15ff700 1 -- 172.21.15.156:0/1685140190 >> 172.21.15.156:0/1685140190 conn(0x7fcfec072620 msgr2=0x7fcfec0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:38.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.983+0000 7fcff15ff700 1 -- 172.21.15.156:0/1685140190 shutdown_connections 2023-11-28T13:15:38.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.983+0000 7fcff15ff700 1 -- 172.21.15.156:0/1685140190 wait complete. 2023-11-28T13:15:38.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.984+0000 7fcff15ff700 1 Processor -- start 2023-11-28T13:15:38.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.984+0000 7fcff15ff700 1 -- start start 2023-11-28T13:15:38.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.985+0000 7fcff15ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfec0cded0 0x7fcfec147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.985+0000 7fcff15ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfec15a760 0x7fcfec0c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.985+0000 7fcff15ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfec15dc30 0x7fcfec0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.985+0000 7fcff15ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec0cc150 con 0x7fcfec15dc30 2023-11-28T13:15:38.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.985+0000 7fcff15ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec0c8680 con 0x7fcfec15a760 2023-11-28T13:15:38.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.985+0000 7fcff15ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec0c8960 con 0x7fcfec0cded0 2023-11-28T13:15:38.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.986+0000 7fcfeb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfec15dc30 0x7fcfec0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.986+0000 7fcfeb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfec15dc30 0x7fcfec0ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57618/0 (socket says 172.21.15.156:57618) 2023-11-28T13:15:38.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.986+0000 7fcfeb7fe700 1 -- 172.21.15.156:0/4170270539 learned_addr learned my addr 172.21.15.156:0/4170270539 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:38.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.986+0000 7fcfe3fff700 1 --2- 172.21.15.156:0/4170270539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfec15a760 0x7fcfec0c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.986+0000 7fcfeaffd700 1 --2- 172.21.15.156:0/4170270539 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfec0cded0 0x7fcfec147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.986+0000 7fcfeb7fe700 1 -- 172.21.15.156:0/4170270539 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfec0cded0 msgr2=0x7fcfec147c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.986+0000 7fcfeb7fe700 1 --2- 172.21.15.156:0/4170270539 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfec0cded0 0x7fcfec147c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.986+0000 7fcfeb7fe700 1 -- 172.21.15.156:0/4170270539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfec15a760 msgr2=0x7fcfec0c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:38.992 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.986+0000 7fcfeb7fe700 1 --2- 172.21.15.156:0/4170270539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfec15a760 0x7fcfec0c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:38.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.986+0000 7fcfeb7fe700 1 -- 172.21.15.156:0/4170270539 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe4009a70 con 0x7fcfec15dc30 2023-11-28T13:15:38.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.986+0000 7fcfeb7fe700 1 --2- 172.21.15.156:0/4170270539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfec15dc30 0x7fcfec0ca240 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4014040 tx=0x7fcfe4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:38.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.986+0000 7fcfe37fe700 1 -- 172.21.15.156:0/4170270539 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe40129f0 con 0x7fcfec15dc30 2023-11-28T13:15:38.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.987+0000 7fcfe37fe700 1 -- 172.21.15.156:0/4170270539 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe4020070 con 0x7fcfec15dc30 2023-11-28T13:15:38.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.987+0000 7fcff15ff700 1 -- 172.21.15.156:0/4170270539 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfec0b5670 con 0x7fcfec15dc30 2023-11-28T13:15:38.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.987+0000 7fcfe37fe700 1 -- 172.21.15.156:0/4170270539 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4012db0 con 0x7fcfec15dc30 2023-11-28T13:15:38.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.987+0000 7fcff15ff700 1 -- 172.21.15.156:0/4170270539 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfec0b5390 con 0x7fcfec15dc30 2023-11-28T13:15:38.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.987+0000 7fcff15ff700 1 -- 172.21.15.156:0/4170270539 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcfec160720 con 0x7fcfec15dc30 2023-11-28T13:15:38.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.989+0000 7fcfe37fe700 1 -- 172.21.15.156:0/4170270539 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcfe4004040 con 0x7fcfec15dc30 2023-11-28T13:15:38.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.989+0000 7fcfe37fe700 1 --2- 172.21.15.156:0/4170270539 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcfcc0682f0 0x7fcfcc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.990+0000 7fcfe37fe700 1 -- 172.21.15.156:0/4170270539 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fcfe4022070 con 0x7fcfec15dc30 2023-11-28T13:15:38.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.990+0000 7fcfe37fe700 1 --2- 172.21.15.156:0/4170270539 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x7fcfcc06e760 0x7fcfcc070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:38.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.990+0000 7fcfe37fe700 1 -- 172.21.15.156:0/4170270539 --> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcfcc071240 con 0x7fcfcc06e760 2023-11-28T13:15:38.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.990+0000 7fcfe37fe700 1 -- 172.21.15.156:0/4170270539 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfe4088200 con 0x7fcfec15dc30 2023-11-28T13:15:38.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.990+0000 7fcfeaffd700 1 --2- 172.21.15.156:0/4170270539 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcfcc0682f0 0x7fcfcc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.990+0000 7fcfe3fff700 1 --2- 172.21.15.156:0/4170270539 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x7fcfcc06e760 0x7fcfcc070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:38.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.991+0000 7fcfeaffd700 1 --2- 172.21.15.156:0/4170270539 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcfcc0682f0 0x7fcfcc06a7a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc007fd0 tx=0x7fcfdc007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:38.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.991+0000 7fcfe3fff700 1 --2- 172.21.15.156:0/4170270539 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x7fcfcc06e760 0x7fcfcc070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:38.996 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:38.992+0000 7fcfe37fe700 1 -- 172.21.15.156:0/4170270539 <== osd.6 v2:172.21.15.186:6824/132946 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fcfcc071240 con 0x7fcfcc06e760 2023-11-28T13:15:39.008 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.007+0000 7fcff15ff700 1 -- 172.21.15.156:0/4170270539 --> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcfec0c8e50 con 0x7fcfcc06e760 2023-11-28T13:15:39.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.007+0000 7fcfe37fe700 1 -- 172.21.15.156:0/4170270539 <== osd.6 v2:172.21.15.186:6824/132946 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fcfec0c8e50 con 0x7fcfcc06e760 2023-11-28T13:15:39.009 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:15:39.009 INFO:teuthology.orchestra.run.smithi156.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:15:39.009 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:15:39.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.008+0000 7fcff15ff700 1 -- 172.21.15.156:0/4170270539 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x7fcfcc06e760 msgr2=0x7fcfcc070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.008+0000 7fcff15ff700 1 --2- 172.21.15.156:0/4170270539 >> [v2:172.21.15.186:6824/132946,v1:172.21.15.186:6825/132946] conn(0x7fcfcc06e760 0x7fcfcc070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.008+0000 7fcff15ff700 1 -- 172.21.15.156:0/4170270539 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcfcc0682f0 msgr2=0x7fcfcc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.008+0000 7fcff15ff700 1 --2- 172.21.15.156:0/4170270539 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fcfcc0682f0 0x7fcfcc06a7a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc007fd0 tx=0x7fcfdc007dc0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.008+0000 7fcff15ff700 1 -- 172.21.15.156:0/4170270539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfec15dc30 msgr2=0x7fcfec0ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.008+0000 7fcff15ff700 1 --2- 172.21.15.156:0/4170270539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfec15dc30 0x7fcfec0ca240 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4014040 tx=0x7fcfe4004ad0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.008+0000 7fcfeb7fe700 1 -- 172.21.15.156:0/4170270539 reap_dead start 2023-11-28T13:15:39.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.008+0000 7fcff15ff700 1 -- 172.21.15.156:0/4170270539 shutdown_connections 2023-11-28T13:15:39.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.008+0000 7fcff15ff700 1 -- 172.21.15.156:0/4170270539 >> 172.21.15.156:0/4170270539 conn(0x7fcfec072620 msgr2=0x7fcfec0b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:39.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.008+0000 7fcff15ff700 1 -- 172.21.15.156:0/4170270539 shutdown_connections 2023-11-28T13:15:39.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.008+0000 7fcff15ff700 1 -- 172.21.15.156:0/4170270539 wait complete. 2023-11-28T13:15:39.027 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-28T13:15:39.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.223+0000 7fe6969ff700 1 Processor -- start 2023-11-28T13:15:39.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.223+0000 7fe6969ff700 1 -- start start 2023-11-28T13:15:39.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.224+0000 7fe6969ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe690161340 0x7fe6900d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.224+0000 7fe6969ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe6900d2320 0x7fe6900cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.224+0000 7fe6969ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6900d1950 con 0x7fe69015c9b0 2023-11-28T13:15:39.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.224+0000 7fe6969ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6900b5800 con 0x7fe690161340 2023-11-28T13:15:39.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.224+0000 7fe6969ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6900b5300 con 0x7fe6900d2320 2023-11-28T13:15:39.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.224+0000 7fe68f7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe690161340 0x7fe6900d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.224+0000 7fe68f7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe690161340 0x7fe6900d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57622/0 (socket says 172.21.15.156:57622) 2023-11-28T13:15:39.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.224+0000 7fe68f7fe700 1 -- 172.21.15.156:0/950301965 learned_addr learned my addr 172.21.15.156:0/950301965 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:39.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.224+0000 7fe694f9c700 1 --2- 172.21.15.156:0/950301965 >> v2:172.21.15.186:3300/0 conn(0x7fe6900d2320 0x7fe6900cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.225+0000 7fe68f7fe700 1 -- 172.21.15.156:0/950301965 >> v2:172.21.15.186:3300/0 conn(0x7fe6900d2320 msgr2=0x7fe6900cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:39.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.225+0000 7fe68f7fe700 1 --2- 172.21.15.156:0/950301965 >> v2:172.21.15.186:3300/0 conn(0x7fe6900d2320 0x7fe6900cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.225+0000 7fe68f7fe700 1 -- 172.21.15.156:0/950301965 >> v1:172.21.15.186:6789/0 conn(0x7fe69015c9b0 legacy=0x7fe6900b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.225+0000 7fe68f7fe700 1 -- 172.21.15.156:0/950301965 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6900b5520 con 0x7fe690161340 2023-11-28T13:15:39.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.225+0000 7fe68f7fe700 1 --2- 172.21.15.156:0/950301965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe690161340 0x7fe6900d1de0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe69006c170 tx=0x7fe684009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d703a2eb05dd69 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:39.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.225+0000 7fe68effd700 1 -- 172.21.15.156:0/950301965 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68400b860 con 0x7fe690161340 2023-11-28T13:15:39.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.225+0000 7fe68effd700 1 -- 172.21.15.156:0/950301965 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68400b9c0 con 0x7fe690161340 2023-11-28T13:15:39.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.225+0000 7fe68effd700 1 -- 172.21.15.156:0/950301965 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68400bd50 con 0x7fe690161340 2023-11-28T13:15:39.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.226+0000 7fe6969ff700 1 -- 172.21.15.156:0/950301965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe690161340 msgr2=0x7fe6900d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.226+0000 7fe6969ff700 1 --2- 172.21.15.156:0/950301965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe690161340 0x7fe6900d1de0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe69006c170 tx=0x7fe684009c00 comp rx=0 tx=0).stop 2023-11-28T13:15:39.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.226+0000 7fe6969ff700 1 -- 172.21.15.156:0/950301965 shutdown_connections 2023-11-28T13:15:39.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.226+0000 7fe6969ff700 1 --2- 172.21.15.156:0/950301965 >> v2:172.21.15.186:3300/0 conn(0x7fe6900d2320 0x7fe6900cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.231 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.226+0000 7fe6969ff700 1 --2- 172.21.15.156:0/950301965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe690161340 0x7fe6900d1de0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.226+0000 7fe6969ff700 1 -- 172.21.15.156:0/950301965 >> 172.21.15.156:0/950301965 conn(0x7fe690072600 msgr2=0x7fe6900cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:39.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.226+0000 7fe6969ff700 1 -- 172.21.15.156:0/950301965 shutdown_connections 2023-11-28T13:15:39.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.226+0000 7fe6969ff700 1 -- 172.21.15.156:0/950301965 wait complete. 2023-11-28T13:15:39.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.228+0000 7fe6969ff700 1 Processor -- start 2023-11-28T13:15:39.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.228+0000 7fe6969ff700 1 -- start start 2023-11-28T13:15:39.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.229+0000 7fe6969ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6900d2320 0x7fe6901578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.229+0000 7fe6969ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe69015c9b0 0x7fe69015e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.229+0000 7fe6969ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe690161340 0x7fe69015e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.229+0000 7fe6969ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6901f2b60 con 0x7fe69015c9b0 2023-11-28T13:15:39.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.229+0000 7fe6969ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6901e6940 con 0x7fe690161340 2023-11-28T13:15:39.238 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.229+0000 7fe6969ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6901e6c20 con 0x7fe6900d2320 2023-11-28T13:15:39.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.229+0000 7fe68f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe69015c9b0 0x7fe69015e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.229+0000 7fe68f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe69015c9b0 0x7fe69015e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57632/0 (socket says 172.21.15.156:57632) 2023-11-28T13:15:39.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.229+0000 7fe68f7fe700 1 -- 172.21.15.156:0/241201835 learned_addr learned my addr 172.21.15.156:0/241201835 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:39.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.229+0000 7fe68ffff700 1 --2- 172.21.15.156:0/241201835 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6900d2320 0x7fe6901578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.229+0000 7fe694f9c700 1 --2- 172.21.15.156:0/241201835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe690161340 0x7fe69015e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.229+0000 7fe68f7fe700 1 -- 172.21.15.156:0/241201835 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6900d2320 msgr2=0x7fe6901578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.239 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.229+0000 7fe68f7fe700 1 --2- 172.21.15.156:0/241201835 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6900d2320 0x7fe6901578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.229+0000 7fe68f7fe700 1 -- 172.21.15.156:0/241201835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe690161340 msgr2=0x7fe69015e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.229+0000 7fe68f7fe700 1 --2- 172.21.15.156:0/241201835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe690161340 0x7fe69015e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.229+0000 7fe68f7fe700 1 -- 172.21.15.156:0/241201835 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe684009550 con 0x7fe69015c9b0 2023-11-28T13:15:39.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.230+0000 7fe68f7fe700 1 --2- 172.21.15.156:0/241201835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe69015c9b0 0x7fe69015e240 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe684009870 tx=0x7fe684004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:39.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.230+0000 7fe68cff9700 1 -- 172.21.15.156:0/241201835 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684004240 con 0x7fe69015c9b0 2023-11-28T13:15:39.240 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.230+0000 7fe68cff9700 1 -- 172.21.15.156:0/241201835 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68401f070 con 0x7fe69015c9b0 2023-11-28T13:15:39.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.230+0000 7fe6969ff700 1 -- 172.21.15.156:0/241201835 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6900d15e0 con 0x7fe69015c9b0 2023-11-28T13:15:39.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.230+0000 7fe68cff9700 1 -- 172.21.15.156:0/241201835 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6840036a0 con 0x7fe69015c9b0 2023-11-28T13:15:39.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.230+0000 7fe6969ff700 1 -- 172.21.15.156:0/241201835 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6900d1300 con 0x7fe69015c9b0 2023-11-28T13:15:39.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.231+0000 7fe6969ff700 1 -- 172.21.15.156:0/241201835 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69015cd80 con 0x7fe69015c9b0 2023-11-28T13:15:39.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.232+0000 7fe68cff9700 1 -- 172.21.15.156:0/241201835 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe68402a020 con 0x7fe69015c9b0 2023-11-28T13:15:39.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.233+0000 7fe68cff9700 1 --2- 172.21.15.156:0/241201835 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe67c0682f0 0x7fe67c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.241 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.233+0000 7fe68cff9700 1 -- 172.21.15.156:0/241201835 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe684086dc0 con 0x7fe69015c9b0 2023-11-28T13:15:39.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.233+0000 7fe68ffff700 1 --2- 172.21.15.156:0/241201835 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe67c0682f0 0x7fe67c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.234+0000 7fe68ffff700 1 --2- 172.21.15.156:0/241201835 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe67c0682f0 0x7fe67c06a7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe680001800 tx=0x7fe680009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:39.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.237+0000 7fe68cff9700 1 -- 172.21.15.156:0/241201835 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe684053920 con 0x7fe69015c9b0 2023-11-28T13:15:39.427 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.426+0000 7fe6969ff700 1 -- 172.21.15.156:0/241201835 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6900745c0 con 0x7fe67c0682f0 2023-11-28T13:15:39.428 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.427+0000 7fe68cff9700 1 -- 172.21.15.156:0/241201835 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe6900745c0 con 0x7fe67c0682f0 2023-11-28T13:15:39.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.429+0000 7fe6969ff700 1 -- 172.21.15.156:0/241201835 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe67c0682f0 msgr2=0x7fe67c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.429+0000 7fe6969ff700 1 --2- 172.21.15.156:0/241201835 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe67c0682f0 0x7fe67c06a7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe680001800 tx=0x7fe680009040 comp rx=0 tx=0).stop 2023-11-28T13:15:39.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.429+0000 7fe6969ff700 1 -- 172.21.15.156:0/241201835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe69015c9b0 msgr2=0x7fe69015e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.429+0000 7fe6969ff700 1 --2- 172.21.15.156:0/241201835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe69015c9b0 0x7fe69015e240 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe684009870 tx=0x7fe684004750 comp rx=0 tx=0).stop 2023-11-28T13:15:39.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.429+0000 7fe6969ff700 1 -- 172.21.15.156:0/241201835 shutdown_connections 2023-11-28T13:15:39.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.429+0000 7fe6969ff700 1 --2- 172.21.15.156:0/241201835 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fe67c0682f0 0x7fe67c06a7a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.431 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.429+0000 7fe6969ff700 1 --2- 172.21.15.156:0/241201835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe690161340 0x7fe69015e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.429+0000 7fe6969ff700 1 --2- 172.21.15.156:0/241201835 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6900d2320 0x7fe6901578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.429+0000 7fe6969ff700 1 --2- 172.21.15.156:0/241201835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe69015c9b0 0x7fe69015e240 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.429+0000 7fe6969ff700 1 -- 172.21.15.156:0/241201835 >> 172.21.15.156:0/241201835 conn(0x7fe690072600 msgr2=0x7fe69015fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:39.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.429+0000 7fe6969ff700 1 -- 172.21.15.156:0/241201835 shutdown_connections 2023-11-28T13:15:39.432 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.429+0000 7fe6969ff700 1 -- 172.21.15.156:0/241201835 wait complete. 2023-11-28T13:15:39.433 INFO:teuthology.orchestra.run.smithi156.stderr:instructed osd(s) 6 to deep-scrub 2023-11-28T13:15:39.449 INFO:tasks.ceph:Scrubbing osd.7 2023-11-28T13:15:39.449 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:15:39.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.649+0000 7fb4b67ff700 1 Processor -- start 2023-11-28T13:15:39.651 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.649+0000 7fb4b67ff700 1 -- start start 2023-11-28T13:15:39.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.651+0000 7fb4b67ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb4b00adb90 0x7fb4b0165210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.651+0000 7fb4b67ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b00b0800 con 0x7fb4b00ceb30 2023-11-28T13:15:39.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.651+0000 7fb4b67ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b00af290 con 0x7fb4b00b1cc0 2023-11-28T13:15:39.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.651+0000 7fb4b67ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b00af570 con 0x7fb4b00adb90 2023-11-28T13:15:39.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.651+0000 7fb4affff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fb4b00ceb30 0x7fb4b00cef00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:40918/0 (socket says 172.21.15.156:40918) 2023-11-28T13:15:39.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.651+0000 7fb4affff700 1 -- 172.21.15.156:0/1113764604 learned_addr learned my addr 172.21.15.156:0/1113764604 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:39.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.651+0000 7fb4b4d9c700 1 --2- 172.21.15.156:0/1113764604 >> v2:172.21.15.186:3300/0 conn(0x7fb4b00adb90 0x7fb4b0165210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.651+0000 7fb4aeffd700 1 -- 172.21.15.156:0/1113764604 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581509625 0 0) 0x7fb4b00b0800 con 0x7fb4b00ceb30 2023-11-28T13:15:39.654 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.651+0000 7fb4aeffd700 1 -- 172.21.15.156:0/1113764604 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4980035b0 con 0x7fb4b00ceb30 2023-11-28T13:15:39.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.651+0000 7fb4b4d9c700 1 -- 172.21.15.156:0/1113764604 >> v1:172.21.15.186:6789/0 conn(0x7fb4b00b1cc0 legacy=0x7fb4b00aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.652+0000 7fb4aeffd700 1 -- 172.21.15.156:0/1113764604 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3204003551 0 0) 0x7fb4980035b0 con 0x7fb4b00ceb30 2023-11-28T13:15:39.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.652+0000 7fb4b4d9c700 1 -- 172.21.15.156:0/1113764604 >> v1:172.21.15.156:6789/0 conn(0x7fb4b00ceb30 legacy=0x7fb4b00cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.652+0000 7fb4b4d9c700 1 -- 172.21.15.156:0/1113764604 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b00af850 con 0x7fb4b00adb90 2023-11-28T13:15:39.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.652+0000 7fb4b4d9c700 1 --2- 172.21.15.156:0/1113764604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4b00adb90 0x7fb4b0165210 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009d90 tx=0x7fb4a800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=777b084c7ed14e6a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:39.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.652+0000 7fb4aeffd700 1 -- 172.21.15.156:0/1113764604 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a800e070 con 0x7fb4b00adb90 2023-11-28T13:15:39.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.652+0000 7fb4aeffd700 1 -- 172.21.15.156:0/1113764604 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a8003a20 con 0x7fb4b00adb90 2023-11-28T13:15:39.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.652+0000 7fb4aeffd700 1 -- 172.21.15.156:0/1113764604 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0001670 con 0x7fb4b00adb90 2023-11-28T13:15:39.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.653+0000 7fb4b67ff700 1 -- 172.21.15.156:0/1113764604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4b00adb90 msgr2=0x7fb4b0165210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.653+0000 7fb4b67ff700 1 --2- 172.21.15.156:0/1113764604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4b00adb90 0x7fb4b0165210 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009d90 tx=0x7fb4a800b330 comp rx=0 tx=0).stop 2023-11-28T13:15:39.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.653+0000 7fb4b67ff700 1 -- 172.21.15.156:0/1113764604 shutdown_connections 2023-11-28T13:15:39.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.653+0000 7fb4b67ff700 1 --2- 172.21.15.156:0/1113764604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4b00adb90 0x7fb4b0165210 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.653+0000 7fb4b67ff700 1 -- 172.21.15.156:0/1113764604 >> 172.21.15.156:0/1113764604 conn(0x7fb4b009a230 msgr2=0x7fb4b008afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:39.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.653+0000 7fb4b67ff700 1 -- 172.21.15.156:0/1113764604 shutdown_connections 2023-11-28T13:15:39.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.653+0000 7fb4b67ff700 1 -- 172.21.15.156:0/1113764604 wait complete. 2023-11-28T13:15:39.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.655+0000 7fb4b67ff700 1 Processor -- start 2023-11-28T13:15:39.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.655+0000 7fb4b67ff700 1 -- start start 2023-11-28T13:15:39.664 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.656+0000 7fb4b67ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4b00adb90 0x7fb4b014e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.656+0000 7fb4b67ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4b00b1cc0 0x7fb4b009aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.656+0000 7fb4b67ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4b00ceb30 0x7fb4b009b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.656+0000 7fb4b67ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b013dbf0 con 0x7fb4b00adb90 2023-11-28T13:15:39.665 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.656+0000 7fb4b67ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b014e680 con 0x7fb4b00b1cc0 2023-11-28T13:15:39.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.656+0000 7fb4b67ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b009dc70 con 0x7fb4b00ceb30 2023-11-28T13:15:39.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.656+0000 7fb4affff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4b00adb90 0x7fb4b014e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.666 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.656+0000 7fb4affff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4b00adb90 0x7fb4b014e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57646/0 (socket says 172.21.15.156:57646) 2023-11-28T13:15:39.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.656+0000 7fb4affff700 1 -- 172.21.15.156:0/2454971200 learned_addr learned my addr 172.21.15.156:0/2454971200 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:39.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.656+0000 7fb4b4d9c700 1 --2- 172.21.15.156:0/2454971200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4b00ceb30 0x7fb4b009b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.656+0000 7fb4af7fe700 1 --2- 172.21.15.156:0/2454971200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4b00b1cc0 0x7fb4b009aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.656+0000 7fb4affff700 1 -- 172.21.15.156:0/2454971200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4b00ceb30 msgr2=0x7fb4b009b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.667 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.656+0000 7fb4affff700 1 --2- 172.21.15.156:0/2454971200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4b00ceb30 0x7fb4b009b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.656+0000 7fb4affff700 1 -- 172.21.15.156:0/2454971200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4b00b1cc0 msgr2=0x7fb4b009aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.656+0000 7fb4affff700 1 --2- 172.21.15.156:0/2454971200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4b00b1cc0 0x7fb4b009aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.657+0000 7fb4affff700 1 -- 172.21.15.156:0/2454971200 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a8009a70 con 0x7fb4b00adb90 2023-11-28T13:15:39.668 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.657+0000 7fb4affff700 1 --2- 172.21.15.156:0/2454971200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4b00adb90 0x7fb4b014e140 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00011e0 tx=0x7fb4a000ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:39.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.657+0000 7fb4acff9700 1 -- 172.21.15.156:0/2454971200 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0004c80 con 0x7fb4b00adb90 2023-11-28T13:15:39.669 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.657+0000 7fb4acff9700 1 -- 172.21.15.156:0/2454971200 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a0004de0 con 0x7fb4b00adb90 2023-11-28T13:15:39.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.657+0000 7fb4acff9700 1 -- 172.21.15.156:0/2454971200 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00072d0 con 0x7fb4b00adb90 2023-11-28T13:15:39.670 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.657+0000 7fb4b67ff700 1 -- 172.21.15.156:0/2454971200 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b00afb30 con 0x7fb4b00adb90 2023-11-28T13:15:39.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.657+0000 7fb4b67ff700 1 -- 172.21.15.156:0/2454971200 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b015aa70 con 0x7fb4b00adb90 2023-11-28T13:15:39.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.658+0000 7fb4b67ff700 1 -- 172.21.15.156:0/2454971200 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb4b0165750 con 0x7fb4b00adb90 2023-11-28T13:15:39.672 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.659+0000 7fb4acff9700 1 -- 172.21.15.156:0/2454971200 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb4a001a020 con 0x7fb4b00adb90 2023-11-28T13:15:39.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.660+0000 7fb4acff9700 1 --2- 172.21.15.156:0/2454971200 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb49c0682f0 0x7fb49c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.674 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.660+0000 7fb4acff9700 1 -- 172.21.15.156:0/2454971200 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb4a00856b0 con 0x7fb4b00adb90 2023-11-28T13:15:39.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.660+0000 7fb4acff9700 1 --2- 172.21.15.156:0/2454971200 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x7fb49c06e760 0x7fb49c070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.675 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.660+0000 7fb4acff9700 1 -- 172.21.15.156:0/2454971200 --> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb49c071240 con 0x7fb49c06e760 2023-11-28T13:15:39.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.660+0000 7fb4acff9700 1 -- 172.21.15.156:0/2454971200 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4a0087d60 con 0x7fb4b00adb90 2023-11-28T13:15:39.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.660+0000 7fb4af7fe700 1 --2- 172.21.15.156:0/2454971200 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb49c0682f0 0x7fb49c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.661+0000 7fb4b4d9c700 1 --2- 172.21.15.156:0/2454971200 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x7fb49c06e760 0x7fb49c070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.661+0000 7fb4af7fe700 1 --2- 172.21.15.156:0/2454971200 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb49c0682f0 0x7fb49c06a7a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb4b01f3260 tx=0x7fb4a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:39.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.661+0000 7fb4b4d9c700 1 --2- 172.21.15.156:0/2454971200 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x7fb49c06e760 0x7fb49c070b70 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:39.676 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.662+0000 7fb4acff9700 1 -- 172.21.15.156:0/2454971200 <== osd.7 v2:172.21.15.186:6808/132936 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb49c071240 con 0x7fb49c06e760 2023-11-28T13:15:39.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.677+0000 7fb4b67ff700 1 -- 172.21.15.156:0/2454971200 --> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb4b004e620 con 0x7fb49c06e760 2023-11-28T13:15:39.679 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.678+0000 7fb4acff9700 1 -- 172.21.15.156:0/2454971200 <== osd.7 v2:172.21.15.186:6808/132936 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb4b004e620 con 0x7fb49c06e760 2023-11-28T13:15:39.679 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-11-28T13:15:39.680 INFO:teuthology.orchestra.run.smithi156.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:15:39.680 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-11-28T13:15:39.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.678+0000 7fb4b67ff700 1 -- 172.21.15.156:0/2454971200 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x7fb49c06e760 msgr2=0x7fb49c070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.678+0000 7fb4b67ff700 1 --2- 172.21.15.156:0/2454971200 >> [v2:172.21.15.186:6808/132936,v1:172.21.15.186:6809/132936] conn(0x7fb49c06e760 0x7fb49c070b70 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.678+0000 7fb4b67ff700 1 -- 172.21.15.156:0/2454971200 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb49c0682f0 msgr2=0x7fb49c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.678+0000 7fb4b67ff700 1 --2- 172.21.15.156:0/2454971200 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fb49c0682f0 0x7fb49c06a7a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb4b01f3260 tx=0x7fb4a4009040 comp rx=0 tx=0).stop 2023-11-28T13:15:39.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.678+0000 7fb4b67ff700 1 -- 172.21.15.156:0/2454971200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4b00adb90 msgr2=0x7fb4b014e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.678+0000 7fb4b67ff700 1 --2- 172.21.15.156:0/2454971200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4b00adb90 0x7fb4b014e140 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00011e0 tx=0x7fb4a000ca50 comp rx=0 tx=0).stop 2023-11-28T13:15:39.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.678+0000 7fb4b4d9c700 1 -- 172.21.15.156:0/2454971200 reap_dead start 2023-11-28T13:15:39.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.678+0000 7fb4b67ff700 1 -- 172.21.15.156:0/2454971200 shutdown_connections 2023-11-28T13:15:39.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.678+0000 7fb4b67ff700 1 -- 172.21.15.156:0/2454971200 >> 172.21.15.156:0/2454971200 conn(0x7fb4b009a230 msgr2=0x7fb4b00b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:39.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.678+0000 7fb4b67ff700 1 -- 172.21.15.156:0/2454971200 shutdown_connections 2023-11-28T13:15:39.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.678+0000 7fb4b67ff700 1 -- 172.21.15.156:0/2454971200 wait complete. 2023-11-28T13:15:39.700 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-28T13:15:39.899 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.896+0000 7f0ff19ff700 1 Processor -- start 2023-11-28T13:15:39.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.896+0000 7f0ff19ff700 1 -- start start 2023-11-28T13:15:39.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.897+0000 7f0ff19ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0fec15c9b0 0x7f0fec0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.897+0000 7f0ff19ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fec0cd7c0 0x7f0fec0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.897+0000 7f0ff19ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fec0d1950 con 0x7f0fec161340 2023-11-28T13:15:39.900 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.897+0000 7f0ff19ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fec0b5800 con 0x7f0fec15c9b0 2023-11-28T13:15:39.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.897+0000 7f0ff19ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec0ce140 con 0x7f0fec0cd7c0 2023-11-28T13:15:39.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.898+0000 7f0feb7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fec0cd7c0 0x7f0fec0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.898+0000 7f0feaffd700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0fec15c9b0 0x7f0fec0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.898+0000 7f0feb7fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fec0cd7c0 0x7f0fec0cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.156:55546/0 (socket says 172.21.15.156:55546) 2023-11-28T13:15:39.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.898+0000 7f0feb7fe700 1 -- 172.21.15.156:0/2467940365 learned_addr learned my addr 172.21.15.156:0/2467940365 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:39.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.898+0000 7f0feb7fe700 1 -- 172.21.15.156:0/2467940365 >> v2:172.21.15.186:3300/0 conn(0x7f0fec15c9b0 msgr2=0x7f0fec0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:39.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.898+0000 7f0feb7fe700 1 --2- 172.21.15.156:0/2467940365 >> v2:172.21.15.186:3300/0 conn(0x7f0fec15c9b0 0x7f0fec0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.898+0000 7f0feb7fe700 1 -- 172.21.15.156:0/2467940365 >> v1:172.21.15.186:6789/0 conn(0x7f0fec161340 legacy=0x7f0fec0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.902 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.898+0000 7f0feb7fe700 1 -- 172.21.15.156:0/2467940365 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec0d24f0 con 0x7f0fec0cd7c0 2023-11-28T13:15:39.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.899+0000 7f0feb7fe700 1 --2- 172.21.15.156:0/2467940365 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fec0cd7c0 0x7f0fec0cdc00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009e50 tx=0x7f0fe400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6df4668bd953030 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:39.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.899+0000 7f0fe9ffb700 1 -- 172.21.15.156:0/2467940365 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe400e070 con 0x7f0fec0cd7c0 2023-11-28T13:15:39.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.899+0000 7f0fe9ffb700 1 -- 172.21.15.156:0/2467940365 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe400b910 con 0x7f0fec0cd7c0 2023-11-28T13:15:39.903 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.899+0000 7f0fe9ffb700 1 -- 172.21.15.156:0/2467940365 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe400bcc0 con 0x7f0fec0cd7c0 2023-11-28T13:15:39.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.899+0000 7f0ff19ff700 1 -- 172.21.15.156:0/2467940365 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fec0cd7c0 msgr2=0x7f0fec0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.899+0000 7f0ff19ff700 1 --2- 172.21.15.156:0/2467940365 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fec0cd7c0 0x7f0fec0cdc00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009e50 tx=0x7f0fe400b330 comp rx=0 tx=0).stop 2023-11-28T13:15:39.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.899+0000 7f0ff19ff700 1 -- 172.21.15.156:0/2467940365 shutdown_connections 2023-11-28T13:15:39.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.899+0000 7f0ff19ff700 1 --2- 172.21.15.156:0/2467940365 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fec0cd7c0 0x7f0fec0cdc00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.899+0000 7f0ff19ff700 1 --2- 172.21.15.156:0/2467940365 >> v2:172.21.15.186:3300/0 conn(0x7f0fec15c9b0 0x7f0fec0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.899+0000 7f0ff19ff700 1 -- 172.21.15.156:0/2467940365 >> 172.21.15.156:0/2467940365 conn(0x7f0fec072600 msgr2=0x7f0fec0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:39.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.900+0000 7f0ff19ff700 1 -- 172.21.15.156:0/2467940365 shutdown_connections 2023-11-28T13:15:39.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.900+0000 7f0ff19ff700 1 -- 172.21.15.156:0/2467940365 wait complete. 2023-11-28T13:15:39.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.902+0000 7f0ff19ff700 1 Processor -- start 2023-11-28T13:15:39.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.902+0000 7f0ff19ff700 1 -- start start 2023-11-28T13:15:39.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.903+0000 7f0ff19ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fec0cd7c0 0x7f0fec14a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.903+0000 7f0ff19ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fec15c9b0 0x7f0fec158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.903+0000 7f0ff19ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fec161340 0x7f0fec158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.903+0000 7f0ff19ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec154990 con 0x7f0fec15c9b0 2023-11-28T13:15:39.907 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.903+0000 7f0ff19ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec0cae20 con 0x7f0fec161340 2023-11-28T13:15:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.903+0000 7f0ff19ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec0cb100 con 0x7f0fec0cd7c0 2023-11-28T13:15:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.904+0000 7f0fea7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fec15c9b0 0x7f0fec158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.904+0000 7f0fea7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fec15c9b0 0x7f0fec158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57662/0 (socket says 172.21.15.156:57662) 2023-11-28T13:15:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.904+0000 7f0fea7fc700 1 -- 172.21.15.156:0/2309572277 learned_addr learned my addr 172.21.15.156:0/2309572277 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.904+0000 7f0feaffd700 1 --2- 172.21.15.156:0/2309572277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fec0cd7c0 0x7f0fec14a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.904+0000 7f0feb7fe700 1 --2- 172.21.15.156:0/2309572277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fec161340 0x7f0fec158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.904+0000 7f0fea7fc700 1 -- 172.21.15.156:0/2309572277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fec0cd7c0 msgr2=0x7f0fec14a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.904+0000 7f0fea7fc700 1 --2- 172.21.15.156:0/2309572277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fec0cd7c0 0x7f0fec14a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.904+0000 7f0fea7fc700 1 -- 172.21.15.156:0/2309572277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fec161340 msgr2=0x7f0fec158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:39.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.904+0000 7f0fea7fc700 1 --2- 172.21.15.156:0/2309572277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fec161340 0x7f0fec158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:39.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.904+0000 7f0fea7fc700 1 -- 172.21.15.156:0/2309572277 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe4009b30 con 0x7f0fec15c9b0 2023-11-28T13:15:39.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.904+0000 7f0fea7fc700 1 --2- 172.21.15.156:0/2309572277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fec15c9b0 0x7f0fec158870 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000c4e0 tx=0x7f0fe000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:39.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.905+0000 7f0fd3fff700 1 -- 172.21.15.156:0/2309572277 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe00052d0 con 0x7f0fec15c9b0 2023-11-28T13:15:39.909 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.905+0000 7f0fd3fff700 1 -- 172.21.15.156:0/2309572277 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe0005430 con 0x7f0fec15c9b0 2023-11-28T13:15:39.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.905+0000 7f0fd3fff700 1 -- 172.21.15.156:0/2309572277 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe00038c0 con 0x7f0fec15c9b0 2023-11-28T13:15:39.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.905+0000 7f0ff19ff700 1 -- 172.21.15.156:0/2309572277 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fec0d1300 con 0x7f0fec15c9b0 2023-11-28T13:15:39.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.905+0000 7f0ff19ff700 1 -- 172.21.15.156:0/2309572277 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fec0b5520 con 0x7f0fec15c9b0 2023-11-28T13:15:39.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.906+0000 7f0ff19ff700 1 -- 172.21.15.156:0/2309572277 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fec0d15e0 con 0x7f0fec15c9b0 2023-11-28T13:15:39.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.907+0000 7f0fd3fff700 1 -- 172.21.15.156:0/2309572277 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0fe0019030 con 0x7f0fec15c9b0 2023-11-28T13:15:39.911 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.907+0000 7f0fd3fff700 1 --2- 172.21.15.156:0/2309572277 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0fd80682f0 0x7f0fd806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:39.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.908+0000 7f0fd3fff700 1 -- 172.21.15.156:0/2309572277 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0fe0084e10 con 0x7f0fec15c9b0 2023-11-28T13:15:39.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.908+0000 7f0feaffd700 1 --2- 172.21.15.156:0/2309572277 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0fd80682f0 0x7f0fd806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:39.912 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.909+0000 7f0feaffd700 1 --2- 172.21.15.156:0/2309572277 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0fd80682f0 0x7f0fd806a7a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009ad0 tx=0x7f0fdc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:39.913 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:39.912+0000 7f0fd3fff700 1 -- 172.21.15.156:0/2309572277 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0fe0088030 con 0x7f0fec15c9b0 2023-11-28T13:15:40.104 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.103+0000 7f0ff19ff700 1 -- 172.21.15.156:0/2309572277 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fec154dd0 con 0x7f0fd80682f0 2023-11-28T13:15:40.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.104+0000 7f0fd3fff700 1 -- 172.21.15.156:0/2309572277 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0fec154dd0 con 0x7f0fd80682f0 2023-11-28T13:15:40.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.106+0000 7f0ff19ff700 1 -- 172.21.15.156:0/2309572277 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0fd80682f0 msgr2=0x7f0fd806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:40.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.106+0000 7f0ff19ff700 1 --2- 172.21.15.156:0/2309572277 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0fd80682f0 0x7f0fd806a7a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009ad0 tx=0x7f0fdc008040 comp rx=0 tx=0).stop 2023-11-28T13:15:40.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.106+0000 7f0ff19ff700 1 -- 172.21.15.156:0/2309572277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fec15c9b0 msgr2=0x7f0fec158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:40.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.106+0000 7f0ff19ff700 1 --2- 172.21.15.156:0/2309572277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fec15c9b0 0x7f0fec158870 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000c4e0 tx=0x7f0fe000c8a0 comp rx=0 tx=0).stop 2023-11-28T13:15:40.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.106+0000 7f0ff19ff700 1 -- 172.21.15.156:0/2309572277 shutdown_connections 2023-11-28T13:15:40.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.106+0000 7f0ff19ff700 1 --2- 172.21.15.156:0/2309572277 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f0fd80682f0 0x7f0fd806a7a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:40.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.106+0000 7f0ff19ff700 1 --2- 172.21.15.156:0/2309572277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fec161340 0x7f0fec158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:40.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.106+0000 7f0ff19ff700 1 --2- 172.21.15.156:0/2309572277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fec0cd7c0 0x7f0fec14a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:40.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.106+0000 7f0ff19ff700 1 --2- 172.21.15.156:0/2309572277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fec15c9b0 0x7f0fec158870 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:40.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.106+0000 7f0ff19ff700 1 -- 172.21.15.156:0/2309572277 >> 172.21.15.156:0/2309572277 conn(0x7f0fec072600 msgr2=0x7f0fec0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:40.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.106+0000 7f0ff19ff700 1 -- 172.21.15.156:0/2309572277 shutdown_connections 2023-11-28T13:15:40.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.106+0000 7f0ff19ff700 1 -- 172.21.15.156:0/2309572277 wait complete. 2023-11-28T13:15:40.110 INFO:teuthology.orchestra.run.smithi156.stderr:instructed osd(s) 7 to deep-scrub 2023-11-28T13:15:40.126 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:15:40.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.321+0000 7fc669dff700 1 Processor -- start 2023-11-28T13:15:40.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.321+0000 7fc669dff700 1 -- start start 2023-11-28T13:15:40.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.322+0000 7fc669dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc66415c9b0 0x7fc6640b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:40.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.322+0000 7fc669dff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc664161340 0x7fc6640d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:40.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.323+0000 7fc669dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6640d1950 con 0x7fc6640d2320 2023-11-28T13:15:40.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.323+0000 7fc669dff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6640b5800 con 0x7fc664161340 2023-11-28T13:15:40.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.323+0000 7fc669dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6640b5520 con 0x7fc66415c9b0 2023-11-28T13:15:40.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.323+0000 7fc6637fe700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc66415c9b0 0x7fc6640b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:40.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.323+0000 7fc662ffd700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc664161340 0x7fc6640d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:40.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.323+0000 7fc663fff700 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fc6640d2320 0x7fc6640cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.156:56588/0 (socket says 172.21.15.156:56588) 2023-11-28T13:15:40.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.323+0000 7fc663fff700 1 -- 172.21.15.156:0/2296406808 learned_addr learned my addr 172.21.15.156:0/2296406808 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:15:40.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.324+0000 7fc6627fc700 1 -- 172.21.15.156:0/2296406808 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524035972 0 0) 0x7fc6640d1950 con 0x7fc6640d2320 2023-11-28T13:15:40.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.324+0000 7fc6627fc700 1 -- 172.21.15.156:0/2296406808 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc64c0035b0 con 0x7fc6640d2320 2023-11-28T13:15:40.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.324+0000 7fc662ffd700 1 -- 172.21.15.156:0/2296406808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc66415c9b0 msgr2=0x7fc6640b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:15:40.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.324+0000 7fc662ffd700 1 --2- 172.21.15.156:0/2296406808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc66415c9b0 0x7fc6640b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:40.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.324+0000 7fc662ffd700 1 -- 172.21.15.156:0/2296406808 >> v1:172.21.15.186:6789/0 conn(0x7fc6640d2320 legacy=0x7fc6640cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:40.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.324+0000 7fc662ffd700 1 -- 172.21.15.156:0/2296406808 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6640d1300 con 0x7fc664161340 2023-11-28T13:15:40.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.324+0000 7fc6637fe700 1 --2- 172.21.15.156:0/2296406808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc66415c9b0 0x7fc6640b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:15:40.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.324+0000 7fc662ffd700 1 --2- 172.21.15.156:0/2296406808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc664161340 0x7fc6640d1de0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc658009840 tx=0x7fc658009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=398fd5493db90217 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:40.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.324+0000 7fc6627fc700 1 -- 172.21.15.156:0/2296406808 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658011830 con 0x7fc664161340 2023-11-28T13:15:40.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.324+0000 7fc6627fc700 1 -- 172.21.15.156:0/2296406808 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc658011990 con 0x7fc664161340 2023-11-28T13:15:40.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.324+0000 7fc6627fc700 1 -- 172.21.15.156:0/2296406808 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658011d00 con 0x7fc664161340 2023-11-28T13:15:40.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.325+0000 7fc669dff700 1 -- 172.21.15.156:0/2296406808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc664161340 msgr2=0x7fc6640d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:40.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.325+0000 7fc669dff700 1 --2- 172.21.15.156:0/2296406808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc664161340 0x7fc6640d1de0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc658009840 tx=0x7fc658009b50 comp rx=0 tx=0).stop 2023-11-28T13:15:40.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.325+0000 7fc669dff700 1 -- 172.21.15.156:0/2296406808 shutdown_connections 2023-11-28T13:15:40.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.325+0000 7fc669dff700 1 --2- 172.21.15.156:0/2296406808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc66415c9b0 0x7fc6640b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:40.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.325+0000 7fc669dff700 1 --2- 172.21.15.156:0/2296406808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc664161340 0x7fc6640d1de0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:40.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.325+0000 7fc669dff700 1 -- 172.21.15.156:0/2296406808 >> 172.21.15.156:0/2296406808 conn(0x7fc664072600 msgr2=0x7fc6640cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:40.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.325+0000 7fc669dff700 1 -- 172.21.15.156:0/2296406808 shutdown_connections 2023-11-28T13:15:40.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.325+0000 7fc669dff700 1 -- 172.21.15.156:0/2296406808 wait complete. 2023-11-28T13:15:40.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.327+0000 7fc669dff700 1 Processor -- start 2023-11-28T13:15:40.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.327+0000 7fc669dff700 1 -- start start 2023-11-28T13:15:40.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.328+0000 7fc669dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6640d2320 0x7fc6640c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:40.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.328+0000 7fc669dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc66415c9b0 0x7fc6640c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:40.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.328+0000 7fc669dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc664161340 0x7fc66415f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:40.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.328+0000 7fc669dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6641529c0 con 0x7fc664161340 2023-11-28T13:15:40.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.328+0000 7fc669dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6640c7d70 con 0x7fc6640d2320 2023-11-28T13:15:40.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.328+0000 7fc669dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6640c74d0 con 0x7fc66415c9b0 2023-11-28T13:15:40.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.328+0000 7fc663fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc664161340 0x7fc66415f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:40.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.328+0000 7fc662ffd700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc66415c9b0 0x7fc6640c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:40.333 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.328+0000 7fc663fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc664161340 0x7fc66415f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57674/0 (socket says 172.21.15.156:57674) 2023-11-28T13:15:40.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.328+0000 7fc663fff700 1 -- 172.21.15.156:0/1536799385 learned_addr learned my addr 172.21.15.156:0/1536799385 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:15:40.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.328+0000 7fc6637fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6640d2320 0x7fc6640c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:40.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.329+0000 7fc663fff700 1 -- 172.21.15.156:0/1536799385 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc66415c9b0 msgr2=0x7fc6640c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:40.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.329+0000 7fc663fff700 1 --2- 172.21.15.156:0/1536799385 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc66415c9b0 0x7fc6640c99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:40.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.329+0000 7fc663fff700 1 -- 172.21.15.156:0/1536799385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6640d2320 msgr2=0x7fc6640c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:40.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.329+0000 7fc663fff700 1 --2- 172.21.15.156:0/1536799385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6640d2320 0x7fc6640c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:40.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.329+0000 7fc663fff700 1 -- 172.21.15.156:0/1536799385 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6540094a0 con 0x7fc664161340 2023-11-28T13:15:40.334 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.329+0000 7fc663fff700 1 --2- 172.21.15.156:0/1536799385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc664161340 0x7fc66415f170 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00caa0 tx=0x7fc65c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:40.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.329+0000 7fc64bfff700 1 -- 172.21.15.156:0/1536799385 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c00a9e0 con 0x7fc664161340 2023-11-28T13:15:40.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.329+0000 7fc64bfff700 1 -- 172.21.15.156:0/1536799385 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c00ab40 con 0x7fc664161340 2023-11-28T13:15:40.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.329+0000 7fc669dff700 1 -- 172.21.15.156:0/1536799385 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6580094a0 con 0x7fc664161340 2023-11-28T13:15:40.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.329+0000 7fc64bfff700 1 -- 172.21.15.156:0/1536799385 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c012400 con 0x7fc664161340 2023-11-28T13:15:40.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.329+0000 7fc669dff700 1 -- 172.21.15.156:0/1536799385 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6640b5130 con 0x7fc664161340 2023-11-28T13:15:40.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.330+0000 7fc669dff700 1 -- 172.21.15.156:0/1536799385 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6640d15e0 con 0x7fc664161340 2023-11-28T13:15:40.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.331+0000 7fc64bfff700 1 -- 172.21.15.156:0/1536799385 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc65c019030 con 0x7fc664161340 2023-11-28T13:15:40.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.332+0000 7fc64bfff700 1 --2- 172.21.15.156:0/1536799385 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6500682f0 0x7fc65006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:15:40.336 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.332+0000 7fc64bfff700 1 -- 172.21.15.156:0/1536799385 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc65c085480 con 0x7fc664161340 2023-11-28T13:15:40.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.333+0000 7fc6637fe700 1 --2- 172.21.15.156:0/1536799385 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6500682f0 0x7fc65006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:15:40.337 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.334+0000 7fc6637fe700 1 --2- 172.21.15.156:0/1536799385 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6500682f0 0x7fc65006a7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc654009cd0 tx=0x7fc654009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:15:40.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.336+0000 7fc64bfff700 1 -- 172.21.15.156:0/1536799385 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc65c052060 con 0x7fc664161340 2023-11-28T13:15:40.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.526+0000 7fc669dff700 1 -- 172.21.15.156:0/1536799385 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6641527b0 con 0x7fc6500682f0 2023-11-28T13:15:40.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.530+0000 7fc64bfff700 1 -- 172.21.15.156:0/1536799385 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72912 (secure 0 0 0) 0x7fc654009e10 con 0x7fc6500682f0 2023-11-28T13:15:40.532 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:15:40.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.532+0000 7fc669dff700 1 -- 172.21.15.156:0/1536799385 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6500682f0 msgr2=0x7fc65006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:40.533 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.532+0000 7fc669dff700 1 --2- 172.21.15.156:0/1536799385 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6500682f0 0x7fc65006a7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc654009cd0 tx=0x7fc654009210 comp rx=0 tx=0).stop 2023-11-28T13:15:40.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.532+0000 7fc669dff700 1 -- 172.21.15.156:0/1536799385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc664161340 msgr2=0x7fc66415f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:15:40.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.532+0000 7fc669dff700 1 --2- 172.21.15.156:0/1536799385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc664161340 0x7fc66415f170 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00caa0 tx=0x7fc65c00cdb0 comp rx=0 tx=0).stop 2023-11-28T13:15:40.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.532+0000 7fc669dff700 1 -- 172.21.15.156:0/1536799385 shutdown_connections 2023-11-28T13:15:40.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.532+0000 7fc669dff700 1 --2- 172.21.15.156:0/1536799385 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7fc6500682f0 0x7fc65006a7a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:40.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.532+0000 7fc669dff700 1 --2- 172.21.15.156:0/1536799385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc664161340 0x7fc66415f170 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:40.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.532+0000 7fc669dff700 1 --2- 172.21.15.156:0/1536799385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6640d2320 0x7fc6640c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:40.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.532+0000 7fc669dff700 1 --2- 172.21.15.156:0/1536799385 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc66415c9b0 0x7fc6640c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:15:40.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.532+0000 7fc669dff700 1 -- 172.21.15.156:0/1536799385 >> 172.21.15.156:0/1536799385 conn(0x7fc664072600 msgr2=0x7fc66415cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:15:40.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.532+0000 7fc669dff700 1 -- 172.21.15.156:0/1536799385 shutdown_connections 2023-11-28T13:15:40.535 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:15:40.532+0000 7fc669dff700 1 -- 172.21.15.156:0/1536799385 wait complete. 2023-11-28T13:15:40.535 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T13:15:40.551 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2023-11-28T13:15:40.135562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":100868,"kb_used_data":7236,"kb_used_omap":0,"kb_used_meta":93568,"kb_avail":754873852,"statfs":{"total":773094113280,"available":772990824448,"internally_reserved":0,"allocated":7409664,"data_stored":3925110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95813632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.516689"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:15:25.993449+0000","last_change":"2023-11-28T13:07:56.890378+0000","last_active":"2023-11-28T13:15:25.993449+0000","last_peered":"2023-11-28T13:15:25.993449+0000","last_clean":"2023-11-28T13:15:25.993449+0000","last_became_active":"2023-11-28T13:07:56.890058+0000","last_became_peered":"2023-11-28T13:07:56.890058+0000","last_unstale":"2023-11-28T13:15:25.993449+0000","last_undegraded":"2023-11-28T13:15:25.993449+0000","last_fullsized":"2023-11-28T13:15:25.993449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:05.344602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":56,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:29.649906+0000","last_change":"2023-11-28T13:07:56.844509+0000","last_active":"2023-11-28T13:15:29.649906+0000","last_peered":"2023-11-28T13:15:29.649906+0000","last_clean":"2023-11-28T13:15:29.649906+0000","last_became_active":"2023-11-28T13:07:56.844081+0000","last_became_peered":"2023-11-28T13:07:56.844081+0000","last_unstale":"2023-11-28T13:15:29.649906+0000","last_undegraded":"2023-11-28T13:15:29.649906+0000","last_fullsized":"2023-11-28T13:15:29.649906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":54,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.858689+0000","last_change":"2023-11-28T13:07:56.844699+0000","last_active":"2023-11-28T13:15:28.858689+0000","last_peered":"2023-11-28T13:15:28.858689+0000","last_clean":"2023-11-28T13:15:28.858689+0000","last_became_active":"2023-11-28T13:07:56.844391+0000","last_became_peered":"2023-11-28T13:07:56.844391+0000","last_unstale":"2023-11-28T13:15:28.858689+0000","last_undegraded":"2023-11-28T13:15:28.858689+0000","last_fullsized":"2023-11-28T13:15:28.858689+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:08.637809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":205,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.619311+0000","last_change":"2023-11-28T13:07:57.844020+0000","last_active":"2023-11-28T13:15:28.619311+0000","last_peered":"2023-11-28T13:15:28.619311+0000","last_clean":"2023-11-28T13:15:28.619311+0000","last_became_active":"2023-11-28T13:07:57.842981+0000","last_became_peered":"2023-11-28T13:07:57.842981+0000","last_unstale":"2023-11-28T13:15:28.619311+0000","last_undegraded":"2023-11-28T13:15:28.619311+0000","last_fullsized":"2023-11-28T13:15:28.619311+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:56.825711+0000","last_clean_scrub_stamp":"2023-11-28T13:07:56.825711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:44.973218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":56,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.621568+0000","last_change":"2023-11-28T13:07:57.306318+0000","last_active":"2023-11-28T13:15:28.621568+0000","last_peered":"2023-11-28T13:15:28.621568+0000","last_clean":"2023-11-28T13:15:28.621568+0000","last_became_active":"2023-11-28T13:07:57.305937+0000","last_became_peered":"2023-11-28T13:07:57.305937+0000","last_unstale":"2023-11-28T13:15:28.621568+0000","last_undegraded":"2023-11-28T13:15:28.621568+0000","last_fullsized":"2023-11-28T13:15:28.621568+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":56,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.619321+0000","last_change":"2023-11-28T13:07:57.306277+0000","last_active":"2023-11-28T13:15:28.619321+0000","last_peered":"2023-11-28T13:15:28.619321+0000","last_clean":"2023-11-28T13:15:28.619321+0000","last_became_active":"2023-11-28T13:07:57.305983+0000","last_became_peered":"2023-11-28T13:07:57.305983+0000","last_unstale":"2023-11-28T13:15:28.619321+0000","last_undegraded":"2023-11-28T13:15:28.619321+0000","last_fullsized":"2023-11-28T13:15:28.619321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.618799+0000","last_change":"2023-11-28T13:07:57.306177+0000","last_active":"2023-11-28T13:15:28.618799+0000","last_peered":"2023-11-28T13:15:28.618799+0000","last_clean":"2023-11-28T13:15:28.618799+0000","last_became_active":"2023-11-28T13:07:57.305853+0000","last_became_peered":"2023-11-28T13:07:57.305853+0000","last_unstale":"2023-11-28T13:15:28.618799+0000","last_undegraded":"2023-11-28T13:15:28.618799+0000","last_fullsized":"2023-11-28T13:15:28.618799+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:59.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":56,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.619919+0000","last_change":"2023-11-28T13:07:57.306321+0000","last_active":"2023-11-28T13:15:28.619919+0000","last_peered":"2023-11-28T13:15:28.619919+0000","last_clean":"2023-11-28T13:15:28.619919+0000","last_became_active":"2023-11-28T13:07:57.305923+0000","last_became_peered":"2023-11-28T13:07:57.305923+0000","last_unstale":"2023-11-28T13:15:28.619919+0000","last_undegraded":"2023-11-28T13:15:28.619919+0000","last_fullsized":"2023-11-28T13:15:28.619919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:41.104679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":56,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:15:28.620302+0000","last_change":"2023-11-28T13:07:56.844305+0000","last_active":"2023-11-28T13:15:28.620302+0000","last_peered":"2023-11-28T13:15:28.620302+0000","last_clean":"2023-11-28T13:15:28.620302+0000","last_became_active":"2023-11-28T13:07:56.843980+0000","last_became_peered":"2023-11-28T13:07:56.843980+0000","last_unstale":"2023-11-28T13:15:28.620302+0000","last_undegraded":"2023-11-28T13:15:28.620302+0000","last_fullsized":"2023-11-28T13:15:28.620302+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:04.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574944,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15108,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94356732,"statfs":{"total":96636764160,"available":96621293568,"internally_reserved":0,"allocated":1372160,"data_stored":933538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68700000000000006,"15min":0.66900000000000004},"min":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89900000000000002,"5min":0.91200000000000003,"15min":0.98399999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68999999999999995,"15min":0.64800000000000002},"min":{"1min":0.498,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82499999999999996,"5min":0.92300000000000004,"15min":1.0569999999999999},"last":0.63100000000000001}]},{"osd":1,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73399999999999999,"15min":0.72299999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84199999999999997,"5min":0.93999999999999995,"15min":1.0289999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.73999999999999999,"15min":0.71399999999999997},"min":{"1min":0.57599999999999996,"5min":0.42399999999999999,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":0.95999999999999996,"15min":1.095},"last":0.69799999999999995}]},{"osd":2,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.66500000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85499999999999998,"5min":0.92300000000000004,"15min":0.996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72599999999999998,"15min":0.69499999999999995},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.61699999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.68999999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.42999999999999999},"max":{"1min":0.89000000000000001,"5min":0.91600000000000004,"15min":0.93899999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73799999999999999,"15min":0.76200000000000001},"min":{"1min":0.60399999999999998,"5min":0.52200000000000002,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.98599999999999999,"15min":1.079},"last":0.51300000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.66400000000000003},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":1.008},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.628,"15min":0.70799999999999996},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":1.006,"15min":1.071},"last":0.72299999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64200000000000002,"15min":0.73499999999999999},"min":{"1min":0.501,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86399999999999999,"5min":0.93600000000000005,"15min":1.1830000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65300000000000002,"15min":0.75900000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96299999999999997,"5min":1.044,"15min":1.1839999999999999},"last":0.79600000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71799999999999997,"15min":0.80600000000000005},"min":{"1min":0.56100000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94099999999999995,"5min":0.98599999999999999,"15min":1.165},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70199999999999996,"15min":0.76500000000000001},"min":{"1min":0.50700000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92800000000000005,"5min":0.95499999999999996,"15min":1.141},"last":0.60499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574943,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9536,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362304,"statfs":{"total":96636764160,"available":96626999296,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.68700000000000006,"15min":0.67700000000000005},"min":{"1min":0.441,"5min":0.432,"15min":0.41599999999999998},"max":{"1min":0.88300000000000001,"5min":0.995,"15min":0.995},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70399999999999996,"15min":0.66400000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91300000000000003,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.93700000000000006}]},{"osd":1,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70899999999999996,"15min":0.68700000000000006},"min":{"1min":0.56100000000000005,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.86899999999999999,"5min":1.113,"15min":1.113},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70699999999999996,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.48599999999999999,"15min":0.47899999999999998},"max":{"1min":0.85499999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.88}]},{"osd":2,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72299999999999998,"15min":0.68899999999999995},"min":{"1min":0.60399999999999998,"5min":0.45000000000000001,"15min":0.41799999999999998},"max":{"1min":0.90300000000000002,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72099999999999997,"15min":0.68500000000000005},"min":{"1min":0.51000000000000001,"5min":0.495,"15min":0.38500000000000001},"max":{"1min":0.89400000000000002,"5min":1.024,"15min":1.024},"last":0.96799999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.73599999999999999,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":1.121,"15min":1.121},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.745,"15min":0.73799999999999999},"min":{"1min":0.63,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.89200000000000002,"5min":1.167,"15min":1.167},"last":0.92000000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.624,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92300000000000004,"5min":1.042,"15min":1.0429999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.79200000000000004},"min":{"1min":0.54200000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96499999999999997,"5min":1.105,"15min":1.3140000000000001},"last":0.90600000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63500000000000001,"15min":0.68500000000000005},"min":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84599999999999997,"5min":0.92100000000000004,"15min":1.0780000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.70699999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70499999999999996,"15min":0.72899999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93400000000000005,"5min":1.034,"15min":1.0980000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73399999999999999,"15min":0.76800000000000002},"min":{"1min":0.501,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94899999999999995,"5min":1.0129999999999999,"15min":1.107},"last":0.76000000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14916,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":13568,"kb_avail":94356924,"statfs":{"total":96636764160,"available":96621490176,"internally_reserved":0,"allocated":1372160,"data_stored":933538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13893632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41799999999999998,"15min":0.43099999999999999},"min":{"1min":0.28100000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.496,"5min":0.75900000000000001,"15min":1.28},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.47899999999999998,"15min":0.45600000000000002},"min":{"1min":0.30499999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64000000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.64000000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.46000000000000002},"min":{"1min":0.33200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60599999999999998,"5min":0.747,"15min":0.747},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.67500000000000004,"5min":0.77000000000000002,"15min":0.84699999999999998},"last":0.59399999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.436,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71099999999999997,"5min":0.88500000000000001,"15min":1.0009999999999999},"last":0.626},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52000000000000002,"15min":0.53800000000000003},"min":{"1min":0.36899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.68899999999999995,"5min":0.746,"15min":1.1120000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73099999999999998,"15min":0.70799999999999996},"min":{"1min":0.60299999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.753,"15min":0.72199999999999998},"min":{"1min":0.624,"5min":0.54100000000000004,"15min":0.49099999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82099999999999995}]},{"osd":5,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.74299999999999999,"15min":0.745},"min":{"1min":0.61199999999999999,"5min":0.51700000000000002,"15min":0.433},"max":{"1min":1.0209999999999999,"5min":1.292,"15min":1.292},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.74299999999999999,"15min":0.72099999999999997},"min":{"1min":0.621,"5min":0.441,"15min":0.441},"max":{"1min":1.141,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.69499999999999995}]},{"osd":6,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.74199999999999999},"min":{"1min":0.64300000000000002,"5min":0.55300000000000005,"15min":0.52700000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.47299999999999998},"max":{"1min":1.1240000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.72399999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.80000000000000004,"15min":0.81000000000000005},"min":{"1min":0.66900000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0820000000000001,"5min":1.357,"15min":1.357},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77800000000000002,"15min":0.78400000000000003},"min":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0389999999999999,"5min":1.24,"15min":1.3580000000000001},"last":0.76600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13884,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357956,"statfs":{"total":96636764160,"available":96622546944,"internally_reserved":0,"allocated":774144,"data_stored":342184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.47799999999999998},"min":{"1min":0.32500000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65400000000000003,"5min":0.69699999999999995,"15min":1.3120000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47999999999999998,"15min":0.51500000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67000000000000004,"5min":0.70599999999999996,"15min":1.48},"last":0.35199999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.504,"15min":0.52100000000000002},"min":{"1min":0.41999999999999998,"5min":0.34200000000000003,"15min":0.33000000000000002},"max":{"1min":0.76700000000000002,"5min":0.874,"15min":0.874},"last":0.505},{"interface":"front","average":{"1min":0.5,"5min":0.53000000000000003,"15min":0.51400000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.749,"5min":0.749,"15min":0.80800000000000005},"last":0.53300000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.35599999999999998,"5min":0.33700000000000002,"15min":0.30099999999999999},"max":{"1min":0.63800000000000001,"5min":0.747,"15min":0.747},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56399999999999995,"15min":0.55500000000000005},"min":{"1min":0.42099999999999999,"5min":0.36699999999999999,"15min":0.35399999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.82199999999999995},"last":0.54600000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.75900000000000001,"15min":0.77500000000000002},"min":{"1min":0.54500000000000004,"5min":0.48399999999999999,"15min":0.45500000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.071},"last":1.03},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.99099999999999999,"5min":1.147,"15min":1.147},"last":1.133}]},{"osd":5,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77300000000000002,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.48299999999999998},"max":{"1min":1.0660000000000001,"5min":1.1040000000000001,"15min":1.3260000000000001},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0309999999999999,"5min":1.127,"15min":1.127},"last":1.097}]},{"osd":6,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.77300000000000002,"15min":0.76100000000000001},"min":{"1min":0.58099999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0660000000000001,"5min":1.127,"15min":1.127},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80800000000000005,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.042,"5min":1.0489999999999999,"15min":1.097},"last":1.0429999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78500000000000003,"15min":0.80000000000000004},"min":{"1min":0.58999999999999997,"5min":0.55600000000000005,"15min":0.53700000000000003},"max":{"1min":1.1200000000000001,"5min":1.149,"15min":1.149},"last":1.056},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79600000000000004,"15min":0.80200000000000005},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.996,"5min":1.163,"15min":1.163},"last":1.01}]}]},{"osd":2,"up_from":13,"seq":55834574943,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13696,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":12928,"kb_avail":94358144,"statfs":{"total":96636764160,"available":96622739456,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13238272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.48099999999999998,"15min":0.44500000000000001},"min":{"1min":0.28499999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.68400000000000005,"5min":6.4130000000000003,"15min":6.4130000000000003},"last":0.435},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.64200000000000002,"15min":0.52200000000000002},"min":{"1min":0.32700000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.625,"5min":19.190000000000001,"15min":19.190000000000001},"last":0.28999999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.33100000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66100000000000003,"5min":5.2009999999999996,"15min":5.2009999999999996},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.68899999999999995,"15min":0.57099999999999995},"min":{"1min":0.40300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.67900000000000005,"5min":19.241,"15min":19.241},"last":0.41199999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55700000000000005,"15min":0.48499999999999999},"min":{"1min":0.34999999999999998,"5min":0.29899999999999999,"15min":0.29799999999999999},"max":{"1min":0.66500000000000004,"5min":7.0410000000000004,"15min":7.0410000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.65200000000000002,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70499999999999996,"5min":11.420999999999999,"15min":11.420999999999999},"last":0.50600000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.84199999999999997,"15min":0.82999999999999996},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.55000000000000004},"max":{"1min":1.006,"5min":5.1630000000000003,"15min":5.1630000000000003},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.80400000000000005,"15min":0.79600000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.496},"max":{"1min":0.95899999999999996,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":0.48999999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.75700000000000001,"15min":0.73999999999999999},"min":{"1min":0.58499999999999996,"5min":0.505,"15min":0.46100000000000002},"max":{"1min":0.86899999999999999,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.81100000000000005,"15min":0.76100000000000001},"min":{"1min":0.52900000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.036,"5min":6.7569999999999997,"15min":6.7569999999999997},"last":0.59699999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.83299999999999996,"15min":0.77700000000000002},"min":{"1min":0.51100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97099999999999997,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.83799999999999997,"15min":0.80800000000000005},"min":{"1min":0.64200000000000002,"5min":0.55700000000000005,"15min":0.49199999999999999},"max":{"1min":0.95399999999999996,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":0.61199999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.79100000000000004,"15min":0.77000000000000002},"min":{"1min":0.57799999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.996,"5min":2.008,"15min":2.008},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.82399999999999995,"15min":0.80800000000000005},"min":{"1min":0.56000000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1379999999999999,"5min":6.6769999999999996,"15min":6.6769999999999996},"last":0.623}]}]},{"osd":3,"up_from":13,"seq":55834574943,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14016,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":13248,"kb_avail":94357824,"statfs":{"total":96636764160,"available":96622411776,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13565952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.33100000000000002,"5min":0.26100000000000001,"15min":0.245},"max":{"1min":0.61699999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.48299999999999998,"15min":0.45700000000000002},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.27300000000000002},"max":{"1min":0.69799999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.46899999999999997,"15min":0.48099999999999998},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71199999999999997,"5min":0.73699999999999999,"15min":0.88100000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.41099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.48099999999999998,"5min":0.372,"15min":0.36399999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56999999999999995,"15min":0.54900000000000004},"min":{"1min":0.40300000000000002,"5min":0.39000000000000001,"15min":0.35399999999999998},"max":{"1min":0.80500000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.61099999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72199999999999998,"15min":0.70499999999999996},"min":{"1min":0.54400000000000004,"5min":0.50900000000000001,"15min":0.50600000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.73399999999999999,"15min":0.71399999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.46200000000000002},"max":{"1min":1.054,"5min":1.087,"15min":1.087},"last":0.73699999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.091},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.77500000000000002,"15min":0.76600000000000001},"min":{"1min":0.63400000000000001,"5min":0.50600000000000001,"15min":0.498},"max":{"1min":1.0369999999999999,"5min":1.0649999999999999,"15min":1.0680000000000001},"last":0.751}]},{"osd":6,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76600000000000001,"15min":0.73499999999999999},"min":{"1min":0.65500000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96799999999999997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.76100000000000001,"15min":0.71999999999999997},"min":{"1min":0.59299999999999997,"5min":0.50600000000000001,"15min":0.46100000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.57799999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79900000000000004,"15min":0.76900000000000002},"min":{"1min":0.64600000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.032,"5min":1.05,"15min":1.05},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.80000000000000004,"15min":0.77300000000000002},"min":{"1min":0.68400000000000005,"5min":0.53600000000000003,"15min":0.52800000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.1299999999999999},"last":0.69899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574942,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9920,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361920,"statfs":{"total":96636764160,"available":96626606080,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70499999999999996,"15min":0.63},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.91900000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72099999999999997,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.42799999999999999,"15min":0.38400000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":1.3089999999999999},"last":0.52100000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73499999999999999,"15min":0.69699999999999995},"min":{"1min":0.59399999999999997,"5min":0.51200000000000001,"15min":0.48299999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":1.161},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69399999999999995,"15min":0.63},"min":{"1min":0.47899999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88900000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.625}]},{"osd":2,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":0.67800000000000005},"min":{"1min":0.46400000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.69899999999999995},"min":{"1min":0.502,"5min":0.47599999999999998,"15min":0.441},"max":{"1min":0.91500000000000004,"5min":1.042,"15min":1.042},"last":0.58699999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71699999999999997,"15min":0.68600000000000005},"min":{"1min":0.48699999999999999,"5min":0.46000000000000002,"15min":0.438},"max":{"1min":0.85499999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.75700000000000001,"15min":0.72799999999999998},"min":{"1min":0.54500000000000004,"5min":0.52900000000000003,"15min":0.505},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65100000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.66000000000000003},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95799999999999996,"5min":1.012,"15min":1.264},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64500000000000002,"15min":0.70299999999999996},"min":{"1min":0.49399999999999999,"5min":0.39100000000000001,"15min":0.38400000000000001},"max":{"1min":0.81999999999999995,"5min":0.93200000000000005,"15min":1.079},"last":0.67400000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65600000000000003,"15min":0.70799999999999996},"min":{"1min":0.46300000000000002,"5min":0.439,"15min":0.32700000000000001},"max":{"1min":0.89000000000000001,"5min":0.91200000000000003,"15min":1.022},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69799999999999995,"15min":0.73099999999999998},"min":{"1min":0.53800000000000003,"5min":0.48699999999999999,"15min":0.47399999999999998},"max":{"1min":0.90900000000000003,"5min":1.044,"15min":1.22},"last":0.68500000000000005}]},{"osd":7,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69099999999999995,"15min":0.745},"min":{"1min":0.51700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98699999999999999,"5min":0.995,"15min":1.4530000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72899999999999998,"15min":0.76700000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.40899999999999997},"max":{"1min":1.0089999999999999,"5min":1.018,"15min":1.018},"last":0.61199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574943,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9792,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362048,"statfs":{"total":96636764160,"available":96626737152,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69199999999999995,"15min":0.63300000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92500000000000004,"5min":1.0640000000000001,"15min":1.659},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69699999999999995,"15min":0.63400000000000001},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.38900000000000001},"max":{"1min":0.95899999999999996,"5min":0.97999999999999998,"15min":2.3050000000000002},"last":0.66800000000000004}]},{"osd":1,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66900000000000004,"15min":0.62},"min":{"1min":0.44600000000000001,"5min":0.38200000000000001,"15min":0.35499999999999998},"max":{"1min":0.94299999999999995,"5min":0.95699999999999996,"15min":1.2390000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72099999999999997,"15min":0.65300000000000002},"min":{"1min":0.54300000000000004,"5min":0.40500000000000003,"15min":0.39400000000000002},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.9630000000000001},"last":0.65400000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69699999999999995,"15min":0.65700000000000003},"min":{"1min":0.53500000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.86899999999999999,"5min":1.0469999999999999,"15min":2.3860000000000001},"last":0.626},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71199999999999997,"15min":0.66400000000000003},"min":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.432},"max":{"1min":0.89300000000000002,"5min":0.97299999999999998,"15min":1.0800000000000001},"last":0.64200000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71399999999999997,"15min":0.65900000000000003},"min":{"1min":0.53800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":1.0169999999999999,"15min":1.8140000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73699999999999999,"15min":0.71399999999999997},"min":{"1min":0.5,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91700000000000004,"5min":0.96699999999999997,"15min":2.3500000000000001},"last":0.56299999999999994}]},{"osd":4,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63200000000000001,"15min":0.65300000000000002},"min":{"1min":0.495,"5min":0.379,"15min":0.379},"max":{"1min":0.79900000000000004,"5min":0.95999999999999996,"15min":1.702},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70799999999999996,"15min":0.72799999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89000000000000001,"5min":1.0529999999999999,"15min":2.0009999999999999},"last":0.71999999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64500000000000002,"15min":0.69599999999999995},"min":{"1min":0.437,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76700000000000002,"5min":0.89400000000000002,"15min":2.4969999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66900000000000004,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86899999999999999,"5min":1.012,"15min":2.0880000000000001},"last":0.51200000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69499999999999995,"15min":0.71499999999999997},"min":{"1min":0.53100000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93999999999999995,"5min":0.999,"15min":2.0449999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.49199999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.85099999999999998,"5min":0.98099999999999998,"15min":2.4249999999999998},"last":0.70899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:15:40.556 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-11-28T13:07:55.823449+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=55, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=15, tm_sec=33, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:15:40.556 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2023-11-28T13:07:55.823449+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=55, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=15, tm_sec=33, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:15:40.556 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2023-11-28T13:07:55.823449+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=55, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=15, tm_sec=33, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:15:40.557 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-28T13:07:56.825711+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=56, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=15, tm_sec=33, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:15:40.557 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-11-28T13:07:55.823449+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=55, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=15, tm_sec=33, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:15:40.557 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-28T13:07:55.823449+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=55, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=15, tm_sec=33, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:15:40.557 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-11-28T13:07:55.823449+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=55, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=15, tm_sec=33, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:15:40.557 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-11-28T13:07:55.823449+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=55, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=15, tm_sec=33, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:15:40.558 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-11-28T13:07:55.823449+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=55, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=15, tm_sec=33, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:15:40.558 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-28T13:16:00.558 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:16:00.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.766+0000 7f62a4dff700 1 Processor -- start 2023-11-28T13:16:00.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.766+0000 7f62a4dff700 1 -- start start 2023-11-28T13:16:00.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.767+0000 7f62a4dff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f62a0158ce0 0x7f62a015d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:00.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.767+0000 7f62a4dff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a006bf10 con 0x7f62a006fb30 2023-11-28T13:16:00.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.767+0000 7f62a4dff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a015d730 con 0x7f62a00ce4b0 2023-11-28T13:16:00.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.767+0000 7f62a4dff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a01590b0 con 0x7f62a0158ce0 2023-11-28T13:16:00.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.768+0000 7f6297fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f62a006fb30 0x7f62a00cdda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:55398/0 (socket says 172.21.15.156:55398) 2023-11-28T13:16:00.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.768+0000 7f6297fff700 1 -- 172.21.15.156:0/1724321508 learned_addr learned my addr 172.21.15.156:0/1724321508 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:16:00.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.768+0000 7f629e59c700 1 --2- 172.21.15.156:0/1724321508 >> v2:172.21.15.186:3300/0 conn(0x7f62a0158ce0 0x7f62a015d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:00.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.768+0000 7f629dd9b700 1 -- 172.21.15.156:0/1724321508 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214588260 0 0) 0x7f62a006bf10 con 0x7f62a006fb30 2023-11-28T13:16:00.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.768+0000 7f629dd9b700 1 -- 172.21.15.156:0/1724321508 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f628c0035b0 con 0x7f62a006fb30 2023-11-28T13:16:00.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.768+0000 7f629dd9b700 1 -- 172.21.15.156:0/1724321508 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138128427 0 0) 0x7f62a015d730 con 0x7f62a00ce4b0 2023-11-28T13:16:00.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.768+0000 7f629dd9b700 1 -- 172.21.15.156:0/1724321508 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62a006bf10 con 0x7f62a00ce4b0 2023-11-28T13:16:00.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.768+0000 7f629dd9b700 1 -- 172.21.15.156:0/1724321508 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 578507485 0 0) 0x7f628c0035b0 con 0x7f62a006fb30 2023-11-28T13:16:00.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.768+0000 7f629dd9b700 1 -- 172.21.15.156:0/1724321508 >> v2:172.21.15.186:3300/0 conn(0x7f62a0158ce0 msgr2=0x7f62a015d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:16:00.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.768+0000 7f629dd9b700 1 --2- 172.21.15.156:0/1724321508 >> v2:172.21.15.186:3300/0 conn(0x7f62a0158ce0 0x7f62a015d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:00.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.768+0000 7f629dd9b700 1 -- 172.21.15.156:0/1724321508 >> v1:172.21.15.186:6789/0 conn(0x7f62a00ce4b0 legacy=0x7f62a006c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:00.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.768+0000 7f629dd9b700 1 -- 172.21.15.156:0/1724321508 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a00cb0d0 con 0x7f62a006fb30 2023-11-28T13:16:00.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f629dd9b700 1 -- 172.21.15.156:0/1724321508 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f6288001670 con 0x7f62a006fb30 2023-11-28T13:16:00.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f629dd9b700 1 -- 172.21.15.156:0/1724321508 >> v1:172.21.15.156:6789/0 conn(0x7f62a006fb30 legacy=0x7f62a00cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:00.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f629dd9b700 1 --2- 172.21.15.156:0/1724321508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628c003be0 0x7f628c003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:00.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f629dd9b700 1 --2- 172.21.15.156:0/1724321508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f628c004850 0x7f628c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:00.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f629dd9b700 1 --2- 172.21.15.156:0/1724321508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f628c005200 0x7f628c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:00.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f629dd9b700 1 -- 172.21.15.156:0/1724321508 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a01590b0 con 0x7f628c003be0 2023-11-28T13:16:00.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f629dd9b700 1 -- 172.21.15.156:0/1724321508 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c00c0e0 con 0x7f628c004850 2023-11-28T13:16:00.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f629dd9b700 1 -- 172.21.15.156:0/1724321508 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c00c270 con 0x7f628c005200 2023-11-28T13:16:00.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f629e59c700 1 --2- 172.21.15.156:0/1724321508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628c003be0 0x7f628c003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:00.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f6297fff700 1 --2- 172.21.15.156:0/1724321508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f628c004850 0x7f628c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:00.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f629ed9d700 1 --2- 172.21.15.156:0/1724321508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f628c005200 0x7f628c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:00.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f629e59c700 1 -- 172.21.15.156:0/1724321508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f628c005200 msgr2=0x7f628c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:16:00.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f629e59c700 1 --2- 172.21.15.156:0/1724321508 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f628c005200 0x7f628c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:00.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f629e59c700 1 -- 172.21.15.156:0/1724321508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f628c004850 msgr2=0x7f628c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:16:00.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f629e59c700 1 --2- 172.21.15.156:0/1724321508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f628c004850 0x7f628c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:00.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f629e59c700 1 -- 172.21.15.156:0/1724321508 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0035b0 con 0x7f628c003be0 2023-11-28T13:16:00.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f6297fff700 1 --2- 172.21.15.156:0/1724321508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f628c004850 0x7f628c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:16:00.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.769+0000 7f629ed9d700 1 -- 172.21.15.156:0/1724321508 reap_dead start 2023-11-28T13:16:00.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.770+0000 7f629e59c700 1 --2- 172.21.15.156:0/1724321508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628c003be0 0x7f628c003fd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f6290009b30 tx=0x7f628c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdfb356cf721dbe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:16:00.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.770+0000 7f629dd9b700 1 -- 172.21.15.156:0/1724321508 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629000e6c0 con 0x7f628c003be0 2023-11-28T13:16:00.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.770+0000 7f629dd9b700 1 -- 172.21.15.156:0/1724321508 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629000e820 con 0x7f628c003be0 2023-11-28T13:16:00.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.770+0000 7f629dd9b700 1 -- 172.21.15.156:0/1724321508 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629000ebb0 con 0x7f628c003be0 2023-11-28T13:16:00.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.770+0000 7f62a4dff700 1 -- 172.21.15.156:0/1724321508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628c003be0 msgr2=0x7f628c003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:00.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.770+0000 7f62a4dff700 1 --2- 172.21.15.156:0/1724321508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628c003be0 0x7f628c003fd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f6290009b30 tx=0x7f628c005880 comp rx=0 tx=0).stop 2023-11-28T13:16:00.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.770+0000 7f62a4dff700 1 -- 172.21.15.156:0/1724321508 shutdown_connections 2023-11-28T13:16:00.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.770+0000 7f62a4dff700 1 --2- 172.21.15.156:0/1724321508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f628c003be0 0x7f628c003fd0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:00.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.770+0000 7f62a4dff700 1 -- 172.21.15.156:0/1724321508 >> 172.21.15.156:0/1724321508 conn(0x7f62a00cb670 msgr2=0x7f62a00cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:16:00.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.771+0000 7f62a4dff700 1 -- 172.21.15.156:0/1724321508 shutdown_connections 2023-11-28T13:16:00.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.771+0000 7f62a4dff700 1 -- 172.21.15.156:0/1724321508 wait complete. 2023-11-28T13:16:00.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.772+0000 7f62a4dff700 1 Processor -- start 2023-11-28T13:16:00.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.772+0000 7f62a4dff700 1 -- start start 2023-11-28T13:16:00.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.773+0000 7f62a4dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f628c003be0 0x7f62a00c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:00.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.773+0000 7f62a4dff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62a01470f0 0x7f62a0152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:00.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.773+0000 7f62a4dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62a00c66b0 0x7f62a00af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:00.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.773+0000 7f62a4dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a00c4f30 con 0x7f62a00c66b0 2023-11-28T13:16:00.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.773+0000 7f62a4dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a01489f0 con 0x7f628c003be0 2023-11-28T13:16:00.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.773+0000 7f62a4dff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a0153230 con 0x7f62a01470f0 2023-11-28T13:16:00.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.774+0000 7f629ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62a00c66b0 0x7f62a00af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:00.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.774+0000 7f629ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62a00c66b0 0x7f62a00af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50306/0 (socket says 172.21.15.156:50306) 2023-11-28T13:16:00.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.774+0000 7f629ed9d700 1 -- 172.21.15.156:0/2761848088 learned_addr learned my addr 172.21.15.156:0/2761848088 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:16:00.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.774+0000 7f629e59c700 1 --2- 172.21.15.156:0/2761848088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f628c003be0 0x7f62a00c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:00.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.774+0000 7f629ed9d700 1 -- 172.21.15.156:0/2761848088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62a01470f0 msgr2=0x7f62a0152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:00.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.774+0000 7f6297fff700 1 --2- 172.21.15.156:0/2761848088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62a01470f0 0x7f62a0152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:00.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.774+0000 7f629ed9d700 1 --2- 172.21.15.156:0/2761848088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62a01470f0 0x7f62a0152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:00.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.774+0000 7f629ed9d700 1 -- 172.21.15.156:0/2761848088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f628c003be0 msgr2=0x7f62a00c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:00.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.774+0000 7f629ed9d700 1 --2- 172.21.15.156:0/2761848088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f628c003be0 0x7f62a00c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:00.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.774+0000 7f629ed9d700 1 -- 172.21.15.156:0/2761848088 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a006b8c0 con 0x7f62a00c66b0 2023-11-28T13:16:00.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.774+0000 7f629e59c700 1 --2- 172.21.15.156:0/2761848088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f628c003be0 0x7f62a00c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:16:00.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.774+0000 7f6297fff700 1 --2- 172.21.15.156:0/2761848088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62a01470f0 0x7f62a0152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:16:00.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.774+0000 7f629ed9d700 1 --2- 172.21.15.156:0/2761848088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62a00c66b0 0x7f62a00af8b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f62a006da30 tx=0x7f629800c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:16:00.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.774+0000 7f6296ffd700 1 -- 172.21.15.156:0/2761848088 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6298003c10 con 0x7f62a00c66b0 2023-11-28T13:16:00.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.774+0000 7f6296ffd700 1 -- 172.21.15.156:0/2761848088 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6298003d90 con 0x7f62a00c66b0 2023-11-28T13:16:00.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.774+0000 7f6296ffd700 1 -- 172.21.15.156:0/2761848088 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629800ca20 con 0x7f62a00c66b0 2023-11-28T13:16:00.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.774+0000 7f62a4dff700 1 -- 172.21.15.156:0/2761848088 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a00cb3b0 con 0x7f62a00c66b0 2023-11-28T13:16:00.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.775+0000 7f62a4dff700 1 -- 172.21.15.156:0/2761848088 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a006bba0 con 0x7f62a00c66b0 2023-11-28T13:16:00.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.776+0000 7f62a4dff700 1 -- 172.21.15.156:0/2761848088 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a006d750 con 0x7f62a00c66b0 2023-11-28T13:16:00.783 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.776+0000 7f6296ffd700 1 -- 172.21.15.156:0/2761848088 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f629800cb80 con 0x7f62a00c66b0 2023-11-28T13:16:00.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.777+0000 7f6296ffd700 1 --2- 172.21.15.156:0/2761848088 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6280068320 0x7f628006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:00.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.777+0000 7f6296ffd700 1 -- 172.21.15.156:0/2761848088 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6298085d90 con 0x7f62a00c66b0 2023-11-28T13:16:00.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.778+0000 7f629e59c700 1 --2- 172.21.15.156:0/2761848088 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6280068320 0x7f628006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:00.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.778+0000 7f629e59c700 1 --2- 172.21.15.156:0/2761848088 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6280068320 0x7f628006a7d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f62900090b0 tx=0x7f6290008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:16:00.784 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.781+0000 7f6296ffd700 1 -- 172.21.15.156:0/2761848088 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6298052820 con 0x7f62a00c66b0 2023-11-28T13:16:00.969 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.969+0000 7f62a4dff700 1 -- 172.21.15.156:0/2761848088 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f62a00745c0 con 0x7f6280068320 2023-11-28T13:16:00.973 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.972+0000 7f6296ffd700 1 -- 172.21.15.156:0/2761848088 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73017 (secure 0 0 0) 0x7f62a00745c0 con 0x7f6280068320 2023-11-28T13:16:00.974 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:16:00.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.974+0000 7f62a4dff700 1 -- 172.21.15.156:0/2761848088 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6280068320 msgr2=0x7f628006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:00.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.974+0000 7f62a4dff700 1 --2- 172.21.15.156:0/2761848088 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6280068320 0x7f628006a7d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f62900090b0 tx=0x7f6290008030 comp rx=0 tx=0).stop 2023-11-28T13:16:00.975 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.975+0000 7f62a4dff700 1 -- 172.21.15.156:0/2761848088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62a00c66b0 msgr2=0x7f62a00af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:00.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.975+0000 7f62a4dff700 1 --2- 172.21.15.156:0/2761848088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62a00c66b0 0x7f62a00af8b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f62a006da30 tx=0x7f629800c0e0 comp rx=0 tx=0).stop 2023-11-28T13:16:00.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.975+0000 7f62a4dff700 1 -- 172.21.15.156:0/2761848088 shutdown_connections 2023-11-28T13:16:00.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.975+0000 7f62a4dff700 1 --2- 172.21.15.156:0/2761848088 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f6280068320 0x7f628006a7d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:00.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.975+0000 7f62a4dff700 1 --2- 172.21.15.156:0/2761848088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f62a00c66b0 0x7f62a00af8b0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:00.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.975+0000 7f62a4dff700 1 --2- 172.21.15.156:0/2761848088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f628c003be0 0x7f62a00c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:00.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.975+0000 7f62a4dff700 1 --2- 172.21.15.156:0/2761848088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62a01470f0 0x7f62a0152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:00.976 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.975+0000 7f62a4dff700 1 -- 172.21.15.156:0/2761848088 >> 172.21.15.156:0/2761848088 conn(0x7f62a00cb670 msgr2=0x7f62a0154040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:16:00.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.975+0000 7f62a4dff700 1 -- 172.21.15.156:0/2761848088 shutdown_connections 2023-11-28T13:16:00.977 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:00.975+0000 7f62a4dff700 1 -- 172.21.15.156:0/2761848088 wait complete. 2023-11-28T13:16:00.977 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T13:16:00.993 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2023-11-28T13:16:00.140080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":100932,"kb_used_data":7236,"kb_used_omap":0,"kb_used_meta":93632,"kb_avail":754873788,"statfs":{"total":773094113280,"available":772990758912,"internally_reserved":0,"allocated":7409664,"data_stored":3925110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95879168},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002364"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:42.758478+0000","last_change":"2023-11-28T13:15:42.758391+0000","last_active":"2023-11-28T13:15:42.758478+0000","last_peered":"2023-11-28T13:15:42.758478+0000","last_clean":"2023-11-28T13:15:42.758478+0000","last_became_active":"2023-11-28T13:07:56.890058+0000","last_became_peered":"2023-11-28T13:07:56.890058+0000","last_unstale":"2023-11-28T13:15:42.758478+0000","last_undegraded":"2023-11-28T13:15:42.758478+0000","last_fullsized":"2023-11-28T13:15:42.758478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:15:42.758168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:15:42.758168+0000","last_clean_scrub_stamp":"2023-11-28T13:15:42.758168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:07.739187+0000","scrub_duration":0.004279354,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:44.399808+0000","last_change":"2023-11-28T13:15:44.399766+0000","last_active":"2023-11-28T13:15:44.399808+0000","last_peered":"2023-11-28T13:15:44.399808+0000","last_clean":"2023-11-28T13:15:44.399808+0000","last_became_active":"2023-11-28T13:07:56.844081+0000","last_became_peered":"2023-11-28T13:07:56.844081+0000","last_unstale":"2023-11-28T13:15:44.399808+0000","last_undegraded":"2023-11-28T13:15:44.399808+0000","last_fullsized":"2023-11-28T13:15:44.399808+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:15:44.399665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:15:44.399665+0000","last_clean_scrub_stamp":"2023-11-28T13:15:44.399665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:18.153360+0000","scrub_duration":0.0044762220000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":69,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:42.747467+0000","last_change":"2023-11-28T13:15:42.747431+0000","last_active":"2023-11-28T13:15:42.747467+0000","last_peered":"2023-11-28T13:15:42.747467+0000","last_clean":"2023-11-28T13:15:42.747467+0000","last_became_active":"2023-11-28T13:07:56.844391+0000","last_became_peered":"2023-11-28T13:07:56.844391+0000","last_unstale":"2023-11-28T13:15:42.747467+0000","last_undegraded":"2023-11-28T13:15:42.747467+0000","last_fullsized":"2023-11-28T13:15:42.747467+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:15:42.747233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:15:42.747233+0000","last_clean_scrub_stamp":"2023-11-28T13:15:42.747233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:25:40.665770+0000","scrub_duration":0.0043988220000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"64'119","reported_seq":222,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:45.990988+0000","last_change":"2023-11-28T13:15:45.990954+0000","last_active":"2023-11-28T13:15:45.990988+0000","last_peered":"2023-11-28T13:15:45.990988+0000","last_clean":"2023-11-28T13:15:45.990988+0000","last_became_active":"2023-11-28T13:07:57.842981+0000","last_became_peered":"2023-11-28T13:07:57.842981+0000","last_unstale":"2023-11-28T13:15:45.990988+0000","last_undegraded":"2023-11-28T13:15:45.990988+0000","last_fullsized":"2023-11-28T13:15:45.990988+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'119","last_scrub_stamp":"2023-11-28T13:15:45.990835+0000","last_deep_scrub":"64'119","last_deep_scrub_stamp":"2023-11-28T13:15:45.990835+0000","last_clean_scrub_stamp":"2023-11-28T13:15:45.990835+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:23.918572+0000","scrub_duration":0.0088478089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":71,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:42.921590+0000","last_change":"2023-11-28T13:15:42.921524+0000","last_active":"2023-11-28T13:15:42.921590+0000","last_peered":"2023-11-28T13:15:42.921590+0000","last_clean":"2023-11-28T13:15:42.921590+0000","last_became_active":"2023-11-28T13:07:57.305937+0000","last_became_peered":"2023-11-28T13:07:57.305937+0000","last_unstale":"2023-11-28T13:15:42.921590+0000","last_undegraded":"2023-11-28T13:15:42.921590+0000","last_fullsized":"2023-11-28T13:15:42.921590+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:15:42.921415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:15:42.921415+0000","last_clean_scrub_stamp":"2023-11-28T13:15:42.921415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:31:45.067182+0000","scrub_duration":0.0041994160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:41.897895+0000","last_change":"2023-11-28T13:15:41.897816+0000","last_active":"2023-11-28T13:15:41.897895+0000","last_peered":"2023-11-28T13:15:41.897895+0000","last_clean":"2023-11-28T13:15:41.897895+0000","last_became_active":"2023-11-28T13:07:57.305983+0000","last_became_peered":"2023-11-28T13:07:57.305983+0000","last_unstale":"2023-11-28T13:15:41.897895+0000","last_undegraded":"2023-11-28T13:15:41.897895+0000","last_fullsized":"2023-11-28T13:15:41.897895+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:15:41.897574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:15:41.897574+0000","last_clean_scrub_stamp":"2023-11-28T13:15:41.897574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:21:03.615064+0000","scrub_duration":0.0041980120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:39.980814+0000","last_change":"2023-11-28T13:15:39.980774+0000","last_active":"2023-11-28T13:15:39.980814+0000","last_peered":"2023-11-28T13:15:39.980814+0000","last_clean":"2023-11-28T13:15:39.980814+0000","last_became_active":"2023-11-28T13:07:57.305853+0000","last_became_peered":"2023-11-28T13:07:57.305853+0000","last_unstale":"2023-11-28T13:15:39.980814+0000","last_undegraded":"2023-11-28T13:15:39.980814+0000","last_fullsized":"2023-11-28T13:15:39.980814+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:15:39.980669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:15:39.980669+0000","last_clean_scrub_stamp":"2023-11-28T13:15:39.980669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:31.460666+0000","scrub_duration":0.003785596,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:47.659084+0000","last_change":"2023-11-28T13:15:47.659050+0000","last_active":"2023-11-28T13:15:47.659084+0000","last_peered":"2023-11-28T13:15:47.659084+0000","last_clean":"2023-11-28T13:15:47.659084+0000","last_became_active":"2023-11-28T13:07:57.305923+0000","last_became_peered":"2023-11-28T13:07:57.305923+0000","last_unstale":"2023-11-28T13:15:47.659084+0000","last_undegraded":"2023-11-28T13:15:47.659084+0000","last_fullsized":"2023-11-28T13:15:47.659084+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:15:47.658945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:15:47.658945+0000","last_clean_scrub_stamp":"2023-11-28T13:15:47.658945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:43.307117+0000","scrub_duration":0.0042506820000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":57,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:30.210599+0000","last_change":"2023-11-28T13:07:56.844305+0000","last_active":"2023-11-28T13:15:30.210599+0000","last_peered":"2023-11-28T13:15:30.210599+0000","last_clean":"2023-11-28T13:15:30.210599+0000","last_became_active":"2023-11-28T13:07:56.843980+0000","last_became_peered":"2023-11-28T13:07:56.843980+0000","last_unstale":"2023-11-28T13:15:30.210599+0000","last_undegraded":"2023-11-28T13:15:30.210599+0000","last_fullsized":"2023-11-28T13:15:30.210599+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.823449+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.823449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574948,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15172,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94356668,"statfs":{"total":96636764160,"available":96621228032,"internally_reserved":0,"allocated":1372160,"data_stored":933538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68700000000000006,"15min":0.66900000000000004},"min":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89900000000000002,"5min":0.91200000000000003,"15min":0.98399999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68999999999999995,"15min":0.64800000000000002},"min":{"1min":0.498,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82499999999999996,"5min":0.92300000000000004,"15min":1.0569999999999999},"last":0.73299999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73399999999999999,"15min":0.72299999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84199999999999997,"5min":0.93999999999999995,"15min":1.0289999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.73999999999999999,"15min":0.71399999999999997},"min":{"1min":0.57599999999999996,"5min":0.42399999999999999,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":0.95999999999999996,"15min":1.095},"last":0.76500000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.66500000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85499999999999998,"5min":0.92300000000000004,"15min":0.996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72599999999999998,"15min":0.69499999999999995},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.74299999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.68999999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.42999999999999999},"max":{"1min":0.89000000000000001,"5min":0.91600000000000004,"15min":0.93899999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73799999999999999,"15min":0.76200000000000001},"min":{"1min":0.60399999999999998,"5min":0.52200000000000002,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.98599999999999999,"15min":1.079},"last":0.77800000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.66400000000000003},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":1.008},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.628,"15min":0.70799999999999996},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":1.006,"15min":1.071},"last":0.51200000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64200000000000002,"15min":0.73499999999999999},"min":{"1min":0.501,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86399999999999999,"5min":0.93600000000000005,"15min":1.1830000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65300000000000002,"15min":0.75900000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96299999999999997,"5min":1.044,"15min":1.1839999999999999},"last":0.53100000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71799999999999997,"15min":0.80600000000000005},"min":{"1min":0.56100000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94099999999999995,"5min":0.98599999999999999,"15min":1.165},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70199999999999996,"15min":0.76500000000000001},"min":{"1min":0.50700000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92800000000000005,"5min":0.95499999999999996,"15min":1.141},"last":0.55300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574947,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9536,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362304,"statfs":{"total":96636764160,"available":96626999296,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.68700000000000006,"15min":0.67700000000000005},"min":{"1min":0.441,"5min":0.432,"15min":0.41599999999999998},"max":{"1min":0.88300000000000001,"5min":0.995,"15min":0.995},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70399999999999996,"15min":0.66400000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91300000000000003,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.68700000000000006}]},{"osd":1,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70899999999999996,"15min":0.68700000000000006},"min":{"1min":0.56100000000000005,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.86899999999999999,"5min":1.113,"15min":1.113},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70699999999999996,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.48599999999999999,"15min":0.47899999999999998},"max":{"1min":0.85499999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.51300000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72299999999999998,"15min":0.68899999999999995},"min":{"1min":0.60399999999999998,"5min":0.45000000000000001,"15min":0.41799999999999998},"max":{"1min":0.90300000000000002,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72099999999999997,"15min":0.68500000000000005},"min":{"1min":0.51000000000000001,"5min":0.495,"15min":0.38500000000000001},"max":{"1min":0.89400000000000002,"5min":1.024,"15min":1.024},"last":0.55000000000000004}]},{"osd":3,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.73599999999999999,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":1.121,"15min":1.121},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.745,"15min":0.73799999999999999},"min":{"1min":0.63,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.89200000000000002,"5min":1.167,"15min":1.167},"last":0.60999999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.624,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92300000000000004,"5min":1.042,"15min":1.0429999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.79200000000000004},"min":{"1min":0.54200000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96499999999999997,"5min":1.105,"15min":1.3140000000000001},"last":0.79100000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63500000000000001,"15min":0.68500000000000005},"min":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84599999999999997,"5min":0.92100000000000004,"15min":1.0780000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.70699999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.751}]},{"osd":7,"last update":"Tue Nov 28 13:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70499999999999996,"15min":0.72899999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93400000000000005,"5min":1.034,"15min":1.0980000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73399999999999999,"15min":0.76800000000000002},"min":{"1min":0.501,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94899999999999995,"5min":1.0129999999999999,"15min":1.107},"last":0.81000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14916,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":13568,"kb_avail":94356924,"statfs":{"total":96636764160,"available":96621490176,"internally_reserved":0,"allocated":1372160,"data_stored":933538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13893632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41799999999999998,"15min":0.43099999999999999},"min":{"1min":0.28100000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.496,"5min":0.75900000000000001,"15min":1.28},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.47899999999999998,"15min":0.45600000000000002},"min":{"1min":0.30499999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64000000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.48899999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.46000000000000002},"min":{"1min":0.33200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60599999999999998,"5min":0.747,"15min":0.747},"last":0.623},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.67500000000000004,"5min":0.77000000000000002,"15min":0.84699999999999998},"last":0.66800000000000004}]},{"osd":3,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.436,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71099999999999997,"5min":0.88500000000000001,"15min":1.0009999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52000000000000002,"15min":0.53800000000000003},"min":{"1min":0.36899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.68899999999999995,"5min":0.746,"15min":1.1120000000000001},"last":0.64600000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73099999999999998,"15min":0.70799999999999996},"min":{"1min":0.60299999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.753,"15min":0.72199999999999998},"min":{"1min":0.624,"5min":0.54100000000000004,"15min":0.49099999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.92400000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.74299999999999999,"15min":0.745},"min":{"1min":0.61199999999999999,"5min":0.51700000000000002,"15min":0.433},"max":{"1min":1.0209999999999999,"5min":1.292,"15min":1.292},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.74299999999999999,"15min":0.72099999999999997},"min":{"1min":0.621,"5min":0.441,"15min":0.441},"max":{"1min":1.141,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.84799999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.74199999999999999},"min":{"1min":0.64300000000000002,"5min":0.55300000000000005,"15min":0.52700000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.47299999999999998},"max":{"1min":1.1240000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.91000000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.80000000000000004,"15min":0.81000000000000005},"min":{"1min":0.66900000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0820000000000001,"5min":1.357,"15min":1.357},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77800000000000002,"15min":0.78400000000000003},"min":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0389999999999999,"5min":1.24,"15min":1.3580000000000001},"last":0.89400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574948,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13884,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357956,"statfs":{"total":96636764160,"available":96622546944,"internally_reserved":0,"allocated":774144,"data_stored":342184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.47799999999999998},"min":{"1min":0.32500000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65400000000000003,"5min":0.69699999999999995,"15min":1.3120000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47999999999999998,"15min":0.51500000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67000000000000004,"5min":0.70599999999999996,"15min":1.48},"last":0.72699999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.504,"15min":0.52100000000000002},"min":{"1min":0.41999999999999998,"5min":0.34200000000000003,"15min":0.33000000000000002},"max":{"1min":0.76700000000000002,"5min":0.874,"15min":0.874},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.5,"5min":0.53000000000000003,"15min":0.51400000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.749,"5min":0.749,"15min":0.80800000000000005},"last":0.58599999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.35599999999999998,"5min":0.33700000000000002,"15min":0.30099999999999999},"max":{"1min":0.63800000000000001,"5min":0.747,"15min":0.747},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56399999999999995,"15min":0.55500000000000005},"min":{"1min":0.42099999999999999,"5min":0.36699999999999999,"15min":0.35399999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.82199999999999995},"last":0.54600000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.75900000000000001,"15min":0.77500000000000002},"min":{"1min":0.54500000000000004,"5min":0.48399999999999999,"15min":0.45500000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.071},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.99099999999999999,"5min":1.147,"15min":1.147},"last":0.75700000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77300000000000002,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.48299999999999998},"max":{"1min":1.0660000000000001,"5min":1.1040000000000001,"15min":1.3260000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0309999999999999,"5min":1.127,"15min":1.127},"last":0.871}]},{"osd":6,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.77300000000000002,"15min":0.76100000000000001},"min":{"1min":0.58099999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0660000000000001,"5min":1.127,"15min":1.127},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80800000000000005,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.042,"5min":1.0489999999999999,"15min":1.097},"last":0.81999999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78500000000000003,"15min":0.80000000000000004},"min":{"1min":0.58999999999999997,"5min":0.55600000000000005,"15min":0.53700000000000003},"max":{"1min":1.1200000000000001,"5min":1.149,"15min":1.149},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79600000000000004,"15min":0.80200000000000005},"min":{"1min":0.55300000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.996,"5min":1.163,"15min":1.163},"last":0.83699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574947,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13696,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":12928,"kb_avail":94358144,"statfs":{"total":96636764160,"available":96622739456,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13238272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.48099999999999998,"15min":0.44500000000000001},"min":{"1min":0.28499999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.68400000000000005,"5min":6.4130000000000003,"15min":6.4130000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.64200000000000002,"15min":0.52200000000000002},"min":{"1min":0.32700000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.625,"5min":19.190000000000001,"15min":19.190000000000001},"last":0.35999999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.33100000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66100000000000003,"5min":5.2009999999999996,"15min":5.2009999999999996},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.68899999999999995,"15min":0.57099999999999995},"min":{"1min":0.40300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.67900000000000005,"5min":19.241,"15min":19.241},"last":0.61299999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55700000000000005,"15min":0.48499999999999999},"min":{"1min":0.34999999999999998,"5min":0.29899999999999999,"15min":0.29799999999999999},"max":{"1min":0.66500000000000004,"5min":7.0410000000000004,"15min":7.0410000000000004},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.65200000000000002,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70499999999999996,"5min":11.420999999999999,"15min":11.420999999999999},"last":0.47599999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.84199999999999997,"15min":0.82999999999999996},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.55000000000000004},"max":{"1min":1.006,"5min":5.1630000000000003,"15min":5.1630000000000003},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.80400000000000005,"15min":0.79600000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.496},"max":{"1min":0.95899999999999996,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.75700000000000001,"15min":0.73999999999999999},"min":{"1min":0.58499999999999996,"5min":0.505,"15min":0.46100000000000002},"max":{"1min":0.86899999999999999,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.81100000000000005,"15min":0.76100000000000001},"min":{"1min":0.52900000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.036,"5min":6.7569999999999997,"15min":6.7569999999999997},"last":0.78300000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.83299999999999996,"15min":0.77700000000000002},"min":{"1min":0.51100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97099999999999997,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.83799999999999997,"15min":0.80800000000000005},"min":{"1min":0.64200000000000002,"5min":0.55700000000000005,"15min":0.49199999999999999},"max":{"1min":0.95399999999999996,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":0.746}]},{"osd":7,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.79100000000000004,"15min":0.77000000000000002},"min":{"1min":0.57799999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.996,"5min":2.008,"15min":2.008},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.82399999999999995,"15min":0.80800000000000005},"min":{"1min":0.56000000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1379999999999999,"5min":6.6769999999999996,"15min":6.6769999999999996},"last":0.82099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574947,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14016,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":13248,"kb_avail":94357824,"statfs":{"total":96636764160,"available":96622411776,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13565952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.33100000000000002,"5min":0.26100000000000001,"15min":0.245},"max":{"1min":0.61699999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.48299999999999998,"15min":0.45700000000000002},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.27300000000000002},"max":{"1min":0.69799999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54300000000000004}]},{"osd":1,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.46899999999999997,"15min":0.48099999999999998},"min":{"1min":0.34899999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71199999999999997,"5min":0.73699999999999999,"15min":0.88100000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.41099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.41099999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.48099999999999998,"5min":0.372,"15min":0.36399999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.432},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56999999999999995,"15min":0.54900000000000004},"min":{"1min":0.40300000000000002,"5min":0.39000000000000001,"15min":0.35399999999999998},"max":{"1min":0.80500000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.49199999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72199999999999998,"15min":0.70499999999999996},"min":{"1min":0.54400000000000004,"5min":0.50900000000000001,"15min":0.50600000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.75,"5min":0.73399999999999999,"15min":0.71399999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.46200000000000002},"max":{"1min":1.054,"5min":1.087,"15min":1.087},"last":0.66600000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.091},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.77500000000000002,"15min":0.76600000000000001},"min":{"1min":0.63400000000000001,"5min":0.50600000000000001,"15min":0.498},"max":{"1min":1.0369999999999999,"5min":1.0649999999999999,"15min":1.0680000000000001},"last":0.57499999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76600000000000001,"15min":0.73499999999999999},"min":{"1min":0.65500000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96799999999999997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.629},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.76100000000000001,"15min":0.71999999999999997},"min":{"1min":0.59299999999999997,"5min":0.50600000000000001,"15min":0.46100000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.61199999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79900000000000004,"15min":0.76900000000000002},"min":{"1min":0.64600000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.032,"5min":1.05,"15min":1.05},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.80000000000000004,"15min":0.77300000000000002},"min":{"1min":0.68400000000000005,"5min":0.53600000000000003,"15min":0.52800000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.1299999999999999},"last":0.75}]}]},{"osd":4,"up_from":13,"seq":55834574946,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9920,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361920,"statfs":{"total":96636764160,"available":96626606080,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70499999999999996,"15min":0.63},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.91900000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72099999999999997,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.42799999999999999,"15min":0.38400000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":1.3089999999999999},"last":0.66100000000000003}]},{"osd":1,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73499999999999999,"15min":0.69699999999999995},"min":{"1min":0.59399999999999997,"5min":0.51200000000000001,"15min":0.48299999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":1.161},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69399999999999995,"15min":0.63},"min":{"1min":0.47899999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88900000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.86299999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":0.67800000000000005},"min":{"1min":0.46400000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.69899999999999995},"min":{"1min":0.502,"5min":0.47599999999999998,"15min":0.441},"max":{"1min":0.91500000000000004,"5min":1.042,"15min":1.042},"last":0.78000000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71699999999999997,"15min":0.68600000000000005},"min":{"1min":0.48699999999999999,"5min":0.46000000000000002,"15min":0.438},"max":{"1min":0.85499999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.75700000000000001,"15min":0.72799999999999998},"min":{"1min":0.54500000000000004,"5min":0.52900000000000003,"15min":0.505},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.751}]},{"osd":5,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.66000000000000003},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95799999999999996,"5min":1.012,"15min":1.264},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64500000000000002,"15min":0.70299999999999996},"min":{"1min":0.49399999999999999,"5min":0.39100000000000001,"15min":0.38400000000000001},"max":{"1min":0.81999999999999995,"5min":0.93200000000000005,"15min":1.079},"last":0.69799999999999995}]},{"osd":6,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65600000000000003,"15min":0.70799999999999996},"min":{"1min":0.46300000000000002,"5min":0.439,"15min":0.32700000000000001},"max":{"1min":0.89000000000000001,"5min":0.91200000000000003,"15min":1.022},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69799999999999995,"15min":0.73099999999999998},"min":{"1min":0.53800000000000003,"5min":0.48699999999999999,"15min":0.47399999999999998},"max":{"1min":0.90900000000000003,"5min":1.044,"15min":1.22},"last":0.72699999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69099999999999995,"15min":0.745},"min":{"1min":0.51700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98699999999999999,"5min":0.995,"15min":1.4530000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72899999999999998,"15min":0.76700000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.40899999999999997},"max":{"1min":1.0089999999999999,"5min":1.018,"15min":1.018},"last":0.84699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574947,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9792,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362048,"statfs":{"total":96636764160,"available":96626737152,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69199999999999995,"15min":0.63300000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92500000000000004,"5min":1.0640000000000001,"15min":1.659},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69699999999999995,"15min":0.63400000000000001},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.38900000000000001},"max":{"1min":0.95899999999999996,"5min":0.97999999999999998,"15min":2.3050000000000002},"last":0.77000000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66900000000000004,"15min":0.62},"min":{"1min":0.44600000000000001,"5min":0.38200000000000001,"15min":0.35499999999999998},"max":{"1min":0.94299999999999995,"5min":0.95699999999999996,"15min":1.2390000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72099999999999997,"15min":0.65300000000000002},"min":{"1min":0.54300000000000004,"5min":0.40500000000000003,"15min":0.39400000000000002},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.9630000000000001},"last":0.79100000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69699999999999995,"15min":0.65700000000000003},"min":{"1min":0.53500000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.86899999999999999,"5min":1.0469999999999999,"15min":2.3860000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71199999999999997,"15min":0.66400000000000003},"min":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.432},"max":{"1min":0.89300000000000002,"5min":0.97299999999999998,"15min":1.0800000000000001},"last":0.68600000000000005}]},{"osd":3,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71399999999999997,"15min":0.65900000000000003},"min":{"1min":0.53800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":1.0169999999999999,"15min":1.8140000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73699999999999999,"15min":0.71399999999999997},"min":{"1min":0.5,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91700000000000004,"5min":0.96699999999999997,"15min":2.3500000000000001},"last":0.746}]},{"osd":4,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63200000000000001,"15min":0.65300000000000002},"min":{"1min":0.495,"5min":0.379,"15min":0.379},"max":{"1min":0.79900000000000004,"5min":0.95999999999999996,"15min":1.702},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70799999999999996,"15min":0.72799999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89000000000000001,"5min":1.0529999999999999,"15min":2.0009999999999999},"last":0.499}]},{"osd":6,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64500000000000002,"15min":0.69599999999999995},"min":{"1min":0.437,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76700000000000002,"5min":0.89400000000000002,"15min":2.4969999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66900000000000004,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86899999999999999,"5min":1.012,"15min":2.0880000000000001},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69499999999999995,"15min":0.71499999999999997},"min":{"1min":0.53100000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93999999999999995,"5min":0.999,"15min":2.0449999999999999},"last":0.622},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.49199999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.85099999999999998,"5min":0.98099999999999998,"15min":2.4249999999999998},"last":0.59199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:16:00.996 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-11-28T13:07:55.823449+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=55, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=15, tm_sec=33, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:16:00.996 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-28T13:16:04.998 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:16:05.005 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:16:20.997 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:16:21.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.204+0000 7f92b3bff700 1 Processor -- start 2023-11-28T13:16:21.206 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.204+0000 7f92b3bff700 1 -- start start 2023-11-28T13:16:21.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.205+0000 7f92b3bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92ac0cd7c0 0x7f92ac0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:21.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.205+0000 7f92b3bff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac0d1950 con 0x7f92ac161340 2023-11-28T13:16:21.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.205+0000 7f92b3bff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac0ce140 con 0x7f92ac15c9b0 2023-11-28T13:16:21.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.205+0000 7f92b3bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac0b5800 con 0x7f92ac0cd7c0 2023-11-28T13:16:21.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.205+0000 7f92b119a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f92ac161340 0x7f92ac0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:59364/0 (socket says 172.21.15.156:59364) 2023-11-28T13:16:21.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.205+0000 7f92b119a700 1 -- 172.21.15.156:0/861155114 learned_addr learned my addr 172.21.15.156:0/861155114 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:16:21.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.205+0000 7f92b219c700 1 --2- 172.21.15.156:0/861155114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92ac0cd7c0 0x7f92ac0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:21.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b0999700 1 -- 172.21.15.156:0/861155114 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2711722779 0 0) 0x7f92ac0d1950 con 0x7f92ac161340 2023-11-28T13:16:21.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b0999700 1 -- 172.21.15.156:0/861155114 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92980035b0 con 0x7f92ac161340 2023-11-28T13:16:21.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b0999700 1 -- 172.21.15.156:0/861155114 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3606372302 0 0) 0x7f92ac0ce140 con 0x7f92ac15c9b0 2023-11-28T13:16:21.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b0999700 1 -- 172.21.15.156:0/861155114 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92ac0d1950 con 0x7f92ac15c9b0 2023-11-28T13:16:21.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b0999700 1 -- 172.21.15.156:0/861155114 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2296898588 0 0) 0x7f92980035b0 con 0x7f92ac161340 2023-11-28T13:16:21.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b0999700 1 -- 172.21.15.156:0/861155114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92ac0cd7c0 msgr2=0x7f92ac0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:16:21.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b0999700 1 --2- 172.21.15.156:0/861155114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92ac0cd7c0 0x7f92ac0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b0999700 1 -- 172.21.15.156:0/861155114 >> v1:172.21.15.186:6789/0 conn(0x7f92ac15c9b0 legacy=0x7f92ac0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:21.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b0999700 1 -- 172.21.15.156:0/861155114 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac0d24f0 con 0x7f92ac161340 2023-11-28T13:16:21.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b0999700 1 -- 172.21.15.156:0/861155114 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f929c001670 con 0x7f92ac161340 2023-11-28T13:16:21.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b0999700 1 -- 172.21.15.156:0/861155114 >> v1:172.21.15.156:6789/0 conn(0x7f92ac161340 legacy=0x7f92ac0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:21.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b0999700 1 --2- 172.21.15.156:0/861155114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9298003bc0 0x7f9298003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:21.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b0999700 1 --2- 172.21.15.156:0/861155114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9298004850 0x7f9298004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:21.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b0999700 1 --2- 172.21.15.156:0/861155114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92ac0cd7c0 0x7f9298005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:21.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b0999700 1 -- 172.21.15.156:0/861155114 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac0b5800 con 0x7f9298004850 2023-11-28T13:16:21.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b0999700 1 -- 172.21.15.156:0/861155114 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929800bbd0 con 0x7f9298003bc0 2023-11-28T13:16:21.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b0999700 1 -- 172.21.15.156:0/861155114 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929800bd90 con 0x7f92ac0cd7c0 2023-11-28T13:16:21.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b119a700 1 --2- 172.21.15.156:0/861155114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9298004850 0x7f9298004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:21.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.206+0000 7f92b199b700 1 --2- 172.21.15.156:0/861155114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9298003bc0 0x7f9298003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:21.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.207+0000 7f92b219c700 1 --2- 172.21.15.156:0/861155114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92ac0cd7c0 0x7f9298005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:21.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.207+0000 7f92b119a700 1 -- 172.21.15.156:0/861155114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92ac0cd7c0 msgr2=0x7f9298005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:16:21.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.207+0000 7f92b119a700 1 --2- 172.21.15.156:0/861155114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92ac0cd7c0 0x7f9298005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.207+0000 7f92b119a700 1 -- 172.21.15.156:0/861155114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9298003bc0 msgr2=0x7f9298003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:16:21.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.207+0000 7f92b119a700 1 --2- 172.21.15.156:0/861155114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9298003bc0 0x7f9298003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.207+0000 7f92b119a700 1 -- 172.21.15.156:0/861155114 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92980035b0 con 0x7f9298004850 2023-11-28T13:16:21.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.207+0000 7f92b219c700 1 --2- 172.21.15.156:0/861155114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92ac0cd7c0 0x7f9298005260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:16:21.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.207+0000 7f92b119a700 1 --2- 172.21.15.156:0/861155114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9298004850 0x7f9298004cc0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f929c00c110 tx=0x7f9298005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34344cddda750e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:16:21.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.207+0000 7f92b0999700 1 -- 172.21.15.156:0/861155114 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c00ccb0 con 0x7f9298004850 2023-11-28T13:16:21.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.207+0000 7f92b0999700 1 -- 172.21.15.156:0/861155114 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929c00ce10 con 0x7f9298004850 2023-11-28T13:16:21.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.207+0000 7f92b0999700 1 -- 172.21.15.156:0/861155114 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c005ba0 con 0x7f9298004850 2023-11-28T13:16:21.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.208+0000 7f92b3bff700 1 -- 172.21.15.156:0/861155114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9298004850 msgr2=0x7f9298004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:21.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.208+0000 7f92b3bff700 1 --2- 172.21.15.156:0/861155114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9298004850 0x7f9298004cc0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f929c00c110 tx=0x7f9298005880 comp rx=0 tx=0).stop 2023-11-28T13:16:21.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.208+0000 7f92b219c700 1 -- 172.21.15.156:0/861155114 reap_dead start 2023-11-28T13:16:21.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.208+0000 7f92b3bff700 1 -- 172.21.15.156:0/861155114 shutdown_connections 2023-11-28T13:16:21.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.208+0000 7f92b3bff700 1 -- 172.21.15.156:0/861155114 >> 172.21.15.156:0/861155114 conn(0x7f92ac072600 msgr2=0x7f92ac0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:16:21.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.208+0000 7f92b3bff700 1 -- 172.21.15.156:0/861155114 shutdown_connections 2023-11-28T13:16:21.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.208+0000 7f92b3bff700 1 -- 172.21.15.156:0/861155114 wait complete. 2023-11-28T13:16:21.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.210+0000 7f92b3bff700 1 Processor -- start 2023-11-28T13:16:21.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.210+0000 7f92b3bff700 1 -- start start 2023-11-28T13:16:21.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.211+0000 7f92b3bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9298004850 0x7f92ac1f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:21.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.211+0000 7f92b3bff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92ac1e6fc0 0x7f92ac157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:21.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.211+0000 7f92b3bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92ac1e7430 0x7f92ac1577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:21.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.211+0000 7f92b3bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac1f3420 con 0x7f92ac1e7430 2023-11-28T13:16:21.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.211+0000 7f92b3bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac0c81a0 con 0x7f9298004850 2023-11-28T13:16:21.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.211+0000 7f92b3bff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac0c8480 con 0x7f92ac1e6fc0 2023-11-28T13:16:21.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.211+0000 7f92b219c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92ac1e7430 0x7f92ac1577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:21.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.211+0000 7f92b219c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92ac1e7430 0x7f92ac1577d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35492/0 (socket says 172.21.15.156:35492) 2023-11-28T13:16:21.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.211+0000 7f92b219c700 1 -- 172.21.15.156:0/2232933608 learned_addr learned my addr 172.21.15.156:0/2232933608 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:16:21.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.211+0000 7f92b199b700 1 --2- 172.21.15.156:0/2232933608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9298004850 0x7f92ac1f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:21.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.211+0000 7f92b119a700 1 --2- 172.21.15.156:0/2232933608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92ac1e6fc0 0x7f92ac157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:21.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.211+0000 7f92b219c700 1 -- 172.21.15.156:0/2232933608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92ac1e6fc0 msgr2=0x7f92ac157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:21.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.211+0000 7f92b219c700 1 --2- 172.21.15.156:0/2232933608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92ac1e6fc0 0x7f92ac157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.211+0000 7f92b219c700 1 -- 172.21.15.156:0/2232933608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9298004850 msgr2=0x7f92ac1f2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:21.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.211+0000 7f92b219c700 1 --2- 172.21.15.156:0/2232933608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9298004850 0x7f92ac1f2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.211+0000 7f92b219c700 1 -- 172.21.15.156:0/2232933608 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac0d1300 con 0x7f92ac1e7430 2023-11-28T13:16:21.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.212+0000 7f92b219c700 1 --2- 172.21.15.156:0/2232933608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92ac1e7430 0x7f92ac1577d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f92ac06bbb0 tx=0x7f92a40104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:16:21.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.212+0000 7f92a27fc700 1 -- 172.21.15.156:0/2232933608 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a401b070 con 0x7f92ac1e7430 2023-11-28T13:16:21.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.212+0000 7f92a27fc700 1 -- 172.21.15.156:0/2232933608 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a400f040 con 0x7f92ac1e7430 2023-11-28T13:16:21.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.212+0000 7f92b3bff700 1 -- 172.21.15.156:0/2232933608 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ac0b5520 con 0x7f92ac1e7430 2023-11-28T13:16:21.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.212+0000 7f92a27fc700 1 -- 172.21.15.156:0/2232933608 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a400e050 con 0x7f92ac1e7430 2023-11-28T13:16:21.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.212+0000 7f92b3bff700 1 -- 172.21.15.156:0/2232933608 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ac0d15e0 con 0x7f92ac1e7430 2023-11-28T13:16:21.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.213+0000 7f92b3bff700 1 -- 172.21.15.156:0/2232933608 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ac0745c0 con 0x7f92ac1e7430 2023-11-28T13:16:21.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.214+0000 7f92a27fc700 1 -- 172.21.15.156:0/2232933608 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f92a4015440 con 0x7f92ac1e7430 2023-11-28T13:16:21.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.215+0000 7f92a27fc700 1 --2- 172.21.15.156:0/2232933608 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f92900682f0 0x7f929006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:21.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.215+0000 7f92a27fc700 1 -- 172.21.15.156:0/2232933608 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f92a4085b40 con 0x7f92ac1e7430 2023-11-28T13:16:21.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.215+0000 7f92b199b700 1 --2- 172.21.15.156:0/2232933608 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f92900682f0 0x7f929006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:21.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.216+0000 7f92b199b700 1 --2- 172.21.15.156:0/2232933608 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f92900682f0 0x7f929006a7a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f92a8007fd0 tx=0x7f92a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:16:21.224 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.219+0000 7f92a27fc700 1 -- 172.21.15.156:0/2232933608 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f92a4052640 con 0x7f92ac1e7430 2023-11-28T13:16:21.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.407+0000 7f92b3bff700 1 -- 172.21.15.156:0/2232933608 --> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f92ac0d08a0 con 0x7f92900682f0 2023-11-28T13:16:21.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.411+0000 7f92a27fc700 1 -- 172.21.15.156:0/2232933608 <== mgr.4101 v2:172.21.15.186:6832/130711 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73038 (secure 0 0 0) 0x7f92ac0d08a0 con 0x7f92900682f0 2023-11-28T13:16:21.413 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:16:21.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.413+0000 7f92b3bff700 1 -- 172.21.15.156:0/2232933608 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f92900682f0 msgr2=0x7f929006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:21.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.413+0000 7f92b3bff700 1 --2- 172.21.15.156:0/2232933608 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f92900682f0 0x7f929006a7a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f92a8007fd0 tx=0x7f92a8009040 comp rx=0 tx=0).stop 2023-11-28T13:16:21.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.413+0000 7f92b3bff700 1 -- 172.21.15.156:0/2232933608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92ac1e7430 msgr2=0x7f92ac1577d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:21.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.413+0000 7f92b3bff700 1 --2- 172.21.15.156:0/2232933608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92ac1e7430 0x7f92ac1577d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f92ac06bbb0 tx=0x7f92a40104a0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.415 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.413+0000 7f92b3bff700 1 -- 172.21.15.156:0/2232933608 shutdown_connections 2023-11-28T13:16:21.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.413+0000 7f92b3bff700 1 --2- 172.21.15.156:0/2232933608 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f92900682f0 0x7f929006a7a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.413+0000 7f92b3bff700 1 --2- 172.21.15.156:0/2232933608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92ac1e7430 0x7f92ac1577d0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.413+0000 7f92b3bff700 1 --2- 172.21.15.156:0/2232933608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9298004850 0x7f92ac1f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.413+0000 7f92b3bff700 1 --2- 172.21.15.156:0/2232933608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f92ac1e6fc0 0x7f92ac157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.413+0000 7f92b3bff700 1 -- 172.21.15.156:0/2232933608 >> 172.21.15.156:0/2232933608 conn(0x7f92ac072600 msgr2=0x7f92ac15acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:16:21.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.413+0000 7f92b3bff700 1 -- 172.21.15.156:0/2232933608 shutdown_connections 2023-11-28T13:16:21.416 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.414+0000 7f92b3bff700 1 -- 172.21.15.156:0/2232933608 wait complete. 2023-11-28T13:16:21.417 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2023-11-28T13:16:21.433 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2023-11-28T13:16:20.144425+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":100932,"kb_used_data":7236,"kb_used_omap":0,"kb_used_meta":93632,"kb_avail":754873788,"statfs":{"total":773094113280,"available":772990758912,"internally_reserved":0,"allocated":7409664,"data_stored":3925110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95879168},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002721"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:42.758478+0000","last_change":"2023-11-28T13:15:42.758391+0000","last_active":"2023-11-28T13:15:42.758478+0000","last_peered":"2023-11-28T13:15:42.758478+0000","last_clean":"2023-11-28T13:15:42.758478+0000","last_became_active":"2023-11-28T13:07:56.890058+0000","last_became_peered":"2023-11-28T13:07:56.890058+0000","last_unstale":"2023-11-28T13:15:42.758478+0000","last_undegraded":"2023-11-28T13:15:42.758478+0000","last_fullsized":"2023-11-28T13:15:42.758478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:15:42.758168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:15:42.758168+0000","last_clean_scrub_stamp":"2023-11-28T13:15:42.758168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:07.739187+0000","scrub_duration":0.004279354,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:44.399808+0000","last_change":"2023-11-28T13:15:44.399766+0000","last_active":"2023-11-28T13:15:44.399808+0000","last_peered":"2023-11-28T13:15:44.399808+0000","last_clean":"2023-11-28T13:15:44.399808+0000","last_became_active":"2023-11-28T13:07:56.844081+0000","last_became_peered":"2023-11-28T13:07:56.844081+0000","last_unstale":"2023-11-28T13:15:44.399808+0000","last_undegraded":"2023-11-28T13:15:44.399808+0000","last_fullsized":"2023-11-28T13:15:44.399808+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:15:44.399665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:15:44.399665+0000","last_clean_scrub_stamp":"2023-11-28T13:15:44.399665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:18.153360+0000","scrub_duration":0.0044762220000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":69,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:42.747467+0000","last_change":"2023-11-28T13:15:42.747431+0000","last_active":"2023-11-28T13:15:42.747467+0000","last_peered":"2023-11-28T13:15:42.747467+0000","last_clean":"2023-11-28T13:15:42.747467+0000","last_became_active":"2023-11-28T13:07:56.844391+0000","last_became_peered":"2023-11-28T13:07:56.844391+0000","last_unstale":"2023-11-28T13:15:42.747467+0000","last_undegraded":"2023-11-28T13:15:42.747467+0000","last_fullsized":"2023-11-28T13:15:42.747467+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:15:42.747233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:15:42.747233+0000","last_clean_scrub_stamp":"2023-11-28T13:15:42.747233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:25:40.665770+0000","scrub_duration":0.0043988220000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"64'119","reported_seq":223,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:45.991013+0000","last_change":"2023-11-28T13:15:45.990954+0000","last_active":"2023-11-28T13:15:45.991013+0000","last_peered":"2023-11-28T13:15:45.991013+0000","last_clean":"2023-11-28T13:15:45.991013+0000","last_became_active":"2023-11-28T13:07:57.842981+0000","last_became_peered":"2023-11-28T13:07:57.842981+0000","last_unstale":"2023-11-28T13:15:45.991013+0000","last_undegraded":"2023-11-28T13:15:45.991013+0000","last_fullsized":"2023-11-28T13:15:45.991013+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'119","last_scrub_stamp":"2023-11-28T13:15:45.990835+0000","last_deep_scrub":"64'119","last_deep_scrub_stamp":"2023-11-28T13:15:45.990835+0000","last_clean_scrub_stamp":"2023-11-28T13:15:45.990835+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:23.918572+0000","scrub_duration":0.0088478089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":71,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:42.921590+0000","last_change":"2023-11-28T13:15:42.921524+0000","last_active":"2023-11-28T13:15:42.921590+0000","last_peered":"2023-11-28T13:15:42.921590+0000","last_clean":"2023-11-28T13:15:42.921590+0000","last_became_active":"2023-11-28T13:07:57.305937+0000","last_became_peered":"2023-11-28T13:07:57.305937+0000","last_unstale":"2023-11-28T13:15:42.921590+0000","last_undegraded":"2023-11-28T13:15:42.921590+0000","last_fullsized":"2023-11-28T13:15:42.921590+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:15:42.921415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:15:42.921415+0000","last_clean_scrub_stamp":"2023-11-28T13:15:42.921415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:31:45.067182+0000","scrub_duration":0.0041994160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:41.897912+0000","last_change":"2023-11-28T13:15:41.897816+0000","last_active":"2023-11-28T13:15:41.897912+0000","last_peered":"2023-11-28T13:15:41.897912+0000","last_clean":"2023-11-28T13:15:41.897912+0000","last_became_active":"2023-11-28T13:07:57.305983+0000","last_became_peered":"2023-11-28T13:07:57.305983+0000","last_unstale":"2023-11-28T13:15:41.897912+0000","last_undegraded":"2023-11-28T13:15:41.897912+0000","last_fullsized":"2023-11-28T13:15:41.897912+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:15:41.897574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:15:41.897574+0000","last_clean_scrub_stamp":"2023-11-28T13:15:41.897574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:21:03.615064+0000","scrub_duration":0.0041980120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:39.980814+0000","last_change":"2023-11-28T13:15:39.980774+0000","last_active":"2023-11-28T13:15:39.980814+0000","last_peered":"2023-11-28T13:15:39.980814+0000","last_clean":"2023-11-28T13:15:39.980814+0000","last_became_active":"2023-11-28T13:07:57.305853+0000","last_became_peered":"2023-11-28T13:07:57.305853+0000","last_unstale":"2023-11-28T13:15:39.980814+0000","last_undegraded":"2023-11-28T13:15:39.980814+0000","last_fullsized":"2023-11-28T13:15:39.980814+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:15:39.980669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:15:39.980669+0000","last_clean_scrub_stamp":"2023-11-28T13:15:39.980669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:31.460666+0000","scrub_duration":0.003785596,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:47.659084+0000","last_change":"2023-11-28T13:15:47.659050+0000","last_active":"2023-11-28T13:15:47.659084+0000","last_peered":"2023-11-28T13:15:47.659084+0000","last_clean":"2023-11-28T13:15:47.659084+0000","last_became_active":"2023-11-28T13:07:57.305923+0000","last_became_peered":"2023-11-28T13:07:57.305923+0000","last_unstale":"2023-11-28T13:15:47.659084+0000","last_undegraded":"2023-11-28T13:15:47.659084+0000","last_fullsized":"2023-11-28T13:15:47.659084+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:15:47.658945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:15:47.658945+0000","last_clean_scrub_stamp":"2023-11-28T13:15:47.658945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:43.307117+0000","scrub_duration":0.0042506820000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":71,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T13:15:55.546980+0000","last_change":"2023-11-28T13:15:55.546938+0000","last_active":"2023-11-28T13:15:55.546980+0000","last_peered":"2023-11-28T13:15:55.546980+0000","last_clean":"2023-11-28T13:15:55.546980+0000","last_became_active":"2023-11-28T13:07:56.843980+0000","last_became_peered":"2023-11-28T13:07:56.843980+0000","last_unstale":"2023-11-28T13:15:55.546980+0000","last_undegraded":"2023-11-28T13:15:55.546980+0000","last_fullsized":"2023-11-28T13:15:55.546980+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:15:55.546841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:15:55.546841+0000","last_clean_scrub_stamp":"2023-11-28T13:15:55.546841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:21:17.264331+0000","scrub_duration":0.0042606509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15172,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94356668,"statfs":{"total":96636764160,"available":96621228032,"internally_reserved":0,"allocated":1372160,"data_stored":933538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68700000000000006,"15min":0.66900000000000004},"min":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89900000000000002,"5min":0.91200000000000003,"15min":0.98399999999999999},"last":0.626},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68999999999999995,"15min":0.64800000000000002},"min":{"1min":0.498,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82499999999999996,"5min":0.92300000000000004,"15min":1.0569999999999999},"last":0.47399999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73399999999999999,"15min":0.72299999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84199999999999997,"5min":0.93999999999999995,"15min":1.0289999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.73999999999999999,"15min":0.71399999999999997},"min":{"1min":0.57599999999999996,"5min":0.42399999999999999,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":0.95999999999999996,"15min":1.095},"last":0.59999999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.66500000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85499999999999998,"5min":0.92300000000000004,"15min":0.996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72599999999999998,"15min":0.69499999999999995},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.68999999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.42999999999999999},"max":{"1min":0.89000000000000001,"5min":0.91600000000000004,"15min":0.93899999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73799999999999999,"15min":0.76200000000000001},"min":{"1min":0.60399999999999998,"5min":0.52200000000000002,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.98599999999999999,"15min":1.079},"last":0.65700000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.66400000000000003},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":1.008},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.628,"15min":0.70799999999999996},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":1.006,"15min":1.071},"last":0.499}]},{"osd":5,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64200000000000002,"15min":0.73499999999999999},"min":{"1min":0.501,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86399999999999999,"5min":0.93600000000000005,"15min":1.1830000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65300000000000002,"15min":0.75900000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96299999999999997,"5min":1.044,"15min":1.1839999999999999},"last":0.52400000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71799999999999997,"15min":0.80600000000000005},"min":{"1min":0.56100000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94099999999999995,"5min":0.98599999999999999,"15min":1.165},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70199999999999996,"15min":0.76500000000000001},"min":{"1min":0.50700000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92800000000000005,"5min":0.95499999999999996,"15min":1.141},"last":0.61599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9536,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362304,"statfs":{"total":96636764160,"available":96626999296,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69899999999999995,"15min":0.67800000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97299999999999998,"5min":0.995,"15min":0.995},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.71999999999999997,"15min":0.67200000000000004},"min":{"1min":0.5,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.98099999999999998,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.97199999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71899999999999997,"15min":0.69199999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.046,"5min":1.113,"15min":1.113},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71599999999999997,"15min":0.67500000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93400000000000005,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.82799999999999996}]},{"osd":2,"last update":"Tue Nov 28 13:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72799999999999998,"15min":0.69299999999999995},"min":{"1min":0.54000000000000004,"5min":0.45000000000000001,"15min":0.41799999999999998},"max":{"1min":0.95999999999999996,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.874},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72599999999999998,"15min":0.68999999999999995},"min":{"1min":0.55000000000000004,"5min":0.51000000000000001,"15min":0.38500000000000001},"max":{"1min":0.96799999999999997,"5min":0.97699999999999998,"15min":1.024},"last":0.84299999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.746,"15min":0.70099999999999996},"min":{"1min":0.56299999999999994,"5min":0.46100000000000002,"15min":0.436},"max":{"1min":0.94899999999999995,"5min":1.121,"15min":1.121},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.93600000000000005,"5min":1.167,"15min":1.167},"last":0.90700000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.34499999999999997},"max":{"1min":0.89500000000000002,"5min":1.042,"15min":1.0429999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.74399999999999999,"15min":0.79000000000000004},"min":{"1min":0.64300000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97699999999999998,"5min":1.0289999999999999,"15min":1.3140000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Tue Nov 28 13:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63600000000000001,"15min":0.68100000000000005},"min":{"1min":0.42199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91000000000000003,"5min":0.92100000000000004,"15min":1.0780000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.70399999999999996},"min":{"1min":0.48899999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92300000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.73099999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70199999999999996,"15min":0.72799999999999998},"min":{"1min":0.53700000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94199999999999995,"5min":1.034,"15min":1.0980000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.76500000000000001},"min":{"1min":0.499,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96899999999999997,"5min":1.0129999999999999,"15min":1.107},"last":0.80300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14916,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":13568,"kb_avail":94356924,"statfs":{"total":96636764160,"available":96621490176,"internally_reserved":0,"allocated":1372160,"data_stored":933538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13893632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41799999999999998,"15min":0.43099999999999999},"min":{"1min":0.28100000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.496,"5min":0.75900000000000001,"15min":1.28},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.47899999999999998,"15min":0.45600000000000002},"min":{"1min":0.30499999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64000000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.40799999999999997}]},{"osd":2,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.46000000000000002},"min":{"1min":0.33200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60599999999999998,"5min":0.747,"15min":0.747},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.67500000000000004,"5min":0.77000000000000002,"15min":0.84699999999999998},"last":0.57399999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.436,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71099999999999997,"5min":0.88500000000000001,"15min":1.0009999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52000000000000002,"15min":0.53800000000000003},"min":{"1min":0.36899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.68899999999999995,"5min":0.746,"15min":1.1120000000000001},"last":0.61799999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73099999999999998,"15min":0.70799999999999996},"min":{"1min":0.60299999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.753,"15min":0.72199999999999998},"min":{"1min":0.624,"5min":0.54100000000000004,"15min":0.49099999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.98599999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.74299999999999999,"15min":0.745},"min":{"1min":0.61199999999999999,"5min":0.51700000000000002,"15min":0.433},"max":{"1min":1.0209999999999999,"5min":1.292,"15min":1.292},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.74299999999999999,"15min":0.72099999999999997},"min":{"1min":0.621,"5min":0.441,"15min":0.441},"max":{"1min":1.141,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.79300000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.74199999999999999},"min":{"1min":0.64300000000000002,"5min":0.55300000000000005,"15min":0.52700000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.47299999999999998},"max":{"1min":1.1240000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.96199999999999997}]},{"osd":7,"last update":"Tue Nov 28 13:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.80000000000000004,"15min":0.81000000000000005},"min":{"1min":0.66900000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0820000000000001,"5min":1.357,"15min":1.357},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77800000000000002,"15min":0.78400000000000003},"min":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0389999999999999,"5min":1.24,"15min":1.3580000000000001},"last":0.91300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574952,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13884,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":13120,"kb_avail":94357956,"statfs":{"total":96636764160,"available":96622546944,"internally_reserved":0,"allocated":774144,"data_stored":342184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13434880},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.433,"15min":0.47199999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.61799999999999999,"5min":0.65400000000000003,"15min":1.3120000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.47699999999999998,"15min":0.51000000000000001},"min":{"1min":0.28100000000000003,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":1.48},"last":0.50900000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.501,"15min":0.51800000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70399999999999996,"5min":0.874,"15min":0.874},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52800000000000002,"15min":0.51400000000000001},"min":{"1min":0.38500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.65700000000000003,"5min":0.749,"15min":0.80800000000000005},"last":0.65700000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48199999999999998,"15min":0.47599999999999998},"min":{"1min":0.34899999999999998,"5min":0.33700000000000002,"15min":0.30099999999999999},"max":{"1min":0.63500000000000001,"5min":0.747,"15min":0.747},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.36699999999999999,"15min":0.35399999999999998},"max":{"1min":0.746,"5min":0.81200000000000006,"15min":0.82199999999999995},"last":0.624}]},{"osd":4,"last update":"Tue Nov 28 13:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77000000000000002,"15min":0.77500000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.45500000000000002},"max":{"1min":1.03,"5min":1.056,"15min":1.071},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78300000000000003,"15min":0.77800000000000002},"min":{"1min":0.52900000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.133,"5min":1.147,"15min":1.147},"last":0.92400000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77800000000000002,"15min":0.78700000000000003},"min":{"1min":0.54800000000000004,"5min":0.51200000000000001,"15min":0.48299999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.3260000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80400000000000005,"15min":0.79400000000000004},"min":{"1min":0.58099999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.097,"5min":1.127,"15min":1.127},"last":0.86599999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78000000000000003,"15min":0.76300000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0369999999999999,"5min":1.127,"15min":1.127},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81100000000000005,"15min":0.81399999999999995},"min":{"1min":0.56200000000000006,"5min":0.55600000000000005,"15min":0.52200000000000002},"max":{"1min":1.0429999999999999,"5min":1.0489999999999999,"15min":1.097},"last":0.95999999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79900000000000004,"15min":0.80200000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.056,"5min":1.149,"15min":1.149},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80100000000000005,"15min":0.80000000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.01,"5min":1.163,"15min":1.163},"last":0.90300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574951,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13696,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":12928,"kb_avail":94358144,"statfs":{"total":96636764160,"available":96622739456,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13238272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.438,"15min":0.44700000000000001},"min":{"1min":0.28999999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":6.4130000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.46400000000000002,"15min":0.52000000000000002},"min":{"1min":0.28999999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57499999999999996,"5min":0.73899999999999999,"15min":19.190000000000001},"last":0.48299999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.48199999999999998,"15min":0.50700000000000001},"min":{"1min":0.33600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.69099999999999995,"5min":0.70099999999999996,"15min":5.2009999999999996},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52400000000000002,"15min":0.57299999999999995},"min":{"1min":0.35599999999999998,"5min":0.35099999999999998,"15min":0.308},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":19.241},"last":0.59499999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.48799999999999999},"min":{"1min":0.318,"5min":0.29899999999999999,"15min":0.29799999999999999},"max":{"1min":0.68400000000000005,"5min":0.70099999999999996,"15min":7.0410000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.58399999999999996},"min":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.745,"5min":0.83299999999999996,"15min":11.420999999999999},"last":0.68100000000000005}]},{"osd":4,"last update":"Tue Nov 28 13:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.81000000000000005,"15min":0.82899999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.55000000000000004},"max":{"1min":1.1799999999999999,"5min":1.3340000000000001,"15min":5.1630000000000003},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.79800000000000004,"15min":0.79500000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1299999999999999,"5min":1.232,"15min":1.9690000000000001},"last":1.109}]},{"osd":5,"last update":"Tue Nov 28 13:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.755,"15min":0.74199999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.46100000000000002},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":2.0390000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.57399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.091,"5min":1.2669999999999999,"15min":6.7569999999999997},"last":1.119}]},{"osd":6,"last update":"Tue Nov 28 13:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.79000000000000004,"15min":0.78200000000000003},"min":{"1min":0.55900000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1830000000000001,"5min":1.3500000000000001,"15min":6.7220000000000004},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.81299999999999994,"15min":0.81000000000000005},"min":{"1min":0.56100000000000005,"5min":0.55700000000000005,"15min":0.49199999999999999},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":5.1269999999999998},"last":1.018}]},{"osd":7,"last update":"Tue Nov 28 13:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.78300000000000003,"15min":0.77000000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":2.008},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.77500000000000002,"15min":0.80700000000000005},"min":{"1min":0.59799999999999998,"5min":0.52700000000000002,"15min":0.48399999999999999},"max":{"1min":1.0640000000000001,"5min":1.2330000000000001,"15min":6.6769999999999996},"last":0.93799999999999994}]}]},{"osd":3,"up_from":13,"seq":55834574951,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14016,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":13248,"kb_avail":94357824,"statfs":{"total":96636764160,"available":96622411776,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13565952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.439,"15min":0.42499999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.245},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.46100000000000002},"min":{"1min":0.32400000000000001,"5min":0.30599999999999999,"15min":0.27300000000000002},"max":{"1min":0.71299999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54700000000000004}]},{"osd":1,"last update":"Tue Nov 28 13:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47599999999999998,"15min":0.48099999999999998},"min":{"1min":0.32600000000000001,"5min":0.32000000000000001,"15min":0.28699999999999998},"max":{"1min":0.68999999999999995,"5min":0.73699999999999999,"15min":0.88100000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52200000000000002,"15min":0.51400000000000001},"min":{"1min":0.36899999999999999,"5min":0.33000000000000002,"15min":0.30499999999999999},"max":{"1min":0.65800000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55200000000000005,"15min":0.53300000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.36399999999999999},"max":{"1min":0.66700000000000004,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57999999999999996,"15min":0.55000000000000004},"min":{"1min":0.40300000000000002,"5min":0.39000000000000001,"15min":0.35399999999999998},"max":{"1min":0.77000000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.59199999999999997}]},{"osd":4,"last update":"Tue Nov 28 13:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72799999999999998,"15min":0.70699999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73299999999999998,"15min":0.71499999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.46200000000000002},"max":{"1min":0.94999999999999996,"5min":1.087,"15min":1.087},"last":0.70599999999999996}]},{"osd":5,"last update":"Tue Nov 28 13:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.77400000000000002,"15min":0.76800000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.95999999999999996,"5min":1.05,"15min":1.091},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78300000000000003,"15min":0.76700000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.498},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.83599999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76700000000000002,"15min":0.73699999999999999},"min":{"1min":0.60099999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75800000000000001,"15min":0.72199999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.46100000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.752}]},{"osd":7,"last update":"Tue Nov 28 13:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79900000000000004,"15min":0.77100000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80800000000000005,"15min":0.77500000000000002},"min":{"1min":0.58899999999999997,"5min":0.53600000000000003,"15min":0.52800000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.1299999999999999},"last":0.86299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574950,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9920,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361920,"statfs":{"total":96636764160,"available":96626606080,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.69599999999999995,"15min":0.63300000000000001},"min":{"1min":0.45000000000000001,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.89300000000000002,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70799999999999996,"15min":0.68700000000000006},"min":{"1min":0.45100000000000001,"5min":0.42799999999999999,"15min":0.38400000000000001},"max":{"1min":0.92900000000000005,"5min":0.96399999999999997,"15min":1.3089999999999999},"last":0.76100000000000001}]},{"osd":1,"last update":"Tue Nov 28 13:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72699999999999998,"15min":0.69799999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.48299999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.161},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69299999999999995,"15min":0.63600000000000001},"min":{"1min":0.47699999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.878,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.72599999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.70799999999999996,"15min":0.67700000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.755,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.74199999999999999,"15min":0.69999999999999996},"min":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.441},"max":{"1min":0.88800000000000001,"5min":0.95499999999999996,"15min":1.042},"last":0.79300000000000004}]},{"osd":3,"last update":"Tue Nov 28 13:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70999999999999996,"15min":0.68600000000000005},"min":{"1min":0.48899999999999999,"5min":0.46000000000000002,"15min":0.438},"max":{"1min":0.85299999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.74399999999999999,"15min":0.72599999999999998},"min":{"1min":0.55700000000000005,"5min":0.52900000000000003,"15min":0.505},"max":{"1min":0.81299999999999994,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.746}]},{"osd":5,"last update":"Tue Nov 28 13:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60899999999999999,"15min":0.65400000000000003},"min":{"1min":0.41399999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77200000000000002,"5min":0.95799999999999996,"15min":1.264},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64000000000000001,"15min":0.69699999999999995},"min":{"1min":0.42799999999999999,"5min":0.41299999999999998,"15min":0.38400000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":1.079},"last":0.46600000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.64100000000000001,"15min":0.69799999999999995},"min":{"1min":0.441,"5min":0.439,"15min":0.32700000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":1.022},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68799999999999994,"15min":0.72699999999999998},"min":{"1min":0.52300000000000002,"5min":0.48699999999999999,"15min":0.47399999999999998},"max":{"1min":0.86799999999999999,"5min":0.90900000000000003,"15min":1.22},"last":0.58199999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68799999999999994,"15min":0.73999999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.45000000000000001},"max":{"1min":0.92600000000000005,"5min":0.98699999999999999,"15min":1.4530000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.71399999999999997,"15min":0.76000000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40899999999999997},"max":{"1min":0.84699999999999998,"5min":1.0089999999999999,"15min":1.018},"last":0.48699999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9792,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362048,"statfs":{"total":96636764160,"available":96626737152,"internally_reserved":0,"allocated":778240,"data_stored":343170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67900000000000005,"15min":0.63700000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.875,"5min":0.94499999999999995,"15min":1.659},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68999999999999995,"15min":0.64000000000000001},"min":{"1min":0.496,"5min":0.41399999999999998,"15min":0.38900000000000001},"max":{"1min":0.92200000000000004,"5min":0.95899999999999996,"15min":2.3050000000000002},"last":0.873}]},{"osd":1,"last update":"Tue Nov 28 13:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67400000000000004,"15min":0.626},"min":{"1min":0.46200000000000002,"5min":0.44600000000000001,"15min":0.35499999999999998},"max":{"1min":0.89200000000000002,"5min":0.94299999999999995,"15min":1.2390000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.70299999999999996,"15min":0.65600000000000003},"min":{"1min":0.45200000000000001,"5min":0.443,"15min":0.39400000000000002},"max":{"1min":0.79100000000000004,"5min":0.97599999999999998,"15min":1.9630000000000001},"last":0.95499999999999996}]},{"osd":2,"last update":"Tue Nov 28 13:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.69199999999999995,"15min":0.66200000000000003},"min":{"1min":0.52500000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90300000000000002,"5min":0.90700000000000003,"15min":2.3860000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70199999999999996,"15min":0.66600000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91100000000000003,"5min":0.97299999999999998,"15min":1.0800000000000001},"last":0.85499999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70899999999999996,"15min":0.66500000000000004},"min":{"1min":0.52500000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.84999999999999998,"5min":0.99099999999999999,"15min":1.8140000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72499999999999998,"15min":0.71399999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95899999999999996,"15min":2.3500000000000001},"last":0.90700000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.65000000000000002},"min":{"1min":0.47799999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.82499999999999996,"5min":0.89800000000000002,"15min":1.702},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.70499999999999996,"15min":0.72699999999999998},"min":{"1min":0.48299999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89600000000000002,"5min":0.97199999999999998,"15min":2.0009999999999999},"last":0.75700000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.64100000000000001,"15min":0.68999999999999995},"min":{"1min":0.46400000000000002,"5min":0.42999999999999999,"15min":0.38800000000000001},"max":{"1min":0.83199999999999996,"5min":0.89400000000000002,"15min":2.4969999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66100000000000003,"15min":0.70299999999999996},"min":{"1min":0.51200000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89200000000000002,"5min":1.0069999999999999,"15min":2.0880000000000001},"last":0.621}]},{"osd":7,"last update":"Tue Nov 28 13:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68999999999999995,"15min":0.71199999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.41099999999999998},"max":{"1min":0.87,"5min":0.999,"15min":2.0449999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73599999999999999,"15min":0.75600000000000001},"min":{"1min":0.55700000000000005,"5min":0.49199999999999999,"15min":0.443},"max":{"1min":0.90600000000000003,"5min":0.96999999999999997,"15min":2.4249999999999998},"last":0.68000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:16:21.436 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-28T13:16:21.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.627+0000 7f51341ff700 1 Processor -- start 2023-11-28T13:16:21.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.627+0000 7f51341ff700 1 -- start start 2023-11-28T13:16:21.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.628+0000 7f51341ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f512c159490 0x7f512c0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:21.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.628+0000 7f51341ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f512c0cdc80 0x7f512c0b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:21.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.628+0000 7f51341ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512c0ccc00 con 0x7f512c159860 2023-11-28T13:16:21.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.628+0000 7f51341ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512c0ccec0 con 0x7f512c159490 2023-11-28T13:16:21.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.628+0000 7f51341ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512c0cd1a0 con 0x7f512c0cdc80 2023-11-28T13:16:21.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.628+0000 7f5131f9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f512c159490 0x7f512c0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:21.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.628+0000 7f5131f9b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f512c159490 0x7f512c0b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35504/0 (socket says 172.21.15.156:35504) 2023-11-28T13:16:21.630 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.628+0000 7f5131f9b700 1 -- 172.21.15.156:0/3388677323 learned_addr learned my addr 172.21.15.156:0/3388677323 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:16:21.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.628+0000 7f513179a700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f512c159860 0x7f512c0c94c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:59366/0 (socket says 172.21.15.156:59366) 2023-11-28T13:16:21.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.629+0000 7f513279c700 1 --2- 172.21.15.156:0/3388677323 >> v2:172.21.15.186:3300/0 conn(0x7f512c0cdc80 0x7f512c0b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:21.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.629+0000 7f5131f9b700 1 -- 172.21.15.156:0/3388677323 >> v2:172.21.15.186:3300/0 conn(0x7f512c0cdc80 msgr2=0x7f512c0b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:16:21.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.629+0000 7f5131f9b700 1 --2- 172.21.15.156:0/3388677323 >> v2:172.21.15.186:3300/0 conn(0x7f512c0cdc80 0x7f512c0b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.629+0000 7f5131f9b700 1 -- 172.21.15.156:0/3388677323 >> v1:172.21.15.156:6789/0 conn(0x7f512c159860 legacy=0x7f512c0c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:21.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.629+0000 7f5130f99700 1 -- 172.21.15.156:0/3388677323 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778412173 0 0) 0x7f512c0ccc00 con 0x7f512c159860 2023-11-28T13:16:21.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.629+0000 7f5131f9b700 1 -- 172.21.15.156:0/3388677323 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c0cd480 con 0x7f512c159490 2023-11-28T13:16:21.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.629+0000 7f513279c700 1 --2- 172.21.15.156:0/3388677323 >> v2:172.21.15.186:3300/0 conn(0x7f512c0cdc80 0x7f512c0b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:16:21.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.629+0000 7f5131f9b700 1 --2- 172.21.15.156:0/3388677323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f512c159490 0x7f512c0b0670 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f512c0c5be0 tx=0x7f5128009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a042311835f1d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:16:21.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.629+0000 7f5130f99700 1 -- 172.21.15.156:0/3388677323 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512800b870 con 0x7f512c159490 2023-11-28T13:16:21.633 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.629+0000 7f5130f99700 1 -- 172.21.15.156:0/3388677323 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512800b9d0 con 0x7f512c159490 2023-11-28T13:16:21.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.629+0000 7f5130f99700 1 -- 172.21.15.156:0/3388677323 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512800bd60 con 0x7f512c159490 2023-11-28T13:16:21.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.630+0000 7f51341ff700 1 -- 172.21.15.156:0/3388677323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f512c159490 msgr2=0x7f512c0b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:21.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.630+0000 7f51341ff700 1 --2- 172.21.15.156:0/3388677323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f512c159490 0x7f512c0b0670 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f512c0c5be0 tx=0x7f5128009bd0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.634 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.630+0000 7f51341ff700 1 -- 172.21.15.156:0/3388677323 shutdown_connections 2023-11-28T13:16:21.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.630+0000 7f51341ff700 1 --2- 172.21.15.156:0/3388677323 >> v2:172.21.15.186:3300/0 conn(0x7f512c0cdc80 0x7f512c0b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.630+0000 7f51341ff700 1 --2- 172.21.15.156:0/3388677323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f512c159490 0x7f512c0b0670 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.630+0000 7f51341ff700 1 -- 172.21.15.156:0/3388677323 >> 172.21.15.156:0/3388677323 conn(0x7f512c0cb670 msgr2=0x7f512c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:16:21.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.630+0000 7f51341ff700 1 -- 172.21.15.156:0/3388677323 shutdown_connections 2023-11-28T13:16:21.635 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.630+0000 7f51341ff700 1 -- 172.21.15.156:0/3388677323 wait complete. 2023-11-28T13:16:21.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.632+0000 7f51341ff700 1 Processor -- start 2023-11-28T13:16:21.637 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.633+0000 7f51341ff700 1 -- start start 2023-11-28T13:16:21.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.634+0000 7f51341ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f512c0cdc80 0x7f512c0c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:21.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.634+0000 7f51341ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f512c159490 0x7f512c1f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:21.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.634+0000 7f51341ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f512c152840 0x7f512c152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:21.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.634+0000 7f51341ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0c7070 con 0x7f512c159490 2023-11-28T13:16:21.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.634+0000 7f51341ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0ac8c0 con 0x7f512c152840 2023-11-28T13:16:21.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.634+0000 7f51341ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0c6170 con 0x7f512c0cdc80 2023-11-28T13:16:21.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.634+0000 7f513179a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f512c159490 0x7f512c1f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:21.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.634+0000 7f513179a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f512c159490 0x7f512c1f1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35518/0 (socket says 172.21.15.156:35518) 2023-11-28T13:16:21.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.634+0000 7f513179a700 1 -- 172.21.15.156:0/271308474 learned_addr learned my addr 172.21.15.156:0/271308474 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:16:21.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.634+0000 7f5131f9b700 1 --2- 172.21.15.156:0/271308474 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f512c0cdc80 0x7f512c0c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:21.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.634+0000 7f513279c700 1 --2- 172.21.15.156:0/271308474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f512c152840 0x7f512c152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:21.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.635+0000 7f513179a700 1 -- 172.21.15.156:0/271308474 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f512c0cdc80 msgr2=0x7f512c0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:21.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.635+0000 7f513179a700 1 --2- 172.21.15.156:0/271308474 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f512c0cdc80 0x7f512c0c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.635+0000 7f513179a700 1 -- 172.21.15.156:0/271308474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f512c152840 msgr2=0x7f512c152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:21.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.635+0000 7f513179a700 1 --2- 172.21.15.156:0/271308474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f512c152840 0x7f512c152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.635+0000 7f513179a700 1 -- 172.21.15.156:0/271308474 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5128009550 con 0x7f512c159490 2023-11-28T13:16:21.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.635+0000 7f513179a700 1 --2- 172.21.15.156:0/271308474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f512c159490 0x7f512c1f1ca0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f511c00c550 tx=0x7f511c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:16:21.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.635+0000 7f51227fc700 1 -- 172.21.15.156:0/271308474 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c004c80 con 0x7f512c159490 2023-11-28T13:16:21.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.635+0000 7f51227fc700 1 -- 172.21.15.156:0/271308474 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511c004e00 con 0x7f512c159490 2023-11-28T13:16:21.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.635+0000 7f51341ff700 1 -- 172.21.15.156:0/271308474 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512c0c9bd0 con 0x7f512c159490 2023-11-28T13:16:21.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.635+0000 7f51227fc700 1 -- 172.21.15.156:0/271308474 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c007310 con 0x7f512c159490 2023-11-28T13:16:21.640 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.635+0000 7f51341ff700 1 -- 172.21.15.156:0/271308474 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512c0cd760 con 0x7f512c159490 2023-11-28T13:16:21.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.636+0000 7f51341ff700 1 -- 172.21.15.156:0/271308474 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512c0c9d60 con 0x7f512c159490 2023-11-28T13:16:21.641 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.637+0000 7f51227fc700 1 -- 172.21.15.156:0/271308474 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f511c01a020 con 0x7f512c159490 2023-11-28T13:16:21.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.638+0000 7f51227fc700 1 --2- 172.21.15.156:0/271308474 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f51100682f0 0x7f511006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:21.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.638+0000 7f51227fc700 1 -- 172.21.15.156:0/271308474 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f511c085530 con 0x7f512c159490 2023-11-28T13:16:21.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.638+0000 7f5131f9b700 1 --2- 172.21.15.156:0/271308474 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f51100682f0 0x7f511006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:21.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.639+0000 7f5131f9b700 1 --2- 172.21.15.156:0/271308474 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f51100682f0 0x7f511006a7a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5128000c00 tx=0x7f5128008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:16:21.644 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.642+0000 7f51227fc700 1 -- 172.21.15.156:0/271308474 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f511c052000 con 0x7f512c159490 2023-11-28T13:16:21.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.833+0000 7f51341ff700 1 -- 172.21.15.156:0/271308474 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f512c0c71b0 con 0x7f512c159490 2023-11-28T13:16:21.839 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.838+0000 7f51227fc700 1 -- 172.21.15.156:0/271308474 <== mon.0 v2:172.21.15.156:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f511c0011d0 con 0x7f512c159490 2023-11-28T13:16:21.840 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.839+0000 7f51227fc700 1 -- 172.21.15.156:0/271308474 <== mon.0 v2:172.21.15.156:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7f512c0c71b0 con 0x7f512c159490 2023-11-28T13:16:21.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.841+0000 7f51341ff700 1 -- 172.21.15.156:0/271308474 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f51100682f0 msgr2=0x7f511006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:21.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.841+0000 7f51341ff700 1 --2- 172.21.15.156:0/271308474 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f51100682f0 0x7f511006a7a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5128000c00 tx=0x7f5128008030 comp rx=0 tx=0).stop 2023-11-28T13:16:21.843 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.841+0000 7f51341ff700 1 -- 172.21.15.156:0/271308474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f512c159490 msgr2=0x7f512c1f1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:21.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.841+0000 7f51341ff700 1 --2- 172.21.15.156:0/271308474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f512c159490 0x7f512c1f1ca0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f511c00c550 tx=0x7f511c00c860 comp rx=0 tx=0).stop 2023-11-28T13:16:21.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.841+0000 7f51341ff700 1 -- 172.21.15.156:0/271308474 shutdown_connections 2023-11-28T13:16:21.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.841+0000 7f51341ff700 1 --2- 172.21.15.156:0/271308474 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f51100682f0 0x7f511006a7a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.841+0000 7f51341ff700 1 --2- 172.21.15.156:0/271308474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f512c152840 0x7f512c152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.841+0000 7f51341ff700 1 --2- 172.21.15.156:0/271308474 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f512c0cdc80 0x7f512c0c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.841+0000 7f51341ff700 1 --2- 172.21.15.156:0/271308474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f512c159490 0x7f512c1f1ca0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:21.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.841+0000 7f51341ff700 1 -- 172.21.15.156:0/271308474 >> 172.21.15.156:0/271308474 conn(0x7f512c0cb670 msgr2=0x7f512c0b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:16:21.844 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.841+0000 7f51341ff700 1 -- 172.21.15.156:0/271308474 shutdown_connections 2023-11-28T13:16:21.845 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:21.841+0000 7f51341ff700 1 -- 172.21.15.156:0/271308474 wait complete. 2023-11-28T13:16:21.861 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T13:16:21.862 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:16:22.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.057+0000 7f785b5ff700 1 Processor -- start 2023-11-28T13:16:22.063 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.057+0000 7f785b5ff700 1 -- start start 2023-11-28T13:16:22.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.058+0000 7f785b5ff700 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7854161340 0x7f78540d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:22.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.059+0000 7f785b5ff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78540d1950 con 0x7f78540d2320 2023-11-28T13:16:22.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.059+0000 7f785b5ff700 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78540b5520 con 0x7f785415c9b0 2023-11-28T13:16:22.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.059+0000 7f785b5ff700 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78540b5800 con 0x7f7854161340 2023-11-28T13:16:22.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.059+0000 7f7859b9c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f78540d2320 0x7f78540cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:59372/0 (socket says 172.21.15.156:59372) 2023-11-28T13:16:22.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.059+0000 7f7859b9c700 1 -- 172.21.15.156:0/2429767501 learned_addr learned my addr 172.21.15.156:0/2429767501 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-28T13:16:22.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.059+0000 7f7858b9a700 1 --2- 172.21.15.156:0/2429767501 >> v2:172.21.15.186:3300/0 conn(0x7f7854161340 0x7f78540d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:22.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.059+0000 7f784bfff700 1 -- 172.21.15.156:0/2429767501 <== mon.0 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473760300 0 0) 0x7f78540d1950 con 0x7f78540d2320 2023-11-28T13:16:22.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.059+0000 7f784bfff700 1 -- 172.21.15.156:0/2429767501 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78400035b0 con 0x7f78540d2320 2023-11-28T13:16:22.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.059+0000 7f784bfff700 1 -- 172.21.15.156:0/2429767501 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4073407535 0 0) 0x7f78540b5520 con 0x7f785415c9b0 2023-11-28T13:16:22.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.059+0000 7f784bfff700 1 -- 172.21.15.156:0/2429767501 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78540d1950 con 0x7f785415c9b0 2023-11-28T13:16:22.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f784bfff700 1 -- 172.21.15.156:0/2429767501 <== mon.0 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3173702548 0 0) 0x7f78400035b0 con 0x7f78540d2320 2023-11-28T13:16:22.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f784bfff700 1 -- 172.21.15.156:0/2429767501 >> v2:172.21.15.186:3300/0 conn(0x7f7854161340 msgr2=0x7f78540d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:16:22.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f784bfff700 1 --2- 172.21.15.156:0/2429767501 >> v2:172.21.15.186:3300/0 conn(0x7f7854161340 0x7f78540d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:22.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f784bfff700 1 -- 172.21.15.156:0/2429767501 >> v1:172.21.15.186:6789/0 conn(0x7f785415c9b0 legacy=0x7f78540b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:22.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f784bfff700 1 -- 172.21.15.156:0/2429767501 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78540d1300 con 0x7f78540d2320 2023-11-28T13:16:22.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f784bfff700 1 -- 172.21.15.156:0/2429767501 <== mon.0 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 522657264 0 0) 0x7f784c0019f0 con 0x7f78540d2320 2023-11-28T13:16:22.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f784bfff700 1 -- 172.21.15.156:0/2429767501 >> v1:172.21.15.156:6789/0 conn(0x7f78540d2320 legacy=0x7f78540cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:22.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f784bfff700 1 --2- 172.21.15.156:0/2429767501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7840003be0 0x7f7840003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:22.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f784bfff700 1 --2- 172.21.15.156:0/2429767501 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7840004850 0x7f7840004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:22.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f784bfff700 1 --2- 172.21.15.156:0/2429767501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7840005250 0x7f784000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:22.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f784bfff700 1 -- 172.21.15.156:0/2429767501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78540b5800 con 0x7f7840003be0 2023-11-28T13:16:22.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f784bfff700 1 -- 172.21.15.156:0/2429767501 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784000c160 con 0x7f7840005250 2023-11-28T13:16:22.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f784bfff700 1 -- 172.21.15.156:0/2429767501 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784000c320 con 0x7f7840004850 2023-11-28T13:16:22.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f785939b700 1 --2- 172.21.15.156:0/2429767501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7840003be0 0x7f7840003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:22.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f7858b9a700 1 --2- 172.21.15.156:0/2429767501 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7840004850 0x7f7840004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:22.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f7859b9c700 1 --2- 172.21.15.156:0/2429767501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7840005250 0x7f784000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:22.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f785939b700 1 -- 172.21.15.156:0/2429767501 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7840004850 msgr2=0x7f7840004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:16:22.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f785939b700 1 --2- 172.21.15.156:0/2429767501 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7840004850 0x7f7840004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:22.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f785939b700 1 -- 172.21.15.156:0/2429767501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7840005250 msgr2=0x7f784000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:16:22.067 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f785939b700 1 --2- 172.21.15.156:0/2429767501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7840005250 0x7f784000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:22.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f785939b700 1 -- 172.21.15.156:0/2429767501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78400035b0 con 0x7f7840003be0 2023-11-28T13:16:22.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.060+0000 7f7859b9c700 1 -- 172.21.15.156:0/2429767501 reap_dead start 2023-11-28T13:16:22.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.061+0000 7f785939b700 1 --2- 172.21.15.156:0/2429767501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7840003be0 0x7f7840003fd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7850007fd0 tx=0x7f7840005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2392a9e2581eda server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:16:22.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.061+0000 7f784bfff700 1 -- 172.21.15.156:0/2429767501 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7850001430 con 0x7f7840003be0 2023-11-28T13:16:22.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.061+0000 7f784bfff700 1 -- 172.21.15.156:0/2429767501 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f78500052f0 con 0x7f7840003be0 2023-11-28T13:16:22.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.061+0000 7f784bfff700 1 -- 172.21.15.156:0/2429767501 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7850001d40 con 0x7f7840003be0 2023-11-28T13:16:22.068 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.061+0000 7f785b5ff700 1 -- 172.21.15.156:0/2429767501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7840003be0 msgr2=0x7f7840003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:22.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.061+0000 7f785b5ff700 1 --2- 172.21.15.156:0/2429767501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7840003be0 0x7f7840003fd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7850007fd0 tx=0x7f7840005880 comp rx=0 tx=0).stop 2023-11-28T13:16:22.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.061+0000 7f785b5ff700 1 -- 172.21.15.156:0/2429767501 shutdown_connections 2023-11-28T13:16:22.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.062+0000 7f785b5ff700 1 --2- 172.21.15.156:0/2429767501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7840003be0 0x7f7840003fd0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:22.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.062+0000 7f785b5ff700 1 -- 172.21.15.156:0/2429767501 >> 172.21.15.156:0/2429767501 conn(0x7f7854072600 msgr2=0x7f78540cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:16:22.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.062+0000 7f785b5ff700 1 -- 172.21.15.156:0/2429767501 shutdown_connections 2023-11-28T13:16:22.070 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.062+0000 7f785b5ff700 1 -- 172.21.15.156:0/2429767501 wait complete. 2023-11-28T13:16:22.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.063+0000 7f785b5ff700 1 Processor -- start 2023-11-28T13:16:22.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.063+0000 7f785b5ff700 1 -- start start 2023-11-28T13:16:22.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.064+0000 7f785b5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7840003be0 0x7f785414a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:22.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.064+0000 7f785b5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f785415f060 0x7f78540c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:22.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.064+0000 7f785b5ff700 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78540c93b0 0x7f785415d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:22.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.064+0000 7f785b5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78540c9820 con 0x7f785415f060 2023-11-28T13:16:22.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.064+0000 7f785b5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785415dfc0 con 0x7f7840003be0 2023-11-28T13:16:22.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.064+0000 7f785b5ff700 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785415e2a0 con 0x7f78540c93b0 2023-11-28T13:16:22.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.065+0000 7f7858b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f785415f060 0x7f78540c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:22.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.065+0000 7f7858b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f785415f060 0x7f78540c8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35530/0 (socket says 172.21.15.156:35530) 2023-11-28T13:16:22.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.065+0000 7f7858b9a700 1 -- 172.21.15.156:0/2020599739 learned_addr learned my addr 172.21.15.156:0/2020599739 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-28T13:16:22.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.065+0000 7f785939b700 1 --2- 172.21.15.156:0/2020599739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7840003be0 0x7f785414a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:22.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.065+0000 7f7859b9c700 1 --2- 172.21.15.156:0/2020599739 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78540c93b0 0x7f785415d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:22.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.065+0000 7f7858b9a700 1 -- 172.21.15.156:0/2020599739 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78540c93b0 msgr2=0x7f785415d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:22.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.065+0000 7f7858b9a700 1 --2- 172.21.15.156:0/2020599739 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78540c93b0 0x7f785415d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:22.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.065+0000 7f7858b9a700 1 -- 172.21.15.156:0/2020599739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7840003be0 msgr2=0x7f785414a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:22.074 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.065+0000 7f7858b9a700 1 --2- 172.21.15.156:0/2020599739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7840003be0 0x7f785414a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:22.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.065+0000 7f7858b9a700 1 -- 172.21.15.156:0/2020599739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785415cd80 con 0x7f785415f060 2023-11-28T13:16:22.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.065+0000 7f7858b9a700 1 --2- 172.21.15.156:0/2020599739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f785415f060 0x7f78540c8e70 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7844011aa0 tx=0x7f7844011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:16:22.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.065+0000 7f7849ffb700 1 -- 172.21.15.156:0/2020599739 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844011ee0 con 0x7f785415f060 2023-11-28T13:16:22.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.066+0000 7f7849ffb700 1 -- 172.21.15.156:0/2020599739 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f784400f940 con 0x7f785415f060 2023-11-28T13:16:22.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.066+0000 7f785b5ff700 1 -- 172.21.15.156:0/2020599739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78540d15e0 con 0x7f785415f060 2023-11-28T13:16:22.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.066+0000 7f7849ffb700 1 -- 172.21.15.156:0/2020599739 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784400fd30 con 0x7f785415f060 2023-11-28T13:16:22.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.066+0000 7f785b5ff700 1 -- 172.21.15.156:0/2020599739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785415cf10 con 0x7f785415f060 2023-11-28T13:16:22.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.067+0000 7f785b5ff700 1 -- 172.21.15.156:0/2020599739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f785415eeb0 con 0x7f785415f060 2023-11-28T13:16:22.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.067+0000 7f7849ffb700 1 -- 172.21.15.156:0/2020599739 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7844014440 con 0x7f785415f060 2023-11-28T13:16:22.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.068+0000 7f7849ffb700 1 --2- 172.21.15.156:0/2020599739 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78380682f0 0x7f783806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:16:22.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.068+0000 7f7849ffb700 1 -- 172.21.15.156:0/2020599739 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7844086200 con 0x7f785415f060 2023-11-28T13:16:22.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.069+0000 7f785939b700 1 --2- 172.21.15.156:0/2020599739 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78380682f0 0x7f783806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:16:22.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.069+0000 7f785939b700 1 --2- 172.21.15.156:0/2020599739 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78380682f0 0x7f783806a7a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7850015040 tx=0x7f785000a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:16:22.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.072+0000 7f7849ffb700 1 -- 172.21.15.156:0/2020599739 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7844052da0 con 0x7f785415f060 2023-11-28T13:16:22.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.303+0000 7f785b5ff700 1 -- 172.21.15.156:0/2020599739 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78540745c0 con 0x7f785415f060 2023-11-28T13:16:22.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.304+0000 7f7849ffb700 1 -- 172.21.15.156:0/2020599739 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f7844022df0 con 0x7f785415f060 2023-11-28T13:16:22.306 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-28T13:16:22.306 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3942362276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3942362276},{"type":"v1","addr":"172.21.15.186:6835","nonce":3942362276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/722198303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":722198303},{"type":"v1","addr":"172.21.15.186:6837","nonce":722198303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5916,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3601975324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3601975324},{"type":"v1","addr":"172.21.15.156:6835","nonce":3601975324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6543,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/2219753729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2219753729},{"type":"v1","addr":"172.21.15.156:6834","nonce":2219753729}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[]} 2023-11-28T13:16:22.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.306+0000 7f785b5ff700 1 -- 172.21.15.156:0/2020599739 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78380682f0 msgr2=0x7f783806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:22.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.306+0000 7f785b5ff700 1 --2- 172.21.15.156:0/2020599739 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78380682f0 0x7f783806a7a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7850015040 tx=0x7f785000a000 comp rx=0 tx=0).stop 2023-11-28T13:16:22.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.306+0000 7f785b5ff700 1 -- 172.21.15.156:0/2020599739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f785415f060 msgr2=0x7f78540c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:16:22.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.306+0000 7f785b5ff700 1 --2- 172.21.15.156:0/2020599739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f785415f060 0x7f78540c8e70 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7844011aa0 tx=0x7f7844011db0 comp rx=0 tx=0).stop 2023-11-28T13:16:22.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.306+0000 7f785b5ff700 1 -- 172.21.15.156:0/2020599739 shutdown_connections 2023-11-28T13:16:22.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.306+0000 7f785b5ff700 1 --2- 172.21.15.156:0/2020599739 >> [v2:172.21.15.186:6832/130711,v1:172.21.15.186:6833/130711] conn(0x7f78380682f0 0x7f783806a7a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:22.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.306+0000 7f785b5ff700 1 --2- 172.21.15.156:0/2020599739 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f78540c93b0 0x7f785415d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:22.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.306+0000 7f785b5ff700 1 --2- 172.21.15.156:0/2020599739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7840003be0 0x7f785414a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:22.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.306+0000 7f785b5ff700 1 --2- 172.21.15.156:0/2020599739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f785415f060 0x7f78540c8e70 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:16:22.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.306+0000 7f785b5ff700 1 -- 172.21.15.156:0/2020599739 >> 172.21.15.156:0/2020599739 conn(0x7f7854072600 msgr2=0x7f78540b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:16:22.309 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.306+0000 7f785b5ff700 1 -- 172.21.15.156:0/2020599739 shutdown_connections 2023-11-28T13:16:22.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-28T13:16:22.306+0000 7f785b5ff700 1 -- 172.21.15.156:0/2020599739 wait complete. 2023-11-28T13:16:22.310 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 34 2023-11-28T13:16:22.326 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-28T13:16:22.327 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-28T13:16:22.327 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T13:16:22.327 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:16:22.341 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T13:16:22.341 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T13:16:22.341 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:16:22.395 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T13:16:22.395 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T13:16:22.395 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:16:22.412 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T13:16:22.412 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T13:16:22.413 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:16:22.469 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T13:16:22.469 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-28T13:16:22.470 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-28T13:16:22.470 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:16:22.508 INFO:tasks.ceph.osd.0:Stopped 2023-11-28T13:16:22.509 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-28T13:16:22.509 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:16:22.591 INFO:tasks.ceph.osd.1:Stopped 2023-11-28T13:16:22.591 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-28T13:16:22.591 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:16:22.640 INFO:tasks.ceph.osd.2:Stopped 2023-11-28T13:16:22.641 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-28T13:16:22.641 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:16:22.690 INFO:tasks.ceph.osd.3:Stopped 2023-11-28T13:16:22.690 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-28T13:16:22.690 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:16:22.736 INFO:tasks.ceph.osd.4:Stopped 2023-11-28T13:16:22.736 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-28T13:16:22.737 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:16:22.782 INFO:tasks.ceph.osd.5:Stopped 2023-11-28T13:16:22.783 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-28T13:16:22.783 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:16:22.830 INFO:tasks.ceph.osd.6:Stopped 2023-11-28T13:16:22.831 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-28T13:16:22.831 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:16:22.875 INFO:tasks.ceph.osd.7:Stopped 2023-11-28T13:16:22.875 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-28T13:16:22.876 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-28T13:16:22.876 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:16:22.937 INFO:tasks.ceph.mgr.y:Stopped 2023-11-28T13:16:22.937 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-28T13:16:22.938 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:16:23.004 INFO:tasks.ceph.mgr.x:Stopped 2023-11-28T13:16:23.004 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-28T13:16:23.005 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-28T13:16:23.005 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:16:23.026 INFO:tasks.ceph.mon.a:Stopped 2023-11-28T13:16:23.026 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-28T13:16:23.027 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:16:23.043 INFO:tasks.ceph.mon.b:Stopped 2023-11-28T13:16:23.044 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-28T13:16:23.044 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:16:23.102 INFO:tasks.ceph.mon.c:Stopped 2023-11-28T13:16:23.103 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-28T13:16:23.103 DEBUG:teuthology.orchestra.run.smithi156:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2023-11-28T13:16:23.192 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T13:16:23.192 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-28T13:16:23.452 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T13:16:23.452 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-28T13:16:23.613 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T13:16:23.614 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-28T13:16:23.763 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T13:16:23.763 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-28T13:16:23.924 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T13:16:23.925 DEBUG:teuthology.orchestra.run.smithi186:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-28T13:16:24.171 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T13:16:24.171 DEBUG:teuthology.orchestra.run.smithi186:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-28T13:16:24.290 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T13:16:24.290 DEBUG:teuthology.orchestra.run.smithi186:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-28T13:16:24.411 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T13:16:24.412 DEBUG:teuthology.orchestra.run.smithi186:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-28T13:16:24.563 INFO:tasks.ceph:Archiving mon data... 2023-11-28T13:16:24.565 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469974/data/mon.a.tgz 2023-11-28T13:16:24.565 DEBUG:teuthology.orchestra.run.smithi156:> mktemp 2023-11-28T13:16:24.611 INFO:teuthology.orchestra.run.smithi156.stdout:/tmp/tmp.3XAvAmDK6v 2023-11-28T13:16:24.611 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.3XAvAmDK6v 2023-11-28T13:16:24.797 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0666 /tmp/tmp.3XAvAmDK6v 2023-11-28T13:16:24.912 DEBUG:teuthology.orchestra.remote:smithi156:/tmp/tmp.3XAvAmDK6v is 298KB 2023-11-28T13:16:25.010 DEBUG:teuthology.orchestra.run.smithi156:> rm -fr /tmp/tmp.3XAvAmDK6v 2023-11-28T13:16:25.054 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469974/data/mon.b.tgz 2023-11-28T13:16:25.054 DEBUG:teuthology.orchestra.run.smithi186:> mktemp 2023-11-28T13:16:25.099 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.h8Z31JBvmt 2023-11-28T13:16:25.099 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.h8Z31JBvmt 2023-11-28T13:16:25.283 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.h8Z31JBvmt 2023-11-28T13:16:25.397 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.h8Z31JBvmt is 304KB 2023-11-28T13:16:25.496 DEBUG:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.h8Z31JBvmt 2023-11-28T13:16:25.539 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469974/data/mon.c.tgz 2023-11-28T13:16:25.540 DEBUG:teuthology.orchestra.run.smithi186:> mktemp 2023-11-28T13:16:25.622 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.G5c3G4hOhp 2023-11-28T13:16:25.623 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.G5c3G4hOhp 2023-11-28T13:16:25.794 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.G5c3G4hOhp 2023-11-28T13:16:25.908 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.G5c3G4hOhp is 304KB 2023-11-28T13:16:26.009 DEBUG:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.G5c3G4hOhp 2023-11-28T13:16:26.052 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-28T13:16:26.052 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T13:16:26.059 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T13:16:26.161 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-28T13:16:26.161 DEBUG:teuthology.orchestra.run.smithi156:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T13:16:26.167 DEBUG:teuthology.orchestra.run.smithi186:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T13:16:26.242 INFO:teuthology.orchestra.run.smithi156.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T13:16:26.253 INFO:teuthology.orchestra.run.smithi186.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T13:16:26.269 INFO:tasks.ceph:Archiving crash dumps... 2023-11-28T13:16:26.273 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469974/remote/smithi156/crash 2023-11-28T13:16:26.274 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T13:16:26.342 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469974/remote/smithi186/crash 2023-11-28T13:16:26.343 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T13:16:26.410 INFO:tasks.ceph:Shutting down logrotate 2023-11-28T13:16:26.410 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:16:26.415 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:16:26.545 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T13:16:26.613 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T13:16:26.682 INFO:tasks.ceph:Compressing logs... 2023-11-28T13:16:26.682 DEBUG:teuthology.orchestra.run.smithi156:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T13:16:26.690 DEBUG:teuthology.orchestra.run.smithi186:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T13:16:28.796 INFO:tasks.ceph:Archiving logs... 2023-11-28T13:16:28.796 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469974/remote/smithi156/log 2023-11-28T13:16:28.797 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T13:16:29.424 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469974/remote/smithi186/log 2023-11-28T13:16:29.425 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T13:16:29.719 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-28T13:16:29.729 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-28T13:16:29.730 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T13:16:29.734 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T13:16:29.866 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-28T13:16:29.876 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-28T13:16:29.876 DEBUG:teuthology.orchestra.run.smithi156:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T13:16:29.881 DEBUG:teuthology.orchestra.run.smithi186:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T13:16:29.919 INFO:teuthology.orchestra.run.smithi156.stderr:bash: ntpq: command not found 2023-11-28T13:16:29.923 INFO:teuthology.orchestra.run.smithi186.stderr:bash: ntpq: command not found 2023-11-28T13:16:29.926 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T13:16:29.927 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2023-11-28T13:16:29.927 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 64 +799us[ +825us] +/- 76ms 2023-11-28T13:16:29.927 INFO:teuthology.orchestra.run.smithi156.stdout:^* hv02.front.sepia.ceph.com 3 6 377 64 -415us[ -388us] +/- 73ms 2023-11-28T13:16:29.927 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 0 -300us[ -300us] +/- 89ms 2023-11-28T13:16:29.928 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T13:16:29.930 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T13:16:29.930 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2023-11-28T13:16:29.930 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 62 +772us[ +772us] +/- 76ms 2023-11-28T13:16:29.931 INFO:teuthology.orchestra.run.smithi186.stdout:^* hv02.front.sepia.ceph.com 3 6 377 63 -400us[ -347us] +/- 73ms 2023-11-28T13:16:29.931 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 -322us[ -269us] +/- 88ms 2023-11-28T13:16:29.931 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T13:16:29.932 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-28T13:16:29.944 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-28T13:16:29.946 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-28T13:16:29.959 DEBUG:teuthology.orchestra.run.smithi156:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T13:16:30.143 DEBUG:teuthology.orchestra.run.smithi186:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T13:16:30.307 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T13:16:30.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:16:30.386 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T13:16:30.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:16:30.462 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-28T13:16:30.472 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-28T13:16:30.482 INFO:teuthology.task.internal:Duration was 1495.459910 seconds 2023-11-28T13:16:30.482 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-28T13:16:30.492 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-28T13:16:30.492 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T13:16:30.497 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T13:16:30.611 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T13:16:30.616 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T13:16:30.781 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-28T13:16:30.781 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi156.front.sepia.ceph.com 2023-11-28T13:16:30.782 DEBUG:teuthology.orchestra.run.smithi156:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T13:16:30.841 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi186.front.sepia.ceph.com 2023-11-28T13:16:30.841 DEBUG:teuthology.orchestra.run.smithi186:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T13:16:30.900 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-28T13:16:30.900 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T13:16:30.905 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T13:16:30.991 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-28T13:16:30.991 DEBUG:teuthology.orchestra.run.smithi156:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T13:16:30.996 DEBUG:teuthology.orchestra.run.smithi186:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T13:16:31.126 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-28T13:16:31.136 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-28T13:16:31.137 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T13:16:31.206 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T13:16:31.277 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-28T13:16:31.288 DEBUG:teuthology.orchestra.run.smithi156:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T13:16:31.293 DEBUG:teuthology.orchestra.run.smithi186:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T13:16:31.356 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = core 2023-11-28T13:16:31.360 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = core 2023-11-28T13:16:31.401 DEBUG:teuthology.orchestra.run.smithi156:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T13:16:31.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:16:31.442 DEBUG:teuthology.orchestra.run.smithi186:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T13:16:31.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:16:31.484 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-28T13:16:31.494 INFO:teuthology.task.internal:Transferring archived files... 2023-11-28T13:16:31.495 DEBUG:teuthology.misc:Transferring archived files from smithi156:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469974/remote/smithi156 2023-11-28T13:16:31.496 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T13:16:31.579 DEBUG:teuthology.misc:Transferring archived files from smithi186:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469974/remote/smithi186 2023-11-28T13:16:31.580 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T13:16:31.663 INFO:teuthology.task.internal:Removing archive directory... 2023-11-28T13:16:31.664 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T13:16:31.669 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T13:16:31.712 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-28T13:16:31.731 INFO:teuthology.task.internal:Not uploading archives. 2023-11-28T13:16:31.732 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-28T13:16:31.745 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-28T13:16:31.745 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T13:16:31.753 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T13:16:31.796 INFO:teuthology.orchestra.run.smithi156.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 13:16 /home/ubuntu/cephtest 2023-11-28T13:16:31.797 INFO:teuthology.orchestra.run.smithi186.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 13:16 /home/ubuntu/cephtest 2023-11-28T13:16:31.800 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-28T13:16:31.813 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-28T13:16:31.836 INFO:teuthology.nuke:Checking targets against current locks 2023-11-28T13:16:31.863 DEBUG:teuthology.nuke:shortname: smithi156 2023-11-28T13:16:31.864 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T13:16:31.889 DEBUG:teuthology.nuke:shortname: smithi186 2023-11-28T13:16:31.890 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T13:16:31.893 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469974', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 12:42:17.443992', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T13:16:31.912 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469974', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 12:42:17.445454', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T13:16:31.946 INFO:teuthology.orchestra.console.smithi156:Power off 2023-11-28T13:16:31.947 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T13:16:31.970 DEBUG:teuthology.orchestra.console.smithi156:power off output: Chassis Power Control: Down/Off 2023-11-28T13:16:31.971 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:16:31.987 INFO:teuthology.orchestra.console.smithi186:Power off 2023-11-28T13:16:31.987 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T13:16:32.001 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2023-11-28T13:16:32.015 DEBUG:teuthology.orchestra.console.smithi186:power off output: Chassis Power Control: Down/Off 2023-11-28T13:16:32.015 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:16:32.041 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-11-28T13:16:36.002 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:16:36.042 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:16:36.114 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2023-11-28T13:16:36.154 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-11-28T13:16:40.116 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:16:40.155 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:16:40.226 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is off 2023-11-28T13:16:40.266 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2023-11-28T13:16:40.327 INFO:teuthology.orchestra.console.smithi156:Power off completed 2023-11-28T13:16:40.367 INFO:teuthology.orchestra.console.smithi186:Power off completed 2023-11-28T13:16:40.472 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 client.1-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} duration: 1495.4599103927612 failure_reason: 'Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=7ecd282109e64400a18f20efba006070 status: fail success: false 2023-11-28T13:16:40.473 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T13:16:40.556 INFO:teuthology.run:FAIL